Yes but it's not easy to find random Wikipedia user pages, or even find the specific talk page comment in the archives without knowing what to look for. Go ahead, find a friend, give them no clues, and see if they find it.
"The answer to this is very simple. It was a joke. It had to be a number, an ordinary, smallish number, and I chose that one. Binary representations, base thirteen, Tibetan monks are all complete nonsense. I sat at my desk, stared into the garden and thought '42 will do' I typed it out. End of story." from the man himself[1]
...but let us not ruin a good story with the truth. Remember why earth was built. The "real" answer might then be flowing in the ether.
Talk pages aren’t valid sources in general. In this case the author is dead and an established expert having published in the field, so I guess it’s fine, but I wouldn’t bet.
If you had told me an article ostensibly about a file format would have me teary-eyed by the end I wouldn't have believed you. This is beautiful, thank you!
A huge number of us labor behind the scenes with no public acknowledgment or credit. For each idea brought to life we might hope for -at most- an epitaph carved in expired patent claims.
It is, indeed. Thanks for writing that up, it touched me.
And thanks for being the historian of our culture-that-eschews culture (or so it seems to me sometimes, that tech tries to exist in a perennial present without acknowledging it's roots and history)
Thanks! If you look at his (logged-in) edits on Wikipedia (https://en.wikipedia.org/wiki/Special:Contributions/Scarlsen ), then apart from the lone comment on the talk page (about the reason for "42") and creating that user page, he has two edits to the TIFF article:
- and the other is even more interesting: though he is being scrupulous and removing a sentence that has no published citations, in his edit summary he confirms that it is basically true:
> The author of the original TIFF specification wanted TIFF to stand for "The Image File Format", but he was overruled by Aldus' president Paul Brainerd on the grounds that it sounded presumptuous.
(The edit summary says: Removed the "The Image File Format" sentence, since it only has eye-witness support (me, for one), but no published citatations)
> Primary sources that have been reputably published may be used in Wikipedia, but only with care
and I imagine a Wikipedia edit summary does not count as a reliable source. (For one thing, despite it being very plausible that the Wikipedia user Scarlsen who signed himself as Stephen E Carlsen is indeed that person—I believe it completely!—it cannot be guaranteed that it wasn't an impostor, for example.)
That would be what Wikipedia calls "original research". A big no-no on wikipedia. At a minimum he would have to tweet or blog about it and link the tweet or blog. And even then that's a primary source, which wikipedia considers less valuable. Ideally he would get someone else to report on his tweet/blog and use that as source. Then the wikipedia gods are happy
Technically yes, but I'm fairly sure Wikipedia wants cited sources, not "I'm the guy, I said so" anecdotal sources.
Of course, if he was still alive he could have written a blog post or something like that and use that as a source, much like how it's likely this blog post will be used as a source for things surrounding the format and person.
>Yes it is true: the second word of a TIFF file, 42, was indeed taken from the answer to the Ultimate Question of Life, the Universe, and Everything, from Hitchhikers_guide_to_the_galaxy. StephenECarlsen 23:38, 12 October 2007 (UTC)
Hey John, I'm just curious how people find these comments about "would be nice if X saw this" on HN. I don't think there's any pinging behavior. Did somebody message you? Did you just happen to read it? Do you have an eldritch curse that summons you when called by name?
I bet Musk hacked something together (or has a column in TweetDeck if that's still around) that continuously searches for mentions. I wonder if there's a tool like that that covers more of the internet, although the primary users would only be famous people and/or their agents / social media staff.
There are lots of tools that do this if you are willing to pay for it. "Social Media Monitoring" or "Brand Monitoring" would be the keywords to search for
From what I've seen, sometimes people see that their blog post got a lot of referral traffic from Hacker News and they come have a look to see where it came from.
Not who you're asking for, but generally I think it's just a case of the author also being an HN regular. Although, I suppose you could set up some Google Alerts for mentions of your blog posts.
I have a habit of filling my head with all kinds of trivia/information. Sometimes I think this is a useless habit.
Not today. I will try to remember the name of Mr. Stephen Carlsen as the inventor of TIFF format as long as I can. As a mediocre programmer, it is the least amount of respect I can pay for an unsung but talented engineer of an era that is fast going past us.
The great thing about TIF was it's extensibility. Flexible (data could be stored as tiles or in stripes), multiple compression options etc.
Well documented spec, easy to bolt on extras either as public tags - GeoTIFFs added projection metadata - or private, for your own needs.
Back in the day, to improve a desktop application's performance I found it was simple to create a custom reader and writer to handle cases where tiles were completely one single colour removing the need to decompress at run time.
Perhaps the greatest thing about TIFF, but also the most horrible things, and probably why TIFF is mostly historical. It was so extendable that no two programs ever accepted the exact same TIFF extensions. (omitting the war story)
But most of these variations were part of the spec (endianness with II or MM, later magic 43 for bigTIFF 64bit extension). I work with tiff and tiff-derived formats in digital microscopy where its very much not historical. And the alternatives (DICOM supp 145, vendor-specific garbage ... and thats it) are worse.
I quite like the format, the only thing I would change is to have the option not to store directory information in a linked list spread throughout the file but in a simple array. Duplicate it at the beginning and end of the file and you've got resilience too (important in the age of floppies)
We have hundreds of thousands of TIFF files where I work which are scans of questionnaires filled out by clinical trial participants. The one annoying thing is that web browsers don’t natively display them. I did some incredibly inefficient JavaScript bs to decode the pixel data, plop it in a canvas, get a PNG data url from it, and set that as the src for an img element xD (why not just display the canvas? because I was too lazy to manually handle resizes…) good times
TIFF is still very much alive in certain circles, see for example https://cogeo.org/
The format is basically a TIFF file with attached georeferencing information and with the data organized by geographical sector, enabling fast downloads of regional subsets.
The Adobe DNG standard for raw camera images is based on TIFF as well. DNG is used in lots of places, including the raw capture support built into all modern iOS and Android smartphones.
I’ve been using both TIFF and DNG this very week in my work (https://filmlabapp.com), so I was happy to read this post and learn about Steve Carlsen aka Mr. TIFF, whose work we’re still building on 39 years later.
Not the .IA files from old Sinar digital backs! Those are based on DOOM PWADs (lol). But otherwise mostly yes this is true for nearly every other format as far as I am aware.
I'm not super knowledgeable about this stuff.. but out of curiosity, what advantage does it have over NetCDF?
I wrote a program processing GeoTIFF data. When I had started this project I chose GeoTIFF mostly b/c i wanted something simple. And I could load them in to Java's BufferImage class and manipulate them that way. But it seems all the pros exclusively use NetCDF and GeoTiffs are for noobs (working with atmospheric science data here)
GeoTIFF does extend "images" to cover more usecases, but a lot of stuff doesn't fit (like say a wind vector) and then you need some other container or metadata b/c you generally have many images. So I get the sense the complexity just ends up being moved elsewhere.
Generally speaking I would classify TIFF and its variants as imaging formats (or for very simple numerical datasets), and NetCDF as more suited for raw data, in particular multi-dimensional data with time series, etc.
For forecast and climatological data I find NetCDF is vastly superior, but also much more complicated to work with due to the capabilities and how open the format is. Just have a look at the complexity of the CF Conventions to see what I mean: https://cfconventions.org/cf-conventions/release/v1.12.0/cf-...
For visualizing orthophotos and the like, I would choose GeoTIFF any day of the week, as they're easy to visualize across platforms using existing libraries. Using COGs you also get the functionality of a spatial index within each GeoTIFF file, meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
Almost any digital camera RAW format is TIFF inside. And you can see how much kludges good metadata library needs to read all of them: offsets from the IFD, offsets from beginning of file with or without header, offsets from fields in IFD, etc, etc, etc. You take TIFF, you change header to make your format, and then you cannot implement this TIFF properly!
Even DNG (which is tiff inside) is mangled by camera firmware authors!
And the (early) availability of well made library, LibTIFF by Sam Leffler.
I used it extensively from 1995 on, but only found out that according to Wikipedia is dates back to 1988!
The article is great but the web site is supposedly related to a book "inventing the future".. which is nowhere to be found. Other than a big, slowly loading graphic, 3 posts and indexes for the book... the site doesn't provide a clue about where to acquire the actual (PDF only?) book.
I assume you have to sign up to find out more ?
On the web I can only find articles about the book.
So.. what is the deal in making the actual book hard to find ?
Based on the quality of the article, the subject matter of the book being right in the center of my wheelhouse and the references I could find on the internet, I just ordered a copy (apparently a paper copy), look forward reading it.
I had a similar issue, clicking the author's name gets you to a decent page, but yeah I'd actually prefer if he made it a bit easier to buy the book! I'll have to get it now after such a nice article
This page [0] still links to (when clicking the main image) an oddish seeming corner of your site that mentions purchasing the book but has no link to it
I had downloaded the final Aldus TIFF specifications document, hoping to find the author’s name. However, the name is seemingly written in white text on white paper - making it invisible. What?
Is there an explanation for this that I missed? Was it an Easter Egg left by the author?
Just as a side note, there are two versions of tiff6.pdf (titled “TIFF, Revision 6.0, Final — June 3, 1992”) on the ’net: one[1] that mentions Aldus on the title page and one[2] that mentions Adobe. Only the Aldus one has the invisitext. (Curiously, the metadata says it’s newer.)
[2] SHA256: 8cb1e1a2226e423ba8b88f57366a30ef1b7ad6109443ebdda072b952739a8d76, modify date 1995-09-14, created by Acrobat Distiller 2.1 for Power Macintosh, e.g. https://download.osgeo.org/libtiff/doc/TIFF6.pdf
I've found a number of hidden items in PDFs masked in the same way. The one that comes to mind immediately are Dell product code names hiding in official spec PDFs. (The PowerConnect 6200-series switches are "Kinnick", for example.) I always assumed it was a lazy redaction method and people weren't necessarily aware the text wasn't actually redacted.
the article shows scans of the research reports listing only Carlsen as their author, you could have just linked to one in the first paragraph of the Wikipedia page to support his sole inventorhood, right
I participated in creating a history book, in regard to an organization in which I’m involved.
It took eight years, and was a lot of work. The process that he mentioned is quite familiar. Many of the folks we interviewed have since passed away. Some, before the book was complete.
Threads like this are, in a sense, like a digital wake - we can all mourn Carlsen a little bit now. I remember the first time I saw a .TIF - on Rainbow Paint, a free paint program bundled with a Dexxa mouse my old man bought as an upgrade for our 286. To me, running across a random .tif somewhere was such a delight, something I could open as a surprise, maybe re-use parts of, zoom into, etc. or share on a disk with a friend. It's quaint, now...
Respect to those unsung engineers who made such lasting contributions, and to the author as well. This kind of work is not easy, but truly meaningful.
I do have a question, though: shouldn’t the creation of industry standards also allow individual attribution, similar to how patents credit inventors?
Did a similar deep dive for one of the posters for the cult classic movie Possession (1981). Just giving random phone numbers a call is incredibly effective, lots of people are happy to reminisce about old work and have great stories.
Very often these people are so humble and so amazed to find that anyone cares so much about some little project they did. I've brought some people to comic cons and they have been blown away by the fans they never knew they had. (and they always have fascinating industry stories to tell)
It’s so inspiring to see someone spend years uncovering the real people behind tech we use every day. This kind of dedication keeps our digital history alive.
TIFF indeed -- I recall the floppy disk for Mac mailed from Seattle with the TIFF spec printed on paper. A few weeks later, another graphics editor with TIFF support. I never, ever heard the name Carlsen until today. Thank you for this article
I had exposure to TIFF files shortly after the format creation in 1985/86, before the final form specification in 1992.
Not mentioned in either the article or the tail end wikipedia article iamge was the early adoption of TIFF by the mapping and geodetic community to store raster line data (maps, images, and raw sat and instrument platform multichannel line data).
The tagging format made the embedding of spheroids, datums, projections, origins, lens and focal specifications relatively easy (plus or minus the usual Tower of Babel Tag Naming and Meaning Confusion).
Please do not let my comment take away your enjoyment of the article.
I hate to nit-pick on such a beautiful story but that it ended with a faux-Ghibli profile picture is just sad.
How can someone working so hard to humanize technology and preserve history, justify this soul-less commodification of art? Do the animators deserve to get treated as anonymous model trainers without their consent, names and frames lost in a dead ocean of bit-vectors?
Crazy this information would have probably been lost in time if one single person on this planet didn’t give a shit like the rest of us.
What a journey and congratulations to SC (don't want to spoil it) on your 15 minutes and rightful restoration as inventor of TIFF, take your place in history.
thanks righthand, i guess it was just curiosity that led me down the path. most people do give a sh## but i hear you. i also had the time to search, as i wasn't super busy with work.
I'd like to add, it's really nice when good people get recognized purely for their work, rather than because they're loud enough to be noticed. I very much appreciate people like yourself that take the time to look for the quiet ones like this.
Thank you so much again for your efforts, I sound brash but this really is inspiring and as you demonstrated and have indicated, we can all use our free time to easily make the world a little more accurate and better.
Glad that the information was preserved in the magazines, usenet messages and just text files. That will not happen with the modern web software, the internet is the dark ages of our time. All those Java,Flash amazing pieces of software and the stories of their creators will be gone long before the internet dies from LLM slop.
And Geocities, Vine, Google+, Anglefire, Tripod, Xoom, Homestead, Lycos communities, AOL Hometown, MSN Groups, 50megs.com, etc, etc.... not to mention small specialty sites like em411.com. All that content/history, just poof.
They found the Vine archives recently. Doesn't mean they'll get uploaded as Musk wants the new Vine to just be AI waifus. But the files still exist on a disk or tape somewhere.
I just remembered Orkut. Though I suspect Google has backups or Orkut and Google+ somewhere. I wonder if Yahoo Answers is still on a tape somewhere?
Back when I was working in the book publishing industry (writing and typesetting computer books using Quark XPress [1] on the old MacOS [version 7, 8 and 9 IIRC], before Adobe's InDesign [2] ruled the earth), TIFF was all the rage. Probably still is.
I think the reason TIFF was so prevalent was it already had support for CMYK color space (even though many books were printed in black and white) and for lossless compression (as TFA mentions).
It was a "one size fits all" format and so our 100 or 250 MB (!) Zip drives [3] exchanged between authors/publisher/typesetters often contained TIFF files.
> For as long as I have published my books, one of my overarching goals was to give credit to those who actually invented the hardware and software that we use.
https://web.archive.org/web/20210108174645/https://www.adobe...
"The answer to this is very simple. It was a joke. It had to be a number, an ordinary, smallish number, and I chose that one. Binary representations, base thirteen, Tibetan monks are all complete nonsense. I sat at my desk, stared into the garden and thought '42 will do' I typed it out. End of story." from the man himself[1]
...but let us not ruin a good story with the truth. Remember why earth was built. The "real" answer might then be flowing in the ether.
[1] https://en.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%27...
Edit: from the other comment, it appears it was in fact random...
Umm. The ASCII code for h is 102 ;)
[0] https://news.ycombinator.com/user?id=taviso
[1] https://en.wikipedia.org/wiki/Tavis_Ormandy
https://en.wikipedia.org/wiki/Wikipedia:Verifiability#Self-p...
[1]https://www.mountainviewtacoma.com/obituaries/stephen-carlse...
This story is touching.
And thanks for being the historian of our culture-that-eschews culture (or so it seems to me sometimes, that tech tries to exist in a perennial present without acknowledging it's roots and history)
https://en.wikipedia.org/wiki/User:Scarlsen
Turns out the answer was on Wikipedia already :).
- one of them clarifies the (non-)involvement of Microsoft: https://en.wikipedia.org/w/index.php?title=TIFF&diff=prev&ol...
- and the other is even more interesting: though he is being scrupulous and removing a sentence that has no published citations, in his edit summary he confirms that it is basically true:
> The author of the original TIFF specification wanted TIFF to stand for "The Image File Format", but he was overruled by Aldus' president Paul Brainerd on the grounds that it sounded presumptuous.
(The edit summary says: Removed the "The Image File Format" sentence, since it only has eye-witness support (me, for one), but no published citatations)
> Primary sources that have been reputably published may be used in Wikipedia, but only with care
and I imagine a Wikipedia edit summary does not count as a reliable source. (For one thing, despite it being very plausible that the Wikipedia user Scarlsen who signed himself as Stephen E Carlsen is indeed that person—I believe it completely!—it cannot be guaranteed that it wasn't an impostor, for example.)
https://en.wikipedia.org/wiki/Wikipedia:No_original_research
Of course, if he was still alive he could have written a blog post or something like that and use that as a source, much like how it's likely this blog post will be used as a source for things surrounding the format and person.
>Yes it is true: the second word of a TIFF file, 42, was indeed taken from the answer to the Ultimate Question of Life, the Universe, and Everything, from Hitchhikers_guide_to_the_galaxy. StephenECarlsen 23:38, 12 October 2007 (UTC)
Not today. I will try to remember the name of Mr. Stephen Carlsen as the inventor of TIFF format as long as I can. As a mediocre programmer, it is the least amount of respect I can pay for an unsung but talented engineer of an era that is fast going past us.
Well documented spec, easy to bolt on extras either as public tags - GeoTIFFs added projection metadata - or private, for your own needs.
Back in the day, to improve a desktop application's performance I found it was simple to create a custom reader and writer to handle cases where tiles were completely one single colour removing the need to decompress at run time.
Thank you TIFf!
edit: forgot about byte order...
I quite like the format, the only thing I would change is to have the option not to store directory information in a linked list spread throughout the file but in a simple array. Duplicate it at the beginning and end of the file and you've got resilience too (important in the age of floppies)
I’ve been using both TIFF and DNG this very week in my work (https://filmlabapp.com), so I was happy to read this post and learn about Steve Carlsen aka Mr. TIFF, whose work we’re still building on 39 years later.
I wrote a program processing GeoTIFF data. When I had started this project I chose GeoTIFF mostly b/c i wanted something simple. And I could load them in to Java's BufferImage class and manipulate them that way. But it seems all the pros exclusively use NetCDF and GeoTiffs are for noobs (working with atmospheric science data here)
GeoTIFF does extend "images" to cover more usecases, but a lot of stuff doesn't fit (like say a wind vector) and then you need some other container or metadata b/c you generally have many images. So I get the sense the complexity just ends up being moved elsewhere.
For forecast and climatological data I find NetCDF is vastly superior, but also much more complicated to work with due to the capabilities and how open the format is. Just have a look at the complexity of the CF Conventions to see what I mean: https://cfconventions.org/cf-conventions/release/v1.12.0/cf-...
For visualizing orthophotos and the like, I would choose GeoTIFF any day of the week, as they're easy to visualize across platforms using existing libraries. Using COGs you also get the functionality of a spatial index within each GeoTIFF file, meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
Almost any digital camera RAW format is TIFF inside. And you can see how much kludges good metadata library needs to read all of them: offsets from the IFD, offsets from beginning of file with or without header, offsets from fields in IFD, etc, etc, etc. You take TIFF, you change header to make your format, and then you cannot implement this TIFF properly!
Even DNG (which is tiff inside) is mangled by camera firmware authors!
https://en.wikibooks.org/wiki/PSP/Homebrew_History?useskin=v...
The article is great but the web site is supposedly related to a book "inventing the future".. which is nowhere to be found. Other than a big, slowly loading graphic, 3 posts and indexes for the book... the site doesn't provide a clue about where to acquire the actual (PDF only?) book.
I assume you have to sign up to find out more ?
On the web I can only find articles about the book.
So.. what is the deal in making the actual book hard to find ?
Edit: I think I cracked the code: Click Home, Open "Close Your Rings" article, scroll all the way down, find link: https://books.by/john-buck?ref=inventingthefuture.ghost.io
Based on the quality of the article, the subject matter of the book being right in the center of my wheelhouse and the references I could find on the internet, I just ordered a copy (apparently a paper copy), look forward reading it.
This page [0] still links to (when clicking the main image) an oddish seeming corner of your site that mentions purchasing the book but has no link to it
[0] https://inventingthefuture.ghost.io/
[1] SHA256: dbcdf729182937ecff415dfd06806894bf03bfd741291aa3ad7ba45335673def, modify date 2002-05-10, created by Acrobat Distiller 4.05 for Windows, e.g. https://www.itu.int/itudoc/itu-t/com16/tiff-fx/docs/tiff6.pd...
[2] SHA256: 8cb1e1a2226e423ba8b88f57366a30ef1b7ad6109443ebdda072b952739a8d76, modify date 1995-09-14, created by Acrobat Distiller 2.1 for Power Macintosh, e.g. https://download.osgeo.org/libtiff/doc/TIFF6.pdf
It took eight years, and was a lot of work. The process that he mentioned is quite familiar. Many of the folks we interviewed have since passed away. Some, before the book was complete.
Not mentioned in either the article or the tail end wikipedia article iamge was the early adoption of TIFF by the mapping and geodetic community to store raster line data (maps, images, and raw sat and instrument platform multichannel line data).
The tagging format made the embedding of spheroids, datums, projections, origins, lens and focal specifications relatively easy (plus or minus the usual Tower of Babel Tag Naming and Meaning Confusion).
I hate to nit-pick on such a beautiful story but that it ended with a faux-Ghibli profile picture is just sad.
How can someone working so hard to humanize technology and preserve history, justify this soul-less commodification of art? Do the animators deserve to get treated as anonymous model trainers without their consent, names and frames lost in a dead ocean of bit-vectors?
Thank you for all the efforts that went into preserving the memories of those that built the world around us.
What a journey and congratulations to SC (don't want to spoil it) on your 15 minutes and rightful restoration as inventor of TIFF, take your place in history.
It's weird to see times one has lived through presented as ancient history....
I just remembered Orkut. Though I suspect Google has backups or Orkut and Google+ somewhere. I wonder if Yahoo Answers is still on a tape somewhere?
I now adopted the practice of recovering the texts I deem worthy from way back machine and downloading all yt videos I really like locally.
But ofc one day I’ll also hit the bucket; still have to work out a contingency plan for my archive for that …
I think the reason TIFF was so prevalent was it already had support for CMYK color space (even though many books were printed in black and white) and for lossless compression (as TFA mentions).
It was a "one size fits all" format and so our 100 or 250 MB (!) Zip drives [3] exchanged between authors/publisher/typesetters often contained TIFF files.
> For as long as I have published my books, one of my overarching goals was to give credit to those who actually invented the hardware and software that we use.
So thank you Mr. Stephen "TIFF" Carlsen!
[1] https://en.wikipedia.org/wiki/QuarkXPress
[2] https://en.wikipedia.org/wiki/Adobe_InDesign
[3] https://en.wikipedia.org/wiki/Zip_drive