Reminds me of when I tried to use the library of babel as a data compression tool. It led me down a fun rabbit hole and was my first introduction to information theory.
The conclusion being that you basically need the same amount of data to represent the address of your data as the data itself, so it's not really effective at compression, just a fun thought experiment.
The cool part of this in modern times is that LLMs are basically a form of lossy compression that actually achieves the gist of what these tools fail at. Although it is lossy, and requires a massive substrate. This is related to the idea of AI/LLMs being a form of language compression.
It is worth noting that as the length of data increases it becomes extremely unlikely that the index and length of the sequence within pi would actually be smaller than the data.
> The SDCS is only possible if keys are allowed to become infinite, or the data store is allowed to become infinite (...) This would, of course, make the idea useless.
But Pi is infinite. And thus this genius contraption will work as long as we have Moore's law on our side :)
It is actually not proven that the decimal expansion (or any rational base expansion) of pi contains all possible sequences of numbers. It sounds like it intuitively would be since the expansion is infinite, but it is not necessarily true. For example, the number 0.12345678910111213... (i.e., decimal formed by concatenating the digits of all decimal numbers in ascending order) is infinite, never-ending, and irrational but does not contain every sequence of numbers.
I vaguely remember an entry to a compression-benchmark that gamed the benchmark by treating the filename as part of the input to the decompression-algorithm, thus beating the metric that only measured the size of the file.
So does every other random infinite sequence of bits. The unintuitive part comes from infinity, not pi.
It also doesn't contain all past and future knowledge because it also contains all possible falsehoods about the past and future in a way that's indiscernible from the truth.
Encoding information as an offset into a pseudorandom sequence is no more storage efficient than storing the information directly.
And also all the days you don’t, so, by itself not very meaningful. Especially since you can’t tell which one is right in advance. In some sense, so does a calendar
> Matches that occur early enough in π to attain significant compression will not be varied. That is, it isn't possible to use π to compress interesting, real-world data because real-word strings are unlikely to arise early.
for
> Calculate the number of bits to encode that value using log2(938933556), which is ~29.8
This is roughly same as saying: "If you rewrite 938933556 as a binary number / usize, it will need 30 bits".
Sanity check: 1101111111|0110111111|0100110100 (| delimits every 10 bigits).
> Since the file is 128 bits long, one would expect this place to be around the 2*128th bit.
This statement is a bit more subtle - pi is not actually equidistributed really well. But as a first lvl approximation, we can see pi sort of as a RNG.
If we write pi (ignore the decimal point), as a binary number, we get:
11011001111111011110010101011110001010101111101101110001001100001...
You can... kind of squint and pretend this is a random sequence of 1s and 0s.
Now, if you had a file that is 128 bits (so lots of intermingling 0s and 1s), and each next digit of pi is effectively a coin flip. Pretend 1s are heads, and 0s are tails. You basically have to get the exact 128 consecutive coin flips of the same result as your file to get your file back.
Imagine now, PI not as a number, but a sequence of experiments of flipping the coin 128 times.
(11011..01000)(10000...00100)....
^attempt 1 ^attempt 2
You have to try, on expectation, quite a few times to win this game! Now, you could easily get lucky for sure. But on average, your chance of winning per attempt is roughly 0.5^128! So, how many times do you have to try to win this game?
> Why is this thing so slow? It took me five minutes to store a 400 line text file!
> Well, this is just an initial prototype, and don't worry, there's always Moore's law!
Seriously? They're only storing individual bytes in pi:
> In this implementation, to maximise performance, we consider each individual byte of the file separately, and look it up in π.
So the whole transformation should be trivially reducible to a 256-element lookup table from source byte to location in pi and a similar table used to convert back the other way. Maybe a fancy formula could be used for the (never actually encountered) case in which a byte is encoded by one of the infinite available noncanonical encodings.
This is probably a dumb question, but do we actually know that pi has an infinite number of decimal digits or are we assuming that it does because we haven’t developed a sufficiently powerful computer to calculate the last digit of pi?
I’m guessing this is something that could be formally proven?
It's amazing how inscrutable calculus can be when you return to reading it after not doing so for a period of time, much like lisp or forth. I don't think I've actually done an integral or taken a derivative in years. I can see the elegance of that proof but I'll be damned if I can actually follow the mathematics from one step to the next.
jshell> "πfs".toUpperCase()
$1 ==> "ΠFS"
Welcome to Node.js v26.3.0.
Type ".help" for more information.
> "πfs".toUpperCase()
'ΠFS'
Python 3.14.5 (main, May 10 2026, 10:21:34) [Clang 21.0.0 (clang-2100.0.123.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> "πfs".upper()
'ΠFS'
echo 'πfs' | awk '{print toupper($0)}'
ΠFS
Part of the joke is that, in this implementation, the metadata is guaranteed to be larger than the file:
> Now, we all know that it can take a while to find a long sequence of digits in π, so for practical reasons, we should break the files up into smaller chunks that can be more readily found.
> In this implementation, to maximise performance, we consider each individual byte of the file separately, and look it up in π.
Looked at the repo but it says NOTHING about what value this project offers.
I mean, I get that it's "fun" to store information within the digits of pi. But is this just amusement, or is there a value prop for production use here?
(Speaking as a math major, by the way. I'm sympathetic to the cause.)
This is interesting, but I feel like my use cases would better align with a different irrational number. Could I get an option to do this with e instead? /s
The conclusion being that you basically need the same amount of data to represent the address of your data as the data itself, so it's not really effective at compression, just a fun thought experiment.
The cool part of this in modern times is that LLMs are basically a form of lossy compression that actually achieves the gist of what these tools fail at. Although it is lossy, and requires a massive substrate. This is related to the idea of AI/LLMs being a form of language compression.
πfs – A data-free filesystem - https://news.ycombinator.com/item?id=36357466 - June 2023 (107 comments)
πfs – A data-free filesystem - https://news.ycombinator.com/item?id=28699499 - Sept 2021 (30 comments)
PiFS – The Data-Free Filesystem - https://news.ycombinator.com/item?id=26208704 - Feb 2021 (1 comment)
Πfs: Never worry about data again - https://news.ycombinator.com/item?id=21359338 - Oct 2019 (1 comment)
The π Filesystem for FUSE: Store Your Data in π - https://news.ycombinator.com/item?id=19223032 - Feb 2019 (1 comment)
pifs - Avoid disk space usage by saving your files in the digits of Pi - https://news.ycombinator.com/item?id=18687275 - Dec 2018 (1 comment)
πfs – A data-free filesystem - https://news.ycombinator.com/item?id=13869691 - March 2017 (105 comments)
Πfs: Stores your data in π - https://news.ycombinator.com/item?id=10856108 - Jan 2016 (1 comment)
Πfs: Never worry about data again - https://news.ycombinator.com/item?id=10847693 - Jan 2016 (1 comment)
File system that stores location of file in Pi - https://news.ycombinator.com/item?id=8018818 - July 2014 (98 comments)
100% Compression Using Pi - https://news.ycombinator.com/item?id=6698852 - Nov 2013 (32 comments)
(Reposts are fine after a year or so; links to past threads are just to satisfy extra-curious readers)
https://news.ycombinator.com/from?site=github.com/philipl
https://hn.algolia.com/
Further reading: https://en.wikipedia.org/wiki/Sloot_Digital_Coding_System
But Pi is infinite. And thus this genius contraption will work as long as we have Moore's law on our side :)
It also doesn't contain all past and future knowledge because it also contains all possible falsehoods about the past and future in a way that's indiscernible from the truth.
Encoding information as an offset into a pseudorandom sequence is no more storage efficient than storing the information directly.
(Fun fact: "Chrispratt" is an ancient Californian word that means "Joel McHale didn't want the role.")
Perfect crypto!
https://en.wikipedia.org/wiki/Normal_number
> Matches that occur early enough in π to attain significant compression will not be varied. That is, it isn't possible to use π to compress interesting, real-world data because real-word strings are unlikely to arise early.
> Calculate the number of bits to encode that value using log2(938933556), which is ~29.8
Can someone explain these two statements to me?
This is roughly same as saying: "If you rewrite 938933556 as a binary number / usize, it will need 30 bits".
Sanity check: 1101111111|0110111111|0100110100 (| delimits every 10 bigits).
> Since the file is 128 bits long, one would expect this place to be around the 2*128th bit.
This statement is a bit more subtle - pi is not actually equidistributed really well. But as a first lvl approximation, we can see pi sort of as a RNG.
If we write pi (ignore the decimal point), as a binary number, we get: 11011001111111011110010101011110001010101111101101110001001100001...
You can... kind of squint and pretend this is a random sequence of 1s and 0s.
Now, if you had a file that is 128 bits (so lots of intermingling 0s and 1s), and each next digit of pi is effectively a coin flip. Pretend 1s are heads, and 0s are tails. You basically have to get the exact 128 consecutive coin flips of the same result as your file to get your file back.
Imagine now, PI not as a number, but a sequence of experiments of flipping the coin 128 times. (11011..01000)(10000...00100).... ^attempt 1 ^attempt 2 You have to try, on expectation, quite a few times to win this game! Now, you could easily get lucky for sure. But on average, your chance of winning per attempt is roughly 0.5^128! So, how many times do you have to try to win this game?
[1] https://www.youtube.com/watch?v=JcJSW7Rprio
> Well, this is just an initial prototype, and don't worry, there's always Moore's law!
Seriously? They're only storing individual bytes in pi:
> In this implementation, to maximise performance, we consider each individual byte of the file separately, and look it up in π.
So the whole transformation should be trivially reducible to a 256-element lookup table from source byte to location in pi and a similar table used to convert back the other way. Maybe a fancy formula could be used for the (never actually encountered) case in which a byte is encoded by one of the infinite available noncanonical encodings.
I’m guessing this is something that could be formally proven?
0x123456789ABCDEF0
use this number as a shorter nibble storage alternative...
> Now, we all know that it can take a while to find a long sequence of digits in π, so for practical reasons, we should break the files up into smaller chunks that can be more readily found.
> In this implementation, to maximise performance, we consider each individual byte of the file separately, and look it up in π.
My favourite issue being about GDPR compliance https://github.com/philipl/pifs/issues/56
I mean, I get that it's "fun" to store information within the digits of pi. But is this just amusement, or is there a value prop for production use here?
(Speaking as a math major, by the way. I'm sympathetic to the cause.)