I'm thinking about using this and have a few questions:
1. How are hardlinks and duplicate files (same content, different paths) handled?
2. Does deduplication work on a block/chunk level for partially matching files, or does it only look at whole files?
3. Is there any specific integration or handling for Copy-on-Write (CoW)?
Thanks for building this, I am just about to give it ago with my self-hosted Garage cluster.
Does running `stat` against a file require pulling the whole file from s3, or can that be handled by the metadata?
Do you know what backup performance is like for something like borg/borgmatic or restic, especially on follow up runs where most files are just checked.
Is there any particular Redis/Valkey config you recommend when using it for `conditional_put`, or just default config?
stat doesn’t pull the file contents from S3; it only accesses the metadata tree, which is usually cached.
I haven’t benchmarked Borg or Restic specifically. Sequential writes can comfortably reach several Gbit/s. For follow-up runs, if they only stat unchanged files, that should stay entirely in metadata.
The default Redis/Valkey configuration should work fine for conditional_put.
NFSv4 is unlikely for now. It would add a lot of surface area, and I’m pretty happy with where the 9P extensions are today.
I really want this, but it feels a little scary to trust all my files. I wouls like it if there was some contineous suite trying to corrupt the files and then see the failure cases!
That’s a fair concern. The closest thing right now is a deterministic simulation suite that injects storage faults and crashes at arbitrary points, then checks the recovered data against reference models. It runs hourly with fresh seeds.
Also would love to know how was we could iterate over the files. ie a bit how duckdb allows for paruet fikes scanning and reading would be nice to see how fast we can query the fs for ai/ml training workloads
It’s been quite a ride building ZeroFS, and I’m happy to answer any questions.
Does running `stat` against a file require pulling the whole file from s3, or can that be handled by the metadata?
Do you know what backup performance is like for something like borg/borgmatic or restic, especially on follow up runs where most files are just checked.
Is there any particular Redis/Valkey config you recommend when using it for `conditional_put`, or just default config?
Is there any chance for NFSv4 support?
stat doesn’t pull the file contents from S3; it only accesses the metadata tree, which is usually cached.
I haven’t benchmarked Borg or Restic specifically. Sequential writes can comfortably reach several Gbit/s. For follow-up runs, if they only stat unchanged files, that should stay entirely in metadata.
The default Redis/Valkey configuration should work fine for conditional_put. NFSv4 is unlikely for now. It would add a lot of surface area, and I’m pretty happy with where the 9P extensions are today.
CI also runs pjdfstest, xfstests, stress-ng, ZFS scrubs, and Jepsen crash/failover tests: https://github.com/Barre/ZeroFS#testing
ZeroFS is still pretty young as storage projects go, so I completely understand wanting to see it prove itself over time.
Opened with Safari in iOS