Schrödinger’s chunks
A writer overwrites the cat chunk by chunk in random order. Two readers read all the data they can see at any moment. Reads and writes are not instant, so a read can overlap a write. Zarr and Icechunk differ in what the reader sees during a write, because Icechunk offers "ACID transactions" where native Zarr does not.
- Zarr: A read that starts mid-write returns a mix of old and new chunks. The state of the data is inconsistent, the cat is neither alive nor dead!
- Icechunk: Data is not visible to readers until it is committed, so readers never observe partially-written data. The cat is either alive or dead!
ACID transactions: In Icechunk the data is always in a consistent state (the C in ACID). This works because writers write intermediate data to a session, before atomically (the A in ACID) flipping the latest available snapshot at the moment of the commit. If a new commit lands mid-read it keeps that snapshot and finishes reading the version it started on (isolation, the I in ACID). Older commits stay available as part of the repo history, i.e. the commit is durable (the D in ACID).