Virtual chunks
A virtual dataset leaves the original files untouched in object storage and keeps a tiny manifest mapping each logical chunk to its (file, offset, length). The reader pulls the manifest, then range-GETs only the bytes it needs, instead of downloading whole files and throwing most of them away. That saves bytes on two very different archive formats, and it also saves the round trips a reader would otherwise spend just working out where the bytes are:
Query the archive in place
NetCDF / GRIB files
A pile of NetCDF or GRIB files in object storage. A native reader downloads whole files to grab a small slice; virtual chunks range-GET just the chunks the query touches.
Open visualization →Read one region of a giant slide
Whole-slide microscopy (NDPI / TIFF)
A gigapixel pathology slide, mostly empty background. A whole-slide reader downloads the entire file to crop one tissue region; virtual chunks read only that region, one tile-row at a time, and the image fills in as it arrives.
Open visualization →The cost of a chained walk
Latency, not bandwidth
One plane, deep inside a microscopy stack. A native TIFF reader walks the IFD chain one round trip at a time just to find it. Icechunk looks it up in a manifest over that same file, in a number of hops that never grows with the data.
Open visualization →