Virtual chunks: query the archive in place
Big archival datasets are usually a pile of NetCDF or GRIB files in object storage. To read a small slice with the native reader you have to download whole files and throw 99% of the bytes away. With VirtualiZarr and Icechunk, a tiny manifest maps every logical chunk to a (file, offset, length) in the archive. The reader pulls the manifest first, then range-GETs only the bytes it needs.
Native reader bill
$0
< 1 s
0 B downloaded
Virtual chunks bill
$0
< 1 s
0 B downloaded
☁️ Archival (NetCDF / GRIB)
Native NetCDF / GRIB reader
Your laptop
decode
result array
Icechunk virtual dataset
Icechunk repo
manifest
(path, offset, length)
Your laptop
result array