How the pieces fit¶

icechunk-java lets Java code and Fiji read and write icechunk repositories. icechunk keeps Zarr arrays under version control, and its engine is a Rust library. icechunk-java wraps that library for Java, and its adapter modules plug it in underneath zarr-java and N5, the libraries that Fiji's Zarr tools are built on.

What provides what¶

The Java libraries and applications are unchanged except for opening by URL, which needs forks of n5-universe and n5-ij (Changes in other projects).

Use it from Java¶

Pick the modules for the library your code already uses:

• Add icechunk-n5-codecs to either N5 row when arrays use numcodecs.pcodec or numcodecs.zlib, which zarr-python can write and n5-zarr cannot decode on its own.

Use it in Fiji¶

• Open: Fiji's importer and BigDataViewer's N5 viewer take a repository URL, such as s3://bucket/repo|icechunk://branch.main/em/raw, wherever they take a path. URLs open read-only.
• Save: File > Save As > icechunk... writes the active image as OME-Zarr and commits it to a branch.
• Install: Fiji needs six jars built from source, two of which replace jars Fiji ships. Use in Fiji builds and installs them.

The projects¶

Storage¶

• Zarr is a format for chunked, compressed N-dimensional arrays. Arrays are split into chunks stored as separate objects and arranged in groups, like directories. A Zarr store maps keys such as temperature/zarr.json or temperature/c/0/0 to bytes. icechunk holds Zarr version 3 only.
• icechunk is a transactional storage engine for Zarr, written in Rust. A repository keeps a Zarr hierarchy and its history: each commit is a snapshot, and branches and tags name snapshots, as in git. Virtual chunks read byte ranges inside existing NetCDF, HDF5 or TIFF files where they live. A repository is the same from either language: what icechunk-python writes with zarr-python and xarray, icechunk-java reads, and the reverse.
• Arraylake is Earthmover's managed service that hosts icechunk repositories and holds their credentials. The separate icechunk-arraylake-java project opens Arraylake repositories as icechunk-java Repository objects.

Java array libraries¶

• zarr-java is a Zarr library for Java that reads and writes arrays through a store interface.
• N5 is a Java API for chunked N-dimensional arrays from the Saalfeld lab at HHMI Janelia. It does its I/O through a KeyValueAccess, an interface for reading and writing bytes at paths, which N5's libraries implement for local files, S3, Google Cloud Storage and HTTP.
• n5-zarr is N5's reader and writer for Zarr versions 2 and 3.
• n5-universe opens N5, Zarr and HDF5 containers from a path or URL. Its N5Factory picks the storage backend and format for the location, and it reads OME-NGFF metadata.

Applications¶

• Fiji is a distribution of ImageJ, the image analysis application widely used in microscopy and bioimaging, with plugins bundled.
• n5-ij is Fiji's importer and exporter for HDF5, N5, Zarr and OME-NGFF (File > Import > HDF5/N5/Zarr/OME-NGFF ...). It opens locations through N5Factory.
• BigDataViewer is a Fiji viewer for large and multiscale images that loads only the chunks on screen. Its N5 viewer opens containers through n5-ij's dialog and N5Factory (Plugins > BigDataViewer > HDF5/N5/Zarr/OME-NGFF Viewer).
• Paintera is a tool for painting and proofreading labels in large 3D images, built on BigDataViewer and N5.
• OME-NGFF, also called OME-Zarr, is the convention for storing microscopy images in Zarr: multiscale pyramids, axes, channels and labels. OME-Zarr stored in icechunk is still OME-Zarr, so n5-ij and the N5 viewer read its metadata as usual.

From Fiji to storage¶

A Fiji tool opens a repository from a URL such as s3://bucket/repo|icechunk://branch.main/em/raw, and a read of one chunk goes down this stack. Open from a URL has the URL syntax.

n5-ij importer, or the N5 viewer in BigDataViewer
  │  opens s3://bucket/repo|icechunk://branch.main/em/raw
  ▼
n5-universe          N5Factory asks each KeyValueAccessProvider on the classpath for the URL
  ▼
icechunk-n5-universe IcechunkKeyValueAccessProvider opens the repository and a read-only session
  ▼
n5-zarr              the Zarr v3 reader that N5Factory builds on that session
  ▼
icechunk-n5          IcechunkKeyValueAccess turns N5 paths into Zarr keys
  ▼
icechunk-java        Session and Store, which call the Rust library through JNI
  ▼
icechunk (Rust)      finds the chunk in the snapshot's manifests and fetches it
  │
  ├── repository storage: a local directory, S3, Google Cloud Storage, HTTP, or Arraylake
  └── virtual chunks: byte ranges in other files, read from where they live

• URLs open read-only, because a write needs a commit and a URL has no place for one. To write from N5, use a writable session in code, as in Write and commit. Fiji writes through File > Save As > icechunk..., which commits (Use in Fiji).
• Arraylake repositories open the same way, as al:org/repo, once icechunk-arraylake-java is on the classpath. It supplies the storage and credentials, including those for virtual chunks.