Zarr (data format)

Updated 2026-07-02

Zarr is an open-source specification and file format for the storage of chunked, compressed, N-dimensional typed arrays (also known as data cubes, ND-arrays, or tensors). It was created in 2016 by Alistair Miles at the Wellcome Sanger Institute, originally for genomics research, and has since grown into a community-governed project with implementations in multiple programming languages. Zarr is fiscally sponsored by NumFOCUS.

Overview

Zarr is designed to address the challenge of storing and accessing large multidimensional array datasets in a way that is compatible with parallel and distributed computing. It stores array data as optionally compressed binary chunks, with metadata stored as JSON documents. The format is storage-agnostic: it can be used with any system that provides a key/value interface, including local file systems, cloud object stores (e.g., Amazon S3, Google Cloud Storage, Azure Blob Storage), key-value databases (e.g., LMDB), and in-memory stores.

Key Concepts

Hierarchy: A Zarr dataset is organized as a tree structure (hierarchy) where each node is either a group (which may have child nodes) or an array (which may not have children). The root of a hierarchy can be either a group or an array.

Array: A node in the hierarchy with zero or more dimensions, a shape (tuple of dimension lengths), a data type, and zero or more elements. All elements in an array conform to the same data type.

Chunk: An array is divided into chunks—hyperrectangular sub-regions defined by intervals along each dimension. Chunks are the unit of storage and I/O. All chunks in an array typically share the same shape, forming a regular grid.

Codec: A bidirectional transform (encode/decode) applied to chunk data. Codecs can be array-to-array transforms, array-to-bytes transforms (required exactly one per array), or bytes-to-bytes transforms (e.g., compression). Multiple codecs can be chained sequentially.

Store: The underlying storage system that holds metadata documents and encoded chunk data. The store interface defines operations such as get, set, erase, list, list_prefix, and list_dir.

Metadata Document: Each array or group is represented by a JSON metadata document (named zarr.json in v3, .zarray/.zgroup in v2) containing essential processing information such as shape, data type, chunk grid, codecs, and fill value.

Storage Transformer: An optional intermediate layer that can intercept and modify storage keys and bytes before they reach the underlying store, enabling features like sharding or caching.

Specification Versions

Zarr v2

The version 2 specification (widely adopted and implemented) stores array metadata under the key .zarray and group metadata under .zgroup. Data types follow the NumPy array protocol type string format (e.g., "<f8" for little-endian 64-bit float). Chunks are stored under keys formed from grid indices separated by a period (e.g., "0.0") or slash (e.g., "0/0"). The v2 specification was endorsed as an Open Geospatial Consortium (OGC) Community Standard in June 2022 (OGC Doc No. 21-050r1).

Zarr v3

Version 3 (accepted via ZEP0001 on May 15, 2023) introduced several changes to improve extensibility, interoperability, and performance with high-latency storage. Key differences from v2 include:

Metadata is stored under the key zarr.json for both arrays and groups.
The dtype field was renamed to data_type; endianness is handled by a codec rather than being part of the data type.
The chunks field was replaced with chunk_grid (supporting extension for non-regular grids).
The separate filters and compressor fields were combined into a single codecs list.
The order field was replaced by a transpose codec.
Explicit extension points were defined for data types, chunk grids, chunk key encodings, codecs, and storage transformers.
The default chunk key separator changed from . to /.
Chunk data keys use a c/ prefix (e.g., c/1/0).

The current version of the core specification is 3.1.

Supported Data Types

Zarr v3 core data types include: bool, int8, int16, int32, int64, uint8, uint16, uint32, uint64, float16 (optional), float32, float64, complex64, complex128, and raw bits types (r*). Additional data types can be defined via extensions.

Implementations

Zarr has been implemented in multiple programming languages, including:

Python (zarr-python) — the reference implementation, supporting both v2 and v3
C++ (zarr-cpp)
Julia (Zarr.jl)
Java (zarr-java)
JavaScript (zarr-js)

Applications and Use Cases

Zarr is used across a wide range of scientific and technical domains, including genomics, bioimaging, astronomy, climate science, oceanography, atmospheric science, geospatial imaging, physics, and quantitative finance. Notable examples include the Google Cloud CMIP6 Public Dataset (climate data) and OME-Zarr (bioimaging). Its compatibility with cloud object storage makes it well-suited for analysis-ready, cloud-optimized geospatial data.

Governance

The Zarr specification is developed and maintained by the Zarr community through the Zarr Enhancement Proposal (ZEP) process. The project is hosted under the zarr-developers GitHub organization.