.. only:: doctest >>> import shutil >>> shutil.rmtree('data', ignore_errors=True) .. _user-guide-arrays: Working with arrays =================== Creating an array ----------------- Zarr has several functions for creating arrays. For example:: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> z = zarr.create_array(store=store, shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') >>> z The code above creates a 2-dimensional array of 32-bit integers with 10000 rows and 10000 columns, divided into chunks where each chunk has 1000 rows and 1000 columns (and so there will be 100 chunks in total). The data is written to a :class:`zarr.storage.MemoryStore` (e.g. an in-memory dict). See :ref:`user-guide-persist` for details on storing arrays in other stores, and see :ref:`user-guide-data-types` for an in-depth look at the data types supported by Zarr. For a complete list of array creation routines see the :mod:`zarr` module documentation. .. _user-guide-array: Reading and writing data ------------------------ Zarr arrays support a similar interface to `NumPy `_ arrays for reading and writing data. For example, the entire array can be filled with a scalar value:: >>> z[:] = 42 Regions of the array can also be written to, e.g.:: >>> import numpy as np >>> >>> z[0, :] = np.arange(10000) >>> z[:, 0] = np.arange(10000) The contents of the array can be retrieved by slicing, which will load the requested region into memory as a NumPy array, e.g.:: >>> z[0, 0] array(0, dtype=int32) >>> z[-1, -1] array(42, dtype=int32) >>> z[0, :] array([ 0, 1, 2, ..., 9997, 9998, 9999], shape=(10000,), dtype=int32) >>> z[:, 0] array([ 0, 1, 2, ..., 9997, 9998, 9999], shape=(10000,), dtype=int32) >>> z[:] array([[ 0, 1, 2, ..., 9997, 9998, 9999], [ 1, 42, 42, ..., 42, 42, 42], [ 2, 42, 42, ..., 42, 42, 42], ..., [9997, 42, 42, ..., 42, 42, 42], [9998, 42, 42, ..., 42, 42, 42], [9999, 42, 42, ..., 42, 42, 42]], shape=(10000, 10000), dtype=int32) Read more about NumPy-style indexing can be found in the `NumPy documentation `_. .. _user-guide-persist: Persistent arrays ----------------- In the examples above, compressed data for each chunk of the array was stored in main memory. Zarr arrays can also be stored on a file system, enabling persistence of data between sessions. To do this, we can change the store argument to point to a filesystem path:: >>> z1 = zarr.create_array(store='data/example-1.zarr', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') The array above will store its configuration metadata and all compressed chunk data in a directory called ``'data/example-1.zarr'`` relative to the current working directory. The :func:`zarr.create_array` function provides a convenient way to create a new persistent array or continue working with an existing array. Note, there is no need to close an array: data are automatically flushed to disk, and files are automatically closed whenever an array is modified. Persistent arrays support the same interface for reading and writing data, e.g.:: >>> z1[:] = 42 >>> z1[0, :] = np.arange(10000) >>> z1[:, 0] = np.arange(10000) Check that the data have been written and can be read again:: >>> z2 = zarr.open_array('data/example-1.zarr', mode='r') >>> np.all(z1[:] == z2[:]) np.True_ If you are just looking for a fast and convenient way to save NumPy arrays to disk then load back into memory later, the functions :func:`zarr.save` and :func:`zarr.load` may be useful. E.g.:: >>> a = np.arange(10) >>> zarr.save('data/example-2.zarr', a) >>> zarr.load('data/example-2.zarr') array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) Please note that there are a number of other options for persistent array storage, see the :ref:`Storage Guide ` guide for more details. .. _user-guide-resize: Resizing and appending ---------------------- A Zarr array can be resized, which means that any of its dimensions can be increased or decreased in length. For example:: >>> z = zarr.create_array(store='data/example-3.zarr', shape=(10000, 10000), dtype='int32',chunks=(1000, 1000)) >>> z[:] = 42 >>> z.shape (10000, 10000) >>> z.resize((20000, 10000)) >>> z.shape (20000, 10000) Note that when an array is resized, the underlying data are not rearranged in any way. If one or more dimensions are shrunk, any chunks falling outside the new array shape will be deleted from the underlying store. :func:`zarr.Array.append` is provided as a convenience function, which can be used to append data to any axis. E.g.:: >>> a = np.arange(10000000, dtype='int32').reshape(10000, 1000) >>> z = zarr.create_array(store='data/example-4.zarr', shape=a.shape, dtype=a.dtype, chunks=(1000, 100)) >>> z[:] = a >>> z.shape (10000, 1000) >>> z.append(a) (20000, 1000) >>> z.append(np.vstack([a, a]), axis=1) (20000, 2000) >>> z.shape (20000, 2000) .. _user-guide-compress: Compressors ----------- A number of different compressors can be used with Zarr. Zarr includes Blosc, Zstandard and Gzip compressors. Additional compressors are available through a separate package called NumCodecs_ which provides various compressor libraries including LZ4, Zlib, BZ2 and LZMA. Different compressors can be provided via the ``compressors`` keyword argument accepted by all array creation functions. For example:: >>> compressors = zarr.codecs.BloscCodec(cname='zstd', clevel=3, shuffle=zarr.codecs.BloscShuffle.bitshuffle) >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000) >>> z = zarr.create_array(store='data/example-5.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=compressors) >>> z[:] = data >>> z.compressors (BloscCodec(typesize=4, cname=, clevel=3, shuffle=, blocksize=0),) This array above will use Blosc as the primary compressor, using the Zstandard algorithm (compression level 3) internally within Blosc, and with the bit-shuffle filter applied. When using a compressor, it can be useful to get some diagnostics on the compression ratio. Zarr arrays provide the :attr:`zarr.Array.info` property which can be used to print useful diagnostics, e.g.:: >>> z.info Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : C Read-only : False Store type : LocalStore Filters : () Serializer : BytesCodec(endian=) Compressors : (BloscCodec(typesize=4, cname=, clevel=3, shuffle=, blocksize=0),) No. bytes : 400000000 (381.5M) The :func:`zarr.Array.info_complete` method inspects the underlying store and prints additional diagnostics, e.g.:: >>> z.info_complete() Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : C Read-only : False Store type : LocalStore Filters : () Serializer : BytesCodec(endian=) Compressors : (BloscCodec(typesize=4, cname=, clevel=3, shuffle=, blocksize=0),) No. bytes : 400000000 (381.5M) No. bytes stored : 3558573 (3.4M) Storage ratio : 112.4 Chunks Initialized : 100 .. note:: :func:`zarr.Array.info_complete` will inspect the underlying store and may be slow for large arrays. Use :attr:`zarr.Array.info` if detailed storage statistics are not needed. If you don't specify a compressor, by default Zarr uses the Zstandard compressor. In addition to Blosc and Zstandard, other compression libraries can also be used. For example, here is an array using Gzip compression, level 1:: >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000) >>> z = zarr.create_array(store='data/example-6.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=zarr.codecs.GzipCodec(level=1)) >>> z[:] = data >>> z.compressors (GzipCodec(level=1),) Here is an example using LZMA from NumCodecs_ with a custom filter pipeline including LZMA's built-in delta filter:: >>> import lzma >>> from numcodecs.zarr3 import LZMA >>> import warnings >>> warnings.filterwarnings("ignore", category=UserWarning) >>> >>> lzma_filters = [dict(id=lzma.FILTER_DELTA, dist=4), dict(id=lzma.FILTER_LZMA2, preset=1)] >>> compressors = LZMA(filters=lzma_filters) >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000) >>> z = zarr.create_array(store='data/example-7.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=compressors) >>> z.compressors (LZMA(codec_name='numcodecs.lzma', codec_config={'filters': [{'id': 3, 'dist': 4}, {'id': 33, 'preset': 1}]}),) To disable compression, set ``compressors=None`` when creating an array, e.g.:: >>> z = zarr.create_array(store='data/example-8.zarr', shape=(100000000,), chunks=(1000000,), dtype='int32', compressors=None) >>> z.compressors () .. _user-guide-filters: Filters ------- In some cases, compression can be improved by transforming the data in some way. For example, if nearby values tend to be correlated, then shuffling the bytes within each numerical value or storing the difference between adjacent values may increase compression ratio. Some compressors provide built-in filters that apply transformations to the data prior to compression. For example, the Blosc compressor has built-in implementations of byte- and bit-shuffle filters, and the LZMA compressor has a built-in implementation of a delta filter. However, to provide additional flexibility for implementing and using filters in combination with different compressors, Zarr also provides a mechanism for configuring filters outside of the primary compressor. Here is an example using a delta filter with the Blosc compressor:: >>> from numcodecs.zarr3 import Delta >>> >>> filters = [Delta(dtype='int32')] >>> compressors = zarr.codecs.BloscCodec(cname='zstd', clevel=1, shuffle=zarr.codecs.BloscShuffle.shuffle) >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000) >>> z = zarr.create_array(store='data/example-9.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), filters=filters, compressors=compressors) >>> z.info_complete() Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : C Read-only : False Store type : LocalStore Filters : (Delta(codec_name='numcodecs.delta', codec_config={'dtype': 'int32'}),) Serializer : BytesCodec(endian=) Compressors : (BloscCodec(typesize=4, cname=, clevel=1, shuffle=, blocksize=0),) No. bytes : 400000000 (381.5M) No. bytes stored : 826 Storage ratio : 484261.5 Chunks Initialized : 0 For more information about available filter codecs, see the `Numcodecs `_ documentation. .. _user-guide-indexing: Advanced indexing ----------------- Zarr arrays support several methods for advanced or "fancy" indexing, which enable a subset of data items to be extracted or updated in an array without loading the entire array into memory. Note that although this functionality is similar to some of the advanced indexing capabilities available on NumPy arrays and on h5py datasets, **the Zarr API for advanced indexing is different from both NumPy and h5py**, so please read this section carefully. For a complete description of the indexing API, see the documentation for the :class:`zarr.Array` class. Indexing with coordinate arrays ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Items from a Zarr array can be extracted by providing an integer array of coordinates. E.g.:: >>> data = np.arange(10) ** 2 >>> z = zarr.create_array(store='data/example-10.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81]) >>> z.get_coordinate_selection([2, 5]) array([ 4, 25]) Coordinate arrays can also be used to update data, e.g.:: >>> z.set_coordinate_selection([2, 5], [-1, -2]) >>> z[:] array([ 0, 1, -1, 9, 16, -2, 36, 49, 64, 81]) For multidimensional arrays, coordinates must be provided for each dimension, e.g.:: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-11.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14]]) >>> z.get_coordinate_selection(([0, 2], [1, 3])) array([ 1, 13]) >>> z.set_coordinate_selection(([0, 2], [1, 3]), [-1, -2]) >>> z[:] array([[ 0, -1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, -2, 14]]) For convenience, coordinate indexing is also available via the ``vindex`` property, as well as the square bracket operator, e.g.:: >>> z.vindex[[0, 2], [1, 3]] array([-1, -2]) >>> z.vindex[[0, 2], [1, 3]] = [-3, -4] >>> z[:] array([[ 0, -3, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, -4, 14]]) >>> z[[0, 2], [1, 3]] array([-3, -4]) When the indexing arrays have different shapes, they are broadcast together. That is, the following two calls are equivalent:: >>> z[1, [1, 3]] array([6, 8]) >>> z[[1, 1], [1, 3]] array([6, 8]) Indexing with a mask array ~~~~~~~~~~~~~~~~~~~~~~~~~~ Items can also be extracted by providing a Boolean mask. E.g.:: >>> data = np.arange(10) ** 2 >>> z = zarr.create_array(store='data/example-12.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81]) >>> sel = np.zeros_like(z, dtype=bool) >>> sel[2] = True >>> sel[5] = True >>> z.get_mask_selection(sel) array([ 4, 25]) >>> z.set_mask_selection(sel, [-1, -2]) >>> z[:] array([ 0, 1, -1, 9, 16, -2, 36, 49, 64, 81]) Here's a multidimensional example:: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-13.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14]]) >>> sel = np.zeros_like(z, dtype=bool) >>> sel[0, 1] = True >>> sel[2, 3] = True >>> z.get_mask_selection(sel) array([ 1, 13]) >>> z.set_mask_selection(sel, [-1, -2]) >>> z[:] array([[ 0, -1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, -2, 14]]) For convenience, mask indexing is also available via the ``vindex`` property, e.g.:: >>> z.vindex[sel] array([-1, -2]) >>> z.vindex[sel] = [-3, -4] >>> z[:] array([[ 0, -3, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, -4, 14]]) Mask indexing is conceptually the same as coordinate indexing, and is implemented internally via the same machinery. Both styles of indexing allow selecting arbitrary items from an array, also known as point selection. Orthogonal indexing ~~~~~~~~~~~~~~~~~~~ Zarr arrays also support methods for orthogonal indexing, which allows selections to be made along each dimension of an array independently. For example, this allows selecting a subset of rows and/or columns from a 2-dimensional array. E.g.:: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-14.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14]]) >>> z.get_orthogonal_selection(([0, 2], slice(None))) # select first and third rows array([[ 0, 1, 2, 3, 4], [10, 11, 12, 13, 14]]) >>> z.get_orthogonal_selection((slice(None), [1, 3])) # select second and fourth columns array([[ 1, 3], [ 6, 8], [11, 13]]) >>> z.get_orthogonal_selection(([0, 2], [1, 3])) # select rows [0, 2] and columns [1, 4] array([[ 1, 3], [11, 13]]) Data can also be modified, e.g.:: >>> z.set_orthogonal_selection(([0, 2], [1, 3]), [[-1, -2], [-3, -4]]) For convenience, the orthogonal indexing functionality is also available via the ``oindex`` property, e.g.:: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-15.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z.oindex[[0, 2], :] # select first and third rows array([[ 0, 1, 2, 3, 4], [10, 11, 12, 13, 14]]) >>> z.oindex[:, [1, 3]] # select second and fourth columns array([[ 1, 3], [ 6, 8], [11, 13]]) >>> z.oindex[[0, 2], [1, 3]] # select rows [0, 2] and columns [1, 4] array([[ 1, 3], [11, 13]]) >>> z.oindex[[0, 2], [1, 3]] = [[-1, -2], [-3, -4]] >>> z[:] array([[ 0, -1, 2, -2, 4], [ 5, 6, 7, 8, 9], [10, -3, 12, -4, 14]]) Any combination of integer, slice, 1D integer array and/or 1D Boolean array can be used for orthogonal indexing. If the index contains at most one iterable, and otherwise contains only slices and integers, orthogonal indexing is also available directly on the array:: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-16.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> np.all(z.oindex[[0, 2], :] == z[[0, 2], :]) np.True_ Block Indexing ~~~~~~~~~~~~~~ Zarr also support block indexing, which allows selections of whole chunks based on their logical indices along each dimension of an array. For example, this allows selecting a subset of chunk aligned rows and/or columns from a 2-dimensional array. E.g.:: >>> data = np.arange(100).reshape(10, 10) >>> z = zarr.create_array(store='data/example-17.zarr', shape=data.shape, dtype=data.dtype, chunks=(3, 3)) >>> z[:] = data Retrieve items by specifying their block coordinates:: >>> z.get_block_selection(1) array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) Equivalent slicing:: >>> z[3:6] array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) For convenience, the block selection functionality is also available via the `blocks` property, e.g.:: >>> z.blocks[1] array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) Block index arrays may be multidimensional to index multidimensional arrays. For example:: >>> z.blocks[0, 1:3] array([[ 3, 4, 5, 6, 7, 8], [13, 14, 15, 16, 17, 18], [23, 24, 25, 26, 27, 28]]) Data can also be modified. Let's start by a simple 2D array:: >>> z = zarr.create_array(store='data/example-18.zarr', shape=(6, 6), dtype=int, chunks=(2, 2)) Set data for a selection of items:: >>> z.set_block_selection((1, 0), 1) >>> z[...] array([[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]]) For convenience, this functionality is also available via the ``blocks`` property. E.g.:: >>> z.blocks[:, 2] = 7 >>> z[...] array([[0, 0, 0, 0, 7, 7], [0, 0, 0, 0, 7, 7], [1, 1, 0, 0, 7, 7], [1, 1, 0, 0, 7, 7], [0, 0, 0, 0, 7, 7], [0, 0, 0, 0, 7, 7]]) Any combination of integer and slice can be used for block indexing:: >>> z.blocks[2, 1:3] array([[0, 0, 7, 7], [0, 0, 7, 7]]) >>> >>> root = zarr.create_group('data/example-19.zarr') >>> foo = root.create_array(name='foo', shape=(1000, 100), chunks=(10, 10), dtype='float32') >>> bar = root.create_array(name='bar', shape=(100,), dtype='int32') >>> foo[:, :] = np.random.random((1000, 100)) >>> bar[:] = np.arange(100) >>> root.tree() / ├── bar (100,) int32 └── foo (1000, 100) float32 .. _user-guide-sharding: Sharding -------- Using small chunk shapes in very large arrays can lead to a very large number of chunks. This can become a performance issue for file systems and object storage. With Zarr format 3, a new sharding feature has been added to address this issue. With sharding, multiple chunks can be stored in a single storage object (e.g. a file). Within a shard, chunks are compressed and serialized separately. This allows individual chunks to be read independently. However, when writing data, a full shard must be written in one go for optimal performance and to avoid concurrency issues. That means that shards are the units of writing and chunks are the units of reading. Users need to configure the chunk and shard shapes accordingly. Sharded arrays can be created by providing the ``shards`` parameter to :func:`zarr.create_array`. >>> a = zarr.create_array('data/example-20.zarr', shape=(10000, 10000), shards=(1000, 1000), chunks=(100, 100), dtype='uint8') >>> a[:] = (np.arange(10000 * 10000) % 256).astype('uint8').reshape(10000, 10000) >>> a.info_complete() Type : Array Zarr format : 3 Data type : UInt8() Fill value : 0 Shape : (10000, 10000) Shard shape : (1000, 1000) Chunk shape : (100, 100) Order : C Read-only : False Store type : LocalStore Filters : () Serializer : BytesCodec(endian=None) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 100000000 (95.4M) No. bytes stored : 3981473 (3.8M) Storage ratio : 25.1 Shards Initialized : 100 In this example a shard shape of (1000, 1000) and a chunk shape of (100, 100) is used. This means that 10*10 chunks are stored in each shard, and there are 10*10 shards in total. Without the ``shards`` argument, there would be 10,000 chunks stored as individual files. Missing features in 3.0 ----------------------- The following features have not been ported to 3.0 yet. Copying and migrating data ~~~~~~~~~~~~~~~~~~~~~~~~~~ See the Zarr-Python 2 documentation on `Copying and migrating data `_ for more details. .. _user-guide-attrs: Working with attributes ======================= Zarr arrays and groups support custom key/value attributes, which can be useful for storing application-specific metadata. For example:: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> root = zarr.create_group(store=store) >>> root.attrs['foo'] = 'bar' >>> z = root.create_array(name='zzz', shape=(10000, 10000), dtype='int32') >>> z.attrs['baz'] = 42 >>> z.attrs['qux'] = [1, 4, 7, 12] >>> sorted(root.attrs) ['foo'] >>> 'foo' in root.attrs True >>> root.attrs['foo'] 'bar' >>> sorted(z.attrs) ['baz', 'qux'] >>> z.attrs['baz'] 42 >>> z.attrs['qux'] [1, 4, 7, 12] Internally Zarr uses JSON to store array attributes, so attribute values must be JSON serializable. .. _user-guide-cli: Command-line interface ======================== Zarr-Python provides a command-line interface that enables: - migration of Zarr v2 metadata to v3 - removal of v2 or v3 metadata To see available commands run the following in a terminal: .. code-block:: bash $ zarr --help or to get help on individual commands: .. code-block:: bash $ zarr migrate --help $ zarr remove-metadata --help Migrate metadata from v2 to v3 ------------------------------ Migrate to a separate location ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To migrate a Zarr array/group's metadata from v2 to v3 run: .. code-block:: bash $ zarr migrate v3 path/to/input.zarr path/to/output.zarr This will write new ``zarr.json`` files to ``output.zarr``, leaving ``input.zarr`` un-touched. Note - this will migrate the entire Zarr hierarchy, so if ``input.zarr`` contains multiple groups/arrays, new ``zarr.json`` will be made for all of them. Migrate in-place ~~~~~~~~~~~~~~~~ If you'd prefer to migrate the metadata in-place run: .. code-block:: bash $ zarr migrate v3 path/to/input.zarr This will write new ``zarr.json`` files to ``input.zarr``, leaving the existing v2 metadata un-touched. To open the array/group using the new metadata use: .. code-block:: python >>> import zarr >>> zarr_with_v3_metadata = zarr.open('path/to/input.zarr', zarr_format=3) Once you are happy with the conversion, you can run the following to remove the old v2 metadata: .. code-block:: bash $ zarr remove-metadata v2 path/to/input.zarr Note there is also a shortcut to migrate and remove v2 metadata in one step: .. code-block:: bash $ zarr migrate v3 path/to/input.zarr --remove-v2-metadata Remove metadata ---------------- Remove v2 metadata using: .. code-block:: bash $ zarr remove-metadata v2 path/to/input.zarr or v3 with: .. code-block:: bash $ zarr remove-metadata v3 path/to/input.zarr By default, this will only allow removal of metadata if a valid alternative exists. For example, you can't remove v2 metadata unless v3 metadata exists at that location. To override this behaviour use ``--force``: .. code-block:: bash $ zarr remove-metadata v3 path/to/input.zarr --force Dry run -------- All commands provide a ``--dry-run`` option that will log changes that would be made on a real run, without creating or modifying any files. .. code-block:: bash $ zarr migrate v3 path/to/input.zarr --dry-run Dry run enabled - no new files will be created or changed. Log of files that would be created on a real run: Saving metadata to path/to/input.zarr/zarr.json Verbose -------- You can also add ``--verbose`` **before** any command, to see a full log of its actions: .. code-block:: bash $ zarr --verbose migrate v3 path/to/input.zarr $ zarr --verbose remove-metadata v2 path/to/input.zarr Equivalent functions -------------------- All features of the command-line interface are also available via functions under :mod:`zarr.metadata`. .. _user-guide-config: Runtime configuration ===================== ``zarr.config`` is responsible for managing the configuration of zarr and is based on the `donfig `_ Python library. Configuration values can be set using code like the following:: >>> import zarr >>> >>> zarr.config.set({'array.order': 'F'}) >>> >>> # revert this change so it doesn't impact the rest of the docs >>> zarr.config.set({'array.order': 'C'}) Alternatively, configuration values can be set using environment variables, e.g. ``ZARR_ARRAY__ORDER=F``. The configuration can also be read from a YAML file in standard locations. For more information, see the `donfig documentation `_. Configuration options include the following: - Default Zarr format ``default_zarr_version`` - Default array order in memory ``array.order`` - Whether empty chunks are written to storage ``array.write_empty_chunks`` - Async and threading options, e.g. ``async.concurrency`` and ``threading.max_workers`` - Selections of implementations of codecs, codec pipelines and buffers - Enabling GPU support with ``zarr.config.enable_gpu()``. See :ref:`user-guide-gpu` for more. For selecting custom implementations of codecs, pipelines, buffers and ndbuffers, first register the implementations in the registry and then select them in the config. For example, an implementation of the bytes codec in a class ``'custompackage.NewBytesCodec'``, requires the value of ``codecs.bytes.name`` to be ``'custompackage.NewBytesCodec'``. This is the current default configuration:: >>> zarr.config.pprint() {'array': {'order': 'C', 'write_empty_chunks': False}, 'async': {'concurrency': 10, 'timeout': None}, 'buffer': 'zarr.buffer.cpu.Buffer', 'codec_pipeline': {'batch_size': 1, 'path': 'zarr.core.codec_pipeline.BatchedCodecPipeline'}, 'codecs': {'blosc': 'zarr.codecs.blosc.BloscCodec', 'bytes': 'zarr.codecs.bytes.BytesCodec', 'crc32c': 'zarr.codecs.crc32c_.Crc32cCodec', 'endian': 'zarr.codecs.bytes.BytesCodec', 'gzip': 'zarr.codecs.gzip.GzipCodec', 'numcodecs.adler32': 'zarr.codecs.numcodecs.Adler32', 'numcodecs.astype': 'zarr.codecs.numcodecs.AsType', 'numcodecs.bitround': 'zarr.codecs.numcodecs.BitRound', 'numcodecs.blosc': 'zarr.codecs.numcodecs.Blosc', 'numcodecs.bz2': 'zarr.codecs.numcodecs.BZ2', 'numcodecs.crc32': 'zarr.codecs.numcodecs.CRC32', 'numcodecs.crc32c': 'zarr.codecs.numcodecs.CRC32C', 'numcodecs.delta': 'zarr.codecs.numcodecs.Delta', 'numcodecs.fixedscaleoffset': 'zarr.codecs.numcodecs.FixedScaleOffset', 'numcodecs.fletcher32': 'zarr.codecs.numcodecs.Fletcher32', 'numcodecs.gzip': 'zarr.codecs.numcodecs.GZip', 'numcodecs.jenkins_lookup3': 'zarr.codecs.numcodecs.JenkinsLookup3', 'numcodecs.lz4': 'zarr.codecs.numcodecs.LZ4', 'numcodecs.lzma': 'zarr.codecs.numcodecs.LZMA', 'numcodecs.packbits': 'zarr.codecs.numcodecs.PackBits', 'numcodecs.pcodec': 'zarr.codecs.numcodecs.PCodec', 'numcodecs.quantize': 'zarr.codecs.numcodecs.Quantize', 'numcodecs.shuffle': 'zarr.codecs.numcodecs.Shuffle', 'numcodecs.zfpy': 'zarr.codecs.numcodecs.ZFPY', 'numcodecs.zlib': 'zarr.codecs.numcodecs.Zlib', 'numcodecs.zstd': 'zarr.codecs.numcodecs.Zstd', 'sharding_indexed': 'zarr.codecs.sharding.ShardingCodec', 'transpose': 'zarr.codecs.transpose.TransposeCodec', 'vlen-bytes': 'zarr.codecs.vlen_utf8.VLenBytesCodec', 'vlen-utf8': 'zarr.codecs.vlen_utf8.VLenUTF8Codec', 'zstd': 'zarr.codecs.zstd.ZstdCodec'}, 'default_zarr_format': 3, 'json_indent': 2, 'ndbuffer': 'zarr.buffer.cpu.NDBuffer', 'threading': {'max_workers': None}} .. _user-guide-consolidated-metadata: Consolidated metadata ===================== .. warning:: The Consolidated Metadata feature in Zarr-Python is considered experimental for v3 stores. `zarr-specs#309 `_ has proposed a formal extension to the v3 specification to support consolidated metadata. Zarr-Python implements the `Consolidated Metadata`_ for v2 and v3 stores. Consolidated metadata can reduce the time needed to load the metadata for an entire hierarchy, especially when the metadata is being served over a network. Consolidated metadata essentially stores all the metadata for a hierarchy in the metadata of the root Group. Usage ----- If consolidated metadata is present in a Zarr Group's metadata then it is used by default. The initial read to open the group will need to communicate with the store (reading from a file for a :class:`zarr.storage.LocalStore`, making a network request for a :class:`zarr.storage.FsspecStore`). After that, any subsequent metadata reads get child Group or Array nodes will *not* require reads from the store. In Python, the consolidated metadata is available on the ``.consolidated_metadata`` attribute of the ``GroupMetadata`` object. >>> import zarr >>> import warnings >>> warnings.filterwarnings("ignore", category=UserWarning) >>> >>> store = zarr.storage.MemoryStore() >>> group = zarr.create_group(store=store) >>> group.create_array(shape=(1,), name='a', dtype='float64') >>> group.create_array(shape=(2, 2), name='b', dtype='float64') >>> group.create_array(shape=(3, 3, 3), name='c', dtype='float64') >>> zarr.consolidate_metadata(store) If we open that group, the Group's metadata has a :class:`zarr.core.group.ConsolidatedMetadata` that can be used.: >>> consolidated = zarr.open_group(store=store) >>> consolidated_metadata = consolidated.metadata.consolidated_metadata.metadata >>> from pprint import pprint >>> pprint(dict(consolidated_metadata.items())) {'a': ArrayV3Metadata(shape=(1,), data_type=Float64(endianness='little'), chunk_grid=RegularChunkGrid(chunk_shape=(1,)), chunk_key_encoding=DefaultChunkKeyEncoding(separator='/'), fill_value=np.float64(0.0), codecs=(BytesCodec(endian=), ZstdCodec(level=0, checksum=False)), attributes={}, dimension_names=None, zarr_format=3, node_type='array', storage_transformers=()), 'b': ArrayV3Metadata(shape=(2, 2), data_type=Float64(endianness='little'), chunk_grid=RegularChunkGrid(chunk_shape=(2, 2)), chunk_key_encoding=DefaultChunkKeyEncoding(separator='/'), fill_value=np.float64(0.0), codecs=(BytesCodec(endian=), ZstdCodec(level=0, checksum=False)), attributes={}, dimension_names=None, zarr_format=3, node_type='array', storage_transformers=()), 'c': ArrayV3Metadata(shape=(3, 3, 3), data_type=Float64(endianness='little'), chunk_grid=RegularChunkGrid(chunk_shape=(3, 3, 3)), chunk_key_encoding=DefaultChunkKeyEncoding(separator='/'), fill_value=np.float64(0.0), codecs=(BytesCodec(endian=), ZstdCodec(level=0, checksum=False)), attributes={}, dimension_names=None, zarr_format=3, node_type='array', storage_transformers=())} Operations on the group to get children automatically use the consolidated metadata.: >>> consolidated['a'] # no read / HTTP request to the Store is required With nested groups, the consolidated metadata is available on the children, recursively.: >>> child = group.create_group('child', attributes={'kind': 'child'}) >>> grandchild = child.create_group('child', attributes={'kind': 'grandchild'}) >>> consolidated = zarr.consolidate_metadata(store) >>> >>> consolidated['child'].metadata.consolidated_metadata ConsolidatedMetadata(metadata={'child': GroupMetadata(attributes={'kind': 'grandchild'}, zarr_format=3, consolidated_metadata=ConsolidatedMetadata(metadata={}, kind='inline', must_understand=False), node_type='group')}, kind='inline', must_understand=False) .. versionadded:: 3.1.1 The keys in the consolidated metadata are sorted prior to writing. Keys are sorted in ascending order by path depth, where a path is defined as a sequence of strings joined by ``"/"``. For keys with the same path length, lexicographic order is used to break the tie. This behaviour ensures deterministic metadata output for a given group. Synchronization and Concurrency ------------------------------- Consolidated metadata is intended for read-heavy use cases on slowly changing hierarchies. For hierarchies where new nodes are constantly being added, removed, or modified, consolidated metadata may not be desirable. 1. It will add some overhead to each update operation, since the metadata would need to be re-consolidated to keep it in sync with the store. 2. Readers using consolidated metadata will regularly see a "past" version of the metadata, at the time they read the root node with its consolidated metadata. .. _Consolidated Metadata: https://github.com/zarr-developers/zarr-specs/pull/309 Stores Without Support for Consolidated Metadata ------------------------------------------------ Some stores may want to opt out of the consolidated metadata mechanism. This may be for several reasons like: * They want to maintain read-write consistency, which is challenging with consolidated metadata. * They have their own consolidated metadata mechanism. * They offer good enough performance without need for consolidation. This type of store can declare it doesn't want consolidation by implementing `Store.supports_consolidated_metadata` and returning `False`. For stores that don't support consolidation, Zarr will: * Raise an error on `consolidate_metadata` calls, maintaining the store in its unconsolidated state. * Raise an error in `AsyncGroup.open(..., use_consolidated=True)` * Not use consolidated metadata in `AsyncGroup.open(..., use_consolidated=None)` .. _user-guide-data-types: Array data types ================ Zarr's Data Type Model ---------------------- Zarr is designed for interoperability with NumPy, so if you are familiar with NumPy or any other N-dimensional array library, Zarr's model for array data types should seem familiar. However, Zarr data types have some unique features that are described in this document. Zarr arrays operate under an essential design constraint: unlike NumPy arrays, Zarr arrays are designed to be stored and accessed by other Zarr implementations. This means that, among other things, Zarr data types must be serializable to metadata documents in accordance with the Zarr specifications, which adds some unique aspects to the Zarr data type model. The following sections explain Zarr's data type model in greater detail and demonstrate the Zarr Python APIs for working with Zarr data types. Array Data Types ^^^^^^^^^^^^^^^^ Every Zarr array has a data type, which defines the meaning of the array's elements. An array's data type is encoded in the JSON metadata for the array. This means that the data type of an array must be JSON-serializable. In Zarr V2, the data type of an array is stored in the ``dtype`` field in array metadata. Zarr V3 changed the name of this field to ``data_type`` and also defined new rules for the values that can be assigned to the ``data_type`` field. For example, in Zarr V2, the boolean array data type was represented in array metadata as the string ``"|b1"``. In Zarr V3, the same type is represented as the string ``"bool"``. Scalars ^^^^^^^ Zarr also specifies how array elements, i.e., scalars, are encoded in array metadata. This is necessary because Zarr uses a field in array metadata to define a default value for chunks that are not stored. This field, called ``fill_value`` in both Zarr V2 and Zarr V3 metadata documents, contains a JSON value that can be decoded to a scalar value compatible with the array's data type. For the boolean data type, the scalar encoding is simple—booleans are natively supported by JSON, so Zarr saves booleans as JSON booleans. Other scalars, like floats or raw bytes, have more elaborate encoding schemes, and in some cases, this scheme depends on the Zarr format version. Data Types in Zarr Version 2 ---------------------------- Version 2 of the Zarr format defined its data types relative to `NumPy's data types `_, and added a few non-NumPy data types as well. With one exception (`structured data types <#structured-data-type>`_), the Zarr V2 JSON identifier for a data type is just the NumPy ``str`` attribute of that data type: .. code-block:: python >>> import zarr >>> import numpy as np >>> import json >>> >>> store = {} >>> np_dtype = np.dtype('int64') >>> np_dtype.str '>> z = zarr.create_array(store=store, shape=(1,), dtype=np_dtype, zarr_format=2) >>> dtype_meta = json.loads(store['.zarray'].to_bytes())["dtype"] >>> dtype_meta '`_, or "byte order," of the data type. As per the NumPy model, in Zarr version 2 each data type has an endianness where applicable. However, Zarr version 3 data types do not store endianness information. There are two special cases to consider: `"structured" data types <#structured-data-type>`_, and `"object" <#object-data-type>`_ data types. Structured Data Type ^^^^^^^^^^^^^^^^^^^^ NumPy allows the construction of a so-called "structured" data types comprised of ordered collections of named fields, where each field is itself a distinct NumPy data type. See the NumPy documentation `here `_. Crucially, NumPy does not use a special data type for structured data types—instead, NumPy implements structured data types as an optional feature of the so-called "Void" data type, which models arbitrary fixed-size byte strings. The ``str`` attribute of a regular NumPy void data type is the same as the ``str`` of a NumPy structured data type. This means that the ``str`` attribute does not convey information about the fields contained in a structured data type. For these reasons, Zarr V2 uses a special data type encoding for structured data types. They are stored in JSON as lists of pairs, where the first element is a string, and the second element is a Zarr V2 data type specification. This representation supports recursion. For example: .. code-block:: python >>> store = {} >>> np_dtype = np.dtype([('field_a', '>i2'), ('field_b', [('subfield_c', '>f4'), ('subfield_d', 'i2')])]) >>> np_dtype.str '|V8' >>> z = zarr.create_array(store=store, shape=(1,), dtype=np_dtype, zarr_format=2) >>> dtype_meta = json.loads(store['.zarray'].to_bytes())["dtype"] >>> dtype_meta [['field_a', '>i2'], ['field_b', [['subfield_c', '>f4'], ['subfield_d', '`_. Data Types in Zarr Python ------------------------- The two Zarr formats that Zarr Python supports specify data types in different ways: data types in Zarr version 2 are encoded as NumPy-compatible strings (or lists, in the case of structured data types), while data types in Zarr V3 are encoded as either strings or JSON objects. Zarr V3 data types do not have any associated endianness information, unlike Zarr V2 data types. Zarr Python needs to support both Zarr V2 and V3, which means we need to abstract over these differences. We do this with an abstract Zarr data type class: `ZDType <../api/zarr/dtype/index.html#zarr.dtype.ZDType>`_, which provides Zarr V2 and Zarr V3 compatibility routines for "native" data types. In this context, a "native" data type is a Python class, typically defined in another library, that models an array's data type. For example, ``np.dtypes.UInt8DType`` is a native data type defined in NumPy. Zarr Python wraps the NumPy ``uint8`` with a ``ZDType`` instance called `UInt8 <../api/zarr/dtype/index.html#zarr.dtype.ZDType>`_. As of this writing, the only native data types Zarr Python supports are NumPy data types. We could avoid the "native data type" jargon and just say "NumPy data type," but we do not want to rule out the possibility of using non-NumPy array backends in the future. Each data type supported by Zarr Python is modeled by a ``ZDType`` subclass, which provides an API for the following operations: - Encoding and decoding a native data type - Encoding and decoding a data type to and from Zarr V2 and Zarr V3 array metadata - Encoding and decoding a scalar value to and from Zarr V2 and Zarr V3 array metadata - Casting a Python object to a scalar value consistent with the data type List of data types ^^^^^^^^^^^^^^^^^^ The following section lists the data types built in to Zarr Python. With a few exceptions, Zarr Python supports nearly all of the data types in NumPy. If you need a data type that is not listed here, it's possible to create it yourself: see :ref:`adding-new-data-types`. Boolean """"""" - `Boolean <../api/zarr/dtype/index.html#zarr.dtype.Bool>`_ Integral """""""" - `Signed 8-bit integer <../api/zarr/dtype/index.html#zarr.dtype.Int8>`_ - `Signed 16-bit integer <../api/zarr/dtype/index.html#zarr.dtype.Int16>`_ - `Signed 32-bit integer <../api/zarr/dtype/index.html#zarr.dtype.Int32>`_ - `Signed 64-bit integer <../api/zarr/dtype/index.html#zarr.dtype.Int64>`_ - `Unsigned 8-bit integer <../api/zarr/dtype/index.html#zarr.dtype.UInt8>`_ - `Unsigned 16-bit integer <../api/zarr/dtype/index.html#zarr.dtype.UInt16>`_ - `Unsigned 32-bit integer <../api/zarr/dtype/index.html#zarr.dtype.UInt32>`_ - `Unsigned 64-bit integer <../api/zarr/dtype/index.html#zarr.dtype.UInt64>`_ Floating-point """""""""""""" - `16-bit floating-point <../api/zarr/dtype/index.html#zarr.dtype.Float16>`_ - `32-bit floating-point <../api/zarr/dtype/index.html#zarr.dtype.Float32>`_ - `64-bit floating-point <../api/zarr/dtype/index.html#zarr.dtype.Float64>`_ - `64-bit complex floating-point <../api/zarr/dtype/index.html#zarr.dtype.Complex64>`_ - `128-bit complex floating-point <../api/zarr/dtype/index.html#zarr.dtype.Complex128>`_ String """""" - `Fixed-length UTF-32 string <../api/zarr/dtype/index.html#zarr.dtype.FixedLengthUTF32>`_ - `Variable-length UTF-8 string <../api/zarr/dtype/index.html#zarr.dtype.VariableLengthUTF8>`_ Bytes """"" - `Fixed-length null-terminated bytes <../api/zarr/dtype/index.html#zarr.dtype.NullTerminatedBytes>`_ - `Fixed-length raw bytes <../api/zarr/dtype/index.html#zarr.dtype.RawBytes>`_ - `Variable-length bytes <../api/zarr/dtype/index.html#zarr.dtype.VariableLengthBytes>`_ Temporal """""""" - `DateTime64 <../api/zarr/dtype/index.html#zarr.dtype.DateTime64>`_ - `TimeDelta64 <../api/zarr/dtype/index.html#zarr.dtype.TimeDelta64>`_ Struct-like """"""""""" - `Structured <../api/zarr/dtype/index.html#zarr.dtype.Structured>`_ Example Usage ^^^^^^^^^^^^^ This section will demonstrates the basic usage of Zarr data types. Create a ``ZDType`` from a native data type: .. code-block:: python >>> from zarr.core.dtype import Int8 >>> import numpy as np >>> int8 = Int8.from_native_dtype(np.dtype('int8')) Convert back to a native data type: .. code-block:: python >>> native_dtype = int8.to_native_dtype() >>> assert native_dtype == np.dtype('int8') Get the default scalar value for the data type: .. code-block:: python >>> default_value = int8.default_scalar() >>> assert default_value == np.int8(0) Serialize to JSON for Zarr V2: .. code-block:: python >>> json_v2 = int8.to_json(zarr_format=2) >>> json_v2 {'name': '|i1', 'object_codec_id': None} .. note:: The representation returned by ``to_json(zarr_format=2)`` is more abstract than the literal contents of Zarr V2 array metadata, because the JSON representation used by the ``ZDType`` classes must be distinct across different data types. As noted `earlier <#object-data-type>`_, Zarr V2 identifies multiple distinct data types with the "object" data type identifier ``"|O"``. Extra information is needed to disambiguate these data types from one another. That's the reason for the ``object_codec_id`` field you see here. And for V3: .. code-block:: python >>> json_v3 = int8.to_json(zarr_format=3) >>> json_v3 'int8' Serialize a scalar value to JSON: .. code-block:: python >>> json_value = int8.to_json_scalar(42, zarr_format=3) >>> json_value 42 Deserialize a scalar value from JSON: .. code-block:: python >>> scalar_value = int8.from_json_scalar(42, zarr_format=3) >>> assert scalar_value == np.int8(42) .. _adding-new-data-types: Adding New Data Types ^^^^^^^^^^^^^^^^^^^^^ Each Zarr data type is a separate Python class that inherits from `ZDType <../api/zarr/dtype/index.html#zarr.dtype.ZDType>`_. You can define a custom data type by writing your own subclass of `ZDType <../api/zarr/dtype/index.html#zarr.dtype.ZDType>`_ and adding your data type to the data type registry. A complete example of this process is included below. The source code for this example can be found in the ``examples/custom_dtype.py`` file in the Zarr Python project directory. .. literalinclude:: ../../examples/custom_dtype.py :language: python Data Type Resolution ^^^^^^^^^^^^^^^^^^^^ Although Zarr Python uses a different data type model from NumPy, you can still define a Zarr array with a NumPy data type object: .. code-block:: python >>> from zarr import create_array >>> import numpy as np >>> a = create_array({}, shape=(10,), dtype=np.dtype('int')) >>> a Or a string representation of a NumPy data type: .. code-block:: python >>> a = create_array({}, shape=(10,), dtype='>> a The ``Array`` object presents itself like a NumPy array, including exposing a NumPy data type as its ``dtype`` attribute: .. code-block:: python >>> type(a.dtype) But if we inspect the metadata for the array, we can see the Zarr data type object: .. code-block:: python >>> type(a.metadata.data_type) This example illustrates a general problem Zarr Python has to solve: how can we allow users to specify a data type as a string or a NumPy ``dtype`` object, and produce the right Zarr data type from that input? We call this process "data type resolution." Zarr Python also performs data type resolution when reading stored arrays, although in this case the input is a JSON value instead of a NumPy data type. For simple data types like ``int``, the solution could be extremely simple: just maintain a lookup table that maps a NumPy data type to the Zarr data type equivalent. But not all data types are so simple. Consider this case: .. code-block:: python >>> from zarr import create_array >>> import warnings >>> import numpy as np >>> warnings.simplefilter("ignore", category=FutureWarning) >>> a = create_array({}, shape=(10,), dtype=[('a', 'f8'), ('b', 'i8')]) >>> a.dtype # this is the NumPy data type dtype([('a', '>> a.metadata.data_type # this is the Zarr data type Structured(fields=(('a', Float64(endianness='little')), ('b', Int64(endianness='little')))) In this example, we created a `NumPy structured data type `_. This data type is a container that can hold any NumPy data type, which makes it recursive. It is not possible to make a lookup table that relates all NumPy structured data types to their Zarr equivalents, as there is a nearly unbounded number of different structured data types. So instead of a static lookup table, Zarr Python relies on a dynamic approach to data type resolution. Zarr Python defines a collection of Zarr data types. This collection, called a "data type registry," is essentially a dictionary where the keys are strings (a canonical name for each data type), and the values are the data type classes themselves. Dynamic data type resolution entails iterating over these data type classes, invoking that class' `from_native_dtype <#api/dtype/ZDType.from_native_dtype>`_ method, and returning a concrete data type instance if and only if exactly one of those constructor invocations is successful. In plain language, we take some user input, like a NumPy data type, offer it to all the known data type classes, and return an instance of the one data type class that can accept that user input. We want to avoid a situation where the same native data type matches multiple Zarr data types; that is, a NumPy data type should *uniquely* specify a single Zarr data type. But data type resolution is dynamic, so it's not possible to statically guarantee this uniqueness constraint. Therefore, we attempt data type resolution against *every* data type class, and if, for some reason, a native data type matches multiple Zarr data types, we treat this as an error and raise an exception. If you have a NumPy data type and you want to get the corresponding ``ZDType`` instance, you can use the ``parse_dtype`` function, which will use the dynamic resolution described above. ``parse_dtype`` handles a range of input types: - NumPy data types: .. code-block:: python >>> import numpy as np >>> from zarr.dtype import parse_dtype >>> my_dtype = np.dtype('>M8[10s]') >>> parse_dtype(my_dtype, zarr_format=2) DateTime64(endianness='big', scale_factor=10, unit='s') - NumPy data type-compatible strings: .. code-block:: python >>> dtype_str = '>M8[10s]' >>> parse_dtype(dtype_str, zarr_format=2) DateTime64(endianness='big', scale_factor=10, unit='s') - ``ZDType`` instances: .. code-block:: python >>> from zarr.dtype import DateTime64 >>> zdt = DateTime64(endianness='big', scale_factor=10, unit='s') >>> parse_dtype(zdt, zarr_format=2) # Use a ZDType (this is a no-op) DateTime64(endianness='big', scale_factor=10, unit='s') - Python dictionaries (requires ``zarr_format=3``). These dictionaries must be consistent with the ``JSON`` form of the data type: .. code-block:: python >>> dt_dict = {"name": "numpy.datetime64", "configuration": {"unit": "s", "scale_factor": 10}} >>> parse_dtype(dt_dict, zarr_format=3) DateTime64(endianness='little', scale_factor=10, unit='s') >>> parse_dtype(dt_dict, zarr_format=3).to_json(zarr_format=3) {'name': 'numpy.datetime64', 'configuration': {'unit': 's', 'scale_factor': 10}} Extending Zarr ============== Zarr-Python 3 was designed to be extensible. This means that you can extend the library by writing custom classes and plugins. Currently, Zarr can be extended in the following ways: Custom codecs ------------- .. note:: This section explains how custom codecs can be created for Zarr format 3 arrays. For Zarr format 2, codecs should subclass the `numcodecs.abc.Codec `_ base class and register through `numcodecs.registry.register_codec `_. There are three types of codecs in Zarr: - array-to-array - array-to-bytes - bytes-to-bytes Array-to-array codecs are used to transform the array data before serializing to bytes. Examples include delta encoding or scaling codecs. Array-to-bytes codecs are used for serializing the array data to bytes. In Zarr, the main codec to use for numeric arrays is the :class:`zarr.codecs.BytesCodec`. Bytes-to-bytes codecs transform the serialized bytestreams of the array data. Examples include compression codecs, such as :class:`zarr.codecs.GzipCodec`, :class:`zarr.codecs.BloscCodec` or :class:`zarr.codecs.ZstdCodec`, and codecs that add a checksum to the bytestream, such as :class:`zarr.codecs.Crc32cCodec`. Custom codecs for Zarr are implemented by subclassing the relevant base class, see :class:`zarr.abc.codec.ArrayArrayCodec`, :class:`zarr.abc.codec.ArrayBytesCodec` and :class:`zarr.abc.codec.BytesBytesCodec`. Most custom codecs should implemented the ``_encode_single`` and ``_decode_single`` methods. These methods operate on single chunks of the array data. Alternatively, custom codecs can implement the ``encode`` and ``decode`` methods, which operate on batches of chunks, in case the codec is intended to implement its own batch processing. Custom codecs should also implement the following methods: - ``compute_encoded_size``, which returns the byte size of the encoded data given the byte size of the original data. It should raise ``NotImplementedError`` for codecs with variable-sized outputs, such as compression codecs. - ``validate`` (optional), which can be used to check that the codec metadata is compatible with the array metadata. It should raise errors if not. - ``resolve_metadata`` (optional), which is important for codecs that change the shape, dtype or fill value of a chunk. - ``evolve_from_array_spec`` (optional), which can be useful for automatically filling in codec configuration metadata from the array metadata. To use custom codecs in Zarr, they need to be registered using the `entrypoint mechanism `_. Commonly, entrypoints are declared in the ``pyproject.toml`` of your package under the ``[project.entry-points."zarr.codecs"]`` section. Zarr will automatically discover and load all codecs registered with the entrypoint mechanism from imported modules. .. code-block:: toml [project.entry-points."zarr.codecs"] "custompackage.fancy_codec" = "custompackage:FancyCodec" New codecs need to have their own unique identifier. To avoid naming collisions, it is strongly recommended to prefix the codec identifier with a unique name. For example, the codecs from ``numcodecs`` are prefixed with ``numcodecs.``, e.g. ``numcodecs.delta``. .. note:: Note that the extension mechanism for the Zarr format 3 is still under development. Requirements for custom codecs including the choice of codec identifiers might change in the future. It is also possible to register codecs as replacements for existing codecs. This might be useful for providing specialized implementations, such as GPU-based codecs. In case of multiple codecs, the :mod:`zarr.core.config` mechanism can be used to select the preferred implementation. Custom stores ------------- Coming soon. Custom array buffers -------------------- Zarr-python provides control over where and how arrays stored in memory through :mod:`zarr.buffer`. Currently both CPU (the default) and GPU implementations are provided (see :ref:`user-guide-gpu` for more). You can implement your own buffer classes by implementing the interface defined in :mod:`zarr.abc.buffer`. Other extensions ---------------- In the future, Zarr will support writing custom custom data types and chunk grids. .. _user-guide-gpu: Using GPUs with Zarr ==================== Zarr can use GPUs to accelerate your workload by running :meth:`zarr.config.enable_gpu`. .. note:: `zarr-python` currently supports reading the ndarray data into device (GPU) memory as the final stage of the codec pipeline. Data will still be read into or copied to host (CPU) memory for encoding and decoding. In the future, codecs will be available compressing and decompressing data on the GPU, avoiding the need to move data between the host and device for compression and decompression. Reading data into device memory ------------------------------- :meth:`zarr.config.enable_gpu` configures Zarr to use GPU memory for the data buffers used internally by Zarr. .. code-block:: python >>> import zarr >>> import cupy as cp # doctest: +SKIP >>> zarr.config.enable_gpu() # doctest: +SKIP >>> store = zarr.storage.MemoryStore() # doctest: +SKIP >>> z = zarr.create_array( # doctest: +SKIP ... store=store, shape=(100, 100), chunks=(10, 10), dtype="float32", ... ) >>> type(z[:10, :10]) # doctest: +SKIP cupy.ndarray Note that the output type is a ``cupy.ndarray`` rather than a NumPy array. .. only:: doctest >>> import shutil >>> shutil.rmtree('data', ignore_errors=True) .. _user-guide-groups: Working with groups =================== Zarr supports hierarchical organization of arrays via groups. As with arrays, groups can be stored in memory, on disk, or via other storage systems that support a similar interface. To create a group, use the :func:`zarr.group` function:: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> root = zarr.create_group(store=store) >>> root Groups have a similar API to the Group class from `h5py `_. For example, groups can contain other groups:: >>> foo = root.create_group('foo') >>> bar = foo.create_group('bar') Groups can also contain arrays, e.g.:: >>> z1 = bar.create_array(name='baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') >>> z1 Members of a group can be accessed via the suffix notation, e.g.:: >>> root['foo'] The '/' character can be used to access multiple levels of the hierarchy in one call, e.g.:: >>> root['foo/bar'] >>> root['foo/bar/baz'] The :func:`zarr.Group.tree` method can be used to print a tree representation of the hierarchy, e.g.:: >>> root.tree() / └── foo └── bar └── baz (10000, 10000) int32 The :func:`zarr.open_group` function provides a convenient way to create or re-open a group stored in a directory on the file-system, with sub-groups stored in sub-directories, e.g.:: >>> root = zarr.open_group('data/group.zarr', mode='w') >>> root >>> >>> z = root.create_array(name='foo/bar/baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') >>> z .. TODO: uncomment after __enter__ and __exit__ are implemented .. Groups can be used as context managers (in a ``with`` statement). .. If the underlying store has a ``close`` method, it will be called on exit. For more information on groups see the :class:`zarr.Group` API docs. .. _user-guide-diagnostics: Batch Group Creation -------------------- You can also create multiple groups concurrently with a single function call. :func:`zarr.create_hierarchy` takes a :class:`zarr.storage.Store` instance and a dict of ``key : metadata`` pairs, parses that dict, and writes metadata documents to storage: >>> from zarr import create_hierarchy >>> from zarr.core.group import GroupMetadata >>> from zarr.storage import LocalStore >>> node_spec = {'a/b/c': GroupMetadata()} >>> nodes_created = dict(create_hierarchy(store=LocalStore(root='data'), nodes=node_spec)) >>> print(sorted(nodes_created.items(), key=lambda kv: len(kv[0]))) [('', ), ('a', ), ('a/b', ), ('a/b/c', )] Note that we only specified a single group named ``a/b/c``, but 4 groups were created. These additional groups were created to ensure that the desired node ``a/b/c`` is connected to the root group ``''`` by a sequence of intermediate groups. :func:`zarr.create_hierarchy` normalizes the ``nodes`` keyword argument to ensure that the resulting hierarchy is complete, i.e. all groups or arrays are connected to the root of the hierarchy via intermediate groups. Because :func:`zarr.create_hierarchy` concurrently creates metadata documents, it's more efficient than repeated calls to :func:`create_group` or :func:`create_array`, provided you can statically define the metadata for the groups and arrays you want to create. Array and group diagnostics --------------------------- Diagnostic information about arrays and groups is available via the ``info`` property. E.g.:: >>> store = zarr.storage.MemoryStore() >>> root = zarr.group(store=store) >>> foo = root.create_group('foo') >>> bar = foo.create_array(name='bar', shape=1000000, chunks=100000, dtype='int64') >>> bar[:] = 42 >>> baz = foo.create_array(name='baz', shape=(1000, 1000), chunks=(100, 100), dtype='float32') >>> baz[:] = 4.2 >>> root.info Name : Type : Group Zarr format : 3 Read-only : False Store type : MemoryStore >>> foo.info Name : foo Type : Group Zarr format : 3 Read-only : False Store type : MemoryStore >>> bar.info_complete() Type : Array Zarr format : 3 Data type : Int64(endianness='little') Fill value : 0 Shape : (1000000,) Chunk shape : (100000,) Order : C Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 8000000 (7.6M) No. bytes stored : 1614 (1.6K) Storage ratio : 4956.6 Chunks Initialized : 10 >>> baz.info Type : Array Zarr format : 3 Data type : Float32(endianness='little') Fill value : 0.0 Shape : (1000, 1000) Chunk shape : (100, 100) Order : C Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 4000000 (3.8M) Groups also have the :func:`zarr.Group.tree` method, e.g.:: >>> root.tree() / └── foo ├── bar (1000000,) int64 └── baz (1000, 1000) float32 .. note:: :func:`zarr.Group.tree` requires the optional `rich `_ dependency. It can be installed with the ``[tree]`` extra. .. _user-guide: User guide ========== .. toctree:: :maxdepth: 1 installation arrays groups attributes storage config v3_migration cli Advanced Topics --------------- .. toctree:: :maxdepth: 1 data_types performance consolidated_metadata extending gpu .. Coming soon async Installation ============ Required dependencies --------------------- Required dependencies include: - `Python `_ (3.11 or later) - `packaging `_ (22.0 or later) - `numpy `_ (1.26 or later) - `numcodecs[crc32c] `_ (0.14 or later) - `typing_extensions `_ (4.9 or later) - `donfig `_ (0.8 or later) pip --- Zarr is available on `PyPI `_. Install it using ``pip``: .. code-block:: console $ pip install zarr There are a number of optional dependency groups you can install for extra functionality. These can be installed using ``pip install "zarr[]"``, e.g. ``pip install "zarr[gpu]"`` - ``gpu``: support for GPUs - ``remote``: support for reading/writing to remote data stores Additional optional dependencies include ``rich``, ``universal_pathlib``. These must be installed separately. conda ----- Zarr is also published to `conda-forge `_. Install it using ``conda``: .. code-block:: console $ conda install -c conda-forge zarr Conda does not support optional dependencies, so you will have to manually install any packages needed to enable extra functionality. Nightly wheels -------------- Development wheels are built nightly and published to the `scientific-python-nightly-wheels `_ index. To install the latest nightly build: .. code-block:: console $ pip install --pre \ --extra-index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple \ zarr Note that nightly wheels may be unstable and are intended for testing purposes. Dependency support ------------------ Zarr has endorsed `Scientific-Python SPEC 0 `_ and now follows the version support window as outlined below: - Python: 36 months after initial release - Core package dependencies (e.g. NumPy): 24 months after initial release Development ----------- To install the latest development version of Zarr, see the :ref:`contributing guide `. .. only:: doctest >>> import shutil >>> shutil.rmtree('data', ignore_errors=True) .. _user-guide-performance: Optimizing performance ====================== .. _user-guide-chunks: Chunk optimizations ------------------- .. _user-guide-chunks-shape: Chunk size and shape ~~~~~~~~~~~~~~~~~~~~ In general, chunks of at least 1 megabyte (1M) uncompressed size seem to provide better performance, at least when using the Blosc compression library. The optimal chunk shape will depend on how you want to access the data. E.g., for a 2-dimensional array, if you only ever take slices along the first dimension, then chunk across the second dimension. If you know you want to chunk across an entire dimension you can use the full size of that dimension within the ``chunks`` argument, e.g.:: >>> import zarr >>> z1 = zarr.create_array(store={}, shape=(10000, 10000), chunks=(100, 10000), dtype='int32') >>> z1.chunks (100, 10000) Alternatively, if you only ever take slices along the second dimension, then chunk across the first dimension, e.g.:: >>> z2 = zarr.create_array(store={}, shape=(10000, 10000), chunks=(10000, 100), dtype='int32') >>> z2.chunks (10000, 100) If you require reasonable performance for both access patterns then you need to find a compromise, e.g.:: >>> z3 = zarr.create_array(store={}, shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') >>> z3.chunks (1000, 1000) If you are feeling lazy, you can let Zarr guess a chunk shape for your data by providing ``chunks='auto'``, although please note that the algorithm for guessing a chunk shape is based on simple heuristics and may be far from optimal. E.g.:: >>> z4 = zarr.create_array(store={}, shape=(10000, 10000), chunks='auto', dtype='int32') >>> z4.chunks (625, 625) If you know you are always going to be loading the entire array into memory, you can turn off chunks by providing ``chunks`` equal to ``shape``, in which case there will be one single chunk for the array:: >>> z5 = zarr.create_array(store={}, shape=(10000, 10000), chunks=(10000, 10000), dtype='int32') >>> z5.chunks (10000, 10000) Sharding ~~~~~~~~ If you have large arrays but need small chunks to efficiently access the data, you can use sharding. Sharding provides a mechanism to store multiple chunks in a single storage object or file. This can be useful because traditional file systems and object storage systems may have performance issues storing and accessing many files. Additionally, small files can be inefficient to store if they are smaller than the block size of the file system. Picking a good combination of chunk shape and shard shape is important for performance. The chunk shape determines what unit of your data can be read independently, while the shard shape determines what unit of your data can be written efficiently. For an example, consider you have a 100 GB array and need to read small chunks of 1 MB. Without sharding, each chunk would be one file resulting in 100,000 files. That can already cause performance issues on some file systems. With sharding, you could use a shard size of 1 GB. This would result in 1000 chunks per file and 100 files in total, which seems manageable for most storage systems. You would still be able to read each 1 MB chunk independently, but you would need to write your data in 1 GB increments. To use sharding, you need to specify the ``shards`` parameter when creating the array. >>> z6 = zarr.create_array(store={}, shape=(10000, 10000, 1000), shards=(1000, 1000, 1000), chunks=(100, 100, 100), dtype='uint8') >>> z6.info Type : Array Zarr format : 3 Data type : UInt8() Fill value : 0 Shape : (10000, 10000, 1000) Shard shape : (1000, 1000, 1000) Chunk shape : (100, 100, 100) Order : C Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=None) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 100000000000 (93.1G) .. _user-guide-chunks-order: Chunk memory layout ~~~~~~~~~~~~~~~~~~~ The order of bytes **within each chunk** of an array can be changed via the ``order`` config option, to use either C or Fortran layout. For multi-dimensional arrays, these two layouts may provide different compression ratios, depending on the correlation structure within the data. E.g.:: >>> import numpy as np >>> >>> a = np.arange(100000000, dtype='int32').reshape(10000, 10000).T >>> c = zarr.create_array(store={}, shape=a.shape, chunks=(1000, 1000), dtype=a.dtype, config={'order': 'C'}) >>> c[:] = a >>> c.info_complete() Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : C Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 400000000 (381.5M) No. bytes stored : 342588911 (326.7M) Storage ratio : 1.2 Chunks Initialized : 100 >>> with zarr.config.set({'array.order': 'F'}): ... f = zarr.create_array(store={}, shape=a.shape, chunks=(1000, 1000), dtype=a.dtype) ... f[:] = a >>> f.info_complete() Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : F Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 400000000 (381.5M) No. bytes stored : 342588911 (326.7M) Storage ratio : 1.2 Chunks Initialized : 100 In the above example, Fortran order gives a better compression ratio. This is an artificial example but illustrates the general point that changing the order of bytes within chunks of an array may improve the compression ratio, depending on the structure of the data, the compression algorithm used, and which compression filters (e.g., byte-shuffle) have been applied. .. _user-guide-chunks-empty-chunks: Empty chunks ~~~~~~~~~~~~ It is possible to configure how Zarr handles the storage of chunks that are "empty" (i.e., every element in the chunk is equal to the array's fill value). When creating an array with ``write_empty_chunks=False``, Zarr will check whether a chunk is empty before compression and storage. If a chunk is empty, then Zarr does not store it, and instead deletes the chunk from storage if the chunk had been previously stored. This optimization prevents storing redundant objects and can speed up reads, but the cost is added computation during array writes, since the contents of each chunk must be compared to the fill value, and these advantages are contingent on the content of the array. If you know that your data will form chunks that are almost always non-empty, then there is no advantage to the optimization described above. In this case, creating an array with ``write_empty_chunks=True`` (the default) will instruct Zarr to write every chunk without checking for emptiness. The following example illustrates the effect of the ``write_empty_chunks`` flag on the time required to write an array with different values.:: >>> import zarr >>> import numpy as np >>> import time >>> >>> def timed_write(write_empty_chunks): ... """ ... Measure the time required and number of objects created when writing ... to a Zarr array with random ints or fill value. ... """ ... chunks = (8192,) ... shape = (chunks[0] * 1024,) ... data = np.random.randint(0, 255, shape) ... dtype = 'uint8' ... arr = zarr.create_array( ... f'data/example-{write_empty_chunks}.zarr', ... shape=shape, ... chunks=chunks, ... dtype=dtype, ... fill_value=0, ... config={'write_empty_chunks': write_empty_chunks} ... ) ... # initialize all chunks ... arr[:] = 100 ... result = [] ... for value in (data, arr.fill_value): ... start = time.time() ... arr[:] = value ... elapsed = time.time() - start ... result.append((elapsed, arr.nchunks_initialized)) ... return result ... # log results >>> for write_empty_chunks in (True, False): ... full, empty = timed_write(write_empty_chunks) ... print(f'\nwrite_empty_chunks={write_empty_chunks}:\n\tRandom Data: {full[0]:.4f}s, {full[1]} objects stored\n\t Empty Data: {empty[0]:.4f}s, {empty[1]} objects stored\n') write_empty_chunks=True: Random Data: ..., 1024 objects stored Empty Data: ...s, 1024 objects stored write_empty_chunks=False: Random Data: ...s, 1024 objects stored Empty Data: ...s, 0 objects stored In this example, writing random data is slightly slower with ``write_empty_chunks=True``, but writing empty data is substantially faster and generates far fewer objects in storage. .. _user-guide-rechunking: Changing chunk shapes (rechunking) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Coming soon. .. _user-guide-sync: Parallel computing and synchronization -------------------------------------- Coming soon. .. _user-guide-pickle: Pickle support -------------- Zarr arrays and groups can be pickled, as long as the underlying store object can be pickled. With the exception of the :class:`zarr.storage.MemoryStore`, any of the storage classes provided in the :mod:`zarr.storage` module can be pickled. If an array or group is backed by a persistent store such as the a :class:`zarr.storage.LocalStore`, :class:`zarr.storage.ZipStore` or :class:`zarr.storage.FsspecStore` then the store data **are not** pickled. The only thing that is pickled is the necessary parameters to allow the store to re-open any underlying files or databases upon being unpickled. E.g., pickle/unpickle an local store array:: >>> import pickle >>> data = np.arange(100000) >>> z1 = zarr.create_array(store='data/example-2.zarr', shape=data.shape, chunks=data.shape, dtype=data.dtype) >>> z1[:] = data >>> s = pickle.dumps(z1) >>> z2 = pickle.loads(s) >>> z1 == z2 True >>> np.all(z1[:] == z2[:]) np.True_ .. _user-guide-tips-blosc: Configuring Blosc ----------------- Coming soon. .. only:: doctest >>> import shutil >>> shutil.rmtree('data', ignore_errors=True) .. _user-guide-storage: Storage guide ============= Zarr-Python supports multiple storage backends, including: local file systems, Zip files, remote stores via fsspec_ (S3, HTTP, etc.), and in-memory stores. In Zarr-Python 3, stores must implement the abstract store API from :class:`zarr.abc.store.Store`. .. note:: Unlike Zarr-Python 2 where the store interface was built around a generic ``MutableMapping`` API, Zarr-Python 3 utilizes a custom store API that utilizes Python's AsyncIO library. Implicit Store Creation ----------------------- In most cases, it is not required to create a ``Store`` object explicitly. Passing a string to Zarr's top level API will result in the store being created automatically.: >>> import zarr >>> >>> # Implicitly create a writable LocalStore >>> zarr.create_group(store='data/foo/bar') >>> >>> # Implicitly create a read-only FsspecStore >>> zarr.open_group( ... store='s3://noaa-nwm-retro-v2-zarr-pds', ... mode='r', ... storage_options={'anon': True} ... ) > >>> >>> # Implicitly creates a MemoryStore >>> data = {} >>> zarr.create_group(store=data) Explicit Store Creation ----------------------- In some cases, it may be helpful to create a store instance directly. Zarr-Python offers four built-in store: :class:`zarr.storage.LocalStore`, :class:`zarr.storage.FsspecStore`, :class:`zarr.storage.ZipStore`, :class:`zarr.storage.MemoryStore`, and :class:`zarr.storage.ObjectStore`. Local Store ~~~~~~~~~~~ The :class:`zarr.storage.LocalStore` stores data in a nested set of directories on a local filesystem.: >>> store = zarr.storage.LocalStore('data/foo/bar', read_only=True) >>> zarr.open_group(store=store, mode='r') Zip Store ~~~~~~~~~ The :class:`zarr.storage.ZipStore` stores the contents of a Zarr hierarchy in a single Zip file. The `Zip Store specification`_ is currently in draft form.: >>> store = zarr.storage.ZipStore('data.zip', mode='w') >>> zarr.create_array(store=store, shape=(2,), dtype='float64') Remote Store ~~~~~~~~~~~~ The :class:`zarr.storage.FsspecStore` stores the contents of a Zarr hierarchy in following the same logical layout as the ``LocalStore``, except the store is assumed to be on a remote storage system such as cloud object storage (e.g. AWS S3, Google Cloud Storage, Azure Blob Store). The :class:`zarr.storage.FsspecStore` is backed by `fsspec`_ and can support any backend that implements the `AbstractFileSystem `_ API. ``storage_options`` can be used to configure the fsspec backend.: >>> store = zarr.storage.FsspecStore.from_url( ... 's3://noaa-nwm-retro-v2-zarr-pds', ... read_only=True, ... storage_options={'anon': True} ... ) >>> zarr.open_group(store=store, mode='r') > The type of filesystem (e.g. S3, https, etc..) is inferred from the scheme of the url (e.g. s3 for "**s3**://noaa-nwm-retro-v2-zarr-pds"). In case a specific filesystem is needed, one can explicitly create it. For example to create a S3 filesystem: >>> import fsspec >>> fs = fsspec.filesystem( ... 's3', anon=True, asynchronous=True, ... client_kwargs={'endpoint_url': "https://noaa-nwm-retro-v2-zarr-pds.s3.amazonaws.com"} ... ) >>> store = zarr.storage.FsspecStore(fs) Memory Store ~~~~~~~~~~~~ The :class:`zarr.storage.MemoryStore` a in-memory store that allows for serialization of Zarr data (metadata and chunks) to a dictionary.: >>> data = {} >>> store = zarr.storage.MemoryStore(data) >>> # TODO: replace with create_array after #2463 >>> zarr.create_array(store=store, shape=(2,), dtype='float64') Object Store ~~~~~~~~~~~~ :class:`zarr.storage.ObjectStore` stores the contents of the Zarr hierarchy using any ObjectStore `storage implementation `_, including AWS S3 (:class:`obstore.store.S3Store`), Google Cloud Storage (:class:`obstore.store.GCSStore`), and Azure Blob Storage (:class:`obstore.store.AzureStore`). This store is backed by `obstore `_, which builds on the production quality Rust library `object_store `_. >>> from zarr.storage import ObjectStore >>> from obstore.store import MemoryStore >>> >>> store = ObjectStore(MemoryStore()) >>> zarr.create_array(store=store, shape=(2,), dtype='float64') Here's an example of using ObjectStore for accessing remote data: >>> from zarr.storage import ObjectStore >>> from obstore.store import S3Store >>> >>> s3_store = S3Store('noaa-nwm-retro-v2-zarr-pds', skip_signature=True, region="us-west-2") >>> store = zarr.storage.ObjectStore(store=s3_store, read_only=True) >>> group = zarr.open_group(store=store, mode='r') >>> group.info Name : Type : Group Zarr format : 2 Read-only : True Store type : ObjectStore No. members : 12 No. arrays : 12 No. groups : 0 .. warning:: The :class:`zarr.storage.ObjectStore` class is experimental. .. _user-guide-custom-stores: Developing custom stores ------------------------ Zarr-Python :class:`zarr.abc.store.Store` API is meant to be extended. The Store Abstract Base Class includes all of the methods needed to be a fully operational store in Zarr Python. Zarr also provides a test harness for custom stores: :class:`zarr.testing.store.StoreTests`. .. _Zip Store Specification: https://github.com/zarr-developers/zarr-specs/pull/311 .. _fsspec: https://filesystem-spec.readthedocs.io .. _v3 migration guide: 3.0 Migration Guide =================== Zarr-Python 3 represents a major refactor of the Zarr-Python codebase. Some of the goals motivating this refactor included: * adding support for the Zarr format 3 specification (along with the Zarr format 2 specification) * cleaning up internal and user facing APIs * improving performance (particularly in high latency storage environments like cloud object stores) To accommodate this, Zarr-Python 3 introduces a number of changes to the API, including a number of significant breaking changes and deprecations. This page provides a guide explaining breaking changes and deprecations to help you migrate your code from version 2 to version 3. If we have missed anything, please open a `GitHub issue `_ so we can improve this guide. Compatibility target -------------------- The goals described above necessitated some breaking changes to the API (hence the major version update), but where possible we have maintained backwards compatibility in the most widely used parts of the API. This in the :class:`zarr.Array` and :class:`zarr.Group` classes and the "top-level API" (e.g. :func:`zarr.open_array` and :func:`zarr.open_group`). Getting ready for 3.0 --------------------- Before migrating to Zarr-Python 3, we suggest projects that depend on Zarr-Python take the following actions in order: 1. Pin the supported Zarr-Python version to ``zarr>=2,<3``. This is a best practice and will protect your users from any incompatibilities that may arise during the release of Zarr-Python 3. This pin can be removed after migrating to Zarr-Python 3. 2. Limit your imports from the Zarr-Python package. Most of the primary API ``zarr.*`` will be compatible in Zarr-Python 3. However, the following breaking API changes are planned: - ``numcodecs.*`` will no longer be available in ``zarr.*``. To migrate, import codecs directly from ``numcodecs``: .. code-block:: python from numcodecs import Blosc # instead of: # from zarr import Blosc - The ``zarr.v3_api_available`` feature flag is being removed. In Zarr-Python 3 the v3 API is always available, so you shouldn't need to use this flag. - The following internal modules are being removed or significantly changed. If your application relies on imports from any of the below modules, you will need to either a) modify your application to no longer rely on these imports or b) vendor the parts of the specific modules that you need. * ``zarr.attrs`` has gone, with no replacement * ``zarr.codecs`` has changed, see "Codecs" section below for more information * ``zarr.context`` has gone, with no replacement * ``zarr.core`` remains but should be considered private API * ``zarr.hierarchy`` has gone, with no replacement (use ``zarr.Group`` inplace of ``zarr.hierarchy.Group``) * ``zarr.indexing`` has gone, with no replacement * ``zarr.meta`` has gone, with no replacement * ``zarr.meta_v1`` has gone, with no replacement * ``zarr.sync`` has gone, with no replacement * ``zarr.types`` has gone, with no replacement * ``zarr.util`` has gone, with no replacement * ``zarr.n5`` has gone, see below for an alternative N5 options 3. Test that your package works with version 3. 4. Update the pin to include ``zarr>=3,<4``. Zarr-Python 2 support window ---------------------------- Zarr-Python 2.x is still available, though we recommend migrating to Zarr-Python 3 for its performance improvements and new features. Security and bug fixes will be made to the 2.x series for at least six months following the first Zarr-Python 3 release. If you need to use the latest Zarr-Python 2 release, you can install it with: .. code-block:: console $ pip install "zarr==2.*" .. note:: Development and maintenance of the 2.x release series has moved to the `support/v2 `_ branch. Issues and pull requests related to this branch are tagged with the `V2 `_ label. Migrating to Zarr-Python 3 -------------------------- The following sections provide details on breaking changes in Zarr-Python 3. The Array class ~~~~~~~~~~~~~~~ 1. Disallow direct construction - the signature for initializing the ``Array`` class has changed significantly. Please use :func:`zarr.create_array` or :func:`zarr.open_array` instead of directly constructing the :class:`zarr.Array` class. 2. Defaulting to ``zarr_format=3`` - newly created arrays will use the version 3 of the Zarr specification. To continue using version 2, set ``zarr_format=2`` when creating arrays or set ``default_zarr_version=2`` in Zarr's :ref:`runtime configuration `. The Group class ~~~~~~~~~~~~~~~ 1. Disallow direct construction - use :func:`zarr.open_group` or :func:`zarr.create_group` instead of directly constructing the :class:`zarr.Group` class. 2. Most of the h5py compatibility methods are deprecated and will issue warnings if used. The following functions are drop in replacements that have the same signature and functionality: - Use :func:`zarr.Group.create_array` in place of :func:`zarr.Group.create_dataset` - Use :func:`zarr.Group.require_array` in place of :func:`zarr.Group.require_dataset` 3. Disallow "." syntax for getting group members. To get a member of a group named ``foo``, use ``group["foo"]`` in place of ``group.foo``. The Store class ~~~~~~~~~~~~~~~ The Store API has changed significant in Zarr-Python 3. The most notable changes to the Store API are: Store Import Paths ^^^^^^^^^^^^^^^^^^ Several store implementations have moved from the top-level module to ``zarr.storage``: .. code-block:: diff :caption: Store import changes from v2 to v3 # Before (v2) - from zarr import MemoryStore, DirectoryStore + from zarr.storage import MemoryStore, LocalStore # LocalStore replaces DirectoryStore Common replacements: +-------------------------+------------------------------------+ | v2 Import | v3 Import | +=========================+====================================+ | ``zarr.MemoryStore`` | ``zarr.storage.MemoryStore`` | +-------------------------+------------------------------------+ | ``zarr.DirectoryStore`` | ``zarr.storage.LocalStore`` | +-------------------------+------------------------------------+ | ``zarr.TempStore`` | Use ``tempfile.TemporaryDirectory``| | | with ``LocalStore`` | +-------------------------+------------------------------------+ 1. Replaced the ``MutableMapping`` base class in favor of a custom abstract base class (:class:`zarr.abc.store.Store`). 2. Switched to an asynchronous interface for all store methods that result in IO. This change ensures that all store methods are non-blocking and are as performant as possible. Beyond the changes store interface, a number of deprecated stores were also removed in Zarr-Python 3. See :issue:`1274` for more details on the removal of these stores. - ``N5Store`` - see https://github.com/zarr-developers/n5py for an alternative interface to N5 formatted data. - ``ABSStore`` - use the :class:`zarr.storage.FsspecStore` instead along with fsspec's `adlfs backend `_. The following stores have been removed altogether. Users who need these stores will have to implement their own version in zarr-python v3. - ``DBMStore`` - ``LMDBStore`` - ``SQLiteStore`` - ``MongoDBStore`` - ``RedisStore`` At present, the latter five stores in this list do not have an equivalent in Zarr-Python 3. If you are interested in developing a custom store that targets these backends, see :ref:`developing custom stores ` or open an `issue `_ to discuss your use case. Codecs ~~~~~~ Codecs defined in ``numcodecs`` (and also imported into the ``zarr.codecs`` namespace in Zarr-Python 2) should still be used when creating Zarr format 2 arrays. Codecs for creating Zarr format 3 arrays are available in two locations: - `zarr.codecs` contains Zarr format 3 codecs that are defined in the `codecs section of the Zarr format 3 specification `_. - `numcodecs.zarr3` contains codecs from ``numcodecs`` that can be used to create Zarr format 3 arrays, but are not necessarily part of the Zarr format 3 specification. Dependencies ~~~~~~~~~~~~ When installing using ``pip``: - The new ``remote`` dependency group can be used to install a supported version of ``fsspec``, required for remote data access. - The new ``gpu`` dependency group can be used to install a supported version of ``cuda``, required for GPU functionality. - The ``jupyter`` optional dependency group has been removed, since v3 contains no jupyter specific functionality. Miscellaneous ~~~~~~~~~~~~~ - The keyword argument ``zarr_version`` available in most creation functions in :mod:`zarr` (e.g. :func:`zarr.create`, :func:`zarr.open`, :func:`zarr.group`, :func:`zarr.array`) has been deprecated in favor of ``zarr_format``. 🚧 Work in Progress 🚧 ---------------------- Zarr-Python 3 is still under active development, and is not yet fully complete. The following list summarizes areas of the codebase that we expect to build out after the 3.0.0 release. If features listed below are important to your use case of Zarr-Python, please open (or comment on) a `GitHub issue `_. - The following functions / methods have not been ported to Zarr-Python 3 yet: * :func:`zarr.copy` (:issue:`2407`) * :func:`zarr.copy_all` (:issue:`2407`) * :func:`zarr.copy_store` (:issue:`2407`) * :func:`zarr.Group.move` (:issue:`2108`) - The following features (corresponding to function arguments to functions in :mod:`zarr`) have not been ported to Zarr-Python 3 yet. Using these features will raise a warning or a ``NotImplementedError``: * ``cache_attrs`` * ``cache_metadata`` * ``chunk_store`` (:issue:`2495`) * ``meta_array`` * ``object_codec`` (:issue:`2617`) * ``synchronizer`` (:issue:`1596`) * ``dimension_separator`` - The following features that were supported by Zarr-Python 2 have not been ported to Zarr-Python 3 yet: * Structured arrays / dtypes (:issue:`2134`) * Fixed-length string dtypes (:issue:`2347`) * Datetime and timedelta dtypes (:issue:`2616`) * Object dtypes (:issue:`2617`) * Ragged arrays (:issue:`2618`) * Groups and Arrays do not implement ``__enter__`` and ``__exit__`` protocols (:issue:`2619`) * Big Endian dtypes (:issue:`2324`) * Default filters for object dtypes for Zarr format 2 arrays (:issue:`2627`)Skip to main content Back to top `Ctrl`+`K` * Quickstart * User guide * API reference * Release notes * Developer’s Guide * More * About Search `Ctrl`+`K` * GitHub * Bluesky * Mastodon * Search `Ctrl`+`K` * Quickstart * User guide * API reference * Release notes * Developer’s Guide * About * GitHub * Bluesky * Mastodon * # Zarr-Python# ## Quickstart# Welcome to the Zarr-Python Quickstart guide! This page will help you get up and running with the Zarr library in Python to efficiently manage and analyze multi-dimensional arrays. Zarr is a powerful library for storage of n-dimensional arrays, supporting chunking, compression, and various backends, making it a versatile choice for scientific and large-scale data. ### Installation# Zarr requires Python 3.11 or higher. You can install it via pip: pip install zarr or conda: conda install --channel conda-forge zarr ### Creating an Array# To get started, you can create a simple Zarr array: >>> import zarr >>> import numpy as np >>> >>> # Create a 2D Zarr array >>> z = zarr.create_array( ... store="data/example-1.zarr", ... shape=(100, 100), ... chunks=(10, 10), ... dtype="f4" ... ) >>> >>> # Assign data to the array >>> z[:, :] = np.random.random((100, 100)) >>> z.info Type : Array Zarr format : 3 Data type : DataType.float32 Shape : (100, 100) Chunk shape : (10, 10) Order : C Read-only : False Store type : LocalStore Codecs : [{'endian': }, {'level': 0, 'checksum': False}] No. bytes : 40000 (39.1K) Here, we created a 2D array of shape `(100, 100)`, chunked into blocks of `(10, 10)`, and filled it with random floating-point data. This array was written to a `LocalStore` in the `data/example-1.zarr` directory. #### Compression and Filters# Zarr supports data compression and filters. For example, to use Blosc compression: >>> z = zarr.create_array( ... "data/example-3.zarr", ... mode="w", shape=(100, 100), ... chunks=(10, 10), dtype="f4", ... compressors=zarr.codecs.BloscCodec(cname="zstd", clevel=3, shuffle=zarr.codecs.BloscShuffle.shuffle) ... ) >>> z[:, :] = np.random.random((100, 100)) >>> >>> z.info Type : Array Zarr format : 3 Data type : DataType.float32 Shape : (100, 100) Chunk shape : (10, 10) Order : C Read-only : False Store type : LocalStore Codecs : [{'endian': }, {'level': 0, 'checksum': False}] No. bytes : 40000 (39.1K) This compresses the data using the Zstandard codec with shuffle enabled for better compression. ### Hierarchical Groups# Zarr allows you to create hierarchical groups, similar to directories: >>> # Create nested groups and add arrays >>> root = zarr.group("data/example-2.zarr") >>> foo = root.create_group(name="foo") >>> bar = root.create_array( ... name="bar", shape=(100, 10), chunks=(10, 10), dtype="f4" ... ) >>> spam = foo.create_array(name="spam", shape=(10,), dtype="i4") >>> >>> # Assign values >>> bar[:, :] = np.random.random((100, 10)) >>> spam[:] = np.arange(10) >>> >>> # print the hierarchy >>> root.tree() / ├── bar (100, 10) float32 └── foo └── spam (10,) int32 This creates a group with two datasets: `foo` and `bar`. #### Batch Hierarchy Creation# Zarr provides tools for creating a collection of arrays and groups with a single function call. Suppose we want to copy existing groups and arrays into a new storage backend: >>> # Create nested groups and add arrays >>> root = zarr.group("data/example-3.zarr", attributes={'name': 'root'}) >>> foo = root.create_group(name="foo") >>> bar = root.create_array( ... name="bar", shape=(100, 10), chunks=(10, 10), dtype="f4" ... ) >>> nodes = {'': root.metadata} | {k: v.metadata for k,v in root.members()} >>> print(nodes) >>> from zarr.storage import MemoryStore >>> new_nodes = dict(zarr.create_hierarchy(store=MemoryStore(), nodes=nodes)) >>> new_root = new_nodes[''] >>> assert new_root.attrs == root.attrs Note that `zarr.create_hierarchy()` will only initialize arrays and groups – copying array data must be done in a separate step. ### Persistent Storage# Zarr supports persistent storage to disk or cloud-compatible backends. While examples above utilized a `zarr.storage.LocalStore`, a number of other storage options are available. Zarr integrates seamlessly with cloud object storage such as Amazon S3 and Google Cloud Storage using external libraries like s3fs or gcsfs: >>> import s3fs >>> >>> z = zarr.create_array("s3://example-bucket/foo", mode="w", shape=(100, 100), chunks=(10, 10), dtype="f4") >>> z[:, :] = np.random.random((100, 100)) A single-file store can also be created using the the `zarr.storage.ZipStore`: >>> # Store the array in a ZIP file >>> store = zarr.storage.ZipStore("data/example-3.zip", mode='w') >>> >>> z = zarr.create_array( ... store=store, ... mode="w", ... shape=(100, 100), ... chunks=(10, 10), ... dtype="f4" ... ) >>> >>> # write to the array >>> z[:, :] = np.random.random((100, 100)) >>> >>> # the ZipStore must be explicitly closed >>> store.close() To open an existing array from a ZIP file: >>> # Open the ZipStore in read-only mode >>> store = zarr.storage.ZipStore("data/example-3.zip", read_only=True) >>> >>> z = zarr.open_array(store, mode='r') >>> >>> # read the data as a NumPy Array >>> z[:] array([[0.66734236, 0.15667458, 0.98720884, ..., 0.36229587, 0.67443246, 0.34315267], [0.65787303, 0.9544212 , 0.4830079 , ..., 0.33097172, 0.60423803, 0.45621237], [0.27632037, 0.9947008 , 0.42434934, ..., 0.94860053, 0.6226942 , 0.6386924 ], ..., [0.12854576, 0.934397 , 0.19524333, ..., 0.11838563, 0.4967675 , 0.43074256], [0.82029045, 0.4671437 , 0.8090906 , ..., 0.7814118 , 0.42650765, 0.95929915], [0.4335856 , 0.7565437 , 0.7828931 , ..., 0.48119593, 0.66220033, 0.6652362 ]], shape=(100, 100), dtype=float32) Read more about Zarr’s storage options in the User Guide. ### Next Steps# Now that you’re familiar with the basics, explore the following resources: * User Guide * API Reference ## User guide# ### Installation# #### Required dependencies# Required dependencies include: * Python (3.11 or later) * packaging (22.0 or later) * numpy (1.26 or later) * numcodecs[crc32c] (0.14 or later) * typing_extensions (4.9 or later) * donfig (0.8 or later) #### pip# Zarr is available on PyPI. Install it using `pip`: $ pip install zarr There are a number of optional dependency groups you can install for extra functionality. These can be installed using `pip install "zarr[]"`, e.g. `pip install "zarr[gpu]"` * `gpu`: support for GPUs * `remote`: support for reading/writing to remote data stores Additional optional dependencies include `rich`, `universal_pathlib`. These must be installed separately. #### conda# Zarr is also published to conda-forge. Install it using `conda`: $ conda install -c conda-forge zarr Conda does not support optional dependencies, so you will have to manually install any packages needed to enable extra functionality. #### Nightly wheels# Development wheels are built nightly and published to the scientific-python-nightly-wheels index. To install the latest nightly build: $ pip install --pre \ --extra-index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple \ zarr Note that nightly wheels may be unstable and are intended for testing purposes. #### Dependency support# Zarr has endorsed Scientific-Python SPEC 0 and now follows the version support window as outlined below: * Python: 36 months after initial release * Core package dependencies (e.g. NumPy): 24 months after initial release #### Development# To install the latest development version of Zarr, see the contributing guide. ### Working with arrays# #### Creating an array# Zarr has several functions for creating arrays. For example: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> z = zarr.create_array(store=store, shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') >>> z The code above creates a 2-dimensional array of 32-bit integers with 10000 rows and 10000 columns, divided into chunks where each chunk has 1000 rows and 1000 columns (and so there will be 100 chunks in total). The data is written to a `zarr.storage.MemoryStore` (e.g. an in-memory dict). See Persistent arrays for details on storing arrays in other stores, and see Array data types for an in-depth look at the data types supported by Zarr. For a complete list of array creation routines see the `zarr` module documentation. #### Reading and writing data# Zarr arrays support a similar interface to NumPy arrays for reading and writing data. For example, the entire array can be filled with a scalar value: >>> z[:] = 42 Regions of the array can also be written to, e.g.: >>> import numpy as np >>> >>> z[0, :] = np.arange(10000) >>> z[:, 0] = np.arange(10000) The contents of the array can be retrieved by slicing, which will load the requested region into memory as a NumPy array, e.g.: >>> z[0, 0] array(0, dtype=int32) >>> z[-1, -1] array(42, dtype=int32) >>> z[0, :] array([ 0, 1, 2, ..., 9997, 9998, 9999], shape=(10000,), dtype=int32) >>> z[:, 0] array([ 0, 1, 2, ..., 9997, 9998, 9999], shape=(10000,), dtype=int32) >>> z[:] array([[ 0, 1, 2, ..., 9997, 9998, 9999], [ 1, 42, 42, ..., 42, 42, 42], [ 2, 42, 42, ..., 42, 42, 42], ..., [9997, 42, 42, ..., 42, 42, 42], [9998, 42, 42, ..., 42, 42, 42], [9999, 42, 42, ..., 42, 42, 42]], shape=(10000, 10000), dtype=int32) Read more about NumPy-style indexing can be found in the NumPy documentation. #### Persistent arrays# In the examples above, compressed data for each chunk of the array was stored in main memory. Zarr arrays can also be stored on a file system, enabling persistence of data between sessions. To do this, we can change the store argument to point to a filesystem path: >>> z1 = zarr.create_array(store='data/example-1.zarr', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') The array above will store its configuration metadata and all compressed chunk data in a directory called `'data/example-1.zarr'` relative to the current working directory. The `zarr.create_array()` function provides a convenient way to create a new persistent array or continue working with an existing array. Note, there is no need to close an array: data are automatically flushed to disk, and files are automatically closed whenever an array is modified. Persistent arrays support the same interface for reading and writing data, e.g.: >>> z1[:] = 42 >>> z1[0, :] = np.arange(10000) >>> z1[:, 0] = np.arange(10000) Check that the data have been written and can be read again: >>> z2 = zarr.open_array('data/example-1.zarr', mode='r') >>> np.all(z1[:] == z2[:]) np.True_ If you are just looking for a fast and convenient way to save NumPy arrays to disk then load back into memory later, the functions `zarr.save()` and `zarr.load()` may be useful. E.g.: >>> a = np.arange(10) >>> zarr.save('data/example-2.zarr', a) >>> zarr.load('data/example-2.zarr') array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) Please note that there are a number of other options for persistent array storage, see the Storage Guide guide for more details. #### Resizing and appending# A Zarr array can be resized, which means that any of its dimensions can be increased or decreased in length. For example: >>> z = zarr.create_array(store='data/example-3.zarr', shape=(10000, 10000), dtype='int32',chunks=(1000, 1000)) >>> z[:] = 42 >>> z.shape (10000, 10000) >>> z.resize((20000, 10000)) >>> z.shape (20000, 10000) Note that when an array is resized, the underlying data are not rearranged in any way. If one or more dimensions are shrunk, any chunks falling outside the new array shape will be deleted from the underlying store. `zarr.Array.append()` is provided as a convenience function, which can be used to append data to any axis. E.g.: >>> a = np.arange(10000000, dtype='int32').reshape(10000, 1000) >>> z = zarr.create_array(store='data/example-4.zarr', shape=a.shape, dtype=a.dtype, chunks=(1000, 100)) >>> z[:] = a >>> z.shape (10000, 1000) >>> z.append(a) (20000, 1000) >>> z.append(np.vstack([a, a]), axis=1) (20000, 2000) >>> z.shape (20000, 2000) #### Compressors# A number of different compressors can be used with Zarr. Zarr includes Blosc, Zstandard and Gzip compressors. Additional compressors are available through a separate package called NumCodecs which provides various compressor libraries including LZ4, Zlib, BZ2 and LZMA. Different compressors can be provided via the `compressors` keyword argument accepted by all array creation functions. For example: >>> compressors = zarr.codecs.BloscCodec(cname='zstd', clevel=3, shuffle=zarr.codecs.BloscShuffle.bitshuffle) >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000) >>> z = zarr.create_array(store='data/example-5.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=compressors) >>> z[:] = data >>> z.compressors (BloscCodec(typesize=4, cname=, clevel=3, shuffle=, blocksize=0),) This array above will use Blosc as the primary compressor, using the Zstandard algorithm (compression level 3) internally within Blosc, and with the bit-shuffle filter applied. When using a compressor, it can be useful to get some diagnostics on the compression ratio. Zarr arrays provide the `zarr.Array.info` property which can be used to print useful diagnostics, e.g.: >>> z.info Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : C Read-only : False Store type : LocalStore Filters : () Serializer : BytesCodec(endian=) Compressors : (BloscCodec(typesize=4, cname=, clevel=3, shuffle=, blocksize=0),) No. bytes : 400000000 (381.5M) The `zarr.Array.info_complete()` method inspects the underlying store and prints additional diagnostics, e.g.: >>> z.info_complete() Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : C Read-only : False Store type : LocalStore Filters : () Serializer : BytesCodec(endian=) Compressors : (BloscCodec(typesize=4, cname=, clevel=3, shuffle=, blocksize=0),) No. bytes : 400000000 (381.5M) No. bytes stored : 3558573 (3.4M) Storage ratio : 112.4 Chunks Initialized : 100 Note `zarr.Array.info_complete()` will inspect the underlying store and may be slow for large arrays. Use `zarr.Array.info` if detailed storage statistics are not needed. If you don’t specify a compressor, by default Zarr uses the Zstandard compressor. In addition to Blosc and Zstandard, other compression libraries can also be used. For example, here is an array using Gzip compression, level 1: >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000) >>> z = zarr.create_array(store='data/example-6.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=zarr.codecs.GzipCodec(level=1)) >>> z[:] = data >>> z.compressors (GzipCodec(level=1),) Here is an example using LZMA from NumCodecs with a custom filter pipeline including LZMA’s built-in delta filter: >>> import lzma >>> from numcodecs.zarr3 import LZMA >>> import warnings >>> warnings.filterwarnings("ignore", category=UserWarning) >>> >>> lzma_filters = [dict(id=lzma.FILTER_DELTA, dist=4), dict(id=lzma.FILTER_LZMA2, preset=1)] >>> compressors = LZMA(filters=lzma_filters) >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000) >>> z = zarr.create_array(store='data/example-7.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=compressors) >>> z.compressors (LZMA(codec_name='numcodecs.lzma', codec_config={'filters': [{'id': 3, 'dist': 4}, {'id': 33, 'preset': 1}]}),) To disable compression, set `compressors=None` when creating an array, e.g.: >>> z = zarr.create_array(store='data/example-8.zarr', shape=(100000000,), chunks=(1000000,), dtype='int32', compressors=None) >>> z.compressors () #### Filters# In some cases, compression can be improved by transforming the data in some way. For example, if nearby values tend to be correlated, then shuffling the bytes within each numerical value or storing the difference between adjacent values may increase compression ratio. Some compressors provide built-in filters that apply transformations to the data prior to compression. For example, the Blosc compressor has built-in implementations of byte- and bit-shuffle filters, and the LZMA compressor has a built-in implementation of a delta filter. However, to provide additional flexibility for implementing and using filters in combination with different compressors, Zarr also provides a mechanism for configuring filters outside of the primary compressor. Here is an example using a delta filter with the Blosc compressor: >>> from numcodecs.zarr3 import Delta >>> >>> filters = [Delta(dtype='int32')] >>> compressors = zarr.codecs.BloscCodec(cname='zstd', clevel=1, shuffle=zarr.codecs.BloscShuffle.shuffle) >>> data = np.arange(100000000, dtype='int32').reshape(10000, 10000) >>> z = zarr.create_array(store='data/example-9.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), filters=filters, compressors=compressors) >>> z.info_complete() Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : C Read-only : False Store type : LocalStore Filters : (Delta(codec_name='numcodecs.delta', codec_config={'dtype': 'int32'}),) Serializer : BytesCodec(endian=) Compressors : (BloscCodec(typesize=4, cname=, clevel=1, shuffle=, blocksize=0),) No. bytes : 400000000 (381.5M) No. bytes stored : 826 Storage ratio : 484261.5 Chunks Initialized : 0 For more information about available filter codecs, see the Numcodecs documentation. #### Advanced indexing# Zarr arrays support several methods for advanced or “fancy” indexing, which enable a subset of data items to be extracted or updated in an array without loading the entire array into memory. Note that although this functionality is similar to some of the advanced indexing capabilities available on NumPy arrays and on h5py datasets, the Zarr API for advanced indexing is different from both NumPy and h5py, so please read this section carefully. For a complete description of the indexing API, see the documentation for the `zarr.Array` class. ##### Indexing with coordinate arrays# Items from a Zarr array can be extracted by providing an integer array of coordinates. E.g.: >>> data = np.arange(10) ** 2 >>> z = zarr.create_array(store='data/example-10.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81]) >>> z.get_coordinate_selection([2, 5]) array([ 4, 25]) Coordinate arrays can also be used to update data, e.g.: >>> z.set_coordinate_selection([2, 5], [-1, -2]) >>> z[:] array([ 0, 1, -1, 9, 16, -2, 36, 49, 64, 81]) For multidimensional arrays, coordinates must be provided for each dimension, e.g.: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-11.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14]]) >>> z.get_coordinate_selection(([0, 2], [1, 3])) array([ 1, 13]) >>> z.set_coordinate_selection(([0, 2], [1, 3]), [-1, -2]) >>> z[:] array([[ 0, -1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, -2, 14]]) For convenience, coordinate indexing is also available via the `vindex` property, as well as the square bracket operator, e.g.: >>> z.vindex[[0, 2], [1, 3]] array([-1, -2]) >>> z.vindex[[0, 2], [1, 3]] = [-3, -4] >>> z[:] array([[ 0, -3, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, -4, 14]]) >>> z[[0, 2], [1, 3]] array([-3, -4]) When the indexing arrays have different shapes, they are broadcast together. That is, the following two calls are equivalent: >>> z[1, [1, 3]] array([6, 8]) >>> z[[1, 1], [1, 3]] array([6, 8]) ##### Indexing with a mask array# Items can also be extracted by providing a Boolean mask. E.g.: >>> data = np.arange(10) ** 2 >>> z = zarr.create_array(store='data/example-12.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([ 0, 1, 4, 9, 16, 25, 36, 49, 64, 81]) >>> sel = np.zeros_like(z, dtype=bool) >>> sel[2] = True >>> sel[5] = True >>> z.get_mask_selection(sel) array([ 4, 25]) >>> z.set_mask_selection(sel, [-1, -2]) >>> z[:] array([ 0, 1, -1, 9, 16, -2, 36, 49, 64, 81]) Here’s a multidimensional example: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-13.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14]]) >>> sel = np.zeros_like(z, dtype=bool) >>> sel[0, 1] = True >>> sel[2, 3] = True >>> z.get_mask_selection(sel) array([ 1, 13]) >>> z.set_mask_selection(sel, [-1, -2]) >>> z[:] array([[ 0, -1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, -2, 14]]) For convenience, mask indexing is also available via the `vindex` property, e.g.: >>> z.vindex[sel] array([-1, -2]) >>> z.vindex[sel] = [-3, -4] >>> z[:] array([[ 0, -3, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, -4, 14]]) Mask indexing is conceptually the same as coordinate indexing, and is implemented internally via the same machinery. Both styles of indexing allow selecting arbitrary items from an array, also known as point selection. ##### Orthogonal indexing# Zarr arrays also support methods for orthogonal indexing, which allows selections to be made along each dimension of an array independently. For example, this allows selecting a subset of rows and/or columns from a 2-dimensional array. E.g.: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-14.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z[:] array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14]]) >>> z.get_orthogonal_selection(([0, 2], slice(None))) # select first and third rows array([[ 0, 1, 2, 3, 4], [10, 11, 12, 13, 14]]) >>> z.get_orthogonal_selection((slice(None), [1, 3])) # select second and fourth columns array([[ 1, 3], [ 6, 8], [11, 13]]) >>> z.get_orthogonal_selection(([0, 2], [1, 3])) # select rows [0, 2] and columns [1, 4] array([[ 1, 3], [11, 13]]) Data can also be modified, e.g.: >>> z.set_orthogonal_selection(([0, 2], [1, 3]), [[-1, -2], [-3, -4]]) For convenience, the orthogonal indexing functionality is also available via the `oindex` property, e.g.: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-15.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> z.oindex[[0, 2], :] # select first and third rows array([[ 0, 1, 2, 3, 4], [10, 11, 12, 13, 14]]) >>> z.oindex[:, [1, 3]] # select second and fourth columns array([[ 1, 3], [ 6, 8], [11, 13]]) >>> z.oindex[[0, 2], [1, 3]] # select rows [0, 2] and columns [1, 4] array([[ 1, 3], [11, 13]]) >>> z.oindex[[0, 2], [1, 3]] = [[-1, -2], [-3, -4]] >>> z[:] array([[ 0, -1, 2, -2, 4], [ 5, 6, 7, 8, 9], [10, -3, 12, -4, 14]]) Any combination of integer, slice, 1D integer array and/or 1D Boolean array can be used for orthogonal indexing. If the index contains at most one iterable, and otherwise contains only slices and integers, orthogonal indexing is also available directly on the array: >>> data = np.arange(15).reshape(3, 5) >>> z = zarr.create_array(store='data/example-16.zarr', shape=data.shape, dtype=data.dtype) >>> z[:] = data >>> np.all(z.oindex[[0, 2], :] == z[[0, 2], :]) np.True_ ##### Block Indexing# Zarr also support block indexing, which allows selections of whole chunks based on their logical indices along each dimension of an array. For example, this allows selecting a subset of chunk aligned rows and/or columns from a 2-dimensional array. E.g.: >>> data = np.arange(100).reshape(10, 10) >>> z = zarr.create_array(store='data/example-17.zarr', shape=data.shape, dtype=data.dtype, chunks=(3, 3)) >>> z[:] = data Retrieve items by specifying their block coordinates: >>> z.get_block_selection(1) array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) Equivalent slicing: >>> z[3:6] array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) For convenience, the block selection functionality is also available via the blocks property, e.g.: >>> z.blocks[1] array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) Block index arrays may be multidimensional to index multidimensional arrays. For example: >>> z.blocks[0, 1:3] array([[ 3, 4, 5, 6, 7, 8], [13, 14, 15, 16, 17, 18], [23, 24, 25, 26, 27, 28]]) Data can also be modified. Let’s start by a simple 2D array: >>> z = zarr.create_array(store='data/example-18.zarr', shape=(6, 6), dtype=int, chunks=(2, 2)) Set data for a selection of items: >>> z.set_block_selection((1, 0), 1) >>> z[...] array([[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]]) For convenience, this functionality is also available via the `blocks` property. E.g.: >>> z.blocks[:, 2] = 7 >>> z[...] array([[0, 0, 0, 0, 7, 7], [0, 0, 0, 0, 7, 7], [1, 1, 0, 0, 7, 7], [1, 1, 0, 0, 7, 7], [0, 0, 0, 0, 7, 7], [0, 0, 0, 0, 7, 7]]) Any combination of integer and slice can be used for block indexing: >>> z.blocks[2, 1:3] array([[0, 0, 7, 7], [0, 0, 7, 7]]) >>> >>> root = zarr.create_group('data/example-19.zarr') >>> foo = root.create_array(name='foo', shape=(1000, 100), chunks=(10, 10), dtype='float32') >>> bar = root.create_array(name='bar', shape=(100,), dtype='int32') >>> foo[:, :] = np.random.random((1000, 100)) >>> bar[:] = np.arange(100) >>> root.tree() / ├── bar (100,) int32 └── foo (1000, 100) float32 #### Sharding# Using small chunk shapes in very large arrays can lead to a very large number of chunks. This can become a performance issue for file systems and object storage. With Zarr format 3, a new sharding feature has been added to address this issue. With sharding, multiple chunks can be stored in a single storage object (e.g. a file). Within a shard, chunks are compressed and serialized separately. This allows individual chunks to be read independently. However, when writing data, a full shard must be written in one go for optimal performance and to avoid concurrency issues. That means that shards are the units of writing and chunks are the units of reading. Users need to configure the chunk and shard shapes accordingly. Sharded arrays can be created by providing the `shards` parameter to `zarr.create_array()`. >>> a = zarr.create_array('data/example-20.zarr', shape=(10000, 10000), shards=(1000, 1000), chunks=(100, 100), dtype='uint8') >>> a[:] = (np.arange(10000 * 10000) % 256).astype('uint8').reshape(10000, 10000) >>> a.info_complete() Type : Array Zarr format : 3 Data type : UInt8() Fill value : 0 Shape : (10000, 10000) Shard shape : (1000, 1000) Chunk shape : (100, 100) Order : C Read-only : False Store type : LocalStore Filters : () Serializer : BytesCodec(endian=None) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 100000000 (95.4M) No. bytes stored : 3981473 (3.8M) Storage ratio : 25.1 Shards Initialized : 100 In this example a shard shape of (1000, 1000) and a chunk shape of (100, 100) is used. This means that 10*10 chunks are stored in each shard, and there are 10*10 shards in total. Without the `shards` argument, there would be 10,000 chunks stored as individual files. #### Missing features in 3.0# The following features have not been ported to 3.0 yet. ##### Copying and migrating data# See the Zarr-Python 2 documentation on Copying and migrating data for more details. ### Working with groups# Zarr supports hierarchical organization of arrays via groups. As with arrays, groups can be stored in memory, on disk, or via other storage systems that support a similar interface. To create a group, use the `zarr.group()` function: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> root = zarr.create_group(store=store) >>> root Groups have a similar API to the Group class from h5py. For example, groups can contain other groups: >>> foo = root.create_group('foo') >>> bar = foo.create_group('bar') Groups can also contain arrays, e.g.: >>> z1 = bar.create_array(name='baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') >>> z1 Members of a group can be accessed via the suffix notation, e.g.: >>> root['foo'] The ‘/’ character can be used to access multiple levels of the hierarchy in one call, e.g.: >>> root['foo/bar'] >>> root['foo/bar/baz'] The `zarr.Group.tree()` method can be used to print a tree representation of the hierarchy, e.g.: >>> root.tree() / └── foo └── bar └── baz (10000, 10000) int32 The `zarr.open_group()` function provides a convenient way to create or re-open a group stored in a directory on the file-system, with sub-groups stored in sub-directories, e.g.: >>> root = zarr.open_group('data/group.zarr', mode='w') >>> root >>> >>> z = root.create_array(name='foo/bar/baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') >>> z For more information on groups see the `zarr.Group` API docs. #### Batch Group Creation# You can also create multiple groups concurrently with a single function call. `zarr.create_hierarchy()` takes a `zarr.storage.Store` instance and a dict of `key : metadata` pairs, parses that dict, and writes metadata documents to storage: >>> from zarr import create_hierarchy >>> from zarr.core.group import GroupMetadata >>> from zarr.storage import LocalStore >>> node_spec = {'a/b/c': GroupMetadata()} >>> nodes_created = dict(create_hierarchy(store=LocalStore(root='data'), nodes=node_spec)) >>> print(sorted(nodes_created.items(), key=lambda kv: len(kv[0]))) [('', ), ('a', ), ('a/b', ), ('a/b/c', )] Note that we only specified a single group named `a/b/c`, but 4 groups were created. These additional groups were created to ensure that the desired node `a/b/c` is connected to the root group `''` by a sequence of intermediate groups. `zarr.create_hierarchy()` normalizes the `nodes` keyword argument to ensure that the resulting hierarchy is complete, i.e. all groups or arrays are connected to the root of the hierarchy via intermediate groups. Because `zarr.create_hierarchy()` concurrently creates metadata documents, it’s more efficient than repeated calls to `create_group()` or `create_array()`, provided you can statically define the metadata for the groups and arrays you want to create. #### Array and group diagnostics# Diagnostic information about arrays and groups is available via the `info` property. E.g.: >>> store = zarr.storage.MemoryStore() >>> root = zarr.group(store=store) >>> foo = root.create_group('foo') >>> bar = foo.create_array(name='bar', shape=1000000, chunks=100000, dtype='int64') >>> bar[:] = 42 >>> baz = foo.create_array(name='baz', shape=(1000, 1000), chunks=(100, 100), dtype='float32') >>> baz[:] = 4.2 >>> root.info Name : Type : Group Zarr format : 3 Read-only : False Store type : MemoryStore >>> foo.info Name : foo Type : Group Zarr format : 3 Read-only : False Store type : MemoryStore >>> bar.info_complete() Type : Array Zarr format : 3 Data type : Int64(endianness='little') Fill value : 0 Shape : (1000000,) Chunk shape : (100000,) Order : C Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 8000000 (7.6M) No. bytes stored : 1614 (1.6K) Storage ratio : 4956.6 Chunks Initialized : 10 >>> baz.info Type : Array Zarr format : 3 Data type : Float32(endianness='little') Fill value : 0.0 Shape : (1000, 1000) Chunk shape : (100, 100) Order : C Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 4000000 (3.8M) Groups also have the `zarr.Group.tree()` method, e.g.: >>> root.tree() / └── foo ├── bar (1000000,) int64 └── baz (1000, 1000) float32 Note `zarr.Group.tree()` requires the optional rich dependency. It can be installed with the `[tree]` extra. ### Working with attributes# Zarr arrays and groups support custom key/value attributes, which can be useful for storing application-specific metadata. For example: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> root = zarr.create_group(store=store) >>> root.attrs['foo'] = 'bar' >>> z = root.create_array(name='zzz', shape=(10000, 10000), dtype='int32') >>> z.attrs['baz'] = 42 >>> z.attrs['qux'] = [1, 4, 7, 12] >>> sorted(root.attrs) ['foo'] >>> 'foo' in root.attrs True >>> root.attrs['foo'] 'bar' >>> sorted(z.attrs) ['baz', 'qux'] >>> z.attrs['baz'] 42 >>> z.attrs['qux'] [1, 4, 7, 12] Internally Zarr uses JSON to store array attributes, so attribute values must be JSON serializable. ### Storage guide# Zarr-Python supports multiple storage backends, including: local file systems, Zip files, remote stores via fsspec (S3, HTTP, etc.), and in-memory stores. In Zarr-Python 3, stores must implement the abstract store API from `zarr.abc.store.Store`. Note Unlike Zarr-Python 2 where the store interface was built around a generic `MutableMapping` API, Zarr-Python 3 utilizes a custom store API that utilizes Python’s AsyncIO library. #### Implicit Store Creation# In most cases, it is not required to create a `Store` object explicitly. Passing a string to Zarr’s top level API will result in the store being created automatically.: >>> import zarr >>> >>> # Implicitly create a writable LocalStore >>> zarr.create_group(store='data/foo/bar') >>> >>> # Implicitly create a read-only FsspecStore >>> zarr.open_group( ... store='s3://noaa-nwm-retro-v2-zarr-pds', ... mode='r', ... storage_options={'anon': True} ... ) > >>> >>> # Implicitly creates a MemoryStore >>> data = {} >>> zarr.create_group(store=data) #### Explicit Store Creation# In some cases, it may be helpful to create a store instance directly. Zarr-Python offers four built-in store: `zarr.storage.LocalStore`, `zarr.storage.FsspecStore`, `zarr.storage.ZipStore`, `zarr.storage.MemoryStore`, and `zarr.storage.ObjectStore`. ##### Local Store# The `zarr.storage.LocalStore` stores data in a nested set of directories on a local filesystem.: >>> store = zarr.storage.LocalStore('data/foo/bar', read_only=True) >>> zarr.open_group(store=store, mode='r') ##### Zip Store# The `zarr.storage.ZipStore` stores the contents of a Zarr hierarchy in a single Zip file. The Zip Store specification is currently in draft form.: >>> store = zarr.storage.ZipStore('data.zip', mode='w') >>> zarr.create_array(store=store, shape=(2,), dtype='float64') ##### Remote Store# The `zarr.storage.FsspecStore` stores the contents of a Zarr hierarchy in following the same logical layout as the `LocalStore`, except the store is assumed to be on a remote storage system such as cloud object storage (e.g. AWS S3, Google Cloud Storage, Azure Blob Store). The `zarr.storage.FsspecStore` is backed by fsspec and can support any backend that implements the AbstractFileSystem API. `storage_options` can be used to configure the fsspec backend.: >>> store = zarr.storage.FsspecStore.from_url( ... 's3://noaa-nwm-retro-v2-zarr-pds', ... read_only=True, ... storage_options={'anon': True} ... ) >>> zarr.open_group(store=store, mode='r') > The type of filesystem (e.g. S3, https, etc..) is inferred from the scheme of the url (e.g. s3 for “s3://noaa-nwm-retro-v2-zarr-pds”). In case a specific filesystem is needed, one can explicitly create it. For example to create a S3 filesystem: >>> import fsspec >>> fs = fsspec.filesystem( ... 's3', anon=True, asynchronous=True, ... client_kwargs={'endpoint_url': "https://noaa-nwm-retro-v2-zarr-pds.s3.amazonaws.com"} ... ) >>> store = zarr.storage.FsspecStore(fs) ##### Memory Store# The `zarr.storage.MemoryStore` a in-memory store that allows for serialization of Zarr data (metadata and chunks) to a dictionary.: >>> data = {} >>> store = zarr.storage.MemoryStore(data) >>> # TODO: replace with create_array after #2463 >>> zarr.create_array(store=store, shape=(2,), dtype='float64') ##### Object Store# `zarr.storage.ObjectStore` stores the contents of the Zarr hierarchy using any ObjectStore storage implementation, including AWS S3 (`obstore.store.S3Store`), Google Cloud Storage (`obstore.store.GCSStore`), and Azure Blob Storage (`obstore.store.AzureStore`). This store is backed by obstore, which builds on the production quality Rust library object_store. >>> from zarr.storage import ObjectStore >>> from obstore.store import MemoryStore >>> >>> store = ObjectStore(MemoryStore()) >>> zarr.create_array(store=store, shape=(2,), dtype='float64') Here’s an example of using ObjectStore for accessing remote data: >>> from zarr.storage import ObjectStore >>> from obstore.store import S3Store >>> >>> s3_store = S3Store('noaa-nwm-retro-v2-zarr-pds', skip_signature=True, region="us-west-2") >>> store = zarr.storage.ObjectStore(store=s3_store, read_only=True) >>> group = zarr.open_group(store=store, mode='r') >>> group.info Name : Type : Group Zarr format : 2 Read-only : True Store type : ObjectStore No. members : 12 No. arrays : 12 No. groups : 0 Warning The `zarr.storage.ObjectStore` class is experimental. #### Developing custom stores# Zarr-Python `zarr.abc.store.Store` API is meant to be extended. The Store Abstract Base Class includes all of the methods needed to be a fully operational store in Zarr Python. Zarr also provides a test harness for custom stores: `zarr.testing.store.StoreTests`. ### Runtime configuration# `zarr.config` is responsible for managing the configuration of zarr and is based on the donfig Python library. Configuration values can be set using code like the following: >>> import zarr >>> >>> zarr.config.set({'array.order': 'F'}) >>> >>> # revert this change so it doesn't impact the rest of the docs >>> zarr.config.set({'array.order': 'C'}) Alternatively, configuration values can be set using environment variables, e.g. `ZARR_ARRAY__ORDER=F`. The configuration can also be read from a YAML file in standard locations. For more information, see the donfig documentation. Configuration options include the following: * Default Zarr format `default_zarr_version` * Default array order in memory `array.order` * Whether empty chunks are written to storage `array.write_empty_chunks` * Async and threading options, e.g. `async.concurrency` and `threading.max_workers` * Selections of implementations of codecs, codec pipelines and buffers * Enabling GPU support with `zarr.config.enable_gpu()`. See Using GPUs with Zarr for more. For selecting custom implementations of codecs, pipelines, buffers and ndbuffers, first register the implementations in the registry and then select them in the config. For example, an implementation of the bytes codec in a class `'custompackage.NewBytesCodec'`, requires the value of `codecs.bytes.name` to be `'custompackage.NewBytesCodec'`. This is the current default configuration: >>> zarr.config.pprint() {'array': {'order': 'C', 'write_empty_chunks': False}, 'async': {'concurrency': 10, 'timeout': None}, 'buffer': 'zarr.buffer.cpu.Buffer', 'codec_pipeline': {'batch_size': 1, 'path': 'zarr.core.codec_pipeline.BatchedCodecPipeline'}, 'codecs': {'blosc': 'zarr.codecs.blosc.BloscCodec', 'bytes': 'zarr.codecs.bytes.BytesCodec', 'crc32c': 'zarr.codecs.crc32c_.Crc32cCodec', 'endian': 'zarr.codecs.bytes.BytesCodec', 'gzip': 'zarr.codecs.gzip.GzipCodec', 'numcodecs.adler32': 'zarr.codecs.numcodecs.Adler32', 'numcodecs.astype': 'zarr.codecs.numcodecs.AsType', 'numcodecs.bitround': 'zarr.codecs.numcodecs.BitRound', 'numcodecs.blosc': 'zarr.codecs.numcodecs.Blosc', 'numcodecs.bz2': 'zarr.codecs.numcodecs.BZ2', 'numcodecs.crc32': 'zarr.codecs.numcodecs.CRC32', 'numcodecs.crc32c': 'zarr.codecs.numcodecs.CRC32C', 'numcodecs.delta': 'zarr.codecs.numcodecs.Delta', 'numcodecs.fixedscaleoffset': 'zarr.codecs.numcodecs.FixedScaleOffset', 'numcodecs.fletcher32': 'zarr.codecs.numcodecs.Fletcher32', 'numcodecs.gzip': 'zarr.codecs.numcodecs.GZip', 'numcodecs.jenkins_lookup3': 'zarr.codecs.numcodecs.JenkinsLookup3', 'numcodecs.lz4': 'zarr.codecs.numcodecs.LZ4', 'numcodecs.lzma': 'zarr.codecs.numcodecs.LZMA', 'numcodecs.packbits': 'zarr.codecs.numcodecs.PackBits', 'numcodecs.pcodec': 'zarr.codecs.numcodecs.PCodec', 'numcodecs.quantize': 'zarr.codecs.numcodecs.Quantize', 'numcodecs.shuffle': 'zarr.codecs.numcodecs.Shuffle', 'numcodecs.zfpy': 'zarr.codecs.numcodecs.ZFPY', 'numcodecs.zlib': 'zarr.codecs.numcodecs.Zlib', 'numcodecs.zstd': 'zarr.codecs.numcodecs.Zstd', 'sharding_indexed': 'zarr.codecs.sharding.ShardingCodec', 'transpose': 'zarr.codecs.transpose.TransposeCodec', 'vlen-bytes': 'zarr.codecs.vlen_utf8.VLenBytesCodec', 'vlen-utf8': 'zarr.codecs.vlen_utf8.VLenUTF8Codec', 'zstd': 'zarr.codecs.zstd.ZstdCodec'}, 'default_zarr_format': 3, 'json_indent': 2, 'ndbuffer': 'zarr.buffer.cpu.NDBuffer', 'threading': {'max_workers': None}} ### 3.0 Migration Guide# Zarr-Python 3 represents a major refactor of the Zarr-Python codebase. Some of the goals motivating this refactor included: * adding support for the Zarr format 3 specification (along with the Zarr format 2 specification) * cleaning up internal and user facing APIs * improving performance (particularly in high latency storage environments like cloud object stores) To accommodate this, Zarr-Python 3 introduces a number of changes to the API, including a number of significant breaking changes and deprecations. This page provides a guide explaining breaking changes and deprecations to help you migrate your code from version 2 to version 3. If we have missed anything, please open a GitHub issue so we can improve this guide. #### Compatibility target# The goals described above necessitated some breaking changes to the API (hence the major version update), but where possible we have maintained backwards compatibility in the most widely used parts of the API. This in the `zarr.Array` and `zarr.Group` classes and the “top-level API” (e.g. `zarr.open_array()` and `zarr.open_group()`). #### Getting ready for 3.0# Before migrating to Zarr-Python 3, we suggest projects that depend on Zarr-Python take the following actions in order: 1. Pin the supported Zarr-Python version to `zarr>=2,<3`. This is a best practice and will protect your users from any incompatibilities that may arise during the release of Zarr-Python 3. This pin can be removed after migrating to Zarr-Python 3. 2. Limit your imports from the Zarr-Python package. Most of the primary API `zarr.*` will be compatible in Zarr-Python 3. However, the following breaking API changes are planned: * `numcodecs.*` will no longer be available in `zarr.*`. To migrate, import codecs directly from `numcodecs`: from numcodecs import Blosc # instead of: # from zarr import Blosc * The `zarr.v3_api_available` feature flag is being removed. In Zarr-Python 3 the v3 API is always available, so you shouldn’t need to use this flag. * The following internal modules are being removed or significantly changed. If your application relies on imports from any of the below modules, you will need to either a) modify your application to no longer rely on these imports or b) vendor the parts of the specific modules that you need. * `zarr.attrs` has gone, with no replacement * `zarr.codecs` has changed, see “Codecs” section below for more information * `zarr.context` has gone, with no replacement * `zarr.core` remains but should be considered private API * `zarr.hierarchy` has gone, with no replacement (use `zarr.Group` inplace of `zarr.hierarchy.Group`) * `zarr.indexing` has gone, with no replacement * `zarr.meta` has gone, with no replacement * `zarr.meta_v1` has gone, with no replacement * `zarr.sync` has gone, with no replacement * `zarr.types` has gone, with no replacement * `zarr.util` has gone, with no replacement * `zarr.n5` has gone, see below for an alternative N5 options 3. Test that your package works with version 3. 4. Update the pin to include `zarr>=3,<4`. #### Zarr-Python 2 support window# Zarr-Python 2.x is still available, though we recommend migrating to Zarr-Python 3 for its performance improvements and new features. Security and bug fixes will be made to the 2.x series for at least six months following the first Zarr-Python 3 release. If you need to use the latest Zarr-Python 2 release, you can install it with: $ pip install "zarr==2.*" Note Development and maintenance of the 2.x release series has moved to the support/v2 branch. Issues and pull requests related to this branch are tagged with the V2 label. #### Migrating to Zarr-Python 3# The following sections provide details on breaking changes in Zarr-Python 3. ##### The Array class# 1. Disallow direct construction - the signature for initializing the `Array` class has changed significantly. Please use `zarr.create_array()` or `zarr.open_array()` instead of directly constructing the `zarr.Array` class. 2. Defaulting to `zarr_format=3` \- newly created arrays will use the version 3 of the Zarr specification. To continue using version 2, set `zarr_format=2` when creating arrays or set `default_zarr_version=2` in Zarr’s runtime configuration. ##### The Group class# 1. Disallow direct construction - use `zarr.open_group()` or `zarr.create_group()` instead of directly constructing the `zarr.Group` class. 2. Most of the h5py compatibility methods are deprecated and will issue warnings if used. The following functions are drop in replacements that have the same signature and functionality: * Use `zarr.Group.create_array()` in place of `zarr.Group.create_dataset()` * Use `zarr.Group.require_array()` in place of `zarr.Group.require_dataset()` 3. Disallow “.” syntax for getting group members. To get a member of a group named `foo`, use `group["foo"]` in place of `group.foo`. ##### The Store class# The Store API has changed significant in Zarr-Python 3. The most notable changes to the Store API are: ###### Store Import Paths# Several store implementations have moved from the top-level module to `zarr.storage`: Store import changes from v2 to v3# # Before (v2) - from zarr import MemoryStore, DirectoryStore + from zarr.storage import MemoryStore, LocalStore # LocalStore replaces DirectoryStore Common replacements: v2 Import v3 Import `zarr.MemoryStore` `zarr.storage.MemoryStore` `zarr.DirectoryStore` `zarr.storage.LocalStore` `zarr.TempStore` Use `tempfile.TemporaryDirectory` with `LocalStore` 1. Replaced the `MutableMapping` base class in favor of a custom abstract base class (`zarr.abc.store.Store`). 2. Switched to an asynchronous interface for all store methods that result in IO. This change ensures that all store methods are non-blocking and are as performant as possible. Beyond the changes store interface, a number of deprecated stores were also removed in Zarr-Python 3. See #1274 for more details on the removal of these stores. * `N5Store` \- see zarr-developers/n5py for an alternative interface to N5 formatted data. * `ABSStore` \- use the `zarr.storage.FsspecStore` instead along with fsspec’s adlfs backend. The following stores have been removed altogether. Users who need these stores will have to implement their own version in zarr-python v3. * `DBMStore` * `LMDBStore` * `SQLiteStore` * `MongoDBStore` * `RedisStore` At present, the latter five stores in this list do not have an equivalent in Zarr-Python 3. If you are interested in developing a custom store that targets these backends, see developing custom stores or open an issue to discuss your use case. ##### Codecs# Codecs defined in `numcodecs` (and also imported into the `zarr.codecs` namespace in Zarr-Python 2) should still be used when creating Zarr format 2 arrays. Codecs for creating Zarr format 3 arrays are available in two locations: * zarr.codecs contains Zarr format 3 codecs that are defined in the codecs section of the Zarr format 3 specification. * numcodecs.zarr3 contains codecs from `numcodecs` that can be used to create Zarr format 3 arrays, but are not necessarily part of the Zarr format 3 specification. ##### Dependencies# When installing using `pip`: * The new `remote` dependency group can be used to install a supported version of `fsspec`, required for remote data access. * The new `gpu` dependency group can be used to install a supported version of `cuda`, required for GPU functionality. * The `jupyter` optional dependency group has been removed, since v3 contains no jupyter specific functionality. ##### Miscellaneous# * The keyword argument `zarr_version` available in most creation functions in `zarr` (e.g. `zarr.create()`, `zarr.open()`, `zarr.group()`, `zarr.array()`) has been deprecated in favor of `zarr_format`. #### 🚧 Work in Progress 🚧# Zarr-Python 3 is still under active development, and is not yet fully complete. The following list summarizes areas of the codebase that we expect to build out after the 3.0.0 release. If features listed below are important to your use case of Zarr-Python, please open (or comment on) a GitHub issue. * The following functions / methods have not been ported to Zarr-Python 3 yet: * `zarr.copy()` (#2407) * `zarr.copy_all()` (#2407) * `zarr.copy_store()` (#2407) * `zarr.Group.move()` (#2108) * The following features (corresponding to function arguments to functions in `zarr`) have not been ported to Zarr-Python 3 yet. Using these features will raise a warning or a `NotImplementedError`: * `cache_attrs` * `cache_metadata` * `chunk_store` (#2495) * `meta_array` * `object_codec` (#2617) * `synchronizer` (#1596) * `dimension_separator` * The following features that were supported by Zarr-Python 2 have not been ported to Zarr-Python 3 yet: * Structured arrays / dtypes (#2134) * Fixed-length string dtypes (#2347) * Datetime and timedelta dtypes (#2616) * Object dtypes (#2617) * Ragged arrays (#2618) * Groups and Arrays do not implement `__enter__` and `__exit__` protocols (#2619) * Big Endian dtypes (#2324) * Default filters for object dtypes for Zarr format 2 arrays (#2627) ### Command-line interface# Zarr-Python provides a command-line interface that enables: * migration of Zarr v2 metadata to v3 * removal of v2 or v3 metadata To see available commands run the following in a terminal: $ zarr --help or to get help on individual commands: $ zarr migrate --help $ zarr remove-metadata --help #### Migrate metadata from v2 to v3# ##### Migrate to a separate location# To migrate a Zarr array/group’s metadata from v2 to v3 run: $ zarr migrate v3 path/to/input.zarr path/to/output.zarr This will write new `zarr.json` files to `output.zarr`, leaving `input.zarr` un-touched. Note - this will migrate the entire Zarr hierarchy, so if `input.zarr` contains multiple groups/arrays, new `zarr.json` will be made for all of them. ##### Migrate in-place# If you’d prefer to migrate the metadata in-place run: $ zarr migrate v3 path/to/input.zarr This will write new `zarr.json` files to `input.zarr`, leaving the existing v2 metadata un-touched. To open the array/group using the new metadata use: >>> import zarr >>> zarr_with_v3_metadata = zarr.open('path/to/input.zarr', zarr_format=3) Once you are happy with the conversion, you can run the following to remove the old v2 metadata: $ zarr remove-metadata v2 path/to/input.zarr Note there is also a shortcut to migrate and remove v2 metadata in one step: $ zarr migrate v3 path/to/input.zarr --remove-v2-metadata #### Remove metadata# Remove v2 metadata using: $ zarr remove-metadata v2 path/to/input.zarr or v3 with: $ zarr remove-metadata v3 path/to/input.zarr By default, this will only allow removal of metadata if a valid alternative exists. For example, you can’t remove v2 metadata unless v3 metadata exists at that location. To override this behaviour use `--force`: $ zarr remove-metadata v3 path/to/input.zarr --force #### Dry run# All commands provide a `--dry-run` option that will log changes that would be made on a real run, without creating or modifying any files. $ zarr migrate v3 path/to/input.zarr --dry-run Dry run enabled - no new files will be created or changed. Log of files that would be created on a real run: Saving metadata to path/to/input.zarr/zarr.json #### Verbose# You can also add `--verbose` before any command, to see a full log of its actions: $ zarr --verbose migrate v3 path/to/input.zarr $ zarr --verbose remove-metadata v2 path/to/input.zarr #### Equivalent functions# All features of the command-line interface are also available via functions under `zarr.metadata`. ### Advanced Topics# #### Array data types# ##### Zarr’s Data Type Model# Zarr is designed for interoperability with NumPy, so if you are familiar with NumPy or any other N-dimensional array library, Zarr’s model for array data types should seem familiar. However, Zarr data types have some unique features that are described in this document. Zarr arrays operate under an essential design constraint: unlike NumPy arrays, Zarr arrays are designed to be stored and accessed by other Zarr implementations. This means that, among other things, Zarr data types must be serializable to metadata documents in accordance with the Zarr specifications, which adds some unique aspects to the Zarr data type model. The following sections explain Zarr’s data type model in greater detail and demonstrate the Zarr Python APIs for working with Zarr data types. ###### Array Data Types# Every Zarr array has a data type, which defines the meaning of the array’s elements. An array’s data type is encoded in the JSON metadata for the array. This means that the data type of an array must be JSON-serializable. In Zarr V2, the data type of an array is stored in the `dtype` field in array metadata. Zarr V3 changed the name of this field to `data_type` and also defined new rules for the values that can be assigned to the `data_type` field. For example, in Zarr V2, the boolean array data type was represented in array metadata as the string `"|b1"`. In Zarr V3, the same type is represented as the string `"bool"`. ###### Scalars# Zarr also specifies how array elements, i.e., scalars, are encoded in array metadata. This is necessary because Zarr uses a field in array metadata to define a default value for chunks that are not stored. This field, called `fill_value` in both Zarr V2 and Zarr V3 metadata documents, contains a JSON value that can be decoded to a scalar value compatible with the array’s data type. For the boolean data type, the scalar encoding is simple—booleans are natively supported by JSON, so Zarr saves booleans as JSON booleans. Other scalars, like floats or raw bytes, have more elaborate encoding schemes, and in some cases, this scheme depends on the Zarr format version. ##### Data Types in Zarr Version 2# Version 2 of the Zarr format defined its data types relative to NumPy’s data types, and added a few non-NumPy data types as well. With one exception (structured data types), the Zarr V2 JSON identifier for a data type is just the NumPy `str` attribute of that data type: >>> import zarr >>> import numpy as np >>> import json >>> >>> store = {} >>> np_dtype = np.dtype('int64') >>> np_dtype.str '>> z = zarr.create_array(store=store, shape=(1,), dtype=np_dtype, zarr_format=2) >>> dtype_meta = json.loads(store['.zarray'].to_bytes())["dtype"] >>> dtype_meta '>> store = {} >>> np_dtype = np.dtype([('field_a', '>i2'), ('field_b', [('subfield_c', '>f4'), ('subfield_d', 'i2')])]) >>> np_dtype.str '|V8' >>> z = zarr.create_array(store=store, shape=(1,), dtype=np_dtype, zarr_format=2) >>> dtype_meta = json.loads(store['.zarray'].to_bytes())["dtype"] >>> dtype_meta [['field_a', '>i2'], ['field_b', [['subfield_c', '>f4'], ['subfield_d', '>> from zarr.core.dtype import Int8 >>> import numpy as np >>> int8 = Int8.from_native_dtype(np.dtype('int8')) Convert back to a native data type: >>> native_dtype = int8.to_native_dtype() >>> assert native_dtype == np.dtype('int8') Get the default scalar value for the data type: >>> default_value = int8.default_scalar() >>> assert default_value == np.int8(0) Serialize to JSON for Zarr V2: >>> json_v2 = int8.to_json(zarr_format=2) >>> json_v2 {'name': '|i1', 'object_codec_id': None} Note The representation returned by `to_json(zarr_format=2)` is more abstract than the literal contents of Zarr V2 array metadata, because the JSON representation used by the `ZDType` classes must be distinct across different data types. As noted earlier, Zarr V2 identifies multiple distinct data types with the “object” data type identifier `"|O"`. Extra information is needed to disambiguate these data types from one another. That’s the reason for the `object_codec_id` field you see here. And for V3: >>> json_v3 = int8.to_json(zarr_format=3) >>> json_v3 'int8' Serialize a scalar value to JSON: >>> json_value = int8.to_json_scalar(42, zarr_format=3) >>> json_value 42 Deserialize a scalar value from JSON: >>> scalar_value = int8.from_json_scalar(42, zarr_format=3) >>> assert scalar_value == np.int8(42) ###### Adding New Data Types# Each Zarr data type is a separate Python class that inherits from ZDType. You can define a custom data type by writing your own subclass of ZDType and adding your data type to the data type registry. A complete example of this process is included below. The source code for this example can be found in the `examples/custom_dtype.py` file in the Zarr Python project directory. # /// script # requires-python = ">=3.11" # dependencies = [ # "zarr @ git+https://github.com/zarr-developers/zarr-python.git@main", # "ml_dtypes==0.5.1", # "pytest==8.4.1" # ] # /// # """ Demonstrate how to extend Zarr Python by defining a new data type """ import json import sys from pathlib import Path from typing import ClassVar, Literal, Self, TypeGuard, overload import ml_dtypes # necessary to add extra dtypes to NumPy import numpy as np import pytest import zarr from zarr.core.common import JSON, ZarrFormat from zarr.core.dtype import ZDType, data_type_registry from zarr.core.dtype.common import ( DataTypeValidationError, DTypeConfig_V2, DTypeJSON, check_dtype_spec_v2, ) # This is the int2 array data type int2_dtype_cls = type(np.dtype("int2")) # This is the int2 scalar type int2_scalar_cls = ml_dtypes.int2 class Int2(ZDType[int2_dtype_cls, int2_scalar_cls]): """ This class provides a Zarr compatibility layer around the int2 data type (the ``dtype`` of a NumPy array of type int2) and the int2 scalar type (the ``dtype`` of the scalar value inside an int2 array). """ # This field is as the key for the data type in the internal data type registry, and also # as the identifier for the data type when serializaing the data type to disk for zarr v3 _zarr_v3_name: ClassVar[Literal["int2"]] = "int2" # this field will be used internally _zarr_v2_name: ClassVar[Literal["int2"]] = "int2" # we bind a class variable to the native data type class so we can create instances of it dtype_cls = int2_dtype_cls @classmethod def from_native_dtype(cls, dtype: np.dtype) -> Self: """Create an instance of this ZDType from a native dtype.""" if cls._check_native_dtype(dtype): return cls() raise DataTypeValidationError( f"Invalid data type: {dtype}. Expected an instance of {cls.dtype_cls}" ) def to_native_dtype(self: Self) -> int2_dtype_cls: """Create an int2 dtype instance from this ZDType""" return self.dtype_cls() @classmethod def _check_json_v2(cls, data: DTypeJSON) -> TypeGuard[DTypeConfig_V2[Literal["|b1"], None]]: """ Type check for Zarr v2-flavored JSON. This will check that the input is a dict like this: .. code-block:: json { "name": "int2", "object_codec_id": None } Note that this representation differs from the ``dtype`` field looks like in zarr v2 metadata. Specifically, whatever goes into the ``dtype`` field in metadata is assigned to the ``name`` field here. See the Zarr docs for more information about the JSON encoding for data types. """ return ( check_dtype_spec_v2(data) and data["name"] == "int2" and data["object_codec_id"] is None ) @classmethod def _check_json_v3(cls, data: DTypeJSON) -> TypeGuard[Literal["int2"]]: """ Type check for Zarr V3-flavored JSON. Checks that the input is the string "int2". """ return data == cls._zarr_v3_name @classmethod def _from_json_v2(cls, data: DTypeJSON) -> Self: """ Create an instance of this ZDType from Zarr V3-flavored JSON. """ if cls._check_json_v2(data): return cls() # This first does a type check on the input, and if that passes we create an instance of the ZDType. msg = f"Invalid JSON representation of {cls.__name__}. Got {data!r}, expected the string {cls._zarr_v2_name!r}" raise DataTypeValidationError(msg) @classmethod def _from_json_v3(cls: type[Self], data: DTypeJSON) -> Self: """ Create an instance of this ZDType from Zarr V3-flavored JSON. This first does a type check on the input, and if that passes we create an instance of the ZDType. """ if cls._check_json_v3(data): return cls() msg = f"Invalid JSON representation of {cls.__name__}. Got {data!r}, expected the string {cls._zarr_v3_name!r}" raise DataTypeValidationError(msg) @overload # type: ignore[override] def to_json(self, zarr_format: Literal[2]) -> DTypeConfig_V2[Literal["int2"], None]: ... @overload def to_json(self, zarr_format: Literal[3]) -> Literal["int2"]: ... def to_json( self, zarr_format: ZarrFormat ) -> DTypeConfig_V2[Literal["int2"], None] | Literal["int2"]: """ Serialize this ZDType to v2- or v3-flavored JSON If the zarr_format is 2, then return a dict like this: .. code-block:: json { "name": "int2", "object_codec_id": None } If the zarr_format is 3, then return the string "int2" """ if zarr_format == 2: return {"name": "int2", "object_codec_id": None} elif zarr_format == 3: return self._zarr_v3_name raise ValueError(f"zarr_format must be 2 or 3, got {zarr_format}") # pragma: no cover def _check_scalar(self, data: object) -> TypeGuard[int | ml_dtypes.int2]: """ Check if a python object is a valid int2-compatible scalar The strictness of this type check is an implementation degree of freedom. You could be strict here, and only accept int2 values, or be open and accept any integer or any object and rely on exceptions from the int2 constructor that will be called in cast_scalar. """ return isinstance(data, (int, int2_scalar_cls)) def cast_scalar(self, data: object) -> ml_dtypes.int2: """ Attempt to cast a python object to an int2. We first perform a type check to ensure that the input type is appropriate, and if that passes we call the int2 scalar constructor. """ if self._check_scalar(data): return ml_dtypes.int2(data) msg = ( f"Cannot convert object {data!r} with type {type(data)} to a scalar compatible with the " f"data type {self}." ) raise TypeError(msg) def default_scalar(self) -> ml_dtypes.int2: """ Get the default scalar value. This will be used when automatically selecting a fill value. """ return ml_dtypes.int2(0) def to_json_scalar(self, data: object, *, zarr_format: ZarrFormat) -> int: """ Convert a python object to a JSON representation of an int2 scalar. This is necessary for taking user input for the ``fill_value`` attribute in array metadata. In this implementation, we optimistically convert the input to an int, and then check that it lies in the acceptable range for this data type. """ # We could add a type check here, but we don't need to for this example val: int = int(data) # type: ignore[call-overload] if val not in (-2, -1, 0, 1): raise ValueError("Invalid value. Expected -2, -1, 0, or 1.") return val def from_json_scalar(self, data: JSON, *, zarr_format: ZarrFormat) -> ml_dtypes.int2: """ Read a JSON-serializable value as an int2 scalar. We first perform a type check to ensure that the JSON value is well-formed, then call the int2 scalar constructor. The base definition of this method requires that it take a zarr_format parameter because other data types serialize scalars differently in zarr v2 and v3, but we don't use this here. """ if self._check_scalar(data): return ml_dtypes.int2(data) raise TypeError(f"Invalid type: {data}. Expected an int.") # after defining dtype class, it must be registered with the data type registry so zarr can use it data_type_registry.register(Int2._zarr_v3_name, Int2) # this parametrized function will create arrays in zarr v2 and v3 using our new data type @pytest.mark.parametrize("zarr_format", [2, 3]) def test_custom_dtype(tmp_path: Path, zarr_format: Literal[2, 3]) -> None: # create array and write values z_w = zarr.create_array( store=tmp_path, shape=(4,), dtype="int2", zarr_format=zarr_format, compressors=None ) z_w[:] = [-1, -2, 0, 1] # open the array z_r = zarr.open_array(tmp_path, mode="r") print(z_r.info_complete()) # look at the array metadata if zarr_format == 2: meta_file = tmp_path / ".zarray" else: meta_file = tmp_path / "zarr.json" print(json.dumps(json.loads(meta_file.read_text()), indent=2)) if __name__ == "__main__": # Run the example with printed output, and a dummy pytest configuration file specified. # Without the dummy configuration file, at test time pytest will attempt to use the # configuration file in the project root, which will error because Zarr is using some # plugins that are not installed in this example. sys.exit(pytest.main(["-s", __file__, f"-c {__file__}"])) ###### Data Type Resolution# Although Zarr Python uses a different data type model from NumPy, you can still define a Zarr array with a NumPy data type object: >>> from zarr import create_array >>> import numpy as np >>> a = create_array({}, shape=(10,), dtype=np.dtype('int')) >>> a Or a string representation of a NumPy data type: >>> a = create_array({}, shape=(10,), dtype='>> a The `Array` object presents itself like a NumPy array, including exposing a NumPy data type as its `dtype` attribute: >>> type(a.dtype) But if we inspect the metadata for the array, we can see the Zarr data type object: >>> type(a.metadata.data_type) This example illustrates a general problem Zarr Python has to solve: how can we allow users to specify a data type as a string or a NumPy `dtype` object, and produce the right Zarr data type from that input? We call this process “data type resolution.” Zarr Python also performs data type resolution when reading stored arrays, although in this case the input is a JSON value instead of a NumPy data type. For simple data types like `int`, the solution could be extremely simple: just maintain a lookup table that maps a NumPy data type to the Zarr data type equivalent. But not all data types are so simple. Consider this case: >>> from zarr import create_array >>> import warnings >>> import numpy as np >>> warnings.simplefilter("ignore", category=FutureWarning) >>> a = create_array({}, shape=(10,), dtype=[('a', 'f8'), ('b', 'i8')]) >>> a.dtype # this is the NumPy data type dtype([('a', '>> a.metadata.data_type # this is the Zarr data type Structured(fields=(('a', Float64(endianness='little')), ('b', Int64(endianness='little')))) In this example, we created a NumPy structured data type. This data type is a container that can hold any NumPy data type, which makes it recursive. It is not possible to make a lookup table that relates all NumPy structured data types to their Zarr equivalents, as there is a nearly unbounded number of different structured data types. So instead of a static lookup table, Zarr Python relies on a dynamic approach to data type resolution. Zarr Python defines a collection of Zarr data types. This collection, called a “data type registry,” is essentially a dictionary where the keys are strings (a canonical name for each data type), and the values are the data type classes themselves. Dynamic data type resolution entails iterating over these data type classes, invoking that class’ from_native_dtype method, and returning a concrete data type instance if and only if exactly one of those constructor invocations is successful. In plain language, we take some user input, like a NumPy data type, offer it to all the known data type classes, and return an instance of the one data type class that can accept that user input. We want to avoid a situation where the same native data type matches multiple Zarr data types; that is, a NumPy data type should uniquely specify a single Zarr data type. But data type resolution is dynamic, so it’s not possible to statically guarantee this uniqueness constraint. Therefore, we attempt data type resolution against every data type class, and if, for some reason, a native data type matches multiple Zarr data types, we treat this as an error and raise an exception. If you have a NumPy data type and you want to get the corresponding `ZDType` instance, you can use the `parse_dtype` function, which will use the dynamic resolution described above. `parse_dtype` handles a range of input types: * NumPy data types: >>> import numpy as np >>> from zarr.dtype import parse_dtype >>> my_dtype = np.dtype('>M8[10s]') >>> parse_dtype(my_dtype, zarr_format=2) DateTime64(endianness='big', scale_factor=10, unit='s') * NumPy data type-compatible strings: >>> dtype_str = '>M8[10s]' >>> parse_dtype(dtype_str, zarr_format=2) DateTime64(endianness='big', scale_factor=10, unit='s') * `ZDType` instances: >>> from zarr.dtype import DateTime64 >>> zdt = DateTime64(endianness='big', scale_factor=10, unit='s') >>> parse_dtype(zdt, zarr_format=2) # Use a ZDType (this is a no-op) DateTime64(endianness='big', scale_factor=10, unit='s') * Python dictionaries (requires `zarr_format=3`). These dictionaries must be consistent with the `JSON` form of the data type: >>> dt_dict = {"name": "numpy.datetime64", "configuration": {"unit": "s", "scale_factor": 10}} >>> parse_dtype(dt_dict, zarr_format=3) DateTime64(endianness='little', scale_factor=10, unit='s') >>> parse_dtype(dt_dict, zarr_format=3).to_json(zarr_format=3) {'name': 'numpy.datetime64', 'configuration': {'unit': 's', 'scale_factor': 10}} #### Optimizing performance# ##### Chunk optimizations# ###### Chunk size and shape# In general, chunks of at least 1 megabyte (1M) uncompressed size seem to provide better performance, at least when using the Blosc compression library. The optimal chunk shape will depend on how you want to access the data. E.g., for a 2-dimensional array, if you only ever take slices along the first dimension, then chunk across the second dimension. If you know you want to chunk across an entire dimension you can use the full size of that dimension within the `chunks` argument, e.g.: >>> import zarr >>> z1 = zarr.create_array(store={}, shape=(10000, 10000), chunks=(100, 10000), dtype='int32') >>> z1.chunks (100, 10000) Alternatively, if you only ever take slices along the second dimension, then chunk across the first dimension, e.g.: >>> z2 = zarr.create_array(store={}, shape=(10000, 10000), chunks=(10000, 100), dtype='int32') >>> z2.chunks (10000, 100) If you require reasonable performance for both access patterns then you need to find a compromise, e.g.: >>> z3 = zarr.create_array(store={}, shape=(10000, 10000), chunks=(1000, 1000), dtype='int32') >>> z3.chunks (1000, 1000) If you are feeling lazy, you can let Zarr guess a chunk shape for your data by providing `chunks='auto'`, although please note that the algorithm for guessing a chunk shape is based on simple heuristics and may be far from optimal. E.g.: >>> z4 = zarr.create_array(store={}, shape=(10000, 10000), chunks='auto', dtype='int32') >>> z4.chunks (625, 625) If you know you are always going to be loading the entire array into memory, you can turn off chunks by providing `chunks` equal to `shape`, in which case there will be one single chunk for the array: >>> z5 = zarr.create_array(store={}, shape=(10000, 10000), chunks=(10000, 10000), dtype='int32') >>> z5.chunks (10000, 10000) ###### Sharding# If you have large arrays but need small chunks to efficiently access the data, you can use sharding. Sharding provides a mechanism to store multiple chunks in a single storage object or file. This can be useful because traditional file systems and object storage systems may have performance issues storing and accessing many files. Additionally, small files can be inefficient to store if they are smaller than the block size of the file system. Picking a good combination of chunk shape and shard shape is important for performance. The chunk shape determines what unit of your data can be read independently, while the shard shape determines what unit of your data can be written efficiently. For an example, consider you have a 100 GB array and need to read small chunks of 1 MB. Without sharding, each chunk would be one file resulting in 100,000 files. That can already cause performance issues on some file systems. With sharding, you could use a shard size of 1 GB. This would result in 1000 chunks per file and 100 files in total, which seems manageable for most storage systems. You would still be able to read each 1 MB chunk independently, but you would need to write your data in 1 GB increments. To use sharding, you need to specify the `shards` parameter when creating the array. >>> z6 = zarr.create_array(store={}, shape=(10000, 10000, 1000), shards=(1000, 1000, 1000), chunks=(100, 100, 100), dtype='uint8') >>> z6.info Type : Array Zarr format : 3 Data type : UInt8() Fill value : 0 Shape : (10000, 10000, 1000) Shard shape : (1000, 1000, 1000) Chunk shape : (100, 100, 100) Order : C Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=None) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 100000000000 (93.1G) ###### Chunk memory layout# The order of bytes within each chunk of an array can be changed via the `order` config option, to use either C or Fortran layout. For multi-dimensional arrays, these two layouts may provide different compression ratios, depending on the correlation structure within the data. E.g.: >>> import numpy as np >>> >>> a = np.arange(100000000, dtype='int32').reshape(10000, 10000).T >>> c = zarr.create_array(store={}, shape=a.shape, chunks=(1000, 1000), dtype=a.dtype, config={'order': 'C'}) >>> c[:] = a >>> c.info_complete() Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : C Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 400000000 (381.5M) No. bytes stored : 342588911 (326.7M) Storage ratio : 1.2 Chunks Initialized : 100 >>> with zarr.config.set({'array.order': 'F'}): ... f = zarr.create_array(store={}, shape=a.shape, chunks=(1000, 1000), dtype=a.dtype) ... f[:] = a >>> f.info_complete() Type : Array Zarr format : 3 Data type : Int32(endianness='little') Fill value : 0 Shape : (10000, 10000) Chunk shape : (1000, 1000) Order : F Read-only : False Store type : MemoryStore Filters : () Serializer : BytesCodec(endian=) Compressors : (ZstdCodec(level=0, checksum=False),) No. bytes : 400000000 (381.5M) No. bytes stored : 342588911 (326.7M) Storage ratio : 1.2 Chunks Initialized : 100 In the above example, Fortran order gives a better compression ratio. This is an artificial example but illustrates the general point that changing the order of bytes within chunks of an array may improve the compression ratio, depending on the structure of the data, the compression algorithm used, and which compression filters (e.g., byte-shuffle) have been applied. ###### Empty chunks# It is possible to configure how Zarr handles the storage of chunks that are “empty” (i.e., every element in the chunk is equal to the array’s fill value). When creating an array with `write_empty_chunks=False`, Zarr will check whether a chunk is empty before compression and storage. If a chunk is empty, then Zarr does not store it, and instead deletes the chunk from storage if the chunk had been previously stored. This optimization prevents storing redundant objects and can speed up reads, but the cost is added computation during array writes, since the contents of each chunk must be compared to the fill value, and these advantages are contingent on the content of the array. If you know that your data will form chunks that are almost always non-empty, then there is no advantage to the optimization described above. In this case, creating an array with `write_empty_chunks=True` (the default) will instruct Zarr to write every chunk without checking for emptiness. The following example illustrates the effect of the `write_empty_chunks` flag on the time required to write an array with different values.: >>> import zarr >>> import numpy as np >>> import time >>> >>> def timed_write(write_empty_chunks): ... """ ... Measure the time required and number of objects created when writing ... to a Zarr array with random ints or fill value. ... """ ... chunks = (8192,) ... shape = (chunks[0] * 1024,) ... data = np.random.randint(0, 255, shape) ... dtype = 'uint8' ... arr = zarr.create_array( ... f'data/example-{write_empty_chunks}.zarr', ... shape=shape, ... chunks=chunks, ... dtype=dtype, ... fill_value=0, ... config={'write_empty_chunks': write_empty_chunks} ... ) ... # initialize all chunks ... arr[:] = 100 ... result = [] ... for value in (data, arr.fill_value): ... start = time.time() ... arr[:] = value ... elapsed = time.time() - start ... result.append((elapsed, arr.nchunks_initialized)) ... return result ... # log results >>> for write_empty_chunks in (True, False): ... full, empty = timed_write(write_empty_chunks) ... print(f'\nwrite_empty_chunks={write_empty_chunks}:\n\tRandom Data: {full[0]:.4f}s, {full[1]} objects stored\n\t Empty Data: {empty[0]:.4f}s, {empty[1]} objects stored\n') write_empty_chunks=True: Random Data: ..., 1024 objects stored Empty Data: ...s, 1024 objects stored write_empty_chunks=False: Random Data: ...s, 1024 objects stored Empty Data: ...s, 0 objects stored In this example, writing random data is slightly slower with `write_empty_chunks=True`, but writing empty data is substantially faster and generates far fewer objects in storage. ###### Changing chunk shapes (rechunking)# Coming soon. ##### Parallel computing and synchronization# Coming soon. ##### Pickle support# Zarr arrays and groups can be pickled, as long as the underlying store object can be pickled. With the exception of the `zarr.storage.MemoryStore`, any of the storage classes provided in the `zarr.storage` module can be pickled. If an array or group is backed by a persistent store such as the a `zarr.storage.LocalStore`, `zarr.storage.ZipStore` or `zarr.storage.FsspecStore` then the store data are not pickled. The only thing that is pickled is the necessary parameters to allow the store to re-open any underlying files or databases upon being unpickled. E.g., pickle/unpickle an local store array: >>> import pickle >>> data = np.arange(100000) >>> z1 = zarr.create_array(store='data/example-2.zarr', shape=data.shape, chunks=data.shape, dtype=data.dtype) >>> z1[:] = data >>> s = pickle.dumps(z1) >>> z2 = pickle.loads(s) >>> z1 == z2 True >>> np.all(z1[:] == z2[:]) np.True_ ##### Configuring Blosc# Coming soon. #### Consolidated metadata# Warning The Consolidated Metadata feature in Zarr-Python is considered experimental for v3 stores. zarr-specs#309 has proposed a formal extension to the v3 specification to support consolidated metadata. Zarr-Python implements the Consolidated Metadata for v2 and v3 stores. Consolidated metadata can reduce the time needed to load the metadata for an entire hierarchy, especially when the metadata is being served over a network. Consolidated metadata essentially stores all the metadata for a hierarchy in the metadata of the root Group. ##### Usage# If consolidated metadata is present in a Zarr Group’s metadata then it is used by default. The initial read to open the group will need to communicate with the store (reading from a file for a `zarr.storage.LocalStore`, making a network request for a `zarr.storage.FsspecStore`). After that, any subsequent metadata reads get child Group or Array nodes will not require reads from the store. In Python, the consolidated metadata is available on the `.consolidated_metadata` attribute of the `GroupMetadata` object. >>> import zarr >>> import warnings >>> warnings.filterwarnings("ignore", category=UserWarning) >>> >>> store = zarr.storage.MemoryStore() >>> group = zarr.create_group(store=store) >>> group.create_array(shape=(1,), name='a', dtype='float64') >>> group.create_array(shape=(2, 2), name='b', dtype='float64') >>> group.create_array(shape=(3, 3, 3), name='c', dtype='float64') >>> zarr.consolidate_metadata(store) If we open that group, the Group’s metadata has a `zarr.core.group.ConsolidatedMetadata` that can be used.: >>> consolidated = zarr.open_group(store=store) >>> consolidated_metadata = consolidated.metadata.consolidated_metadata.metadata >>> from pprint import pprint >>> pprint(dict(consolidated_metadata.items())) {'a': ArrayV3Metadata(shape=(1,), data_type=Float64(endianness='little'), chunk_grid=RegularChunkGrid(chunk_shape=(1,)), chunk_key_encoding=DefaultChunkKeyEncoding(separator='/'), fill_value=np.float64(0.0), codecs=(BytesCodec(endian=), ZstdCodec(level=0, checksum=False)), attributes={}, dimension_names=None, zarr_format=3, node_type='array', storage_transformers=()), 'b': ArrayV3Metadata(shape=(2, 2), data_type=Float64(endianness='little'), chunk_grid=RegularChunkGrid(chunk_shape=(2, 2)), chunk_key_encoding=DefaultChunkKeyEncoding(separator='/'), fill_value=np.float64(0.0), codecs=(BytesCodec(endian=), ZstdCodec(level=0, checksum=False)), attributes={}, dimension_names=None, zarr_format=3, node_type='array', storage_transformers=()), 'c': ArrayV3Metadata(shape=(3, 3, 3), data_type=Float64(endianness='little'), chunk_grid=RegularChunkGrid(chunk_shape=(3, 3, 3)), chunk_key_encoding=DefaultChunkKeyEncoding(separator='/'), fill_value=np.float64(0.0), codecs=(BytesCodec(endian=), ZstdCodec(level=0, checksum=False)), attributes={}, dimension_names=None, zarr_format=3, node_type='array', storage_transformers=())} Operations on the group to get children automatically use the consolidated metadata.: >>> consolidated['a'] # no read / HTTP request to the Store is required With nested groups, the consolidated metadata is available on the children, recursively.: >>> child = group.create_group('child', attributes={'kind': 'child'}) >>> grandchild = child.create_group('child', attributes={'kind': 'grandchild'}) >>> consolidated = zarr.consolidate_metadata(store) >>> >>> consolidated['child'].metadata.consolidated_metadata ConsolidatedMetadata(metadata={'child': GroupMetadata(attributes={'kind': 'grandchild'}, zarr_format=3, consolidated_metadata=ConsolidatedMetadata(metadata={}, kind='inline', must_understand=False), node_type='group')}, kind='inline', must_understand=False) Added in version 3.1.1: The keys in the consolidated metadata are sorted prior to writing. Keys are sorted in ascending order by path depth, where a path is defined as a sequence of strings joined by `"/"`. For keys with the same path length, lexicographic order is used to break the tie. This behaviour ensures deterministic metadata output for a given group. ##### Synchronization and Concurrency# Consolidated metadata is intended for read-heavy use cases on slowly changing hierarchies. For hierarchies where new nodes are constantly being added, removed, or modified, consolidated metadata may not be desirable. 1. It will add some overhead to each update operation, since the metadata would need to be re-consolidated to keep it in sync with the store. 2. Readers using consolidated metadata will regularly see a “past” version of the metadata, at the time they read the root node with its consolidated metadata. ##### Stores Without Support for Consolidated Metadata# Some stores may want to opt out of the consolidated metadata mechanism. This may be for several reasons like: * They want to maintain read-write consistency, which is challenging with consolidated metadata. * They have their own consolidated metadata mechanism. * They offer good enough performance without need for consolidation. This type of store can declare it doesn’t want consolidation by implementing Store.supports_consolidated_metadata and returning False. For stores that don’t support consolidation, Zarr will: * Raise an error on consolidate_metadata calls, maintaining the store in its unconsolidated state. * Raise an error in AsyncGroup.open(…, use_consolidated=True) * Not use consolidated metadata in AsyncGroup.open(…, use_consolidated=None) #### Extending Zarr# Zarr-Python 3 was designed to be extensible. This means that you can extend the library by writing custom classes and plugins. Currently, Zarr can be extended in the following ways: ##### Custom codecs# Note This section explains how custom codecs can be created for Zarr format 3 arrays. For Zarr format 2, codecs should subclass the numcodecs.abc.Codec base class and register through numcodecs.registry.register_codec. There are three types of codecs in Zarr: \- array-to-array \- array-to-bytes \- bytes-to-bytes Array-to-array codecs are used to transform the array data before serializing to bytes. Examples include delta encoding or scaling codecs. Array-to-bytes codecs are used for serializing the array data to bytes. In Zarr, the main codec to use for numeric arrays is the `zarr.codecs.BytesCodec`. Bytes-to-bytes codecs transform the serialized bytestreams of the array data. Examples include compression codecs, such as `zarr.codecs.GzipCodec`, `zarr.codecs.BloscCodec` or `zarr.codecs.ZstdCodec`, and codecs that add a checksum to the bytestream, such as `zarr.codecs.Crc32cCodec`. Custom codecs for Zarr are implemented by subclassing the relevant base class, see `zarr.abc.codec.ArrayArrayCodec`, `zarr.abc.codec.ArrayBytesCodec` and `zarr.abc.codec.BytesBytesCodec`. Most custom codecs should implemented the `_encode_single` and `_decode_single` methods. These methods operate on single chunks of the array data. Alternatively, custom codecs can implement the `encode` and `decode` methods, which operate on batches of chunks, in case the codec is intended to implement its own batch processing. Custom codecs should also implement the following methods: * `compute_encoded_size`, which returns the byte size of the encoded data given the byte size of the original data. It should raise `NotImplementedError` for codecs with variable-sized outputs, such as compression codecs. * `validate` (optional), which can be used to check that the codec metadata is compatible with the array metadata. It should raise errors if not. * `resolve_metadata` (optional), which is important for codecs that change the shape, dtype or fill value of a chunk. * `evolve_from_array_spec` (optional), which can be useful for automatically filling in codec configuration metadata from the array metadata. To use custom codecs in Zarr, they need to be registered using the entrypoint mechanism. Commonly, entrypoints are declared in the `pyproject.toml` of your package under the `[project.entry-points."zarr.codecs"]` section. Zarr will automatically discover and load all codecs registered with the entrypoint mechanism from imported modules. [project.entry-points."zarr.codecs"] "custompackage.fancy_codec" = "custompackage:FancyCodec" New codecs need to have their own unique identifier. To avoid naming collisions, it is strongly recommended to prefix the codec identifier with a unique name. For example, the codecs from `numcodecs` are prefixed with `numcodecs.`, e.g. `numcodecs.delta`. Note Note that the extension mechanism for the Zarr format 3 is still under development. Requirements for custom codecs including the choice of codec identifiers might change in the future. It is also possible to register codecs as replacements for existing codecs. This might be useful for providing specialized implementations, such as GPU-based codecs. In case of multiple codecs, the `zarr.core.config` mechanism can be used to select the preferred implementation. ##### Custom stores# Coming soon. ##### Custom array buffers# Zarr-python provides control over where and how arrays stored in memory through `zarr.buffer`. Currently both CPU (the default) and GPU implementations are provided (see Using GPUs with Zarr for more). You can implement your own buffer classes by implementing the interface defined in `zarr.abc.buffer`. ##### Other extensions# In the future, Zarr will support writing custom custom data types and chunk grids. #### Using GPUs with Zarr# Zarr can use GPUs to accelerate your workload by running `zarr.config.enable_gpu()`. Note zarr-python currently supports reading the ndarray data into device (GPU) memory as the final stage of the codec pipeline. Data will still be read into or copied to host (CPU) memory for encoding and decoding. In the future, codecs will be available compressing and decompressing data on the GPU, avoiding the need to move data between the host and device for compression and decompression. ##### Reading data into device memory# `zarr.config.enable_gpu()` configures Zarr to use GPU memory for the data buffers used internally by Zarr. >>> import zarr >>> import cupy as cp >>> zarr.config.enable_gpu() >>> store = zarr.storage.MemoryStore() >>> z = zarr.create_array( ... store=store, shape=(100, 100), chunks=(10, 10), dtype="float32", ... ) >>> type(z[:10, :10]) cupy.ndarray Note that the output type is a `cupy.ndarray` rather than a NumPy array. ## zarr# ### Submodules# #### zarr.abc# ##### Submodules# ###### zarr.abc.buffer# ###### Classes# `ArrayLike` Protocol for the array-like type that underlie Buffer `Buffer` A flat contiguous memory block `BufferPrototype` Prototype of the Buffer and NDBuffer class `NDArrayLike` Protocol for the nd-array-like type that underlie NDBuffer `NDBuffer` An n-dimensional memory block ###### Module Contents# class zarr.abc.buffer.ArrayLike# Bases: `Protocol` Protocol for the array-like type that underlie Buffer property dtype: numpy.dtype[Any]# property ndim: int# property size: int# class zarr.abc.buffer.Buffer(array_like: ArrayLike)# Bases: `abc.ABC` A flat contiguous memory block We use Buffer throughout Zarr to represent a contiguous block of memory. A Buffer is backed by a underlying array-like instance that represents the memory. The memory type is unspecified; can be regular host memory, CUDA device memory, or something else. The only requirement is that the array-like instance can be copied/converted to a regular Numpy array (host memory). Parameters: array_like array-like object that must be 1-dim, contiguous, and byte dtype. Notes This buffer is untyped, so all indexing and sizes are in bytes. as_array_like() -> ArrayLike# Returns the underlying array (host or device memory) of this buffer This will never copy data. Returns: The underlying 1d array such as a NumPy or CuPy array. as_buffer_like() -> zarr.core.common.BytesLike# Returns the buffer as an object that implements the Python buffer protocol. Returns: An object that implements the Python buffer protocol Notes Might have to copy data, since the implementation uses .as_numpy_array(). abstract as_numpy_array() -> numpy.typing.NDArray[Any]# Returns the buffer as a NumPy array (host memory). Returns: NumPy array of this buffer (might be a data copy) Notes Might have to copy data, consider using .as_array_like() instead. classmethod create_zero_length() -> Self# Abstractmethod: Create an empty buffer with length zero Returns: New empty 0-length buffer classmethod from_array_like(array_like: ArrayLike) -> Self# Create a new buffer of an array-like object Parameters: array_like array-like object that must be 1-dim, contiguous, and byte dtype. Returns: New buffer representing array_like classmethod from_buffer(buffer: Buffer) -> Self# Abstractmethod: Create a new buffer of an existing Buffer This is useful if you want to ensure that an existing buffer is of the correct subclass of Buffer. E.g., MemoryStore uses this to return a buffer instance of the subclass specified by its BufferPrototype argument. Typically, this only copies data if the data has to be moved between memory types, such as from host to device memory. Parameters: buffer buffer object. Returns: A new buffer representing the content of the input buffer Notes Subclasses of Buffer must override this method to implement more optimal conversions that avoid copies where possible classmethod from_bytes(bytes_like: zarr.core.common.BytesLike) -> Self# Abstractmethod: Create a new buffer of a bytes-like object (host memory) Parameters: bytes_like bytes-like object Returns: New buffer representing bytes_like to_bytes() -> bytes# Returns the buffer as bytes (host memory). Returns: bytes of this buffer (data copy) Warning Will always copy data, only use this method for small buffers such as metadata buffers. If possible, use .as_numpy_array() or .as_array_like() instead. class zarr.abc.buffer.BufferPrototype# Bases: `NamedTuple` Prototype of the Buffer and NDBuffer class The protocol must be pickable. Attributes: buffer The Buffer class to use when Zarr needs to create new Buffer. nd_buffer The NDBuffer class to use when Zarr needs to create new NDBuffer. buffer: type[Buffer]# nd_buffer: type[NDBuffer]# class zarr.abc.buffer.NDArrayLike# Bases: `Protocol` Protocol for the nd-array-like type that underlie NDBuffer all() -> bool# astype( dtype: numpy.typing.DTypeLike, order: Literal['K', 'A', 'C', 'F'] = ..., *, copy: bool = ..., ) -> Self# copy() -> Self# fill(value: Any) -> None# ravel(order: Literal['K', 'A', 'C', 'F'] = ...) -> Self# reshape( shape: tuple[int, Ellipsis] | Literal[-1], *, order: Literal['A', 'C', 'F'] = ..., ) -> Self# transpose( axes: SupportsIndex | collections.abc.Sequence[SupportsIndex] | None, ) -> Self# view(dtype: numpy.typing.DTypeLike) -> Self# property dtype: numpy.dtype[Any]# property ndim: int# property shape: tuple[int, Ellipsis]# property size: int# class zarr.abc.buffer.NDBuffer(array: NDArrayLike)# An n-dimensional memory block We use NDBuffer throughout Zarr to represent a n-dimensional memory block. A NDBuffer is backed by a underlying ndarray-like instance that represents the memory. The memory type is unspecified; can be regular host memory, CUDA device memory, or something else. The only requirement is that the ndarray-like instance can be copied/converted to a regular Numpy array (host memory). Parameters: arrayndarray_like ndarray-like object that is convertible to a regular Numpy array. Notes The two buffer classes Buffer and NDBuffer are very similar. In fact, Buffer is a special case of NDBuffer where dim=1, stride=1, and dtype=”B”. However, in order to use Python’s type system to differentiate between the contiguous Buffer and the n-dim (non-contiguous) NDBuffer, we keep the definition of the two classes separate. all_equal(other: Any, equal_nan: bool = True) -> bool# Compare to other using np.array_equal. as_ndarray_like() -> NDArrayLike# Returns the underlying array (host or device memory) of this buffer This will never copy data. Returns: The underlying array such as a NumPy or CuPy array. abstract as_numpy_array() -> numpy.typing.NDArray[Any]# Returns the buffer as a NumPy array (host memory). Returns: NumPy array of this buffer (might be a data copy) Warning Might have to copy data, consider using .as_ndarray_like() instead. as_scalar() -> ScalarType# Returns the buffer as a scalar value astype( dtype: numpy.typing.DTypeLike, order: Literal['K', 'A', 'C', 'F'] = 'K', ) -> Self# copy() -> Self# classmethod create( *, shape: collections.abc.Iterable[int], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', fill_value: Any | None = None, ) -> Self# Abstractmethod: Create a new buffer and its underlying ndarray-like object Parameters: shape The shape of the buffer and its underlying ndarray-like object dtype The datatype of the buffer and its underlying ndarray-like object order Whether to store multi-dimensional data in row-major (C-style) or column-major (Fortran-style) order in memory. fill_value If not None, fill the new buffer with a scalar value. Returns: New buffer representing a new ndarray_like object Notes A subclass can overwrite this method to create a ndarray-like object other then the default Numpy array. classmethod empty( shape: tuple[int, Ellipsis], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', ) -> Self# Create an empty buffer with the given shape, dtype, and order. This method can be faster than `NDBuffer.create` because it doesn’t have to initialize the memory used by the underlying ndarray-like object. Parameters: shape The shape of the buffer and its underlying ndarray-like object dtype The datatype of the buffer and its underlying ndarray-like object order Whether to store multi-dimensional data in row-major (C-style) or column-major (Fortran-style) order in memory. Returns: buffer New buffer representing a new ndarray_like object with empty data. See also `NDBuffer.create` Create a new buffer with some initial fill value. fill(value: Any) -> None# classmethod from_ndarray_like(ndarray_like: NDArrayLike) -> Self# Create a new buffer of a ndarray-like object Parameters: ndarray_like ndarray-like object Returns: New buffer representing ndarray_like classmethod from_numpy_array(array_like: numpy.typing.ArrayLike) -> Self# Abstractmethod: Create a new buffer of Numpy array-like object Parameters: array_like Object that can be coerced into a Numpy array Returns: New buffer representing array_like reshape(newshape: tuple[int, Ellipsis] | Literal[-1]) -> Self# squeeze(axis: tuple[int, Ellipsis]) -> Self# transpose( axes: SupportsIndex | collections.abc.Sequence[SupportsIndex] | None, ) -> Self# property byteorder: zarr.codecs.bytes.Endian# property dtype: numpy.dtype[Any]# property shape: tuple[int, Ellipsis]# ###### zarr.abc.codec# ###### Attributes# `CodecInput` `CodecOutput` ###### Classes# `ArrayArrayCodec` Base class for array-to-array codecs. `ArrayBytesCodec` Base class for array-to-bytes codecs. `ArrayBytesCodecPartialDecodeMixin` Mixin for array-to-bytes codecs that implement partial decoding. `ArrayBytesCodecPartialEncodeMixin` Mixin for array-to-bytes codecs that implement partial encoding. `BaseCodec` Generic base class for codecs. `BytesBytesCodec` Base class for bytes-to-bytes codecs. `CodecPipeline` Base class for implementing CodecPipeline. ###### Module Contents# class zarr.abc.codec.ArrayArrayCodec# Bases: `BaseCodec`[`zarr.core.buffer.NDBuffer`, `zarr.core.buffer.NDBuffer`] Base class for array-to-array codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size: bool# class zarr.abc.codec.ArrayBytesCodec# Bases: `BaseCodec`[`zarr.core.buffer.NDBuffer`, `zarr.core.buffer.Buffer`] Base class for array-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size: bool# class zarr.abc.codec.ArrayBytesCodecPartialDecodeMixin# Mixin for array-to-bytes codecs that implement partial decoding. async decode_partial( batch_info: collections.abc.Iterable[tuple[zarr.abc.store.ByteGetter, zarr.core.indexing.SelectorTuple, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[zarr.core.buffer.NDBuffer | None]# Partially decodes a batch of chunks. This method determines parts of a chunk from the slice selection, fetches these parts from the store (via ByteGetter) and decodes them. Parameters: batch_infoIterable[tuple[ByteGetter, SelectorTuple, ArraySpec]] Ordered set of information about slices of encoded chunks. The slice selection determines which parts of the chunk will be fetched. The ByteGetter is used to fetch the necessary bytes. The chunk spec contains information about the construction of an array from the bytes. Returns: Iterable[NDBuffer | None] class zarr.abc.codec.ArrayBytesCodecPartialEncodeMixin# Mixin for array-to-bytes codecs that implement partial encoding. async encode_partial( batch_info: collections.abc.Iterable[tuple[zarr.abc.store.ByteSetter, zarr.core.buffer.NDBuffer, zarr.core.indexing.SelectorTuple, zarr.core.array_spec.ArraySpec]], ) -> None# Partially encodes a batch of chunks. This method determines parts of a chunk from the slice selection, encodes them and writes these parts to the store (via ByteSetter). If merging with existing chunk data in the store is necessary, this method will read from the store first and perform the merge. Parameters: batch_infoIterable[tuple[ByteSetter, NDBuffer, SelectorTuple, ArraySpec]] Ordered set of information about slices of to-be-encoded chunks. The slice selection determines which parts of the chunk will be encoded. The ByteSetter is used to write the necessary bytes and fetch bytes for existing chunk data. The chunk spec contains information about the chunk. class zarr.abc.codec.BaseCodec# Bases: `zarr.abc.metadata.Metadata`, `Generic`[`CodecInput`, `CodecOutput`] Generic base class for codecs. Codecs can be registered via zarr.codecs.registry. Warning This class is not intended to be directly, please use ArrayArrayCodec, ArrayBytesCodec or BytesBytesCodec for subclassing. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size: bool# class zarr.abc.codec.BytesBytesCodec# Bases: `BaseCodec`[`zarr.core.buffer.Buffer`, `zarr.core.buffer.Buffer`] Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size: bool# class zarr.abc.codec.CodecPipeline# Base class for implementing CodecPipeline. A CodecPipeline implements the read and write paths for chunk data. On the read path, it is responsible for fetching chunks from a store (via ByteGetter), decoding them and assembling an output array. On the write path, it encodes the chunks and writes them to a store (via ByteSetter). abstract compute_encoded_size( byte_length: int, array_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: byte_lengthint array_specArraySpec Returns: int abstract decode( chunk_bytes_and_specs: collections.abc.Iterable[tuple[zarr.core.buffer.Buffer | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[zarr.core.buffer.NDBuffer | None]# Async: Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunk_bytes_and_specsIterable[tuple[Buffer | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[NDBuffer | None] abstract encode( chunk_arrays_and_specs: collections.abc.Iterable[tuple[zarr.core.buffer.NDBuffer | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[zarr.core.buffer.Buffer | None]# Async: Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunk_arrays_and_specsIterable[tuple[NDBuffer | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[Buffer | None] abstract evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_array_metadata_and_store( array_metadata: zarr.core.metadata.ArrayMetadata, store: zarr.abc.store.Store, ) -> Self# Abstractmethod: Creates a codec pipeline from array metadata and a store path. Raises NotImplementedError by default, indicating the CodecPipeline must be created with from_codecs instead. Parameters: array_metadataArrayMetadata storeStore Returns: Self classmethod from_codecs(codecs: collections.abc.Iterable[Codec]) -> Self# Abstractmethod: Creates a codec pipeline from an iterable of codecs. Parameters: codecsIterable[Codec] Returns: Self abstract read( batch_info: collections.abc.Iterable[tuple[zarr.abc.store.ByteGetter, zarr.core.array_spec.ArraySpec, zarr.core.indexing.SelectorTuple, zarr.core.indexing.SelectorTuple, bool]], out: zarr.core.buffer.NDBuffer, drop_axes: tuple[int, Ellipsis] = (), ) -> None# Async: Reads chunk data from the store, decodes it and writes it into an output array. Partial decoding may be utilized if the codecs and stores support it. Parameters: batch_infoIterable[tuple[ByteGetter, ArraySpec, SelectorTuple, SelectorTuple]] Ordered set of information about the chunks. The first slice selection determines which parts of the chunk will be fetched. The second slice selection determines where in the output array the chunk data will be written. The ByteGetter is used to fetch the necessary bytes. The chunk spec contains information about the construction of an array from the bytes. If the Store returns `None` for a chunk, then the chunk was not written and the implementation must set the values of that chunk (or `out`) to the fill value for the array. outNDBuffer abstract validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that all codec configurations are compatible with the array metadata. Raises errors when a codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid abstract write( batch_info: collections.abc.Iterable[tuple[zarr.abc.store.ByteSetter, zarr.core.array_spec.ArraySpec, zarr.core.indexing.SelectorTuple, zarr.core.indexing.SelectorTuple, bool]], value: zarr.core.buffer.NDBuffer, drop_axes: tuple[int, Ellipsis] = (), ) -> None# Async: Encodes chunk data and writes it to the store. Merges with existing chunk data by reading first, if necessary. Partial encoding may be utilized if the codecs and stores support it. Parameters: batch_infoIterable[tuple[ByteSetter, ArraySpec, SelectorTuple, SelectorTuple]] Ordered set of information about the chunks. The first slice selection determines which parts of the chunk will be encoded. The second slice selection determines where in the value array the chunk data is located. The ByteSetter is used to fetch and write the necessary bytes. The chunk spec contains information about the chunk. valueNDBuffer property supports_partial_decode: bool# Abstractmethod: property supports_partial_encode: bool# Abstractmethod: zarr.abc.codec.CodecInput# zarr.abc.codec.CodecOutput# ###### zarr.abc.metadata# ###### Classes# `Metadata` ###### Module Contents# class zarr.abc.metadata.Metadata# classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. ###### zarr.abc.numcodec# ###### Classes# `Numcodec` A protocol that models the `numcodecs.abc.Codec` interface. ###### Module Contents# class zarr.abc.numcodec.Numcodec# Bases: `typing_extensions.Protocol` A protocol that models the `numcodecs.abc.Codec` interface. This protocol should be considered experimental. Expect the type annotations for `buf` and `out` to narrow in the future. decode(buf: Any, out: Any | None = None) -> Any# Decode data in `buf`. Parameters: bufAny Encoded data. outAny Writeable buffer to store decoded data. If provided, this buffer must be exactly the right size to store the decoded data. Returns: decAny Decoded data. encode(buf: Any) -> Any# Encode data from `buf`. Parameters: bufAny Data to be encoded. Returns: enc: Any Encoded data. classmethod from_config(config: Any) -> Self# Instantiate a codec from a configuration dictionary. Parameters: configAny A configuration dictionary for this codec. get_config() -> Any# Return a JSON-serializable configuration dictionary for this codec. Must include an `'id'` field with the codec identifier. codec_id: str# ###### zarr.abc.store# ###### Classes# `ByteGetter` Base class for protocol classes. `ByteSetter` Base class for protocol classes. `Store` Abstract base class for Zarr stores. ###### Functions# `set_or_delete`(→ None) Set or delete a value in a byte setter ###### Module Contents# class zarr.abc.store.ByteGetter# Bases: `Protocol` Base class for protocol classes. Protocol classes are defined as: class Proto(Protocol): def meth(self) -> int: ... Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing). For example: class C: def meth(self) -> int: return 0 def func(x: Proto) -> int: return x.meth() func(C()) # Passes static type check See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as: class GenProto[T](Protocol): def meth(self) -> T: ... async get( prototype: zarr.core.buffer.BufferPrototype, byte_range: ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# class zarr.abc.store.ByteSetter# Bases: `Protocol` Base class for protocol classes. Protocol classes are defined as: class Proto(Protocol): def meth(self) -> int: ... Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing). For example: class C: def meth(self) -> int: return 0 def func(x: Proto) -> int: return x.meth() func(C()) # Passes static type check See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as: class GenProto[T](Protocol): def meth(self) -> T: ... async delete() -> None# async get( prototype: zarr.core.buffer.BufferPrototype, byte_range: ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# async set(value: zarr.core.buffer.Buffer) -> None# async set_if_not_exists(default: zarr.core.buffer.Buffer) -> None# class zarr.abc.store.Store(*, read_only: bool = False)# Bases: `abc.ABC` Abstract base class for Zarr stores. async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. abstract delete(key: str) -> None# Async: Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. abstract exists(key: str) -> bool# Async: Check if a key exists in the store. Parameters: keystr Returns: bool abstract get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Async: Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer abstract get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Async: Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. abstract list() -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store. Returns: AsyncIterator[str] abstract list_dir(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] abstract list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] classmethod open(*args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. abstract set(key: str, value: zarr.core.buffer.Buffer) -> None# Async: Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer abstract with_read_only(read_only: bool = False) -> Store# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. property read_only: bool# Is the store read-only? property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. property supports_deletes: bool# Abstractmethod: Does the store support deletes? property supports_listing: bool# Abstractmethod: Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. property supports_writes: bool# Abstractmethod: Does the store support writes? async zarr.abc.store.set_or_delete( byte_setter: ByteSetter, value: zarr.core.buffer.Buffer | None, ) -> None# Set or delete a value in a byte setter Parameters: byte_setterByteSetter valueBuffer | None Notes If value is None, the key will be deleted. #### zarr.api# ##### Submodules# ###### zarr.api.asynchronous# ###### Functions# `array`(...) Create an array filled with data. `consolidate_metadata`(→ zarr.core.group.AsyncGroup) Consolidate the metadata of all nodes in a hierarchy. `copy`(→ tuple[int, int, int]) Not implemented. `copy_all`(→ tuple[int, int, int]) Not implemented. `copy_store`(→ tuple[int, int, int]) Not implemented. `create`(...) Create an array. `create_array`(...) Create an array. `create_hierarchy`(...) Create a complete zarr hierarchy from a collection of metadata objects. `empty`(...) Create an empty array with the specified shape. The contents will be filled with the `empty_like`(...) Create an empty array like a. The contents will be filled with the `from_array`(...) Create an array from an existing array or array-like. `full`(...) Create an array, with fill_value being used as the default value for `full_like`(...) Create a filled array like a. `group`(→ zarr.core.group.AsyncGroup) Create a group. `load`(...) Load data from an array or group into memory. `ones`(...) Create an array, with one being used as the default value for `ones_like`(...) Create an array of ones like a. `open`(...) Convenience function to open a group or array using file-mode-like semantics. `open_array`(...) Open an array using file-mode-like semantics. `open_consolidated`(→ zarr.core.group.AsyncGroup) Alias for `open_group()` with `use_consolidated=True`. `open_group`(→ zarr.core.group.AsyncGroup) Open a group using file-mode-like semantics. `open_like`(...) Open a persistent array like a. `save`(→ None) Convenience function to save an array or group of arrays to the local file system. `save_array`(→ None) Convenience function to save a NumPy array to the local file system, following a `save_group`(→ None) Convenience function to save several NumPy arrays to the local file system, following a `tree`(→ Any) Provide a rich display of the hierarchy. `zeros`(...) Create an array, with zero being used as the default value for `zeros_like`(...) Create an array of zeros like a. ###### Module Contents# async zarr.api.asynchronous.array( data: numpy.typing.ArrayLike | zarr.core.array.Array, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array filled with data. Parameters: dataarray_like The data to fill the array with. **kwargs Passed through to `create()`. Returns: arrayarray The new array. async zarr.api.asynchronous.consolidate_metadata( store: zarr.storage.StoreLike, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, ) -> zarr.core.group.AsyncGroup# Consolidate the metadata of all nodes in a hierarchy. Upon completion, the metadata of the root node in the Zarr hierarchy will be updated to include all the metadata of child nodes. For Stores that do not support consolidated metadata, this operation raises a `TypeError`. Parameters: storeStoreLike The store-like object whose metadata you wish to consolidate. pathstr, optional A path to a group in the store to consolidate at. Only children below that group will be consolidated. By default, the root node is used so all the metadata in the store is consolidated. zarr_format{2, 3, None}, optional The zarr format of the hierarchy. By default the zarr format is inferred. Returns: group: AsyncGroup The group, with the `consolidated_metadata` field set to include the metadata of each child node. If the Store doesn’t support consolidated metadata, this function raises a TypeError. See `Store.supports_consolidated_metadata`. async zarr.api.asynchronous.copy(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. async zarr.api.asynchronous.copy_all(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. async zarr.api.asynchronous.copy_store(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. async zarr.api.asynchronous.create( shape: tuple[int, Ellipsis] | int, *, chunks: tuple[int, Ellipsis] | int | bool | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, compressor: zarr.core.array.CompressorLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, store: zarr.storage.StoreLike | None = None, synchronizer: Any | None = None, overwrite: bool = False, path: PathLike | None = None, chunk_store: zarr.storage.StoreLike | None = None, filters: collections.abc.Iterable[dict[str, zarr.core.common.JSON] | zarr.abc.numcodec.Numcodec] | None = None, cache_metadata: bool | None = None, cache_attrs: bool | None = None, read_only: bool | None = None, object_codec: zarr.abc.codec.Codec | None = None, dimension_separator: Literal['.', '/'] | None = None, write_empty_chunks: bool | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_shape: tuple[int, Ellipsis] | int | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncoding | tuple[Literal['default'], Literal['.', '/']] | tuple[Literal['v2'], Literal['.', '/']] | None = None, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, config: zarr.core.array_spec.ArrayConfigLike | None = None, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array. Parameters: shapeint or tuple of ints Array shape. chunksint or tuple of ints, optional Chunk shape. If True, will be guessed from `shape` and `dtype`. If False, will be set to `shape`, i.e., single chunk for the whole array. If an int, the chunk size in each dimension will be given by the value of `chunks`. Default is True. dtypestr or dtype, optional NumPy dtype. compressorCodec, optional Primary compressor to compress chunk data. Zarr format 2 only. Zarr format 3 arrays should use `codecs` instead. If neither `compressor` nor `filters` are provided, the default compressor `zarr.codecs.ZstdCodec` is used. If `compressor` is set to `None`, no compression is used. fill_valueAny, optional Fill value for the array. order{‘C’, ‘F’}, optional Deprecated in favor of the `config` keyword argument. Pass `{'order': }` to `create` instead of using this parameter. Memory layout to be used within each chunk. If not specified, the `array.order` parameter in the global config will be used. storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. synchronizerobject, optional Array synchronizer. overwritebool, optional If True, delete all pre-existing data in `store` at `path` before creating the array. pathstr, optional Path under which array is stored. chunk_storeStoreLike or None, default=None Separate storage for chunks. If not provided, `store` will be used for storage of both chunks and metadata. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. cache_metadatabool, optional If True, array configuration metadata will be cached for the lifetime of the object. If False, array metadata will be reloaded prior to all data access and modification operations (may incur overhead depending on storage and data access pattern). cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. read_onlybool, optional True if array should be protected against modification. object_codecCodec, optional A codec to encode object arrays, only needed if dtype=object. dimension_separator{‘.’, ‘/’}, optional Separator placed between the dimensions of a chunk. Zarr format 2 only. Zarr format 3 arrays should use `chunk_key_encoding` instead. write_empty_chunksbool, optional Deprecated in favor of the `config` keyword argument. Pass `{'write_empty_chunks': }` to `create` instead of using this parameter. If True, all chunks will be stored regardless of their contents. If False, each chunk is compared to the array’s fill value prior to storing. If a chunk is uniformly equal to the fill value, then that chunk is not be stored, and the store entry for that chunk’s key is deleted. zarr_format{2, 3, None}, optional The Zarr format to use when creating an array. The default is `None`, which instructs Zarr to choose the default Zarr format value defined in the runtime configuration. meta_arrayarray-like, optional Not implemented. attributesdict[str, JSON], optional A dictionary of user attributes to store with the array. chunk_shapeint or tuple of ints, optional The shape of the Array’s chunks (default is None). Zarr format 3 only. Zarr format 2 arrays should use chunks instead. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. Zarr format 3 only. Zarr format 2 arrays should use dimension_separator instead. Default is `("default", "/")`. codecsSequence of Codecs or dicts, optional An iterable of Codec or dict serializations of Codecs. Zarr V3 only. The elements of `codecs` specify the transformation from array values to stored bytes. Zarr format 3 only. Zarr format 2 arrays should use `filters` and `compressor` instead. If no codecs are provided, default codecs will be used based on the data type of the array. For most data types, the default codecs are the tuple `(BytesCodec(), ZstdCodec())`; data types that require a special `zarr.abc.codec.ArrayBytesCodec`, like variable-length strings or bytes, will use the `zarr.abc.codec.ArrayBytesCodec` required for the data type instead of `zarr.codecs.BytesCodec`. dimension_namesIterable[str | None] | None = None An iterable of dimension names. Zarr format 3 only. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. configArrayConfigLike, optional Runtime configuration of the array. If provided, will override the default values from zarr.config.array. Returns: zarray The array. async zarr.api.asynchronous.create_array( store: zarr.storage.StoreLike, *, name: str | None = None, shape: zarr.core.common.ShapeLike | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, data: numpy.ndarray[Any, numpy.dtype[Any]] | None = None, chunks: tuple[int, Ellipsis] | Literal['auto'] = 'auto', shards: ShardsLike | None = None, filters: FiltersLike = 'auto', compressors: CompressorsLike = 'auto', serializer: SerializerLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, zarr_format: zarr.core.common.ZarrFormat | None = 3, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, write_data: bool = True, ) -> AsyncArray[zarr.core.metadata.ArrayV2Metadata] | AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. namestr or None, optional The name of the array within the store. If `name` is `None`, the array will be located at the root of the store. shapeShapeLike, optional Shape of the array. Must be `None` if `data` is provided. dtypeZDTypeLike | None Data type of the array. Must be `None` if `data` is provided. datanp.ndarray, optional Array-like data to use for initializing the array. If this parameter is provided, the `shape` and `dtype` parameters must be `None`. chunkstuple[int, …] | Literal[“auto”], default=”auto” Chunk shape of the array. If chunks is “auto”, a chunk shape is guessed based on the shape of the array and the dtype. shardstuple[int, …], optional Shard shape of the array. The default value of `None` results in no sharding at all. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec], optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. If no `compressors` are provided, a default set of compressors will be used. These defaults can be changed by modifying the value of `array.v3_default_compressors` in `zarr.core.config`. Use `None` to omit default compressors. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. If no `compressor` is provided, a default compressor will be used. in `zarr.core.config`. Use `None` to omit the default compressor. serializerdict[str, JSON] | ArrayBytesCodec, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. If no `serializer` is provided, a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. fill_valueAny, optional Fill value for the array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If no `order` is provided, a default order will be used. This default can be changed by modifying the value of `array.order` in `zarr.core.config`. zarr_format{2, 3}, optional The zarr format to use when saving. attributesdict, optional Attributes for the array. chunk_key_encodingChunkKeyEncodingLike, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. dimension_namesIterable[str], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. If `True`, all existing paths in the store will be deleted. configArrayConfigLike, optional Runtime configuration for the array. write_databool If a pre-existing array-like object was provided to this function via the `data` parameter then `write_data` determines whether the values in that array-like object should be written to the Zarr array created by this function. If `write_data` is `False`, then the array will be left empty. Returns: AsyncArray The array. Examples >>> import zarr >>> store = zarr.storage.MemoryStore(mode='w') >>> async_arr = await zarr.api.asynchronous.create_array( >>> store=store, >>> shape=(100,100), >>> chunks=(10,10), >>> dtype='i4', >>> fill_value=0) async zarr.api.asynchronous.create_hierarchy( *, store: zarr.abc.store.Store, nodes: dict[str, GroupMetadata | zarr.core.metadata.ArrayV2Metadata | zarr.core.metadata.ArrayV3Metadata], overwrite: bool = False, ) -> collections.abc.AsyncIterator[tuple[str, AsyncGroup | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]]]# Create a complete zarr hierarchy from a collection of metadata objects. This function will parse its input to ensure that the hierarchy is complete. Any implicit groups will be inserted as needed. For example, an input like ``{'a/b': GroupMetadata}`` will be parsed to ``{'': GroupMetadata, 'a': GroupMetadata, 'b': Groupmetadata}`` After input parsing, this function then creates all the nodes in the hierarchy concurrently. Arrays and Groups are yielded in the order they are created. This order is not stable and should not be relied on. Parameters: storeStore The storage backend to use. nodesdict[str, GroupMetadata | ArrayV3Metadata | ArrayV2Metadata] A dictionary defining the hierarchy. The keys are the paths of the nodes in the hierarchy, relative to the root of the `Store`. The root of the store can be specified with the empty string `''`. The values are instances of `GroupMetadata` or `ArrayMetadata`. Note that all values must have the same `zarr_format` – it is an error to mix zarr versions in the same hierarchy. Leading “/” characters from keys will be removed. overwritebool Whether to overwrite existing nodes. Defaults to `False`, in which case an error is raised instead of overwriting an existing array or group. This function will not erase an existing group unless that group is explicitly named in `nodes`. If `nodes` defines implicit groups, e.g. `{`'a/b/c': GroupMetadata}`, and a group already exists at path `a`, then this function will leave the group at `a` as-is. Yields: tuple[str, AsyncGroup | AsyncArray] This function yields (path, node) pairs, in the order the nodes were created. Examples >>> from zarr.api.asynchronous import create_hierarchy >>> from zarr.storage import MemoryStore >>> from zarr.core.group import GroupMetadata >>> import asyncio >>> store = MemoryStore() >>> nodes = {'a': GroupMetadata(attributes={'name': 'leaf'})} >>> async def run(): ... print(dict([x async for x in create_hierarchy(store=store, nodes=nodes)])) >>> asyncio.run(run()) # {'a': , '': } async zarr.api.asynchronous.empty( shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an empty array with the specified shape. The contents will be filled with the specified fill value or zeros if no fill value is provided. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. async zarr.api.asynchronous.empty_like( a: ArrayLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an empty array like a. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. async zarr.api.asynchronous.from_array( store: zarr.storage.StoreLike, *, data: Array | numpy.typing.ArrayLike, write_data: bool = True, name: str | None = None, chunks: Literal['auto', 'keep'] | tuple[int, Ellipsis] = 'keep', shards: ShardsLike | None | Literal['keep'] = 'keep', filters: FiltersLike | Literal['keep'] = 'keep', compressors: CompressorsLike | Literal['keep'] = 'keep', serializer: SerializerLike | Literal['keep'] = 'keep', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) -> AsyncArray[zarr.core.metadata.ArrayV2Metadata] | AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array from an existing array or array-like. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. dataArray | array-like The array to copy. write_databool, default True Whether to copy the data from the input array to the new array. If `write_data` is `False`, the new array will be created with the same metadata as the input array, but without any data. namestr or None, optional The name of the array within the store. If `name` is `None`, the array will be located at the root of the store. chunkstuple[int, …] or “auto” or “keep”, optional Chunk shape of the array. Following values are supported: * “auto”: Automatically determine the chunk shape based on the array’s shape and dtype. * “keep”: Retain the chunk shape of the data array if it is a zarr Array. * tuple[int, …]: A tuple of integers representing the chunk shape. If not specified, defaults to “keep” if data is a zarr Array, otherwise “auto”. shardstuple[int, …], optional Shard shape of the array. Following values are supported: * “auto”: Automatically determine the shard shape based on the array’s shape and chunk shape. * “keep”: Retain the shard shape of the data array if it is a zarr Array. * tuple[int, …]: A tuple of integers representing the shard shape. * None: No sharding. If not specified, defaults to “keep” if data is a zarr Array, otherwise None. filtersIterable[Codec] | Literal[“auto”, “keep”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"keep"` instructs Zarr to infer `filters` from `data`. If that inference is not possible, Zarr will fall back to the behavior specified by `"auto"`, which is to choose default filters based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are the empty tuple `()`. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters is a tuple with a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec] or “auto” or “keep”, optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. Following values are supported: * Iterable[Codec]: List of compressors to apply to the array. * “auto”: Automatically determine the compressors based on the array’s dtype. * “keep”: Retain the compressors of the input array if it is a zarr Array. If no `compressors` are provided, defaults to “keep” if data is a zarr Array, otherwise “auto”. serializerdict[str, JSON] | ArrayBytesCodec or “auto” or “keep”, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. Following values are supported: * dict[str, JSON]: A dict representation of an `ArrayBytesCodec`. * ArrayBytesCodec: An instance of `ArrayBytesCodec`. * “auto”: a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. * “keep”: Retain the serializer of the input array if it is a zarr Array. fill_valueAny, optional Fill value for the array. If not specified, defaults to the fill value of the data array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If not specified, defaults to the memory order of the data array. zarr_format{2, 3}, optional The zarr format to use when saving. If not specified, defaults to the zarr format of the data array. attributesdict, optional Attributes for the array. If not specified, defaults to the attributes of the data array. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. If not specified and the data array has the same zarr format as the target array, the chunk key encoding of the data array is used. dimension_namesIterable[str | None] | None The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. If not specified, defaults to the dimension names of the data array. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. configArrayConfig or ArrayConfigLike, optional Runtime configuration for the array. Returns: AsyncArray The array. Examples Create an array from an existing Array: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> store2 = zarr.storage.LocalStore('example.zarr') >>> arr = zarr.create_array( >>> store=store, >>> shape=(100,100), >>> chunks=(10,10), >>> dtype='int32', >>> fill_value=0) >>> arr2 = await zarr.api.asynchronous.from_array(store2, data=arr) Create an array from an existing NumPy array: >>> arr3 = await zarr.api.asynchronous.from_array( >>> zarr.storage.MemoryStore(), >>> data=np.arange(10000, dtype='i4').reshape(100, 100), >>> ) Create an array from any array-like object: >>> arr4 = await zarr.api.asynchronous.from_array( >>> zarr.storage.MemoryStore(), >>> data=[[1, 2], [3, 4]], >>> ) >>> await arr4.getitem(...) array([[1, 2],[3, 4]]) Create an array from an existing Array without copying the data: >>> arr5 = await zarr.api.asynchronous.from_array( >>> zarr.storage.MemoryStore(), >>> data=Array(arr4), >>> write_data=False, >>> ) >>> await arr5.getitem(...) array([[0, 0],[0, 0]]) async zarr.api.asynchronous.full( shape: tuple[int, Ellipsis], fill_value: Any, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array, with fill_value being used as the default value for uninitialized portions of the array. Parameters: shapeint or tuple of int Shape of the empty array. fill_valuescalar Fill value. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. async zarr.api.asynchronous.full_like( a: ArrayLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create a filled array like a. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. async zarr.api.asynchronous.group( *, store: zarr.storage.StoreLike | None = None, overwrite: bool = False, chunk_store: zarr.storage.StoreLike | None = None, cache_attrs: bool | None = None, synchronizer: Any | None = None, path: str | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, storage_options: dict[str, Any] | None = None, ) -> zarr.core.group.AsyncGroup# Create a group. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. overwritebool, optional If True, delete any pre-existing data in store at path before creating the group. chunk_storeStoreLike or None, default=None Separate storage for chunks. Not implemented. cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. synchronizerobject, optional Array synchronizer. pathstr, optional Group path within store. meta_arrayarray-like, optional An array instance to use for determining arrays to create and return to users. Use numpy.empty(()) by default. zarr_format{2, 3, None}, optional The zarr format to use when saving. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. Returns: ggroup The new group. async zarr.api.asynchronous.load( *, store: zarr.storage.StoreLike, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar | dict[str, zarr.core.buffer.NDArrayLikeOrScalar]# Load data from an array or group into memory. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. pathstr or None, optional The path within the store from which to load. Returns: out If the path contains an array, out will be a numpy array. If the path contains a group, out will be a dict-like object where keys are array names and values are numpy arrays. See also `save`, `savez` Notes If loading data from a group of arrays, data will not be immediately loaded into memory. Rather, arrays will be loaded into memory as they are requested. async zarr.api.asynchronous.ones( shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array, with one being used as the default value for uninitialized portions of the array. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. async zarr.api.asynchronous.ones_like( a: ArrayLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array of ones like a. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. async zarr.api.asynchronous.open( *, store: zarr.storage.StoreLike | None = None, mode: zarr.core.common.AccessModeLiteral | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata] | zarr.core.group.AsyncGroup# Convenience function to open a group or array using file-mode-like semantics. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. mode{‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘}, optional Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-’ means create (fail if exists). If the store is read-only, the default is ‘r’; otherwise, it is ‘a’. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional The path within the store to open. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Additional parameters are passed through to `zarr.creation.open_array()` or `zarr.hierarchy.open_group()`. Returns: zarray or group Return type depends on what exists in the given store. async zarr.api.asynchronous.open_array( *, store: zarr.storage.StoreLike | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: PathLike = '', storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Open an array using file-mode-like semantics. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. zarr_version{2, 3, None}, optional The zarr format to use when saving. Deprecated in favor of zarr_format. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr, optional Path in store to array. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Any keyword arguments to pass to `create()`. Returns: AsyncArray The opened array. async zarr.api.asynchronous.open_consolidated( *args: Any, use_consolidated: Literal[True] = True, **kwargs: Any, ) -> zarr.core.group.AsyncGroup# Alias for `open_group()` with `use_consolidated=True`. async zarr.api.asynchronous.open_group( store: zarr.storage.StoreLike | None = None, *, mode: zarr.core.common.AccessModeLiteral = 'a', cache_attrs: bool | None = None, synchronizer: Any = None, path: str | None = None, chunk_store: zarr.storage.StoreLike | None = None, storage_options: dict[str, Any] | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, use_consolidated: bool | str | None = None, ) -> zarr.core.group.AsyncGroup# Open a group using file-mode-like semantics. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. Strings are interpreted as paths on the local file system and used as the `root` argument to `zarr.storage.LocalStore`. Dictionaries are used as the `store_dict` argument in `zarr.storage.MemoryStore``. By default (`store=None`) a new `zarr.storage.MemoryStore` is created. mode{‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘}, optional Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-’ means create (fail if exists). cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. synchronizerobject, optional Array synchronizer. pathstr, optional Group path within store. chunk_storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. meta_arrayarray-like, optional An array instance to use for determining arrays to create and return to users. Use numpy.empty(()) by default. attributesdict A dictionary of JSON-serializable values with user-defined attributes. use_consolidatedbool or str, default None Whether to use consolidated metadata. By default, consolidated metadata is used if it’s present in the store (in the `zarr.json` for Zarr format 3 and in the `.zmetadata` file for Zarr format 2). To explicitly require consolidated metadata, set `use_consolidated=True`, which will raise an exception if consolidated metadata is not found. To explicitly not use consolidated metadata, set `use_consolidated=False`, which will fall back to using the regular, non consolidated metadata. Zarr format 2 allowed configuring the key storing the consolidated metadata (`.zmetadata` by default). Specify the custom key as `use_consolidated` to load consolidated metadata from a non-default key. Returns: ggroup The new group. async zarr.api.asynchronous.open_like( a: ArrayLike, path: str, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata]# Open a persistent array like a. Parameters: aArray The shape and data-type of a define these same attributes of the returned array. pathstr The path to the new array. **kwargs Any keyword arguments to pass to the array constructor. Returns: AsyncArray The opened array. async zarr.api.asynchronous.save( store: zarr.storage.StoreLike, *args: zarr.core.buffer.NDArrayLike, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, **kwargs: Any, ) -> None# Convenience function to save an array or group of arrays to the local file system. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. *argsndarray NumPy arrays with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional The path within the group where the arrays will be saved. **kwargs NumPy arrays with data to save. async zarr.api.asynchronous.save_array( store: zarr.storage.StoreLike, arr: zarr.core.buffer.NDArrayLike, *, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> None# Convenience function to save a NumPy array to the local file system, following a similar API to the NumPy save() function. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. arrndarray NumPy array with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. The default is `None`, which will use the default Zarr format defined in the global configuration object. pathstr or None, optional The path within the store where the array will be saved. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Passed through to `create()`, e.g., compressor. async zarr.api.asynchronous.save_group( store: zarr.storage.StoreLike, *args: zarr.core.buffer.NDArrayLike, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: zarr.core.buffer.NDArrayLike, ) -> None# Convenience function to save several NumPy arrays to the local file system, following a similar API to the NumPy savez()/savez_compressed() functions. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. *argsndarray NumPy arrays with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional Path within the store where the group will be saved. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs NumPy arrays with data to save. async zarr.api.asynchronous.tree( grp: zarr.core.group.AsyncGroup, expand: bool | None = None, level: int | None = None, ) -> Any# Provide a rich display of the hierarchy. Deprecated since version 3.0.0: zarr.tree() is deprecated and will be removed in a future release. Use group.tree() instead. Parameters: grpGroup Zarr or h5py group. expandbool, optional Only relevant for HTML representation. If True, tree will be fully expanded. levelint, optional Maximum depth to descend into hierarchy. Returns: TreeRepr A pretty-printable object displaying the hierarchy. async zarr.api.asynchronous.zeros( shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array, with zero being used as the default value for uninitialized portions of the array. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. async zarr.api.asynchronous.zeros_like( a: ArrayLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array of zeros like a. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. ###### zarr.api.synchronous# ###### Functions# `array`(→ zarr.core.array.Array) Create an array filled with data. `consolidate_metadata`(→ zarr.core.group.Group) Consolidate the metadata of all nodes in a hierarchy. `copy`(→ tuple[int, int, int]) Not implemented. `copy_all`(→ tuple[int, int, int]) Not implemented. `copy_store`(→ tuple[int, int, int]) Not implemented. `create`(→ zarr.core.array.Array) Create an array. `create_array`(→ zarr.core.array.Array) Create an array. `create_hierarchy`(→ collections.abc.Iterator[tuple[str, ...) Create a complete zarr hierarchy from a collection of metadata objects. `empty`(→ zarr.core.array.Array) Create an empty array with the specified shape. The contents will be filled with the `empty_like`(→ zarr.core.array.Array) Create an empty array like another array. The contents will be filled with the `from_array`(→ zarr.core.array.Array) Create an array from an existing array or array-like. `full`(→ zarr.core.array.Array) Create an array with a default fill value. `full_like`(→ zarr.core.array.Array) Create a filled array like another array. `group`(→ zarr.core.group.Group) Create a group. `load`(...) Load data from an array or group into memory. `ones`(→ zarr.core.array.Array) Create an array with a fill value of one. `ones_like`(→ zarr.core.array.Array) Create an array of ones like another array. `open`(→ zarr.core.array.Array | zarr.core.group.Group) Open a group or array using file-mode-like semantics. `open_array`(→ zarr.core.array.Array) Open an array using file-mode-like semantics. `open_consolidated`(→ zarr.core.group.Group) Alias for `open_group()` with `use_consolidated=True`. `open_group`(→ zarr.core.group.Group) Open a group using file-mode-like semantics. `open_like`(→ zarr.core.array.Array) Open a persistent array like another array. `save`(→ None) Save an array or group of arrays to the local file system. `save_array`(→ None) Save a NumPy array to the local file system. `save_group`(→ None) Save several NumPy arrays to the local file system. `tree`(→ Any) Provide a rich display of the hierarchy. `zeros`(→ zarr.core.array.Array) Create an array with a fill value of zero. `zeros_like`(→ zarr.core.array.Array) Create an array of zeros like another array. ###### Module Contents# zarr.api.synchronous.array( data: numpy.typing.ArrayLike | zarr.core.array.Array, **kwargs: Any, ) -> zarr.core.array.Array# Create an array filled with data. Parameters: dataarray_like The data to fill the array with. **kwargs Passed through to `create()`. Returns: arrayArray The new array. zarr.api.synchronous.consolidate_metadata( store: zarr.storage.StoreLike, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, ) -> zarr.core.group.Group# Consolidate the metadata of all nodes in a hierarchy. Upon completion, the metadata of the root node in the Zarr hierarchy will be updated to include all the metadata of child nodes. For Stores that do not use consolidated metadata, this operation raises a TypeError. Parameters: storeStoreLike The store-like object whose metadata you wish to consolidate. pathstr, optional A path to a group in the store to consolidate at. Only children below that group will be consolidated. By default, the root node is used so all the metadata in the store is consolidated. zarr_format{2, 3, None}, optional The zarr format of the hierarchy. By default the zarr format is inferred. Returns: group: Group The group, with the `consolidated_metadata` field set to include the metadata of each child node. If the Store doesn’t support consolidated metadata, this function raises a TypeError. See `Store.supports_consolidated_metadata`. zarr.api.synchronous.copy(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.api.synchronous.copy_all(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.api.synchronous.copy_store(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.api.synchronous.create( shape: tuple[int, Ellipsis] | int, *, chunks: tuple[int, Ellipsis] | int | bool | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, compressor: zarr.core.array.CompressorLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, store: zarr.storage.StoreLike | None = None, synchronizer: Any | None = None, overwrite: bool = False, path: zarr.api.asynchronous.PathLike | None = None, chunk_store: zarr.storage.StoreLike | None = None, filters: collections.abc.Iterable[dict[str, zarr.core.common.JSON] | zarr.abc.numcodec.Numcodec] | None = None, cache_metadata: bool | None = None, cache_attrs: bool | None = None, read_only: bool | None = None, object_codec: zarr.abc.codec.Codec | None = None, dimension_separator: Literal['.', '/'] | None = None, write_empty_chunks: bool | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_shape: tuple[int, Ellipsis] | int | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncoding | tuple[Literal['default'], Literal['.', '/']] | tuple[Literal['v2'], Literal['.', '/']] | None = None, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, config: zarr.core.array_spec.ArrayConfigLike | None = None, **kwargs: Any, ) -> zarr.core.array.Array# Create an array. Parameters: shapeint or tuple of ints Array shape. chunksint or tuple of ints, optional Chunk shape. If True, will be guessed from `shape` and `dtype`. If False, will be set to `shape`, i.e., single chunk for the whole array. If an int, the chunk size in each dimension will be given by the value of `chunks`. Default is True. dtypestr or dtype, optional NumPy dtype. compressorCodec, optional Primary compressor to compress chunk data. Zarr format 2 only. Zarr format 3 arrays should use `codecs` instead. If neither `compressor` nor `filters` are provided, the default compressor `zarr.codecs.ZstdCodec` is used. If `compressor` is set to `None`, no compression is used. fill_valueAny, optional Fill value for the array. order{‘C’, ‘F’}, optional Deprecated in favor of the `config` keyword argument. Pass `{'order': }` to `create` instead of using this parameter. Memory layout to be used within each chunk. If not specified, the `array.order` parameter in the global config will be used. storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. synchronizerobject, optional Array synchronizer. overwritebool, optional If True, delete all pre-existing data in `store` at `path` before creating the array. pathstr, optional Path under which array is stored. chunk_storeStoreLike or None, default=None Separate storage for chunks. If not provided, `store` will be used for storage of both chunks and metadata. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. cache_metadatabool, optional If True, array configuration metadata will be cached for the lifetime of the object. If False, array metadata will be reloaded prior to all data access and modification operations (may incur overhead depending on storage and data access pattern). cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. read_onlybool, optional True if array should be protected against modification. object_codecCodec, optional A codec to encode object arrays, only needed if dtype=object. dimension_separator{‘.’, ‘/’}, optional Separator placed between the dimensions of a chunk. Zarr format 2 only. Zarr format 3 arrays should use `chunk_key_encoding` instead. write_empty_chunksbool, optional Deprecated in favor of the `config` keyword argument. Pass `{'write_empty_chunks': }` to `create` instead of using this parameter. If True, all chunks will be stored regardless of their contents. If False, each chunk is compared to the array’s fill value prior to storing. If a chunk is uniformly equal to the fill value, then that chunk is not be stored, and the store entry for that chunk’s key is deleted. zarr_format{2, 3, None}, optional The Zarr format to use when creating an array. The default is `None`, which instructs Zarr to choose the default Zarr format value defined in the runtime configuration. meta_arrayarray-like, optional Not implemented. attributesdict[str, JSON], optional A dictionary of user attributes to store with the array. chunk_shapeint or tuple of ints, optional The shape of the Array’s chunks (default is None). Zarr format 3 only. Zarr format 2 arrays should use chunks instead. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. Zarr format 3 only. Zarr format 2 arrays should use dimension_separator instead. Default is `("default", "/")`. codecsSequence of Codecs or dicts, optional An iterable of Codec or dict serializations of Codecs. Zarr V3 only. The elements of `codecs` specify the transformation from array values to stored bytes. Zarr format 3 only. Zarr format 2 arrays should use `filters` and `compressor` instead. If no codecs are provided, default codecs will be used based on the data type of the array. For most data types, the default codecs are the tuple `(BytesCodec(), ZstdCodec())`; data types that require a special `zarr.abc.codec.ArrayBytesCodec`, like variable-length strings or bytes, will use the `zarr.abc.codec.ArrayBytesCodec` required for the data type instead of `zarr.codecs.BytesCodec`. dimension_namesIterable[str | None] | None = None An iterable of dimension names. Zarr format 3 only. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. configArrayConfigLike, optional Runtime configuration of the array. If provided, will override the default values from zarr.config.array. Returns: zArray The array. zarr.api.synchronous.create_array( store: zarr.storage.StoreLike, *, name: str | None = None, shape: zarr.core.common.ShapeLike | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, data: numpy.ndarray[Any, numpy.dtype[Any]] | None = None, chunks: tuple[int, Ellipsis] | Literal['auto'] = 'auto', shards: zarr.core.array.ShardsLike | None = None, filters: zarr.core.array.FiltersLike = 'auto', compressors: zarr.core.array.CompressorsLike = 'auto', serializer: zarr.core.array.SerializerLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, zarr_format: zarr.core.common.ZarrFormat | None = 3, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, write_data: bool = True, ) -> zarr.core.array.Array# Create an array. This function wraps `zarr.core.array.create_array()`. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. namestr or None, optional The name of the array within the store. If `name` is `None`, the array will be located at the root of the store. shapeShapeLike, optional Shape of the array. Must be `None` if `data` is provided. dtypeZDTypeLike | None Data type of the array. Must be `None` if `data` is provided. datanp.ndarray, optional Array-like data to use for initializing the array. If this parameter is provided, the `shape` and `dtype` parameters must be `None`. chunkstuple[int, …] | Literal[“auto”], default=”auto” Chunk shape of the array. If chunks is “auto”, a chunk shape is guessed based on the shape of the array and the dtype. shardstuple[int, …], optional Shard shape of the array. The default value of `None` results in no sharding at all. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec], optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. If no `compressors` are provided, a default set of compressors will be used. These defaults can be changed by modifying the value of `array.v3_default_compressors` in `zarr.core.config`. Use `None` to omit default compressors. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. If no `compressor` is provided, a default compressor will be used. in `zarr.core.config`. Use `None` to omit the default compressor. serializerdict[str, JSON] | ArrayBytesCodec, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. If no `serializer` is provided, a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. fill_valueAny, optional Fill value for the array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If no `order` is provided, a default order will be used. This default can be changed by modifying the value of `array.order` in `zarr.core.config`. zarr_format{2, 3}, optional The zarr format to use when saving. attributesdict, optional Attributes for the array. chunk_key_encodingChunkKeyEncodingLike, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. dimension_namesIterable[str], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. If `True`, all existing paths in the store will be deleted. configArrayConfigLike, optional Runtime configuration for the array. write_databool If a pre-existing array-like object was provided to this function via the `data` parameter then `write_data` determines whether the values in that array-like object should be written to the Zarr array created by this function. If `write_data` is `False`, then the array will be left empty. Returns: Array The array. Examples >>> import zarr >>> store = zarr.storage.MemoryStore() >>> arr = await zarr.create_array( >>> store=store, >>> shape=(100,100), >>> chunks=(10,10), >>> dtype='i4', >>> fill_value=0) zarr.api.synchronous.create_hierarchy( *, store: zarr.abc.store.Store, nodes: dict[str, zarr.core.group.GroupMetadata | zarr.core.metadata.ArrayV2Metadata | zarr.core.metadata.ArrayV3Metadata], overwrite: bool = False, ) -> collections.abc.Iterator[tuple[str, zarr.core.group.Group | zarr.core.array.Array]]# Create a complete zarr hierarchy from a collection of metadata objects. This function will parse its input to ensure that the hierarchy is complete. Any implicit groups will be inserted as needed. For example, an input like ``{'a/b': GroupMetadata}`` will be parsed to ``{'': GroupMetadata, 'a': GroupMetadata, 'b': Groupmetadata}`` After input parsing, this function then creates all the nodes in the hierarchy concurrently. Arrays and Groups are yielded in the order they are created. This order is not stable and should not be relied on. Parameters: storeStore The storage backend to use. nodesdict[str, GroupMetadata | ArrayV3Metadata | ArrayV2Metadata] A dictionary defining the hierarchy. The keys are the paths of the nodes in the hierarchy, relative to the root of the `Store`. The root of the store can be specified with the empty string `''`. The values are instances of `GroupMetadata` or `ArrayMetadata`. Note that all values must have the same `zarr_format` – it is an error to mix zarr versions in the same hierarchy. Leading “/” characters from keys will be removed. overwritebool Whether to overwrite existing nodes. Defaults to `False`, in which case an error is raised instead of overwriting an existing array or group. This function will not erase an existing group unless that group is explicitly named in `nodes`. If `nodes` defines implicit groups, e.g. `{`'a/b/c': GroupMetadata}`, and a group already exists at path `a`, then this function will leave the group at `a` as-is. Yields: tuple[str, Group | Array] This function yields (path, node) pairs, in the order the nodes were created. Examples >>> from zarr import create_hierarchy >>> from zarr.storage import MemoryStore >>> from zarr.core.group import GroupMetadata >>> store = MemoryStore() >>> nodes = {'a': GroupMetadata(attributes={'name': 'leaf'})} >>> nodes_created = dict(create_hierarchy(store=store, nodes=nodes)) >>> print(nodes) # {'a': GroupMetadata(attributes={'name': 'leaf'}, zarr_format=3, consolidated_metadata=None, node_type='group')} zarr.api.synchronous.empty(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an empty array with the specified shape. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. zarr.api.synchronous.empty_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an empty array like another array. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. zarr.api.synchronous.from_array( store: zarr.storage.StoreLike, *, data: zarr.core.array.Array | numpy.typing.ArrayLike, write_data: bool = True, name: str | None = None, chunks: Literal['auto', 'keep'] | tuple[int, Ellipsis] = 'keep', shards: zarr.core.array.ShardsLike | None | Literal['keep'] = 'keep', filters: zarr.core.array.FiltersLike | Literal['keep'] = 'keep', compressors: zarr.core.array.CompressorsLike | Literal['keep'] = 'keep', serializer: zarr.core.array.SerializerLike | Literal['keep'] = 'keep', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) -> zarr.core.array.Array# Create an array from an existing array or array-like. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. dataArray | array-like The array to copy. write_databool, default True Whether to copy the data from the input array to the new array. If `write_data` is `False`, the new array will be created with the same metadata as the input array, but without any data. namestr or None, optional The name of the array within the store. If `name` is `None`, the array will be located at the root of the store. chunkstuple[int, …] or “auto” or “keep”, optional Chunk shape of the array. Following values are supported: * “auto”: Automatically determine the chunk shape based on the array’s shape and dtype. * “keep”: Retain the chunk shape of the data array if it is a zarr Array. * tuple[int, …]: A tuple of integers representing the chunk shape. If not specified, defaults to “keep” if data is a zarr Array, otherwise “auto”. shardstuple[int, …], optional Shard shape of the array. Following values are supported: * “auto”: Automatically determine the shard shape based on the array’s shape and chunk shape. * “keep”: Retain the shard shape of the data array if it is a zarr Array. * tuple[int, …]: A tuple of integers representing the shard shape. * None: No sharding. If not specified, defaults to “keep” if data is a zarr Array, otherwise None. filtersIterable[Codec] | Literal[“auto”, “keep”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"keep"` instructs Zarr to infer `filters` from `data`. If that inference is not possible, Zarr will fall back to the behavior specified by `"auto"`, which is to choose default filters based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are the empty tuple `()`. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters is a tuple with a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec] or “auto” or “keep”, optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. Following values are supported: * Iterable[Codec]: List of compressors to apply to the array. * “auto”: Automatically determine the compressors based on the array’s dtype. * “keep”: Retain the compressors of the input array if it is a zarr Array. If no `compressors` are provided, defaults to “keep” if data is a zarr Array, otherwise “auto”. serializerdict[str, JSON] | ArrayBytesCodec or “auto” or “keep”, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. Following values are supported: * dict[str, JSON]: A dict representation of an `ArrayBytesCodec`. * ArrayBytesCodec: An instance of `ArrayBytesCodec`. * “auto”: a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. * “keep”: Retain the serializer of the input array if it is a zarr Array. fill_valueAny, optional Fill value for the array. If not specified, defaults to the fill value of the data array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If not specified, defaults to the memory order of the data array. zarr_format{2, 3}, optional The zarr format to use when saving. If not specified, defaults to the zarr format of the data array. attributesdict, optional Attributes for the array. If not specified, defaults to the attributes of the data array. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. If not specified and the data array has the same zarr format as the target array, the chunk key encoding of the data array is used. dimension_namesIterable[str | None] | None The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. If not specified, defaults to the dimension names of the data array. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. configArrayConfig or ArrayConfigLike, optional Runtime configuration for the array. Returns: Array The array. Examples Create an array from an existing Array: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> store2 = zarr.storage.LocalStore('example.zarr') >>> arr = zarr.create_array( >>> store=store, >>> shape=(100,100), >>> chunks=(10,10), >>> dtype='int32', >>> fill_value=0) >>> arr2 = zarr.from_array(store2, data=arr) Create an array from an existing NumPy array: >>> import numpy as np >>> arr3 = zarr.from_array( zarr.storage.MemoryStore(), >>> data=np.arange(10000, dtype='i4').reshape(100, 100), >>> ) Create an array from any array-like object: >>> arr4 = zarr.from_array( >>> zarr.storage.MemoryStore(), >>> data=[[1, 2], [3, 4]], >>> ) >>> arr4[...] array([[1, 2],[3, 4]]) Create an array from an existing Array without copying the data: >>> arr5 = zarr.from_array( >>> zarr.storage.MemoryStore(), >>> data=arr4, >>> write_data=False, >>> ) >>> arr5[...] array([[0, 0],[0, 0]]) zarr.api.synchronous.full( shape: tuple[int, Ellipsis], fill_value: Any, **kwargs: Any, ) -> zarr.core.array.Array# Create an array with a default fill value. Parameters: shapeint or tuple of int Shape of the empty array. fill_valuescalar Fill value. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.api.synchronous.full_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create a filled array like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.api.synchronous.group( store: zarr.storage.StoreLike | None = None, *, overwrite: bool = False, chunk_store: zarr.storage.StoreLike | None = None, cache_attrs: bool | None = None, synchronizer: Any | None = None, path: str | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, storage_options: dict[str, Any] | None = None, ) -> zarr.core.group.Group# Create a group. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. overwritebool, optional If True, delete any pre-existing data in store at path before creating the group. chunk_storeStoreLike or None, default=None Separate storage for chunks. Not implemented. cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. synchronizerobject, optional Array synchronizer. pathstr, optional Group path within store. meta_arrayarray-like, optional An array instance to use for determining arrays to create and return to users. Use numpy.empty(()) by default. zarr_format{2, 3, None}, optional The zarr format to use when saving. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. Returns: gGroup The new group. zarr.api.synchronous.load( store: zarr.storage.StoreLike, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar | dict[str, zarr.core.buffer.NDArrayLikeOrScalar]# Load data from an array or group into memory. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. pathstr or None, optional The path within the store from which to load. Returns: out If the path contains an array, out will be a numpy array. If the path contains a group, out will be a dict-like object where keys are array names and values are numpy arrays. See also `save`, `savez` Notes If loading data from a group of arrays, data will not be immediately loaded into memory. Rather, arrays will be loaded into memory as they are requested. zarr.api.synchronous.ones(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an array with a fill value of one. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.api.synchronous.ones_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an array of ones like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.api.synchronous.open( store: zarr.storage.StoreLike | None = None, *, mode: zarr.core.common.AccessModeLiteral | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> zarr.core.array.Array | zarr.core.group.Group# Open a group or array using file-mode-like semantics. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. mode{‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘}, optional Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-’ means create (fail if exists). If the store is read-only, the default is ‘r’; otherwise, it is ‘a’. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional The path within the store to open. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Additional parameters are passed through to `zarr.creation.open_array()` or `zarr.hierarchy.open_group()`. Returns: zarray or group Return type depends on what exists in the given store. zarr.api.synchronous.open_array( store: zarr.storage.StoreLike | None = None, *, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: zarr.api.asynchronous.PathLike = '', storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> zarr.core.array.Array# Open an array using file-mode-like semantics. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. zarr_version{2, 3, None}, optional The zarr format to use when saving. Deprecated in favor of zarr_format. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr, optional Path in store to array. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Any keyword arguments to pass to `create()`. Returns: AsyncArray The opened array. zarr.api.synchronous.open_consolidated( *args: Any, use_consolidated: Literal[True] = True, **kwargs: Any, ) -> zarr.core.group.Group# Alias for `open_group()` with `use_consolidated=True`. zarr.api.synchronous.open_group( store: zarr.storage.StoreLike | None = None, *, mode: zarr.core.common.AccessModeLiteral = 'a', cache_attrs: bool | None = None, synchronizer: Any = None, path: str | None = None, chunk_store: zarr.storage.StoreLike | None = None, storage_options: dict[str, Any] | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, use_consolidated: bool | str | None = None, ) -> zarr.core.group.Group# Open a group using file-mode-like semantics. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. Strings are interpreted as paths on the local file system and used as the `root` argument to `zarr.storage.LocalStore`. Dictionaries are used as the `store_dict` argument in `zarr.storage.MemoryStore``. By default (`store=None`) a new `zarr.storage.MemoryStore` is created. mode{‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘}, optional Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-’ means create (fail if exists). cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. synchronizerobject, optional Array synchronizer. pathstr, optional Group path within store. chunk_storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. meta_arrayarray-like, optional An array instance to use for determining arrays to create and return to users. Use numpy.empty(()) by default. attributesdict A dictionary of JSON-serializable values with user-defined attributes. use_consolidatedbool or str, default None Whether to use consolidated metadata. By default, consolidated metadata is used if it’s present in the store (in the `zarr.json` for Zarr format 3 and in the `.zmetadata` file for Zarr format 2). To explicitly require consolidated metadata, set `use_consolidated=True`, which will raise an exception if consolidated metadata is not found. To explicitly not use consolidated metadata, set `use_consolidated=False`, which will fall back to using the regular, non consolidated metadata. Zarr format 2 allowed configuring the key storing the consolidated metadata (`.zmetadata` by default). Specify the custom key as `use_consolidated` to load consolidated metadata from a non-default key. Returns: gGroup The new group. zarr.api.synchronous.open_like( a: zarr.api.asynchronous.ArrayLike, path: str, **kwargs: Any, ) -> zarr.core.array.Array# Open a persistent array like another array. Parameters: aArray The shape and data-type of a define these same attributes of the returned array. pathstr The path to the new array. **kwargs Any keyword arguments to pass to the array constructor. Returns: AsyncArray The opened array. zarr.api.synchronous.save( store: zarr.storage.StoreLike, *args: zarr.core.buffer.NDArrayLike, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, **kwargs: Any, ) -> None# Save an array or group of arrays to the local file system. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. *argsndarray NumPy arrays with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional The path within the group where the arrays will be saved. **kwargs NumPy arrays with data to save. zarr.api.synchronous.save_array( store: zarr.storage.StoreLike, arr: zarr.core.buffer.NDArrayLike, *, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> None# Save a NumPy array to the local file system. Follows a similar API to the NumPy save() function. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. arrndarray NumPy array with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. The default is `None`, which will use the default Zarr format defined in the global configuration object. pathstr or None, optional The path within the store where the array will be saved. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Passed through to `create()`, e.g., compressor. zarr.api.synchronous.save_group( store: zarr.storage.StoreLike, *args: zarr.core.buffer.NDArrayLike, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: zarr.core.buffer.NDArrayLike, ) -> None# Save several NumPy arrays to the local file system. Follows a similar API to the NumPy savez()/savez_compressed() functions. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. *argsndarray NumPy arrays with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional Path within the store where the group will be saved. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs NumPy arrays with data to save. zarr.api.synchronous.tree( grp: zarr.core.group.Group, expand: bool | None = None, level: int | None = None, ) -> Any# Provide a rich display of the hierarchy. Deprecated since version 3.0.0: zarr.tree() is deprecated and will be removed in a future release. Use group.tree() instead. Parameters: grpGroup Zarr or h5py group. expandbool, optional Only relevant for HTML representation. If True, tree will be fully expanded. levelint, optional Maximum depth to descend into hierarchy. Returns: TreeRepr A pretty-printable object displaying the hierarchy. zarr.api.synchronous.zeros(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an array with a fill value of zero. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.api.synchronous.zeros_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an array of zeros like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. #### zarr.buffer# Implementations of the Zarr Buffer interface. See also `zarr.abc.buffer` Abstract base class for the Zarr Buffer interface. ##### Submodules# ###### zarr.buffer.cpu# ###### Attributes# `buffer_prototype` ###### Classes# `Buffer` A flat contiguous memory block `NDBuffer` An n-dimensional memory block ###### Functions# `as_numpy_array_wrapper`(→ zarr.core.buffer.core.Buffer) Converts the input of func to a numpy array and the output back to Buffer. `numpy_buffer_prototype`(...) ###### Module Contents# class zarr.buffer.cpu.Buffer(array_like: zarr.core.buffer.core.ArrayLike)# Bases: `zarr.core.buffer.core.Buffer` A flat contiguous memory block We use Buffer throughout Zarr to represent a contiguous block of memory. A Buffer is backed by a underlying array-like instance that represents the memory. The memory type is unspecified; can be regular host memory, CUDA device memory, or something else. The only requirement is that the array-like instance can be copied/converted to a regular Numpy array (host memory). Parameters: array_like array-like object that must be 1-dim, contiguous, and byte dtype. Notes This buffer is untyped, so all indexing and sizes are in bytes. as_array_like() -> ArrayLike# Returns the underlying array (host or device memory) of this buffer This will never copy data. Returns: The underlying 1d array such as a NumPy or CuPy array. as_buffer_like() -> zarr.core.common.BytesLike# Returns the buffer as an object that implements the Python buffer protocol. Returns: An object that implements the Python buffer protocol Notes Might have to copy data, since the implementation uses .as_numpy_array(). as_numpy_array() -> numpy.typing.NDArray[Any]# Returns the buffer as a NumPy array (host memory). Returns: NumPy array of this buffer (might be a data copy) Notes Might have to copy data, consider using .as_array_like() instead. classmethod create_zero_length() -> Self# Create an empty buffer with length zero Returns: New empty 0-length buffer classmethod from_array_like(array_like: ArrayLike) -> Self# Create a new buffer of an array-like object Parameters: array_like array-like object that must be 1-dim, contiguous, and byte dtype. Returns: New buffer representing array_like classmethod from_buffer(buffer: zarr.core.buffer.core.Buffer) -> Self# Create a new buffer of an existing Buffer This is useful if you want to ensure that an existing buffer is of the correct subclass of Buffer. E.g., MemoryStore uses this to return a buffer instance of the subclass specified by its BufferPrototype argument. Typically, this only copies data if the data has to be moved between memory types, such as from host to device memory. Parameters: buffer buffer object. Returns: A new buffer representing the content of the input buffer Notes Subclasses of Buffer must override this method to implement more optimal conversions that avoid copies where possible classmethod from_bytes(bytes_like: zarr.core.common.BytesLike) -> Self# Create a new buffer of a bytes-like object (host memory) Parameters: bytes_like bytes-like object Returns: New buffer representing bytes_like to_bytes() -> bytes# Returns the buffer as bytes (host memory). Returns: bytes of this buffer (data copy) Warning Will always copy data, only use this method for small buffers such as metadata buffers. If possible, use .as_numpy_array() or .as_array_like() instead. class zarr.buffer.cpu.NDBuffer(array: zarr.core.buffer.core.NDArrayLike)# Bases: `zarr.core.buffer.core.NDBuffer` An n-dimensional memory block We use NDBuffer throughout Zarr to represent a n-dimensional memory block. A NDBuffer is backed by a underlying ndarray-like instance that represents the memory. The memory type is unspecified; can be regular host memory, CUDA device memory, or something else. The only requirement is that the ndarray-like instance can be copied/converted to a regular Numpy array (host memory). Parameters: array ndarray-like object that is convertible to a regular Numpy array. Notes The two buffer classes Buffer and NDBuffer are very similar. In fact, Buffer is a special case of NDBuffer where dim=1, stride=1, and dtype=”B”. However, in order to use Python’s type system to differentiate between the contiguous Buffer and the n-dim (non-contiguous) NDBuffer, we keep the definition of the two classes separate. all_equal(other: Any, equal_nan: bool = True) -> bool# Compare to other using np.array_equal. as_ndarray_like() -> NDArrayLike# Returns the underlying array (host or device memory) of this buffer This will never copy data. Returns: The underlying array such as a NumPy or CuPy array. as_numpy_array() -> numpy.typing.NDArray[Any]# Returns the buffer as a NumPy array (host memory). Returns: NumPy array of this buffer (might be a data copy) Warning Might have to copy data, consider using .as_ndarray_like() instead. as_scalar() -> ScalarType# Returns the buffer as a scalar value astype( dtype: numpy.typing.DTypeLike, order: Literal['K', 'A', 'C', 'F'] = 'K', ) -> Self# copy() -> Self# classmethod create( *, shape: collections.abc.Iterable[int], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', fill_value: Any | None = None, ) -> Self# Create a new buffer and its underlying ndarray-like object Parameters: shape The shape of the buffer and its underlying ndarray-like object dtype The datatype of the buffer and its underlying ndarray-like object order Whether to store multi-dimensional data in row-major (C-style) or column-major (Fortran-style) order in memory. fill_value If not None, fill the new buffer with a scalar value. Returns: New buffer representing a new ndarray_like object Notes A subclass can overwrite this method to create a ndarray-like object other then the default Numpy array. classmethod empty( shape: tuple[int, Ellipsis], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', ) -> Self# Create an empty buffer with the given shape, dtype, and order. This method can be faster than `NDBuffer.create` because it doesn’t have to initialize the memory used by the underlying ndarray-like object. Parameters: shape The shape of the buffer and its underlying ndarray-like object dtype The datatype of the buffer and its underlying ndarray-like object order Whether to store multi-dimensional data in row-major (C-style) or column-major (Fortran-style) order in memory. Returns: buffer New buffer representing a new ndarray_like object with empty data. See also `NDBuffer.create` Create a new buffer with some initial fill value. fill(value: Any) -> None# classmethod from_ndarray_like(ndarray_like: NDArrayLike) -> Self# Create a new buffer of a ndarray-like object Parameters: ndarray_like ndarray-like object Returns: New buffer representing ndarray_like classmethod from_numpy_array(array_like: numpy.typing.ArrayLike) -> Self# Create a new buffer of Numpy array-like object Parameters: array_like Object that can be coerced into a Numpy array Returns: New buffer representing array_like reshape(newshape: tuple[int, Ellipsis] | Literal[-1]) -> Self# squeeze(axis: tuple[int, Ellipsis]) -> Self# transpose( axes: SupportsIndex | collections.abc.Sequence[SupportsIndex] | None, ) -> Self# property byteorder: zarr.codecs.bytes.Endian# property dtype: numpy.dtype[Any]# property shape: tuple[int, Ellipsis]# zarr.buffer.cpu.as_numpy_array_wrapper( func: collections.abc.Callable[[numpy.typing.NDArray[Any]], bytes], buf: zarr.core.buffer.core.Buffer, prototype: zarr.core.buffer.core.BufferPrototype, ) -> zarr.core.buffer.core.Buffer# Converts the input of func to a numpy array and the output back to Buffer. This function is useful when calling a func that only support host memory such as GZip.decode and Blosc.decode. In this case, use this wrapper to convert the input buf to a Numpy array and convert the result back into a Buffer. Parameters: func The callable that will be called with the converted buf as input. func must return bytes, which will be converted into a Buffer before returned. buf The buffer that will be converted to a Numpy array before given as input to func. prototype The prototype of the output buffer. Returns: The result of func converted to a Buffer zarr.buffer.cpu.numpy_buffer_prototype() -> zarr.core.buffer.core.BufferPrototype# zarr.buffer.cpu.buffer_prototype# ###### zarr.buffer.gpu# ###### Attributes# `buffer_prototype` ###### Classes# `Buffer` A flat contiguous memory block on the GPU `NDBuffer` A n-dimensional memory block on the GPU ###### Module Contents# class zarr.buffer.gpu.Buffer(array_like: zarr.core.buffer.core.ArrayLike)# Bases: `zarr.core.buffer.core.Buffer` A flat contiguous memory block on the GPU We use Buffer throughout Zarr to represent a contiguous block of memory. A Buffer is backed by a underlying array-like instance that represents the memory. The memory type is unspecified; can be regular host memory, CUDA device memory, or something else. The only requirement is that the array-like instance can be copied/converted to a regular Numpy array (host memory). Parameters: array_like array-like object that must be 1-dim, contiguous, and byte dtype. Notes This buffer is untyped, so all indexing and sizes are in bytes. as_array_like() -> ArrayLike# Returns the underlying array (host or device memory) of this buffer This will never copy data. Returns: The underlying 1d array such as a NumPy or CuPy array. as_buffer_like() -> zarr.core.common.BytesLike# Returns the buffer as an object that implements the Python buffer protocol. Returns: An object that implements the Python buffer protocol Notes Might have to copy data, since the implementation uses .as_numpy_array(). as_numpy_array() -> numpy.typing.NDArray[Any]# Returns the buffer as a NumPy array (host memory). Returns: NumPy array of this buffer (might be a data copy) Notes Might have to copy data, consider using .as_array_like() instead. classmethod create_zero_length() -> Self# Create an empty buffer with length zero Returns: New empty 0-length buffer classmethod from_array_like(array_like: ArrayLike) -> Self# Create a new buffer of an array-like object Parameters: array_like array-like object that must be 1-dim, contiguous, and byte dtype. Returns: New buffer representing array_like classmethod from_buffer(buffer: zarr.core.buffer.core.Buffer) -> Self# Create an GPU Buffer given an arbitrary Buffer This will try to be zero-copy if buffer is already on the GPU and will trigger a copy if not. Returns: New GPU Buffer constructed from buffer classmethod from_bytes(bytes_like: zarr.core.common.BytesLike) -> Self# Create a new buffer of a bytes-like object (host memory) Parameters: bytes_like bytes-like object Returns: New buffer representing bytes_like to_bytes() -> bytes# Returns the buffer as bytes (host memory). Returns: bytes of this buffer (data copy) Warning Will always copy data, only use this method for small buffers such as metadata buffers. If possible, use .as_numpy_array() or .as_array_like() instead. class zarr.buffer.gpu.NDBuffer(array: zarr.core.buffer.core.NDArrayLike)# Bases: `zarr.core.buffer.core.NDBuffer` A n-dimensional memory block on the GPU We use NDBuffer throughout Zarr to represent a n-dimensional memory block. A NDBuffer is backed by a underlying ndarray-like instance that represents the memory. The memory type is unspecified; can be regular host memory, CUDA device memory, or something else. The only requirement is that the ndarray-like instance can be copied/converted to a regular Numpy array (host memory). Parameters: array ndarray-like object that is convertible to a regular Numpy array. Notes The two buffer classes Buffer and NDBuffer are very similar. In fact, Buffer is a special case of NDBuffer where dim=1, stride=1, and dtype=”B”. However, in order to use Python’s type system to differentiate between the contiguous Buffer and the n-dim (non-contiguous) NDBuffer, we keep the definition of the two classes separate. all_equal(other: Any, equal_nan: bool = True) -> bool# Compare to other using np.array_equal. as_ndarray_like() -> NDArrayLike# Returns the underlying array (host or device memory) of this buffer This will never copy data. Returns: The underlying array such as a NumPy or CuPy array. as_numpy_array() -> numpy.typing.NDArray[Any]# Returns the buffer as a NumPy array (host memory). Returns: NumPy array of this buffer (might be a data copy) Warning Might have to copy data, consider using .as_ndarray_like() instead. as_scalar() -> ScalarType# Returns the buffer as a scalar value astype( dtype: numpy.typing.DTypeLike, order: Literal['K', 'A', 'C', 'F'] = 'K', ) -> Self# copy() -> Self# classmethod create( *, shape: collections.abc.Iterable[int], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', fill_value: Any | None = None, ) -> Self# Create a new buffer and its underlying ndarray-like object Parameters: shape The shape of the buffer and its underlying ndarray-like object dtype The datatype of the buffer and its underlying ndarray-like object order Whether to store multi-dimensional data in row-major (C-style) or column-major (Fortran-style) order in memory. fill_value If not None, fill the new buffer with a scalar value. Returns: New buffer representing a new ndarray_like object Notes A subclass can overwrite this method to create a ndarray-like object other then the default Numpy array. classmethod empty( shape: tuple[int, Ellipsis], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', ) -> Self# Create an empty buffer with the given shape, dtype, and order. This method can be faster than `NDBuffer.create` because it doesn’t have to initialize the memory used by the underlying ndarray-like object. Parameters: shape The shape of the buffer and its underlying ndarray-like object dtype The datatype of the buffer and its underlying ndarray-like object order Whether to store multi-dimensional data in row-major (C-style) or column-major (Fortran-style) order in memory. Returns: buffer New buffer representing a new ndarray_like object with empty data. See also `NDBuffer.create` Create a new buffer with some initial fill value. fill(value: Any) -> None# classmethod from_ndarray_like(ndarray_like: NDArrayLike) -> Self# Create a new buffer of a ndarray-like object Parameters: ndarray_like ndarray-like object Returns: New buffer representing ndarray_like classmethod from_numpy_array(array_like: numpy.typing.ArrayLike) -> Self# Create a new buffer of Numpy array-like object Parameters: array_like Object that can be coerced into a Numpy array Returns: New buffer representing array_like reshape(newshape: tuple[int, Ellipsis] | Literal[-1]) -> Self# squeeze(axis: tuple[int, Ellipsis]) -> Self# transpose( axes: SupportsIndex | collections.abc.Sequence[SupportsIndex] | None, ) -> Self# property byteorder: zarr.codecs.bytes.Endian# property dtype: numpy.dtype[Any]# property shape: tuple[int, Ellipsis]# zarr.buffer.gpu.buffer_prototype# ##### Functions# `default_buffer_prototype`(→ BufferPrototype) ##### Package Contents# zarr.buffer.default_buffer_prototype() -> BufferPrototype# #### zarr.codecs# ##### Submodules# ###### zarr.codecs.numcodecs# ###### Classes# `Adler32` Base class for bytes-to-bytes codecs. `AsType` Base class for array-to-array codecs. `BZ2` Base class for bytes-to-bytes codecs. `BitRound` Base class for array-to-array codecs. `Blosc` Base class for bytes-to-bytes codecs. `CRC32` Base class for bytes-to-bytes codecs. `CRC32C` Base class for bytes-to-bytes codecs. `Delta` Base class for array-to-array codecs. `FixedScaleOffset` Base class for array-to-array codecs. `Fletcher32` Base class for bytes-to-bytes codecs. `GZip` Base class for bytes-to-bytes codecs. `JenkinsLookup3` Base class for bytes-to-bytes codecs. `LZ4` Base class for bytes-to-bytes codecs. `LZMA` Base class for bytes-to-bytes codecs. `PCodec` Base class for array-to-bytes codecs. `PackBits` Base class for array-to-array codecs. `Quantize` Base class for array-to-array codecs. `Shuffle` Base class for bytes-to-bytes codecs. `ZFPY` Base class for array-to-bytes codecs. `Zlib` Base class for bytes-to-bytes codecs. `Zstd` Base class for bytes-to-bytes codecs. ###### Package Contents# class zarr.codecs.numcodecs.Adler32(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsChecksumCodec` Base class for bytes-to-bytes codecs. compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.AsType(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsArrayArrayCodec` Base class for array-to-array codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> AsType# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.BZ2(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsBytesBytesCodec` Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.BitRound(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsArrayArrayCodec` Base class for array-to-array codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.Blosc(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsBytesBytesCodec` Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.CRC32(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsChecksumCodec` Base class for bytes-to-bytes codecs. compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.CRC32C(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsChecksumCodec` Base class for bytes-to-bytes codecs. compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.Delta(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsArrayArrayCodec` Base class for array-to-array codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.FixedScaleOffset(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsArrayArrayCodec` Base class for array-to-array codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec( array_spec: zarr.core.array_spec.ArraySpec, ) -> FixedScaleOffset# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.Fletcher32(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsChecksumCodec` Base class for bytes-to-bytes codecs. compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.GZip(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsBytesBytesCodec` Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.JenkinsLookup3(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsChecksumCodec` Base class for bytes-to-bytes codecs. compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.LZ4(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsBytesBytesCodec` Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.LZMA(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsBytesBytesCodec` Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.PCodec(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsArrayBytesCodec` Base class for array-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.PackBits(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsArrayArrayCodec` Base class for array-to-array codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate(*, dtype: zarr.dtype.ZDType[Any, Any], **_kwargs: Any) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.Quantize(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsArrayArrayCodec` Base class for array-to-array codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Quantize# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.Shuffle(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsBytesBytesCodec` Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Shuffle# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.ZFPY(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsArrayBytesCodec` Base class for array-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.Zlib(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsBytesBytesCodec` Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# class zarr.codecs.numcodecs.Zstd(**codec_config: zarr.core.common.JSON)# Bases: `_NumcodecsBytesBytesCodec` Base class for bytes-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid codec_config: dict[str, zarr.core.common.JSON]# codec_name: str# is_fixed_size: bool# ##### Classes# `BloscCname` Enum for compression library used by blosc. `BloscCodec` blosc codec `BloscShuffle` Enum for shuffle filter used by blosc. `BytesCodec` bytes codec `Crc32cCodec` crc32c codec `Endian` Enum for endian type used by bytes codec. `GzipCodec` gzip codec `ShardingCodec` Sharding codec `ShardingCodecIndexLocation` Enum for index location used by the sharding codec. `TransposeCodec` Transpose codec `VLenBytesCodec` Base class for array-to-bytes codecs. `VLenUTF8Codec` Variable-length UTF8 codec `ZstdCodec` zstd codec ##### Package Contents# class zarr.codecs.BloscCname(*args, **kwds)# Bases: `enum.Enum` Enum for compression library used by blosc. blosclz = 'blosclz'# lz4 = 'lz4'# lz4hc = 'lz4hc'# snappy = 'snappy'# zlib = 'zlib'# zstd = 'zstd'# class zarr.codecs.BloscCodec( *, typesize: int | None = None, cname: BloscCname | str = BloscCname.zstd, clevel: int = 5, shuffle: BloscShuffle | str | None = None, blocksize: int = 0, )# Bases: `zarr.abc.codec.BytesBytesCodec` blosc codec abstract compute_encoded_size( _input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid blocksize: int = 0# clevel: int = 5# cname: BloscCname# is_fixed_size = False# shuffle: BloscShuffle | None# typesize: int | None# class zarr.codecs.BloscShuffle(*args, **kwds)# Bases: `enum.Enum` Enum for shuffle filter used by blosc. classmethod from_int(num: int) -> BloscShuffle# bitshuffle = 'bitshuffle'# noshuffle = 'noshuffle'# shuffle = 'shuffle'# class zarr.codecs.BytesCodec(*, endian: Endian | str | None = default_system_endian)# Bases: `zarr.abc.codec.ArrayBytesCodec` bytes codec compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid endian: Endian | None# is_fixed_size = True# class zarr.codecs.Crc32cCodec# Bases: `zarr.abc.codec.BytesBytesCodec` crc32c codec compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size = True# class zarr.codecs.Endian(*args, **kwds)# Bases: `enum.Enum` Enum for endian type used by bytes codec. big = 'big'# little = 'little'# class zarr.codecs.GzipCodec(*, level: int = 5)# Bases: `zarr.abc.codec.BytesBytesCodec` gzip codec abstract compute_encoded_size( _input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size = False# level: int = 5# class zarr.codecs.ShardingCodec( *, chunk_shape: zarr.core.common.ShapeLike, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] = (BytesCodec(),), index_codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] = (BytesCodec(), Crc32cCodec()), index_location: ShardingCodecIndexLocation | str = ShardingCodecIndexLocation.end, )# Bases: `zarr.abc.codec.ArrayBytesCodec`, `zarr.abc.codec.ArrayBytesCodecPartialDecodeMixin`, `zarr.abc.codec.ArrayBytesCodecPartialEncodeMixin` Sharding codec compute_encoded_size( input_byte_length: int, shard_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async decode_partial( batch_info: collections.abc.Iterable[tuple[zarr.abc.store.ByteGetter, zarr.core.indexing.SelectorTuple, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[zarr.core.buffer.NDBuffer | None]# Partially decodes a batch of chunks. This method determines parts of a chunk from the slice selection, fetches these parts from the store (via ByteGetter) and decodes them. Parameters: batch_infoIterable[tuple[ByteGetter, SelectorTuple, ArraySpec]] Ordered set of information about slices of encoded chunks. The slice selection determines which parts of the chunk will be fetched. The ByteGetter is used to fetch the necessary bytes. The chunk spec contains information about the construction of an array from the bytes. Returns: Iterable[NDBuffer | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] async encode_partial( batch_info: collections.abc.Iterable[tuple[zarr.abc.store.ByteSetter, zarr.core.buffer.NDBuffer, zarr.core.indexing.SelectorTuple, zarr.core.array_spec.ArraySpec]], ) -> None# Partially encodes a batch of chunks. This method determines parts of a chunk from the slice selection, encodes them and writes these parts to the store (via ByteSetter). If merging with existing chunk data in the store is necessary, this method will read from the store first and perform the merge. Parameters: batch_infoIterable[tuple[ByteSetter, NDBuffer, SelectorTuple, ArraySpec]] Ordered set of information about slices of to-be-encoded chunks. The slice selection determines which parts of the chunk will be encoded. The ByteSetter is used to write the necessary bytes and fetch bytes for existing chunk data. The chunk spec contains information about the chunk. evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid chunk_shape: tuple[int, Ellipsis]# property codec_pipeline: zarr.abc.codec.CodecPipeline# codecs: tuple[zarr.abc.codec.Codec, Ellipsis]# index_codecs: tuple[zarr.abc.codec.Codec, Ellipsis]# index_location: ShardingCodecIndexLocation# is_fixed_size: bool# class zarr.codecs.ShardingCodecIndexLocation(*args, **kwds)# Bases: `enum.Enum` Enum for index location used by the sharding codec. end = 'end'# start = 'start'# class zarr.codecs.TransposeCodec(*, order: collections.abc.Iterable[int])# Bases: `zarr.abc.codec.ArrayArrayCodec` Transpose codec compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size = True# order: tuple[int, Ellipsis]# class zarr.codecs.VLenBytesCodec# Bases: `zarr.abc.codec.ArrayBytesCodec` Base class for array-to-bytes codecs. abstract compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size: bool# class zarr.codecs.VLenUTF8Codec# Bases: `zarr.abc.codec.ArrayBytesCodec` Variable-length UTF8 codec abstract compute_encoded_size( input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid is_fixed_size: bool# class zarr.codecs.ZstdCodec(*, level: int = 0, checksum: bool = False)# Bases: `zarr.abc.codec.BytesBytesCodec` zstd codec abstract compute_encoded_size( _input_byte_length: int, _chunk_spec: zarr.core.array_spec.ArraySpec, ) -> int# Given an input byte length, this method returns the output byte length. Raises a NotImplementedError for codecs with variable-sized outputs (e.g. compressors). Parameters: input_byte_lengthint chunk_specArraySpec Returns: int async decode( chunks_and_specs: collections.abc.Iterable[tuple[CodecOutput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecInput | None]# Decodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecOutput | None, ArraySpec]] Ordered set of encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecInput | None] async encode( chunks_and_specs: collections.abc.Iterable[tuple[CodecInput | None, zarr.core.array_spec.ArraySpec]], ) -> collections.abc.Iterable[CodecOutput | None]# Encodes a batch of chunks. Chunks can be None in which case they are ignored by the codec. Parameters: chunks_and_specsIterable[tuple[CodecInput | None, ArraySpec]] Ordered set of to-be-encoded chunks with their accompanying chunk spec. Returns: Iterable[CodecOutput | None] evolve_from_array_spec(array_spec: zarr.core.array_spec.ArraySpec) -> Self# Fills in codec configuration parameters that can be automatically inferred from the array metadata. Parameters: array_specArraySpec Returns: Self classmethod from_dict(data: dict[str, zarr.core.common.JSON]) -> Self# Create an instance of the model from a dictionary resolve_metadata( chunk_spec: zarr.core.array_spec.ArraySpec, ) -> zarr.core.array_spec.ArraySpec# Computed the spec of the chunk after it has been encoded by the codec. This is important for codecs that change the shape, data type or fill value of a chunk. The spec will then be used for subsequent codecs in the pipeline. Parameters: chunk_specArraySpec Returns: ArraySpec to_dict() -> dict[str, zarr.core.common.JSON]# Recursively serialize this model to a dictionary. This method inspects the fields of self and calls x.to_dict() for any fields that are instances of Metadata. Sequences of Metadata are similarly recursed into, and the output of that recursion is collected in a list. validate( *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar], chunk_grid: zarr.core.chunk_grids.ChunkGrid, ) -> None# Validates that the codec configuration is compatible with the array metadata. Raises errors when the codec configuration is not compatible. Parameters: shapetuple[int, …] The array shape dtypenp.dtype[Any] The array data type chunk_gridChunkGrid The array chunk grid checksum: bool = False# is_fixed_size = True# level: int = 0# #### zarr.convenience# Convenience helpers. Warning This sub-module is deprecated. All functions here are defined in the top level zarr namespace instead. ##### Functions# `consolidate_metadata`(→ zarr.core.group.Group) Consolidate the metadata of all nodes in a hierarchy. `copy`(→ tuple[int, int, int]) Not implemented. `copy_all`(→ tuple[int, int, int]) Not implemented. `copy_store`(→ tuple[int, int, int]) Not implemented. `load`(...) Load data from an array or group into memory. `open`(→ zarr.core.array.Array | zarr.core.group.Group) Open a group or array using file-mode-like semantics. `open_consolidated`(→ zarr.core.group.Group) Alias for `open_group()` with `use_consolidated=True`. `save`(→ None) Save an array or group of arrays to the local file system. `save_array`(→ None) Save a NumPy array to the local file system. `save_group`(→ None) Save several NumPy arrays to the local file system. `tree`(→ Any) Provide a rich display of the hierarchy. ##### Module Contents# zarr.convenience.consolidate_metadata( store: zarr.storage.StoreLike, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, ) -> zarr.core.group.Group# Consolidate the metadata of all nodes in a hierarchy. Upon completion, the metadata of the root node in the Zarr hierarchy will be updated to include all the metadata of child nodes. For Stores that do not use consolidated metadata, this operation raises a TypeError. Parameters: storeStoreLike The store-like object whose metadata you wish to consolidate. pathstr, optional A path to a group in the store to consolidate at. Only children below that group will be consolidated. By default, the root node is used so all the metadata in the store is consolidated. zarr_format{2, 3, None}, optional The zarr format of the hierarchy. By default the zarr format is inferred. Returns: group: Group The group, with the `consolidated_metadata` field set to include the metadata of each child node. If the Store doesn’t support consolidated metadata, this function raises a TypeError. See `Store.supports_consolidated_metadata`. zarr.convenience.copy(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.convenience.copy_all(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.convenience.copy_store(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.convenience.load( store: zarr.storage.StoreLike, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar | dict[str, zarr.core.buffer.NDArrayLikeOrScalar]# Load data from an array or group into memory. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. pathstr or None, optional The path within the store from which to load. Returns: out If the path contains an array, out will be a numpy array. If the path contains a group, out will be a dict-like object where keys are array names and values are numpy arrays. See also `save`, `savez` Notes If loading data from a group of arrays, data will not be immediately loaded into memory. Rather, arrays will be loaded into memory as they are requested. zarr.convenience.open( store: zarr.storage.StoreLike | None = None, *, mode: zarr.core.common.AccessModeLiteral | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> zarr.core.array.Array | zarr.core.group.Group# Open a group or array using file-mode-like semantics. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. mode{‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘}, optional Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-’ means create (fail if exists). If the store is read-only, the default is ‘r’; otherwise, it is ‘a’. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional The path within the store to open. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Additional parameters are passed through to `zarr.creation.open_array()` or `zarr.hierarchy.open_group()`. Returns: zarray or group Return type depends on what exists in the given store. zarr.convenience.open_consolidated( *args: Any, use_consolidated: Literal[True] = True, **kwargs: Any, ) -> zarr.core.group.Group# Alias for `open_group()` with `use_consolidated=True`. zarr.convenience.save( store: zarr.storage.StoreLike, *args: zarr.core.buffer.NDArrayLike, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, **kwargs: Any, ) -> None# Save an array or group of arrays to the local file system. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. *argsndarray NumPy arrays with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional The path within the group where the arrays will be saved. **kwargs NumPy arrays with data to save. zarr.convenience.save_array( store: zarr.storage.StoreLike, arr: zarr.core.buffer.NDArrayLike, *, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> None# Save a NumPy array to the local file system. Follows a similar API to the NumPy save() function. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. arrndarray NumPy array with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. The default is `None`, which will use the default Zarr format defined in the global configuration object. pathstr or None, optional The path within the store where the array will be saved. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Passed through to `create()`, e.g., compressor. zarr.convenience.save_group( store: zarr.storage.StoreLike, *args: zarr.core.buffer.NDArrayLike, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: zarr.core.buffer.NDArrayLike, ) -> None# Save several NumPy arrays to the local file system. Follows a similar API to the NumPy savez()/savez_compressed() functions. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. *argsndarray NumPy arrays with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional Path within the store where the group will be saved. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs NumPy arrays with data to save. zarr.convenience.tree( grp: zarr.core.group.Group, expand: bool | None = None, level: int | None = None, ) -> Any# Provide a rich display of the hierarchy. Deprecated since version 3.0.0: zarr.tree() is deprecated and will be removed in a future release. Use group.tree() instead. Parameters: grpGroup Zarr or h5py group. expandbool, optional Only relevant for HTML representation. If True, tree will be fully expanded. levelint, optional Maximum depth to descend into hierarchy. Returns: TreeRepr A pretty-printable object displaying the hierarchy. #### zarr.creation# Helpers for creating arrays. Warning This sub-module is deprecated. All functions here are defined in the top level zarr namespace instead. ##### Functions# `array`(→ zarr.core.array.Array) Create an array filled with data. `create`(→ zarr.core.array.Array) Create an array. `empty`(→ zarr.core.array.Array) Create an empty array with the specified shape. The contents will be filled with the `empty_like`(→ zarr.core.array.Array) Create an empty array like another array. The contents will be filled with the `full`(→ zarr.core.array.Array) Create an array with a default fill value. `full_like`(→ zarr.core.array.Array) Create a filled array like another array. `ones`(→ zarr.core.array.Array) Create an array with a fill value of one. `ones_like`(→ zarr.core.array.Array) Create an array of ones like another array. `open_array`(→ zarr.core.array.Array) Open an array using file-mode-like semantics. `open_like`(→ zarr.core.array.Array) Open a persistent array like another array. `zeros`(→ zarr.core.array.Array) Create an array with a fill value of zero. `zeros_like`(→ zarr.core.array.Array) Create an array of zeros like another array. ##### Module Contents# zarr.creation.array( data: numpy.typing.ArrayLike | zarr.core.array.Array, **kwargs: Any, ) -> zarr.core.array.Array# Create an array filled with data. Parameters: dataarray_like The data to fill the array with. **kwargs Passed through to `create()`. Returns: arrayArray The new array. zarr.creation.create( shape: tuple[int, Ellipsis] | int, *, chunks: tuple[int, Ellipsis] | int | bool | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, compressor: zarr.core.array.CompressorLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, store: zarr.storage.StoreLike | None = None, synchronizer: Any | None = None, overwrite: bool = False, path: zarr.api.asynchronous.PathLike | None = None, chunk_store: zarr.storage.StoreLike | None = None, filters: collections.abc.Iterable[dict[str, zarr.core.common.JSON] | zarr.abc.numcodec.Numcodec] | None = None, cache_metadata: bool | None = None, cache_attrs: bool | None = None, read_only: bool | None = None, object_codec: zarr.abc.codec.Codec | None = None, dimension_separator: Literal['.', '/'] | None = None, write_empty_chunks: bool | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_shape: tuple[int, Ellipsis] | int | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncoding | tuple[Literal['default'], Literal['.', '/']] | tuple[Literal['v2'], Literal['.', '/']] | None = None, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, config: zarr.core.array_spec.ArrayConfigLike | None = None, **kwargs: Any, ) -> zarr.core.array.Array# Create an array. Parameters: shapeint or tuple of ints Array shape. chunksint or tuple of ints, optional Chunk shape. If True, will be guessed from `shape` and `dtype`. If False, will be set to `shape`, i.e., single chunk for the whole array. If an int, the chunk size in each dimension will be given by the value of `chunks`. Default is True. dtypestr or dtype, optional NumPy dtype. compressorCodec, optional Primary compressor to compress chunk data. Zarr format 2 only. Zarr format 3 arrays should use `codecs` instead. If neither `compressor` nor `filters` are provided, the default compressor `zarr.codecs.ZstdCodec` is used. If `compressor` is set to `None`, no compression is used. fill_valueAny, optional Fill value for the array. order{‘C’, ‘F’}, optional Deprecated in favor of the `config` keyword argument. Pass `{'order': }` to `create` instead of using this parameter. Memory layout to be used within each chunk. If not specified, the `array.order` parameter in the global config will be used. storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. synchronizerobject, optional Array synchronizer. overwritebool, optional If True, delete all pre-existing data in `store` at `path` before creating the array. pathstr, optional Path under which array is stored. chunk_storeStoreLike or None, default=None Separate storage for chunks. If not provided, `store` will be used for storage of both chunks and metadata. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. cache_metadatabool, optional If True, array configuration metadata will be cached for the lifetime of the object. If False, array metadata will be reloaded prior to all data access and modification operations (may incur overhead depending on storage and data access pattern). cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. read_onlybool, optional True if array should be protected against modification. object_codecCodec, optional A codec to encode object arrays, only needed if dtype=object. dimension_separator{‘.’, ‘/’}, optional Separator placed between the dimensions of a chunk. Zarr format 2 only. Zarr format 3 arrays should use `chunk_key_encoding` instead. write_empty_chunksbool, optional Deprecated in favor of the `config` keyword argument. Pass `{'write_empty_chunks': }` to `create` instead of using this parameter. If True, all chunks will be stored regardless of their contents. If False, each chunk is compared to the array’s fill value prior to storing. If a chunk is uniformly equal to the fill value, then that chunk is not be stored, and the store entry for that chunk’s key is deleted. zarr_format{2, 3, None}, optional The Zarr format to use when creating an array. The default is `None`, which instructs Zarr to choose the default Zarr format value defined in the runtime configuration. meta_arrayarray-like, optional Not implemented. attributesdict[str, JSON], optional A dictionary of user attributes to store with the array. chunk_shapeint or tuple of ints, optional The shape of the Array’s chunks (default is None). Zarr format 3 only. Zarr format 2 arrays should use chunks instead. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. Zarr format 3 only. Zarr format 2 arrays should use dimension_separator instead. Default is `("default", "/")`. codecsSequence of Codecs or dicts, optional An iterable of Codec or dict serializations of Codecs. Zarr V3 only. The elements of `codecs` specify the transformation from array values to stored bytes. Zarr format 3 only. Zarr format 2 arrays should use `filters` and `compressor` instead. If no codecs are provided, default codecs will be used based on the data type of the array. For most data types, the default codecs are the tuple `(BytesCodec(), ZstdCodec())`; data types that require a special `zarr.abc.codec.ArrayBytesCodec`, like variable-length strings or bytes, will use the `zarr.abc.codec.ArrayBytesCodec` required for the data type instead of `zarr.codecs.BytesCodec`. dimension_namesIterable[str | None] | None = None An iterable of dimension names. Zarr format 3 only. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. configArrayConfigLike, optional Runtime configuration of the array. If provided, will override the default values from zarr.config.array. Returns: zArray The array. zarr.creation.empty(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an empty array with the specified shape. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. zarr.creation.empty_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an empty array like another array. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. zarr.creation.full( shape: tuple[int, Ellipsis], fill_value: Any, **kwargs: Any, ) -> zarr.core.array.Array# Create an array with a default fill value. Parameters: shapeint or tuple of int Shape of the empty array. fill_valuescalar Fill value. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.creation.full_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create a filled array like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.creation.ones(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an array with a fill value of one. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.creation.ones_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an array of ones like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.creation.open_array( store: zarr.storage.StoreLike | None = None, *, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: zarr.api.asynchronous.PathLike = '', storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> zarr.core.array.Array# Open an array using file-mode-like semantics. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. zarr_version{2, 3, None}, optional The zarr format to use when saving. Deprecated in favor of zarr_format. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr, optional Path in store to array. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Any keyword arguments to pass to `create()`. Returns: AsyncArray The opened array. zarr.creation.open_like( a: zarr.api.asynchronous.ArrayLike, path: str, **kwargs: Any, ) -> zarr.core.array.Array# Open a persistent array like another array. Parameters: aArray The shape and data-type of a define these same attributes of the returned array. pathstr The path to the new array. **kwargs Any keyword arguments to pass to the array constructor. Returns: AsyncArray The opened array. zarr.creation.zeros(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an array with a fill value of zero. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.creation.zeros_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an array of zeros like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. #### zarr.dtype# ##### Attributes# `data_type_registry` ##### Exceptions# `DataTypeValidationError` Inappropriate argument value (of correct type). ##### Classes# `Bool` A Zarr data type for arrays containing booleans. `Complex128` A Zarr data type for arrays containing 64 bit complex floats. `Complex64` A Zarr data type for arrays containing 64 bit complex floats. `DateTime64` A Zarr data type for arrays containing NumPy Datetime64 data. `DateTime64JSON_V2` A wrapper around the JSON representation of the `DateTime64` data type in Zarr V2. `DateTime64JSON_V3` The JSON representation of the `numpy.datetime64` data type in Zarr V3. `FixedLengthUTF32` A Zarr data type for arrays containing fixed-length UTF-32 strings. `FixedLengthUTF32JSON_V2` A wrapper around the JSON representation of the `FixedLengthUTF32` data type in Zarr V2. `FixedLengthUTF32JSON_V3` The JSON representation of the `FixedLengthUTF32` data type in Zarr V3. `Float16` A Zarr data type for arrays containing 16-bit floating point numbers. `Float32` A Zarr data type for arrays containing 32-bit floating point numbers. `Float64` A Zarr data type for arrays containing 64-bit floating point numbers. `Int16` A Zarr data type for arrays containing 16-bit signed integers. `Int32` A Zarr data type for arrays containing 32-bit signed integers. `Int64` A Zarr data type for arrays containing 64-bit signed integers. `Int8` A Zarr data type for arrays containing 8-bit signed integers. `NullTerminatedBytes` A Zarr data type for arrays containing fixed-length null-terminated byte sequences. `NullTerminatedBytesJSON_V3` The JSON representation of the `NullTerminatedBytes` data type in Zarr V3. `NullterminatedBytesJSON_V2` A wrapper around the JSON representation of the `NullTerminatedBytes` data type in Zarr V2. `RawBytes` A Zarr data type for arrays containing fixed-length sequences of raw bytes. `RawBytesJSON_V2` A wrapper around the JSON representation of the `RawBytes` data type in Zarr V2. `RawBytesJSON_V3` The JSON representation of the `RawBytes` data type in Zarr V3. `Structured` A Zarr data type for arrays containing structured scalars, AKA "record arrays". `StructuredJSON_V2` A wrapper around the JSON representation of the `Structured` data type in Zarr V2. `StructuredJSON_V3` A JSON representation of a structured data type in Zarr V3. `TimeDelta64` A Zarr data type for arrays containing NumPy TimeDelta64 data. `TimeDelta64JSON_V2` A wrapper around the JSON representation of the `TimeDelta64` data type in Zarr V2. `TimeDelta64JSON_V3` The JSON representation of the `TimeDelta64` data type in Zarr V3. `UInt16` A Zarr data type for arrays containing 16-bit unsigned integers. `UInt32` A Zarr data type for arrays containing 32-bit unsigned integers. `UInt64` A Zarr data type for arrays containing 64-bit unsigned integers. `UInt8` A Zarr data type for arrays containing 8-bit unsigned integers. `VariableLengthBytes` A Zarr data type for arrays containing variable-length sequences of bytes. `VariableLengthBytesJSON_V2` A wrapper around the JSON representation of the `VariableLengthBytes` data type in Zarr V2. `VariableLengthUTF8` A Zarr data type for arrays containing variable-length UTF-8 strings. `VariableLengthUTF8JSON_V2` A wrapper around the JSON representation of the `VariableLengthUTF8` data type in Zarr V2. `ZDType` Abstract base class for wrapping native array data types, e.g. numpy dtypes ##### Functions# `parse_dtype`(→ wrapper.ZDType[wrapper.TBaseDType, ...) Convert the input as a ZDType. ##### Module Contents# exception zarr.dtype.DataTypeValidationError# Bases: `ValueError` Inappropriate argument value (of correct type). class zarr.dtype.Bool# Bases: `zarr.core.dtype.wrapper.ZDType`[`numpy.dtypes.BoolDType`, `numpy.bool_`], `zarr.core.dtype.common.HasItemSize` A Zarr data type for arrays containing booleans. Wraps the `np.dtypes.BoolDType` data type. Scalars for this data type are instances of `np.bool_`. Attributes: _zarr_v3_nameLiteral[“bool”] = “bool” The Zarr v3 name of the dtype. _zarr_v2_name`Literal["|b1"]` = `"|b1"` The Zarr v2 name of the dtype, which is also a string representation of the boolean dtype used by NumPy. dtype_clsClassVar[type[np.dtypes.BoolDType]] = np.dtypes.BoolDType The NumPy dtype class. References This class implements the boolean data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> numpy.bool_# Cast the input to a numpy boolean scalar. Parameters: dataobject The data to cast. Returns: `np.bool_` The numpy boolean scalar. Raises: TypeError If the input cannot be converted to a numpy boolean. default_scalar() -> numpy.bool_# Get the default value for the boolean dtype. Returns: `np.bool_` The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> numpy.bool_# Read a JSON-serializable value as a numpy boolean scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: `np.bool_` The numpy boolean scalar. Raises: TypeError If the input is not a valid boolean type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of Bool from an instance of np.dtypes.BoolDType. Parameters: dtypeTBaseDType The NumPy boolean dtype instance to convert. Returns: Bool An instance of Bool. Raises: DataTypeValidationError If the provided dtype is not compatible with this ZDType. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['|b1'], None]# to_json(zarr_format: Literal[3]) -> Literal['bool'] Serialize this Bool instance to JSON. Parameters: zarr_formatZarrFormat The Zarr format version (2 or 3). Returns: `DTypeConfig_V2[Literal["|b1"], None] | Literal["bool"]` The JSON representation of the Bool instance. Raises: ValueError If the zarr_format is not 2 or 3. to_json_scalar( data: object, *, zarr_format: zarr.core.common.ZarrFormat, ) -> bool# Convert a scalar to a python bool. Parameters: dataobject The value to convert. zarr_formatZarrFormat The zarr format version. Returns: bool The JSON-serializable format. to_native_dtype() -> numpy.dtypes.BoolDType# Create a NumPy boolean dtype instance from this ZDType. Returns: np.dtypes.BoolDType The NumPy boolean dtype. dtype_cls# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.Complex128# Bases: `BaseComplex`[`numpy.dtypes.Complex128DType`, `numpy.complex128`], `zarr.core.dtype.common.HasEndianness` A Zarr data type for arrays containing 64 bit complex floats. Wraps the `np.dtypes.Complex128DType` data type. Scalars for this data type are instances of `np.complex128`. Attributes: dtype_clsType[np.dtypes.Complex128DType] The numpy dtype class for this data type. _zarr_v3_nameClassVar[Literal[“complex128”]] The name of this data type in Zarr V3. _zarr_v2_namesClassVar[tuple[Literal[“>c16”], Literal[“ zarr.core.dtype.npy.common.TComplexScalar_co# Attempt to cast a given object to a numpy complex scalar. Parameters: dataobject The data to be cast to a numpy complex scalar. Returns: TComplexScalar_co The data cast as a numpy complex scalar. Raises: TypeError If the data cannot be converted to a numpy complex scalar. default_scalar() -> zarr.core.dtype.npy.common.TComplexScalar_co# Get the default value, which is 0 cast to this dtype Returns: Int scalar The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> zarr.core.dtype.npy.common.TComplexScalar_co# Read a JSON-serializable value as a numpy float. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: TScalar_co The numpy float. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this data type from a NumPy complex dtype. Parameters: dtypeTBaseDType The native dtype to convert. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the dtype is not compatible with this data type. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[str, None]# to_json(zarr_format: Literal[3]) -> str Serialize this object to a JSON-serializable representation. Parameters: zarr_formatZarrFormat The Zarr format version. Supported values are 2 and 3. Returns: DTypeConfig_V2[str, None] | str If `zarr_format` is 2, a dictionary with `"name"` and `"object_codec_id"` keys is returned. If `zarr_format` is 3, a string representation of the complex data type is returned. Raises: ValueError If zarr_format is not 2 or 3. to_json_scalar( data: object, *, zarr_format: zarr.core.common.ZarrFormat, ) -> zarr.core.common.JSON# Convert an object to a JSON-serializable float. Parameters: data_BaseScalar The value to convert. zarr_formatZarrFormat The zarr format version. Returns: JSON The JSON-serializable form of the complex number, which is a list of two floats, each of which is encoding according to a zarr-format-specific encoding. to_native_dtype() -> zarr.core.dtype.npy.common.TComplexDType_co# Convert this class to a NumPy complex dtype with the appropriate byte order. Returns: TComplexDType_co A NumPy data type object representing the complex data type with the specified byte order. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.Complex64# Bases: `BaseComplex`[`numpy.dtypes.Complex64DType`, `numpy.complex64`] A Zarr data type for arrays containing 64 bit complex floats. Wraps the `np.dtypes.Complex64DType` data type. Scalars for this data type are instances of `np.complex64`. Attributes: dtype_clsType[np.dtypes.Complex64DType] The numpy dtype class for this data type. _zarr_v3_nameClassVar[Literal[“complex64”]] The name of this data type in Zarr V3. _zarr_v2_namesClassVar[tuple[Literal[“>c8”], Literal[“ zarr.core.dtype.npy.common.TComplexScalar_co# Attempt to cast a given object to a numpy complex scalar. Parameters: dataobject The data to be cast to a numpy complex scalar. Returns: TComplexScalar_co The data cast as a numpy complex scalar. Raises: TypeError If the data cannot be converted to a numpy complex scalar. default_scalar() -> zarr.core.dtype.npy.common.TComplexScalar_co# Get the default value, which is 0 cast to this dtype Returns: Int scalar The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> zarr.core.dtype.npy.common.TComplexScalar_co# Read a JSON-serializable value as a numpy float. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: TScalar_co The numpy float. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this data type from a NumPy complex dtype. Parameters: dtypeTBaseDType The native dtype to convert. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the dtype is not compatible with this data type. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[str, None]# to_json(zarr_format: Literal[3]) -> str Serialize this object to a JSON-serializable representation. Parameters: zarr_formatZarrFormat The Zarr format version. Supported values are 2 and 3. Returns: DTypeConfig_V2[str, None] | str If `zarr_format` is 2, a dictionary with `"name"` and `"object_codec_id"` keys is returned. If `zarr_format` is 3, a string representation of the complex data type is returned. Raises: ValueError If zarr_format is not 2 or 3. to_json_scalar( data: object, *, zarr_format: zarr.core.common.ZarrFormat, ) -> zarr.core.common.JSON# Convert an object to a JSON-serializable float. Parameters: data_BaseScalar The value to convert. zarr_formatZarrFormat The zarr format version. Returns: JSON The JSON-serializable form of the complex number, which is a list of two floats, each of which is encoding according to a zarr-format-specific encoding. to_native_dtype() -> zarr.core.dtype.npy.common.TComplexDType_co# Convert this class to a NumPy complex dtype with the appropriate byte order. Returns: TComplexDType_co A NumPy data type object representing the complex data type with the specified byte order. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.DateTime64# Bases: `TimeDTypeBase`[`numpy.dtypes.DateTime64DType`, `numpy.datetime64`], `zarr.core.dtype.common.HasEndianness` A Zarr data type for arrays containing NumPy Datetime64 data. Wraps the `np.dtypes.TimeDelta64DType` data type. Scalars for this data type are instances of `np.datetime64`. Attributes: dtype_clsType[np.dtypesTimeDelta64DType] The numpy dtype class for this data type. unitDateTimeUnit The unit of time for this data type. scale_factorint The scale factor for the time unit. References The Zarr V2 representation of this data type is defined in the Zarr V2 specification document. The Zarr V3 representation of this data type is defined in the `numpy.datetime64` specification document cast_scalar(data: object) -> numpy.datetime64# Cast the input to a scalar of this data type after a type check. Parameters: dataobject The scalar value to cast. Returns: numpy.datetime64 The input cast to a NumPy datetime scalar. Raises: TypeError If the data cannot be converted to a numpy datetime scalar. default_scalar() -> numpy.datetime64# Return the default scalar value for this data type. Returns: numpy.datetime64 The default scalar value, which is a ‘Not-a-Time’ (NaT) value classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> numpy.datetime64# Read a JSON-serializable value as a scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: numpy.datetime64 The numpy datetime scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this class from a native NumPy data type. Parameters: dtypeTBaseDType The native NumPy dtype to convert. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the dtype is not a valid representation of this class. to_json(zarr_format: Literal[2]) -> DateTime64JSON_V2# to_json(zarr_format: Literal[3]) -> DateTime64JSON_V3 Serialize this data type to JSON. Parameters: zarr_formatZarrFormat The Zarr format version (2 or 3). Returns: DateTime64JSON_V2 | DateTime64JSON_V3 The JSON representation of the data type. Raises: ValueError If the zarr_format is not 2 or 3. to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> int# Convert a python object to a JSON representation of a datetime64 or timedelta64 scalar. Parameters: dataobject The python object to convert. zarr_formatZarrFormat The Zarr format version (2 or 3). Returns: int The JSON representation of the scalar. to_native_dtype() -> BaseTimeDType_co# Convert this data type to a NumPy temporal data type with the appropriate unit and scale factor. Returns: BaseTimeDType_co A NumPy data type object representing the time data type with the specified unit, scale factor, and byte order. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. scale_factor: int = 1# unit: zarr.core.dtype.npy.common.DateTimeUnit = 'generic'# class zarr.dtype.DateTime64JSON_V2# Bases: `zarr.core.dtype.common.DTypeConfig_V2`[`str`, `None`] A wrapper around the JSON representation of the `DateTime64` data type in Zarr V2. The `name` field of this class contains the value that would appear under the `dtype` field in Zarr V2 array metadata. References The structure of the `name` field is defined in the Zarr V2 specification document. Examples { "name": " numpy.str_# Cast the scalar value to the native scalar value. Parameters: dataobject The scalar value. Returns: `np.str_` The native scalar value. default_scalar() -> numpy.str_# Return the default scalar value for this data type. Returns: `np.str_` The default scalar value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> numpy.str_# Convert the JSON representation of a scalar value to the native scalar value. Parameters: dataJSON The JSON data. zarr_formatZarrFormat The Zarr format to use. Returns: `np.str_` The native scalar value. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create a FixedLengthUTF32 from a NumPy data type. Parameters: dtypeTBaseDType The NumPy data type. Returns: Self An instance of this data type. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[str, None]# to_json(zarr_format: Literal[3]) -> FixedLengthUTF32JSON_V3 Convert the FixedLengthUTF32 instance to a JSON representation. Parameters: zarr_formatZarrFormat The Zarr format to use. Returns: DTypeConfig_V2[str, None] | FixedLengthUTF32JSON_V3 The JSON representation of the data type. to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> str# Convert the scalar value to a JSON representation. Parameters: dataobject The scalar value. zarr_formatZarrFormat The Zarr format to use. Returns: str The JSON representation of the scalar value. to_native_dtype() -> numpy.dtypes.StrDType[int]# Convert the FixedLengthUTF32 instance to a NumPy data type. Returns: np.dtypes.StrDType[int] The NumPy data type. code_point_bytes: ClassVar[int] = 4# dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. length: int# class zarr.dtype.FixedLengthUTF32JSON_V2# Bases: `zarr.core.dtype.common.DTypeConfig_V2`[`str`, `None`] A wrapper around the JSON representation of the `FixedLengthUTF32` data type in Zarr V2. The `name` field of this class contains the value that would appear under the `dtype` field in Zarr V2 array metadata. References The structure of the `name` field is defined in the Zarr V2 specification document. Examples { "name": " zarr.core.dtype.npy.common.TFloatScalar_co# Cast a scalar value to a NumPy float scalar. Parameters: dataobject The scalar value to cast. Returns: TFloatScalar_co The NumPy float scalar. default_scalar() -> zarr.core.dtype.npy.common.TFloatScalar_co# Get the default value, which is 0 cast to this zdtype. Returns: TFloatScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> zarr.core.dtype.npy.common.TFloatScalar_co# Read a JSON-serializable value as a NumPy float scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: TFloatScalar_co The NumPy float scalar. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this ZDType from a NumPy data type. Parameters: dtypeTBaseDType The NumPy data type. Returns: Self An instance of this data type. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[str, None]# to_json(zarr_format: Literal[3]) -> str Convert the wrapped data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The zarr format version. Returns: DTypeConfig_V2[str, None] or str The JSON-serializable representation of the wrapped data type. Raises: ValueError If zarr_format is not 2 or 3. to_json_scalar( data: object, *, zarr_format: zarr.core.common.ZarrFormat, ) -> float | str# Convert an object to a JSON-serializable float. Parameters: data_BaseScalar The value to convert. zarr_formatZarrFormat The zarr format version. Returns: JSON The JSON-serializable form of the float, which is potentially a number or a string. See the zarr specifications for details on the JSON encoding for floats. to_native_dtype() -> zarr.core.dtype.npy.common.TFloatDType_co# Convert the wrapped data type to a NumPy data type. Returns: TFloatDType_co The NumPy data type. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.Float32# Bases: `BaseFloat`[`numpy.dtypes.Float32DType`, `numpy.float32`] A Zarr data type for arrays containing 32-bit floating point numbers. Wraps the `np.dtypes.Float32DType` data type. Scalars for this data type are instances of `np.float32`. Attributes: dtype_clsType[np.dtypes.Float32DType] The NumPy dtype class for this data type. References This class implements the float32 data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> zarr.core.dtype.npy.common.TFloatScalar_co# Cast a scalar value to a NumPy float scalar. Parameters: dataobject The scalar value to cast. Returns: TFloatScalar_co The NumPy float scalar. default_scalar() -> zarr.core.dtype.npy.common.TFloatScalar_co# Get the default value, which is 0 cast to this zdtype. Returns: TFloatScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> zarr.core.dtype.npy.common.TFloatScalar_co# Read a JSON-serializable value as a NumPy float scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: TFloatScalar_co The NumPy float scalar. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this ZDType from a NumPy data type. Parameters: dtypeTBaseDType The NumPy data type. Returns: Self An instance of this data type. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[str, None]# to_json(zarr_format: Literal[3]) -> str Convert the wrapped data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The zarr format version. Returns: DTypeConfig_V2[str, None] or str The JSON-serializable representation of the wrapped data type. Raises: ValueError If zarr_format is not 2 or 3. to_json_scalar( data: object, *, zarr_format: zarr.core.common.ZarrFormat, ) -> float | str# Convert an object to a JSON-serializable float. Parameters: data_BaseScalar The value to convert. zarr_formatZarrFormat The zarr format version. Returns: JSON The JSON-serializable form of the float, which is potentially a number or a string. See the zarr specifications for details on the JSON encoding for floats. to_native_dtype() -> zarr.core.dtype.npy.common.TFloatDType_co# Convert the wrapped data type to a NumPy data type. Returns: TFloatDType_co The NumPy data type. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.Float64# Bases: `BaseFloat`[`numpy.dtypes.Float64DType`, `numpy.float64`] A Zarr data type for arrays containing 64-bit floating point numbers. Wraps the `np.dtypes.Float64DType` data type. Scalars for this data type are instances of `np.float64`. Attributes: dtype_clsType[np.dtypes.Float64DType] The NumPy dtype class for this data type. References This class implements the float64 data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> zarr.core.dtype.npy.common.TFloatScalar_co# Cast a scalar value to a NumPy float scalar. Parameters: dataobject The scalar value to cast. Returns: TFloatScalar_co The NumPy float scalar. default_scalar() -> zarr.core.dtype.npy.common.TFloatScalar_co# Get the default value, which is 0 cast to this zdtype. Returns: TFloatScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> zarr.core.dtype.npy.common.TFloatScalar_co# Read a JSON-serializable value as a NumPy float scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: TFloatScalar_co The NumPy float scalar. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this ZDType from a NumPy data type. Parameters: dtypeTBaseDType The NumPy data type. Returns: Self An instance of this data type. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[str, None]# to_json(zarr_format: Literal[3]) -> str Convert the wrapped data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The zarr format version. Returns: DTypeConfig_V2[str, None] or str The JSON-serializable representation of the wrapped data type. Raises: ValueError If zarr_format is not 2 or 3. to_json_scalar( data: object, *, zarr_format: zarr.core.common.ZarrFormat, ) -> float | str# Convert an object to a JSON-serializable float. Parameters: data_BaseScalar The value to convert. zarr_formatZarrFormat The zarr format version. Returns: JSON The JSON-serializable form of the float, which is potentially a number or a string. See the zarr specifications for details on the JSON encoding for floats. to_native_dtype() -> zarr.core.dtype.npy.common.TFloatDType_co# Convert the wrapped data type to a NumPy data type. Returns: TFloatDType_co The NumPy data type. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.Int16# Bases: `BaseInt`[`numpy.dtypes.Int16DType`, `numpy.int16`], `zarr.core.dtype.common.HasEndianness` A Zarr data type for arrays containing 16-bit signed integers. Wraps the `np.dtypes.Int16DType` data type. Scalars for this data type are instances of `np.int16`. Attributes: dtype_clsnp.dtypes.Int16DType The class of the underlying NumPy dtype. References This class implements the 16-bit signed integer data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> TIntScalar_co# Attempt to cast a given object to a NumPy integer scalar. Parameters: dataobject The data to be cast to a NumPy integer scalar. Returns: TIntScalar_co The data cast as a NumPy integer scalar. Raises: TypeError If the data cannot be converted to a NumPy integer scalar. default_scalar() -> TIntScalar_co# Get the default value, which is 0 cast to this dtype. Returns: TIntScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TIntScalar_co# Read a JSON-serializable value as a NumPy int scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The Zarr format version. Returns: TIntScalar_co The NumPy int scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this data type from a np.dtype(‘int16’) instance. Parameters: dtypenp.dtype The instance of np.dtype(‘int16’) to create from. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the input data type is not an instance of np.dtype(‘int16’). to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['>i2', ' Literal['int16'] Serialize this ZDType to v2- or v3-flavored JSON Parameters: zarr_formatZarrFormat The Zarr format version (2 or 3). Returns: DTypeConfig_V2[Literal[“>i2”, “ int# Convert an object to a JSON serializable scalar. For the integer data types, the JSON form is a plain integer. Parameters: dataobject The value to convert. zarr_formatZarrFormat The Zarr format version. Returns: int The JSON-serializable form of the scalar. to_native_dtype() -> numpy.dtypes.Int16DType# Convert the data type to a np.dtype(‘int16’) instance. Returns: np.dtype The np.dtype(‘int16’) instance. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.Int32# Bases: `BaseInt`[`numpy.dtypes.Int32DType`, `numpy.int32`], `zarr.core.dtype.common.HasEndianness` A Zarr data type for arrays containing 32-bit signed integers. Wraps the `np.dtypes.Int32DType` data type. Scalars for this data type are instances of `np.int32`. Attributes: dtype_clsnp.dtypes.Int32DType The class of the underlying NumPy dtype. References This class implements the 32-bit signed integer data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> TIntScalar_co# Attempt to cast a given object to a NumPy integer scalar. Parameters: dataobject The data to be cast to a NumPy integer scalar. Returns: TIntScalar_co The data cast as a NumPy integer scalar. Raises: TypeError If the data cannot be converted to a NumPy integer scalar. default_scalar() -> TIntScalar_co# Get the default value, which is 0 cast to this dtype. Returns: TIntScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TIntScalar_co# Read a JSON-serializable value as a NumPy int scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The Zarr format version. Returns: TIntScalar_co The NumPy int scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an Int32 from a np.dtype(‘int32’) instance. Parameters: dtypeTBaseDType The np.dtype(‘int32’) instance. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the input JSON is not a valid representation of this class Int32. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['>i4', ' Literal['int32'] Serialize this ZDType to v2- or v3-flavored JSON Parameters: zarr_formatZarrFormat The Zarr format version (2 or 3). Returns: DTypeConfig_V2[Literal[“>i4”, “ int# Convert an object to a JSON serializable scalar. For the integer data types, the JSON form is a plain integer. Parameters: dataobject The value to convert. zarr_formatZarrFormat The Zarr format version. Returns: int The JSON-serializable form of the scalar. to_native_dtype() -> numpy.dtypes.Int32DType# Convert the Int32 instance to a np.dtype(‘int32’) instance. Returns: np.dtypes.Int32DType The np.dtype(‘int32’) instance. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.Int64# Bases: `BaseInt`[`numpy.dtypes.Int64DType`, `numpy.int64`], `zarr.core.dtype.common.HasEndianness` A Zarr data type for arrays containing 64-bit signed integers. Wraps the `np.dtypes.Int64DType` data type. Scalars for this data type are instances of `np.int64`. Attributes: dtype_clsnp.dtypes.Int64DType The class of the underlying NumPy dtype. References This class implements the 64-bit signed integer data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> TIntScalar_co# Attempt to cast a given object to a NumPy integer scalar. Parameters: dataobject The data to be cast to a NumPy integer scalar. Returns: TIntScalar_co The data cast as a NumPy integer scalar. Raises: TypeError If the data cannot be converted to a NumPy integer scalar. default_scalar() -> TIntScalar_co# Get the default value, which is 0 cast to this dtype. Returns: TIntScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TIntScalar_co# Read a JSON-serializable value as a NumPy int scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The Zarr format version. Returns: TIntScalar_co The NumPy int scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an Int64 from a np.dtype(‘int64’) instance. Parameters: dtypeTBaseDType The NumPy data type. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the input data type is not a valid representation of this class 64-bit signed integer. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['>i8', ' Literal['int64'] Convert the data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The Zarr format version. Returns: DTypeConfig_V2[Literal[“>i8”, “ int# Convert an object to a JSON serializable scalar. For the integer data types, the JSON form is a plain integer. Parameters: dataobject The value to convert. zarr_formatZarrFormat The Zarr format version. Returns: int The JSON-serializable form of the scalar. to_native_dtype() -> numpy.dtypes.Int64DType# Create a NumPy signed 64-bit integer dtype instance from this Int64 ZDType. Returns: np.dtypes.Int64DType The NumPy signed 64-bit integer dtype. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.Int8# Bases: `BaseInt`[`numpy.dtypes.Int8DType`, `numpy.int8`] A Zarr data type for arrays containing 8-bit signed integers. Wraps the `np.dtypes.Int8DType` data type. Scalars for this data type are instances of `np.int8`. Attributes: dtype_clsnp.dtypes.Int8DType The class of the underlying NumPy dtype. References This class implements the 8-bit signed integer data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> TIntScalar_co# Attempt to cast a given object to a NumPy integer scalar. Parameters: dataobject The data to be cast to a NumPy integer scalar. Returns: TIntScalar_co The data cast as a NumPy integer scalar. Raises: TypeError If the data cannot be converted to a NumPy integer scalar. default_scalar() -> TIntScalar_co# Get the default value, which is 0 cast to this dtype. Returns: TIntScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TIntScalar_co# Read a JSON-serializable value as a NumPy int scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The Zarr format version. Returns: TIntScalar_co The NumPy int scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an Int8 from a np.dtype(‘int8’) instance. Parameters: dtypeTBaseDType The np.dtype(‘int8’) instance. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the input data type is not a valid representation of this class Int8. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['|i1'], None]# to_json(zarr_format: Literal[3]) -> Literal['int8'] Convert the data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The Zarr format version. Returns: `DTypeConfig_V2[Literal["|i1"], None] | Literal["int8"]` The JSON-serializable representation of the data type. Raises: ValueError If the zarr_format is not 2 or 3. to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> int# Convert an object to a JSON serializable scalar. For the integer data types, the JSON form is a plain integer. Parameters: dataobject The value to convert. zarr_formatZarrFormat The Zarr format version. Returns: int The JSON-serializable form of the scalar. to_native_dtype() -> numpy.dtypes.Int8DType# Convert the Int8 instance to a np.dtype(‘int8’) instance. Returns: np.dtypes.Int8DType The np.dtype(‘int8’) instance. dtype_cls# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.NullTerminatedBytes# Bases: `zarr.core.dtype.wrapper.ZDType`[`numpy.dtypes.BytesDType`[`int`], `numpy.bytes_`], `zarr.core.dtype.common.HasLength`, `zarr.core.dtype.common.HasItemSize` A Zarr data type for arrays containing fixed-length null-terminated byte sequences. Wraps the `np.dtypes.BytesDType` data type. Scalars for this data type are instances of `np.bytes_`. This data type is parametrized by an integral length which specifies size in bytes of each scalar. Because this data type uses null-terminated semantics, indexing into NumPy arrays with this data type may return fewer than `length` bytes. Attributes: dtype_cls: ClassVar[type[np.dtypes.BytesDType[int]]] = np.dtypes.BytesDType The NumPy data type wrapped by this ZDType. _zarr_v3_nameClassVar[Literal[“null_terminated_bytes”]] lengthint The length of the bytes. Notes This data type is designed for compatibility with NumPy arrays that use the NumPy `bytes` data type. It may not be desirable for usage outside of that context. If compatibility with the NumPy `bytes` data type is not essential, consider using the `RawBytes` or `VariableLengthBytes` data types instead. cast_scalar(data: object) -> numpy.bytes_# Attempt to cast a given object to a NumPy bytes scalar. This method first checks if the provided data is a valid scalar that can be converted to a NumPy bytes scalar. If the check succeeds, the unchecked casting operation is performed. If the data is not valid, a TypeError is raised. Parameters: dataobject The data to be cast to a NumPy bytes scalar. Returns: `np.bytes_` The data cast as a NumPy bytes scalar. Raises: TypeError If the data cannot be converted to a NumPy bytes scalar. default_scalar() -> numpy.bytes_# Return a default scalar value, which for this data type is an empty byte string. Returns: `np.bytes_` The default scalar value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> numpy.bytes_# Read a JSON-serializable value as `np.bytes_`. Parameters: dataJSON The JSON-serializable base64-encoded string. zarr_formatZarrFormat The zarr format version. Returns: `np.bytes_` The NumPy bytes scalar obtained from decoding the base64 string. Raises: TypeError If the input data is not a base64-encoded string. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of NullTerminatedBytes from an instance of np.dtypes.BytesDType. This method checks if the provided data type is an instance of np.dtypes.BytesDType. If so, it returns a new instance of NullTerminatedBytes with a length equal to the length of input data type. Parameters: dtypeTBaseDType The native dtype to convert. Returns: NullTerminatedBytes An instance of NullTerminatedBytes with the specified length. Raises: DataTypeValidationError If the dtype is not compatible with NullTerminatedBytes. to_json(zarr_format: Literal[2]) -> NullterminatedBytesJSON_V2# to_json(zarr_format: Literal[3]) -> NullTerminatedBytesJSON_V3 Generate a JSON representation of this data type. Parameters: zarr_formatZarrFormat The zarr format version. Returns: NullterminatedBytesJSON_V2 | NullTerminatedBytesJSON_V3 The JSON-serializable representation of the data type to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> str# Convert a scalar to a JSON-serializable string representation. This method encodes the given scalar as a NumPy bytes scalar and then encodes the bytes as a base64-encoded string. Parameters: dataobject The scalar to convert. zarr_formatZarrFormat The zarr format version. Returns: str A string representation of the scalar. to_native_dtype() -> numpy.dtypes.BytesDType[int]# Create a NumPy bytes dtype from this NullTerminatedBytes ZDType. Returns: np.dtypes.BytesDType[int] A NumPy data type object representing null-terminated bytes with a specified length. dtype_cls# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. length: int# class zarr.dtype.NullTerminatedBytesJSON_V3# Bases: `zarr.core.common.NamedConfig`[`Literal`[`'null_terminated_bytes'`], `FixedLengthBytesConfig`] The JSON representation of the `NullTerminatedBytes` data type in Zarr V3. References This representation is not currently defined in an external specification. Examples { "name": "null_terminated_bytes", "configuration": { "length_bytes": 12 } } class zarr.dtype.NullterminatedBytesJSON_V2# Bases: `zarr.core.dtype.common.DTypeConfig_V2`[`str`, `None`] A wrapper around the JSON representation of the `NullTerminatedBytes` data type in Zarr V2. The `name` field of this class contains the value that would appear under the `dtype` field in Zarr V2 array metadata. References The structure of the `name` field is defined in the Zarr V2 specification document. Examples { "name": "|S10", "object_codec_id": None } class zarr.dtype.RawBytes# Bases: `zarr.core.dtype.wrapper.ZDType`[`numpy.dtypes.VoidDType`[`int`], `numpy.void`], `zarr.core.dtype.common.HasLength`, `zarr.core.dtype.common.HasItemSize` A Zarr data type for arrays containing fixed-length sequences of raw bytes. Wraps the NumPy `void` data type. Scalars for this data type are instances of `np.void`. This data type is parametrized by an integral length which specifies size in bytes of each scalar belonging to this data type. Attributes: dtype_cls: ClassVar[type[np.dtypes.VoidDType[int]]] = np.dtypes.VoidDtype The NumPy data type wrapped by this ZDType. _zarr_v3_nameClassVar[Literal[“raw_bytes”]] lengthint The length of the bytes. Notes Although the NumPy “Void” data type is used to create “structured” data types in NumPy, this class does not support structured data types. See the `Structured` data type for this functionality. cast_scalar(data: object) -> numpy.void# Attempt to cast a given object to a NumPy void scalar. This method first checks if the provided data is a valid scalar that can be converted to a NumPy void scalar. If the check succeeds, the unchecked casting operation is performed. If the data is not valid, a TypeError is raised. Parameters: dataobject The data to be cast to a NumPy void scalar. Returns: np.void The data cast as a NumPy void scalar. Raises: TypeError If the data cannot be converted to a NumPy void scalar. default_scalar() -> numpy.void# Return the default scalar value for this data type. The default scalar is a NumPy void scalar of the same length as the data type, filled with zero bytes. Returns: np.void The default scalar value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> numpy.void# Read a JSON-serializable value as a np.void. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: np.void The NumPy void scalar. Raises: TypeError If the data is not a string, or if the string is not a valid base64 encoding. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of RawBytes from an instance of np.dtypes.VoidDType. This method checks if the provided data type is compatible with RawBytes. The input must be an instance of np.dtypes.VoidDType, and have no fields. If the input is compatible, this method returns an instance of RawBytes with the specified length. Parameters: dtypeTBaseDType The native dtype to convert. Returns: RawBytes An instance of RawBytes with the specified length. Raises: DataTypeValidationError If the dtype is not compatible with RawBytes. to_json(zarr_format: Literal[2]) -> RawBytesJSON_V2# to_json(zarr_format: Literal[3]) -> RawBytesJSON_V3 Generate a JSON representation of this data type. Parameters: zarr_formatZarrFormat The zarr format version. Returns: RawBytesJSON_V2 | RawBytesJSON_V3 The JSON-serializable representation of the data type. to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> str# Convert a scalar to a JSON-serializable string representation. This method converts the given scalar to bytes and then encodes the bytes as a base64-encoded string. Parameters: dataobject The scalar to convert. zarr_formatZarrFormat The zarr format version. Returns: str A string representation of the scalar. to_native_dtype() -> numpy.dtypes.VoidDType[int]# Create a NumPy void dtype from this RawBytes ZDType. Returns: np.dtypes.VoidDType[int] A NumPy data type object representing raw bytes with a specified length. dtype_cls# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. length: int# class zarr.dtype.RawBytesJSON_V2# Bases: `zarr.core.dtype.common.DTypeConfig_V2`[`str`, `None`] A wrapper around the JSON representation of the `RawBytes` data type in Zarr V2. The `name` field of this class contains the value that would appear under the `dtype` field in Zarr V2 array metadata. References The structure of the `name` field is defined in the Zarr V2 specification document. Examples { "name": "|V10", "object_codec_id": None } name: typing_extensions.ReadOnly[TDTypeNameV2_co]# object_codec_id: typing_extensions.ReadOnly[TObjectCodecID_co]# class zarr.dtype.RawBytesJSON_V3# Bases: `zarr.core.common.NamedConfig`[`Literal`[`'raw_bytes'`], `FixedLengthBytesConfig`] The JSON representation of the `RawBytes` data type in Zarr V3. References This representation is not currently defined in an external specification. Examples { "name": "raw_bytes", "configuration": { "length_bytes": 12 configuration: typing_extensions.ReadOnly[TConfig]# The configuration of the object. name: typing_extensions.ReadOnly[TName]# The name of the object. class zarr.dtype.Structured# Bases: `zarr.core.dtype.wrapper.ZDType`[`numpy.dtypes.VoidDType`[`int`], `numpy.void`], `zarr.core.dtype.common.HasItemSize` A Zarr data type for arrays containing structured scalars, AKA “record arrays”. Wraps the NumPy np.dtypes.VoidDType if the data type has fields. Scalars for this data type are instances of np.void, with a `fields` attribute. Attributes: fieldsSequence[tuple[str, ZDType]] The fields of the structured dtype. References This data type does not have a Zarr V3 specification. The Zarr V2 data type specification can be found here. cast_scalar(data: object) -> numpy.void# Cast a Python object to a NumPy structured scalar. This function attempts to cast the provided data to a NumPy structured scalar. If the data is compatible with the structured scalar type, it is cast without type checking. Otherwise, a TypeError is raised. Parameters: dataobject The data to be cast to a NumPy structured scalar. Returns: np.void The data cast as a NumPy structured scalar. Raises: TypeError If the data cannot be converted to a NumPy structured scalar. default_scalar() -> numpy.void# Get the default scalar value for this structured data type. Returns: np.void The default scalar value, which is the scalar representation of 0 cast to this structured data type. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> numpy.void# Read a JSON-serializable value as a NumPy structured scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The zarr format version. Returns: np.void The NumPy structured scalar. Raises: TypeError If the input is not a base64-encoded string. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create a Structured ZDType from a native NumPy data type. Parameters: dtypeTBaseDType The native data type. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the input data type is not an instance of np.dtypes.VoidDType with a non-null `fields` attribute. Notes This method attempts to resolve the fields of the structured dtype using the data type registry. to_json(zarr_format: Literal[2]) -> StructuredJSON_V2# to_json(zarr_format: Literal[3]) -> StructuredJSON_V3 Convert the structured data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The Zarr format version. Accepted values are 2 and 3. Returns: StructuredJSON_V2 | StructuredJSON_V3 The JSON representation of the structured data type. Raises: ValueError If the zarr_format is not 2 or 3. to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> str# Convert a scalar to a JSON-serializable string representation. Parameters: dataobject The scalar to convert. zarr_formatZarrFormat The zarr format version. Returns: str A string representation of the scalar, which is a base64-encoded string of the bytes that make up the scalar. to_native_dtype() -> numpy.dtypes.VoidDType[int]# Convert the structured Zarr data type to a native NumPy void dtype. This method constructs a NumPy dtype with fields corresponding to the fields of the structured Zarr data type, by converting each field’s data type to its native dtype representation. Returns: np.dtypes.VoidDType[int] The native NumPy void dtype representing the structured data type. dtype_cls# fields: tuple[tuple[str, zarr.core.dtype.wrapper.ZDType[zarr.core.dtype.wrapper.TBaseDType, zarr.core.dtype.wrapper.TBaseScalar]], Ellipsis]# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.StructuredJSON_V2# Bases: `zarr.core.dtype.common.DTypeConfig_V2`[`zarr.core.dtype.common.StructuredName_V2`, `None`] A wrapper around the JSON representation of the `Structured` data type in Zarr V2. The `name` field is a sequence of sequences, where each inner sequence has two values: the field name and the data type name for that field (which could be another sequence). The data type names are strings, and the object codec ID is always None. References The structure of the `name` field is defined in the Zarr V2 specification document. Examples { "name": [ ["f0", " numpy.timedelta64# Cast the input to a numpy timedelta64 scalar. If the input is not a scalar of this data type, raise a TypeError. default_scalar() -> numpy.timedelta64# Return a default scalar of this data type. This method provides a default value for the timedelta64 scalar, which is a ‘Not-a-Time’ (NaT) value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> numpy.timedelta64# Create a scalar of this data type from JSON input. Parameters: dataJSON The JSON representation of the scalar value. zarr_formatint The zarr format to use for the JSON representation. Returns: numpy.timedelta64 The scalar value of this data type. Raises: TypeError If the input JSON is not a valid representation of a scalar for this data type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this class from a native NumPy data type. Parameters: dtypeTBaseDType The native NumPy dtype to convert. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the dtype is not a valid representation of this class. to_json(zarr_format: Literal[2]) -> TimeDelta64JSON_V2# to_json(zarr_format: Literal[3]) -> TimeDelta64JSON_V3 Serialize this data type to JSON. Parameters: zarr_formatZarrFormat The Zarr format version (2 or 3). Returns: TimeDelta64JSON_V2 | TimeDelta64JSON_V3 The JSON representation of the data type. Raises: ValueError If the zarr_format is not 2 or 3. to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> int# Convert a python object to a JSON representation of a datetime64 or timedelta64 scalar. Parameters: dataobject The python object to convert. zarr_formatZarrFormat The Zarr format version (2 or 3). Returns: int The JSON representation of the scalar. to_native_dtype() -> BaseTimeDType_co# Convert this data type to a NumPy temporal data type with the appropriate unit and scale factor. Returns: BaseTimeDType_co A NumPy data type object representing the time data type with the specified unit, scale factor, and byte order. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. scale_factor: int = 1# unit: zarr.core.dtype.npy.common.DateTimeUnit = 'generic'# class zarr.dtype.TimeDelta64JSON_V2# Bases: `zarr.core.dtype.common.DTypeConfig_V2`[`str`, `None`] A wrapper around the JSON representation of the `TimeDelta64` data type in Zarr V2. The `name` field of this class contains the value that would appear under the `dtype` field in Zarr V2 array metadata. References The structure of the `name` field is defined in the Zarr V2 specification document. Examples { "name": " TIntScalar_co# Attempt to cast a given object to a NumPy integer scalar. Parameters: dataobject The data to be cast to a NumPy integer scalar. Returns: TIntScalar_co The data cast as a NumPy integer scalar. Raises: TypeError If the data cannot be converted to a NumPy integer scalar. default_scalar() -> TIntScalar_co# Get the default value, which is 0 cast to this dtype. Returns: TIntScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TIntScalar_co# Read a JSON-serializable value as a NumPy int scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The Zarr format version. Returns: TIntScalar_co The NumPy int scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this data type from a np.dtype(‘uint16’) instance. Parameters: dtypenp.dtype The NumPy data type. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the input data type is not an instance of np.dtype(‘uint16’). to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['>u2', ' Literal['uint16'] Serialize this ZDType to v2- or v3-flavored JSON Parameters: zarr_formatZarrFormat The Zarr format version (2 or 3). Returns: DTypeConfig_V2[Literal[“>u2”, “ int# Convert an object to a JSON serializable scalar. For the integer data types, the JSON form is a plain integer. Parameters: dataobject The value to convert. zarr_formatZarrFormat The Zarr format version. Returns: int The JSON-serializable form of the scalar. to_native_dtype() -> numpy.dtypes.UInt16DType# Convert the data type to a np.dtype(‘uint16’) instance. Returns: np.dtype The np.dtype(‘uint16’) instance. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.UInt32# Bases: `BaseInt`[`numpy.dtypes.UInt32DType`, `numpy.uint32`], `zarr.core.dtype.common.HasEndianness` A Zarr data type for arrays containing 32-bit unsigned integers. Wraps the `np.dtypes.UInt32DType` data type. Scalars for this data type are instances of `np.uint32`. Attributes: dtype_clsnp.dtypes.UInt32DType The class of the underlying NumPy dtype. References This class implements the 32-bit unsigned integer data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> TIntScalar_co# Attempt to cast a given object to a NumPy integer scalar. Parameters: dataobject The data to be cast to a NumPy integer scalar. Returns: TIntScalar_co The data cast as a NumPy integer scalar. Raises: TypeError If the data cannot be converted to a NumPy integer scalar. default_scalar() -> TIntScalar_co# Get the default value, which is 0 cast to this dtype. Returns: TIntScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TIntScalar_co# Read a JSON-serializable value as a NumPy int scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The Zarr format version. Returns: TIntScalar_co The NumPy int scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create a UInt32 from a np.dtype(‘uint32’) instance. Parameters: dtypeTBaseDType The NumPy data type. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the input data type is not a valid representation of this class 32-bit unsigned integer. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['>u4', ' Literal['uint32'] Convert the data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The Zarr format version. Returns: DTypeConfig_V2[Literal[“>u4”, “ int# Convert an object to a JSON serializable scalar. For the integer data types, the JSON form is a plain integer. Parameters: dataobject The value to convert. zarr_formatZarrFormat The Zarr format version. Returns: int The JSON-serializable form of the scalar. to_native_dtype() -> numpy.dtypes.UInt32DType# Create a NumPy unsigned 32-bit integer dtype instance from this UInt32 ZDType. Returns: np.dtypes.UInt32DType The NumPy unsigned 32-bit integer dtype. dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.UInt64# Bases: `BaseInt`[`numpy.dtypes.UInt64DType`, `numpy.uint64`], `zarr.core.dtype.common.HasEndianness` A Zarr data type for arrays containing 64-bit unsigned integers. Wraps the `np.dtypes.UInt64DType` data type. Scalars for this data type are instances of `np.uint64`. Attributes: dtype_cls: np.dtypes.UInt64DType The class of the underlying NumPy dtype. References This class implements the unsigned 64-bit integer data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> TIntScalar_co# Attempt to cast a given object to a NumPy integer scalar. Parameters: dataobject The data to be cast to a NumPy integer scalar. Returns: TIntScalar_co The data cast as a NumPy integer scalar. Raises: TypeError If the data cannot be converted to a NumPy integer scalar. default_scalar() -> TIntScalar_co# Get the default value, which is 0 cast to this dtype. Returns: TIntScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TIntScalar_co# Read a JSON-serializable value as a NumPy int scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The Zarr format version. Returns: TIntScalar_co The NumPy int scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of this data type from a native NumPy dtype. Parameters: dtypeTBaseDType The native NumPy dtype. Returns: Self An instance of this data type. Raises: DataTypeValidationError If the input dtype is not a valid representation of this class unsigned 64-bit integer. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['>u8', ' Literal['uint64'] Convert the data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The Zarr format version. Returns: DTypeConfig_V2[Literal[“>u8”, “ int# Convert an object to a JSON serializable scalar. For the integer data types, the JSON form is a plain integer. Parameters: dataobject The value to convert. zarr_formatZarrFormat The Zarr format version. Returns: int The JSON-serializable form of the scalar. to_native_dtype() -> numpy.dtypes.UInt64DType# Convert the data type to a native NumPy dtype. Returns: np.dtypes.UInt64DType The native NumPy dtype.eeeeeeeeeeeeeeeee dtype_cls# endianness: EndiannessStr = 'little'# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.UInt8# Bases: `BaseInt`[`numpy.dtypes.UInt8DType`, `numpy.uint8`] A Zarr data type for arrays containing 8-bit unsigned integers. Wraps the `np.dtypes.UInt8DType` data type. Scalars for this data type are instances of `np.uint8`. Attributes: dtype_clsnp.dtypes.UInt8DType The class of the underlying NumPy dtype. References This class implements the 8-bit unsigned integer data type defined in Zarr V2 and V3. See the Zarr V2 and Zarr V3 specification documents for details. cast_scalar(data: object) -> TIntScalar_co# Attempt to cast a given object to a NumPy integer scalar. Parameters: dataobject The data to be cast to a NumPy integer scalar. Returns: TIntScalar_co The data cast as a NumPy integer scalar. Raises: TypeError If the data cannot be converted to a NumPy integer scalar. default_scalar() -> TIntScalar_co# Get the default value, which is 0 cast to this dtype. Returns: TIntScalar_co The default value. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TIntScalar_co# Read a JSON-serializable value as a NumPy int scalar. Parameters: dataJSON The JSON-serializable value. zarr_formatZarrFormat The Zarr format version. Returns: TIntScalar_co The NumPy int scalar. Raises: TypeError If the input is not a valid integer type. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create a UInt8 from a np.dtype(‘uint8’) instance. to_json( zarr_format: Literal[2], ) -> zarr.core.dtype.common.DTypeConfig_V2[Literal['|u1'], None]# to_json(zarr_format: Literal[3]) -> Literal['uint8'] Convert the data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The Zarr format version. Supported values are 2 and 3. Returns: `DTypeConfig_V2[Literal["|u1"], None] | Literal["uint8"]` The JSON-serializable representation of the data type. Raises: ValueError If zarr_format is not 2 or 3. to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> int# Convert an object to a JSON serializable scalar. For the integer data types, the JSON form is a plain integer. Parameters: dataobject The value to convert. zarr_formatZarrFormat The Zarr format version. Returns: int The JSON-serializable form of the scalar. to_native_dtype() -> numpy.dtypes.UInt8DType# Create a NumPy unsigned 8-bit integer dtype instance from this UInt8 ZDType. Returns: np.dtypes.UInt8DType The NumPy unsigned 8-bit integer dtype. dtype_cls# property item_size: int# The size of a single scalar in bytes. Returns: int The size of a single scalar in bytes. class zarr.dtype.VariableLengthBytes# Bases: `zarr.core.dtype.wrapper.ZDType`[`numpy.dtypes.ObjectDType`, `bytes`], `zarr.core.dtype.common.HasObjectCodec` A Zarr data type for arrays containing variable-length sequences of bytes. Wraps the NumPy “object” data type. Scalars for this data type are instances of `bytes`. Attributes: dtype_cls: ClassVar[type[np.dtypes.ObjectDType]] = np.dtypes.ObjectDType The NumPy data type wrapped by this ZDType. _zarr_v3_name: ClassVar[Literal[“variable_length_bytes”]] = “variable_length_bytes” The name of this data type in Zarr V3. object_codec_id: ClassVar[Literal[“vlen-bytes”]] = “vlen-bytes” The object codec ID for this data type. Notes Because this data type uses the NumPy “object” data type, it does not guarantee a compact memory representation of array data. Therefore a “vlen-bytes” codec is needed to ensure that the array data can be persisted to storage. cast_scalar(data: object) -> bytes# Attempt to cast a given object to a bytes scalar. This method first checks if the provided data is a valid scalar that can be converted to a bytes scalar. If the check succeeds, the unchecked casting operation is performed. If the data is not valid, a TypeError is raised. Parameters: dataobject The data to be cast to a bytes scalar. Returns: bytes The data cast as a bytes scalar. Raises: TypeError If the data cannot be converted to a bytes scalar. default_scalar() -> bytes# Return the default scalar value for the variable-length bytes data type. Returns: bytes The default scalar value, which is an empty byte string. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> bytes# Decode a base64-encoded JSON string to bytes. Parameters: dataJSON The JSON-serializable base64-encoded string. zarr_formatZarrFormat The zarr format version. Returns: bytes The decoded bytes from the base64 string. Raises: TypeError If the input data is not a base64-encoded string. classmethod from_native_dtype(dtype: zarr.core.dtype.wrapper.TBaseDType) -> Self# Create an instance of VariableLengthBytes from an instance of np.dtypes.ObjectDType. This method checks if the provided data type is an instance of np.dtypes.ObjectDType. If so, it returns an instance of VariableLengthBytes. Parameters: dtypeTBaseDType The native dtype to convert. Returns: VariableLengthBytes An instance of VariableLengthBytes. Raises: DataTypeValidationError If the dtype is not compatible with VariableLengthBytes. to_json(zarr_format: Literal[2]) -> VariableLengthBytesJSON_V2# to_json(zarr_format: Literal[3]) -> Literal['variable_length_bytes'] Convert the variable-length bytes data type to a JSON-serializable form. Parameters: zarr_formatZarrFormat The zarr format version. Accepted values are 2 and 3. Returns: `DTypeConfig_V2[Literal["|O"], Literal["vlen-bytes"]] | Literal["variable_length_bytes"]` The JSON-serializable representation of the variable-length bytes data type. For zarr_format 2, returns a dictionary with “name” and “object_codec_id”. For zarr_format 3, returns a string identifier “variable_length_bytes”. Raises: ValueError If zarr_format is not 2 or 3. to_json_scalar(data: object, *, zarr_format: zarr.core.common.ZarrFormat) -> str# Convert a scalar to a JSON-serializable string representation. This method encodes the given scalar as bytes and then encodes the bytes as a base64-encoded string. Parameters: dataobject The scalar to convert. zarr_formatZarrFormat The zarr format version. Returns: str A string representation of the scalar. to_native_dtype() -> numpy.dtypes.ObjectDType# Create a NumPy object dtype from this VariableLengthBytes ZDType. Returns: np.dtypes.ObjectDType A NumPy data type object representing variable-length bytes. dtype_cls# object_codec_id: ClassVar[Literal['vlen-bytes']] = 'vlen-bytes'# class zarr.dtype.VariableLengthBytesJSON_V2# Bases: `zarr.core.dtype.common.DTypeConfig_V2`[`Literal`[`'|O'`], `Literal`[`'vlen-bytes'`]] A wrapper around the JSON representation of the `VariableLengthBytes` data type in Zarr V2. The `name` field of this class contains the value that would appear under the `dtype` field in Zarr V2 array metadata. The `object_codec_id` field is always `"vlen-bytes"` References The structure of the `name` field is defined in the Zarr V2 specification document. Examples { "name": "|O", "object_codec_id": "vlen-bytes" } name: typing_extensions.ReadOnly[TDTypeNameV2_co]# object_codec_id: typing_extensions.ReadOnly[TObjectCodecID_co]# class zarr.dtype.VariableLengthUTF8# Bases: `UTF8Base`[`numpy.dtypes.StringDType`] A Zarr data type for arrays containing variable-length UTF-8 strings. Wraps the `np.dtypes.StringDType` data type. Scalars for this data type are instances of `str`. Attributes: dtype_clsType[np.dtypes.StringDType] The NumPy dtype class for this data type. _zarr_v3_nameClassVar[Literal[“variable_length_utf8”]] = “variable_length_utf8” The name of this data type in Zarr V3. object_codec_idClassVar[Literal[“vlen-utf8”]] = “vlen-utf8” The object codec ID for this data type. to_native_dtype() -> numpy.dtypes.StringDType# Create a NumPy string dtype from this VariableLengthUTF8 ZDType. Returns: np.dtypes.StringDType The NumPy string dtype. dtype_cls# class zarr.dtype.VariableLengthUTF8JSON_V2# Bases: `zarr.core.dtype.common.DTypeConfig_V2`[`Literal`[`'|O'`], `Literal`[`'vlen-utf8'`]] A wrapper around the JSON representation of the `VariableLengthUTF8` data type in Zarr V2. The `name` field of this class contains the value that would appear under the `dtype` field in Zarr V2 array metadata. The `object_codec_id` field is always `"vlen-utf8"`. References The structure of the `name` field is defined in the Zarr V2 specification document. Examples { "name": "|O", "object_codec_id": "vlen-utf8" } name: typing_extensions.ReadOnly[TDTypeNameV2_co]# object_codec_id: typing_extensions.ReadOnly[TObjectCodecID_co]# class zarr.dtype.ZDType# Bases: `abc.ABC`, `Generic`[`TDType_co`, `TScalar_co`] Abstract base class for wrapping native array data types, e.g. numpy dtypes Attributes: dtype_clsClassVar[type[TDType]] The wrapped dtype class. This is a class variable. _zarr_v3_nameClassVar[str] The name given to the data type by a Zarr v3 data type specification. This is a class variable, and it should generally be unique across different data types. abstract cast_scalar(data: object) -> TScalar_co# Cast a python object to the wrapped scalar type. The type of the provided scalar is first checked for compatibility. If it’s incompatible with the associated scalar type, a `TypeError` will be raised. Parameters: dataobject The python object to cast. Returns: TScalar The cast value. abstract default_scalar() -> TScalar_co# Get the default scalar value for the wrapped data type. This is a method, rather than an attribute, because the default value for some data types depends on parameters that are not known until a concrete data type is wrapped. For example, data types parametrized by a length like fixed-length strings or bytes will generate scalars consistent with that length. Returns: TScalar The default value for this data type. classmethod from_json( data: zarr.core.dtype.common.DTypeJSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> Self# Create an instance of this ZDType from JSON data. Parameters: dataDTypeJSON The JSON representation of the data type. zarr_formatZarrFormat The zarr format version. Returns: Self An instance of this data type. abstract from_json_scalar( data: zarr.core.common.JSON, *, zarr_format: zarr.core.common.ZarrFormat, ) -> TScalar_co# Read a JSON-serializable value as a scalar. Parameters: dataJSON A JSON representation of a scalar value. zarr_formatZarrFormat The zarr format version. This is specified because the JSON serialization of scalars differs between Zarr V2 and Zarr V3. Returns: TScalar The deserialized scalar value. classmethod from_native_dtype(dtype: TBaseDType) -> Self# Abstractmethod: Create a ZDType instance from a native data type. This method is used when taking a user-provided native data type, like a NumPy data type, and creating the corresponding ZDType instance from them. Parameters: dtypeTDType The native data type object to wrap. Returns: Self The ZDType that wraps the native data type. Raises: TypeError If the native data type is not consistent with the wrapped data type. to_json(zarr_format: Literal[2]) -> zarr.core.dtype.common.DTypeSpec_V2# to_json(zarr_format: Literal[3]) -> zarr.core.dtype.common.DTypeSpec_V3 Serialize this ZDType to JSON. Parameters: zarr_formatZarrFormat The zarr format version. Returns: DTypeJSON_V2 | DTypeJSON_V3 The JSON-serializable representation of the wrapped data type abstract to_json_scalar( data: object, *, zarr_format: zarr.core.common.ZarrFormat, ) -> zarr.core.common.JSON# Serialize a python object to the JSON representation of a scalar. The value will first be cast to the scalar type associated with this ZDType, then serialized to JSON. Parameters: dataobject The value to convert. zarr_formatZarrFormat The zarr format version. This is specified because the JSON serialization of scalars differs between Zarr V2 and Zarr V3. Returns: JSON The JSON-serialized scalar. abstract to_native_dtype() -> TDType_co# Return an instance of the wrapped data type. This operation inverts `from_native_dtype`. Returns: TDType The native data type wrapped by this ZDType. dtype_cls: ClassVar[type[TDType_co]]# zarr.dtype.parse_dtype( dtype_spec: ZDTypeLike, *, zarr_format: zarr.core.common.ZarrFormat, ) -> wrapper.ZDType[wrapper.TBaseDType, wrapper.TBaseScalar]# Convert the input as a ZDType. Parameters: dtype_specZDTypeLike The input to be converted to a ZDType. This could be a ZDType, which will be returned directly, or a JSON representation of a ZDType, or a numpy dtype, or a python object that can be converted into a native dtype. zarr_formatZarrFormat The Zarr format version. This parameter is required because this function will attempt to parse the JSON representation of a data type, and the JSON representation of data types varies between Zarr 2 and Zarr 3. Returns: ZDType[TBaseDType, TBaseScalar] The ZDType corresponding to the input. Examples >>> from zarr.dtype import parse_dtype >>> import numpy as np >>> parse_dtype("int32", zarr_format=2) Int32(endianness='little') >>> parse_dtype(np.dtype('S10'), zarr_format=2) NullTerminatedBytes(length=10) >>> parse_dtype({"name": "numpy.datetime64", "configuration": {"unit": "s", "scale_factor": 10}}, zarr_format=3) DateTime64(endianness='little', scale_factor=10, unit='s') zarr.dtype.data_type_registry# #### zarr.errors# ##### Exceptions# `ArrayIndexError` Sequence index out of range. `ArrayNotFoundError` Raised when an array isn't found at a certain path. `BaseZarrError` Base error which all zarr errors are sub-classed from. `BoundsCheckError` Sequence index out of range. `ContainsArrayAndGroupError` Raised when both array and group metadata are found at the same path. `ContainsArrayError` Raised when an array already exists at a certain path. `ContainsGroupError` Raised when a group already exists at a certain path. `GroupNotFoundError` Raised when a group isn't found at a certain path. `MetadataValidationError` Raised when the Zarr metadata is invalid in some way `NegativeStepError` Sequence index out of range. `NodeTypeValidationError` Specialized exception when the node_type of the metadata document is incorrect. `UnstableSpecificationWarning` A warning raised to indicate that a feature is outside the Zarr specification. `VindexInvalidSelectionError` Sequence index out of range. `ZarrDeprecationWarning` A warning raised to indicate that a feature will be removed in a future release. `ZarrFutureWarning` A warning intended for end users raised to indicate deprecated features. `ZarrRuntimeWarning` A warning for dubious runtime behavior. ##### Module Contents# exception zarr.errors.ArrayIndexError# Bases: `IndexError` Sequence index out of range. exception zarr.errors.ArrayNotFoundError(*args: object)# Bases: `NodeNotFoundError` Raised when an array isn’t found at a certain path. exception zarr.errors.BaseZarrError(*args: object)# Bases: `ValueError` Base error which all zarr errors are sub-classed from. exception zarr.errors.BoundsCheckError# Bases: `IndexError` Sequence index out of range. exception zarr.errors.ContainsArrayAndGroupError(*args: object)# Bases: `BaseZarrError` Raised when both array and group metadata are found at the same path. exception zarr.errors.ContainsArrayError(*args: object)# Bases: `BaseZarrError` Raised when an array already exists at a certain path. exception zarr.errors.ContainsGroupError(*args: object)# Bases: `BaseZarrError` Raised when a group already exists at a certain path. exception zarr.errors.GroupNotFoundError(*args: object)# Bases: `NodeNotFoundError` Raised when a group isn’t found at a certain path. exception zarr.errors.MetadataValidationError(*args: object)# Bases: `BaseZarrError` Raised when the Zarr metadata is invalid in some way exception zarr.errors.NegativeStepError# Bases: `IndexError` Sequence index out of range. exception zarr.errors.NodeTypeValidationError(*args: object)# Bases: `MetadataValidationError` Specialized exception when the node_type of the metadata document is incorrect. This can be raised when the value is invalid or unexpected given the context, for example an ‘array’ node when we expected a ‘group’. exception zarr.errors.UnstableSpecificationWarning# Bases: `ZarrFutureWarning` A warning raised to indicate that a feature is outside the Zarr specification. exception zarr.errors.VindexInvalidSelectionError# Bases: `IndexError` Sequence index out of range. exception zarr.errors.ZarrDeprecationWarning# Bases: `DeprecationWarning` A warning raised to indicate that a feature will be removed in a future release. exception zarr.errors.ZarrFutureWarning# Bases: `FutureWarning` A warning intended for end users raised to indicate deprecated features. exception zarr.errors.ZarrRuntimeWarning# Bases: `RuntimeWarning` A warning for dubious runtime behavior. #### zarr.metadata# ##### Submodules# ###### zarr.metadata.migrate_v3# ###### Functions# `migrate_to_v3`(→ None) Migrate all v2 metadata in a Zarr array/group to v3. `migrate_v2_to_v3`(→ None) Migrate all v2 metadata in a Zarr store to v3. `remove_metadata`(→ None) Remove all v2 (.zarray, .zattrs, .zgroup, .zmetadata) or v3 (zarr.json) metadata files from the given Zarr. ###### Module Contents# zarr.metadata.migrate_v3.migrate_to_v3( zarr_v2: zarr.Array | zarr.Group, output_path: zarr.storage.StorePath, dry_run: bool = False, ) -> None# Migrate all v2 metadata in a Zarr array/group to v3. Note - if a group is provided, then all arrays / groups within this group will also be converted. A zarr.json file will be created for each level and written to output_path, with any v2 files (.zarray, .zattrs etc.) left as-is. Parameters: zarr_v2Array | Group An array or group with zarr_format = 2 output_pathStorePath The store path to write generated v3 metadata to. dry_runbool, optional Enable a ‘dry run’ - files that would be created are logged, but no files are created or changed. zarr.metadata.migrate_v3.migrate_v2_to_v3( *, input_store: zarr.abc.store.Store, output_store: zarr.abc.store.Store | None = None, dry_run: bool = False, ) -> None# Migrate all v2 metadata in a Zarr store to v3. This will create a zarr.json file at each level of a Zarr hierarchy (for every group / array). v2 files (.zarray, .zattrs etc.) will be left as-is. Parameters: input_storeStore Input Zarr to migrate. output_storeStore, optional Output location to write v3 metadata (no array data will be copied). If not provided, v3 metadata will be written to input_store. dry_runbool, optional Enable a ‘dry run’ - files that would be created are logged, but no files are created or changed. async zarr.metadata.migrate_v3.remove_metadata( store: zarr.abc.store.Store, zarr_format: zarr.core.common.ZarrFormat, force: bool = False, dry_run: bool = False, ) -> None# Remove all v2 (.zarray, .zattrs, .zgroup, .zmetadata) or v3 (zarr.json) metadata files from the given Zarr. Note - this will remove metadata files at all levels of the hierarchy (every group and array). Parameters: storeStore Zarr to remove metadata from. zarr_formatZarrFormat Which format’s metadata to remove - 2 or 3. forcebool, optional When False, metadata can only be removed if a valid alternative exists e.g. deletion of v2 metadata will only be allowed when v3 metadata is also present. When True, metadata can be removed when there is no alternative. dry_runbool, optional Enable a ‘dry run’ - files that would be deleted are logged, but no files are removed or changed. #### zarr.registry# ##### Classes# `Registry` ##### Functions# `get_buffer_class`(→ type[zarr.core.buffer.Buffer]) `get_chunk_key_encoding_class`(...) `get_codec_class`(→ type[zarr.abc.codec.Codec]) `get_ndbuffer_class`(→ type[zarr.core.buffer.NDBuffer]) `get_pipeline_class`(→ type[zarr.abc.codec.CodecPipeline]) `register_buffer`(→ None) `register_chunk_key_encoding`(→ None) `register_codec`(→ None) `register_ndbuffer`(→ None) `register_pipeline`(→ None) ##### Module Contents# class zarr.registry.Registry# Bases: `dict`[`str`, `type`[`T`]], `Generic`[`T`] lazy_load(use_entrypoint_name: bool = False) -> None# register(cls: type[T], qualname: str | None = None) -> None# lazy_load_list: list[importlib.metadata.EntryPoint] = []# zarr.registry.get_buffer_class(reload_config: bool = False) -> type[zarr.core.buffer.Buffer]# zarr.registry.get_chunk_key_encoding_class( key: str, ) -> type[zarr.core.chunk_key_encodings.ChunkKeyEncoding]# zarr.registry.get_codec_class( key: str, reload_config: bool = False, ) -> type[zarr.abc.codec.Codec]# zarr.registry.get_ndbuffer_class( reload_config: bool = False, ) -> type[zarr.core.buffer.NDBuffer]# zarr.registry.get_pipeline_class( reload_config: bool = False, ) -> type[zarr.abc.codec.CodecPipeline]# zarr.registry.register_buffer( cls: type[zarr.core.buffer.Buffer], qualname: str | None = None, ) -> None# zarr.registry.register_chunk_key_encoding(key: str, cls: type) -> None# zarr.registry.register_codec( key: str, codec_cls: type[zarr.abc.codec.Codec], *, qualname: str | None = None, ) -> None# zarr.registry.register_ndbuffer( cls: type[zarr.core.buffer.NDBuffer], qualname: str | None = None, ) -> None# zarr.registry.register_pipeline(pipe_cls: type[zarr.abc.codec.CodecPipeline]) -> None# #### zarr.storage# ##### Attributes# `StoreLike` ##### Classes# `FsspecStore` Store for remote data based on FSSpec. `GpuMemoryStore` Store for GPU memory. `LocalStore` Store for the local file system. `LoggingStore` Store that logs all calls to another wrapped store. `MemoryStore` Store for local memory. `ObjectStore` Store that uses obstore for fast read/write from AWS, GCP, Azure. `StorePath` Path-like interface for a Store. `WrapperStore` Store that wraps an existing Store. `ZipStore` Store using a ZIP file. ##### Package Contents# class zarr.storage.FsspecStore( fs: fsspec.asyn.AsyncFileSystem, read_only: bool = False, path: str = '/', allowed_exceptions: tuple[type[Exception], Ellipsis] = ALLOWED_EXCEPTIONS, )# Bases: `zarr.abc.store.Store` Store for remote data based on FSSpec. Parameters: fsAsyncFileSystem The Async FSSpec filesystem to use with this store. read_onlybool Whether the store is read-only pathstr The root path of the store. This should be a relative path and must not include the filesystem scheme. allowed_exceptionstuple[type[Exception], …] When fetching data, these cases will be deemed to correspond to missing keys. Attributes: fs allowed_exceptions supports_writes supports_deletes supports_listing Raises: TypeError If the Filesystem does not support async operations. ValueError If the path argument includes a scheme. Warns: ZarrUserWarning If the file system (fs) was not created with asynchronous=True. See also `FsspecStore.from_upath` `FsspecStore.from_url` async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool classmethod from_mapper( fs_map: fsspec.mapping.FSMap, read_only: bool = False, allowed_exceptions: tuple[type[Exception], Ellipsis] = ALLOWED_EXCEPTIONS, ) -> FsspecStore# Create a FsspecStore from a FSMap object. Parameters: fs_mapFSMap Fsspec mutable mapping object. read_onlybool Whether the store is read-only, defaults to False. allowed_exceptionstuple, optional The exceptions that are allowed to be raised when accessing the store. Defaults to ALLOWED_EXCEPTIONS. Returns: FsspecStore classmethod from_upath( upath: Any, read_only: bool = False, allowed_exceptions: tuple[type[Exception], Ellipsis] = ALLOWED_EXCEPTIONS, ) -> FsspecStore# Create a FsspecStore from an upath object. Parameters: upathUPath The upath to the root of the store. read_onlybool Whether the store is read-only, defaults to False. allowed_exceptionstuple, optional The exceptions that are allowed to be raised when accessing the store. Defaults to ALLOWED_EXCEPTIONS. Returns: FsspecStore classmethod from_url( url: str, storage_options: dict[str, Any] | None = None, read_only: bool = False, allowed_exceptions: tuple[type[Exception], Ellipsis] = ALLOWED_EXCEPTIONS, ) -> FsspecStore# Create a FsspecStore from a URL. The type of store is determined from the URL scheme. Parameters: urlstr The URL to the root of the store. storage_optionsdict, optional The options to pass to fsspec when creating the filesystem. read_onlybool Whether the store is read-only, defaults to False. allowed_exceptionstuple, optional The exceptions that are allowed to be raised when accessing the store. Defaults to ALLOWED_EXCEPTIONS. Returns: FsspecStore async get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. async list() -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store. Returns: AsyncIterator[str] async list_dir(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] async list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] classmethod open(*args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. async set( key: str, value: zarr.core.buffer.Buffer, byte_range: tuple[int, int] | None = None, ) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer with_read_only(read_only: bool = False) -> FsspecStore# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. allowed_exceptions: tuple[type[Exception], Ellipsis]# fs: fsspec.asyn.AsyncFileSystem# path: str# property read_only: bool# Is the store read-only? property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. supports_deletes: bool = True# Does the store support deletes? supports_listing: bool = True# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. supports_writes: bool = True# Does the store support writes? class zarr.storage.GpuMemoryStore( store_dict: collections.abc.MutableMapping[str, zarr.core.buffer.gpu.Buffer] | None = None, *, read_only: bool = False, )# Bases: `MemoryStore` Store for GPU memory. Stores every chunk in GPU memory irrespective of the original location. The dictionary of buffers to initialize this memory store with must be GPU Buffers. Writing data to this store through `.set` will move the buffer to the GPU if necessary. Parameters: store_dictMutableMapping, optional A mutable mapping with string keys and `zarr.core.buffer.gpu.Buffer` values. read_onlybool Whether to open the store in read-only mode. async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool classmethod from_dict( store_dict: collections.abc.MutableMapping[str, zarr.core.buffer.Buffer], ) -> Self# Create a GpuMemoryStore from a dictionary of buffers at any location. The dictionary backing the newly created `GpuMemoryStore` will not be the same as `store_dict`. Parameters: store_dictmapping A mapping of strings keys to arbitrary Buffers. The buffer data will be moved into a `gpu.Buffer`. Returns: GpuMemoryStore async get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. async list() -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store. Returns: AsyncIterator[str] async list_dir(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] async list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] classmethod open(*args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. async set( key: str, value: zarr.core.buffer.Buffer, byte_range: tuple[int, int] | None = None, ) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer with_read_only(read_only: bool = False) -> MemoryStore# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. property read_only: bool# Is the store read-only? property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. supports_deletes: bool = True# Does the store support deletes? supports_listing: bool = True# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. supports_writes: bool = True# Does the store support writes? class zarr.storage.LocalStore(root: pathlib.Path | str, *, read_only: bool = False)# Bases: `zarr.abc.store.Store` Store for the local file system. Parameters: rootstr or Path Directory to use as root of store. read_onlybool Whether the store is read-only Attributes: supports_writes supports_deletes supports_listing root async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store. Parameters: keystr Notes If `key` is a directory within this store, the entire directory at `store.root / key` is deleted. async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool async get( key: str, prototype: zarr.core.buffer.BufferPrototype | None = None, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. async list() -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store. Returns: AsyncIterator[str] async list_dir(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] async list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] async move(dest_root: pathlib.Path | str) -> None# Move the store to another path. The old root directory is deleted. classmethod open( root: pathlib.Path | str, *, read_only: bool = False, mode: zarr.core.common.AccessModeLiteral | None = None, ) -> Self# Async: Create and open the store. Parameters: rootstr or Path Directory to use as root of store. read_onlybool Whether the store is read-only mode Mode in which to create the store. This only affects opening the store, and the final read-only state of the store is controlled through the read_only parameter. Returns: Store The opened store instance. async set(key: str, value: zarr.core.buffer.Buffer) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer with_read_only(read_only: bool = False) -> Self# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. property read_only: bool# Is the store read-only? root: pathlib.Path# property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. supports_deletes: bool = True# Does the store support deletes? supports_listing: bool = True# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. supports_writes: bool = True# Does the store support writes? class zarr.storage.LoggingStore( store: T_Store, log_level: str = 'DEBUG', log_handler: logging.Handler | None = None, )# Bases: `zarr.storage._wrapper.WrapperStore`[`T_Store`] Store that logs all calls to another wrapped store. Parameters: storeStore Store to wrap log_levelstr Log level log_handlerlogging.Handler Log handler Attributes: counterdict Counter of number of times each method has been called async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool async get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str = '') -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. async list() -> collections.abc.AsyncGenerator[str, None]# Retrieve all keys in the store. Returns: AsyncIterator[str] async list_dir(prefix: str) -> collections.abc.AsyncGenerator[str, None]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] async list_prefix(prefix: str) -> collections.abc.AsyncGenerator[str, None]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] log(hint: Any = '') -> collections.abc.Generator[None, None, None]# Context manager to log method calls Each call to the wrapped store is logged to the configured logger and added to the counter dict. classmethod open(store_cls: type[T_Store], *args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. async set(key: str, value: zarr.core.buffer.Buffer) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer abstract with_read_only(read_only: bool = False) -> Store# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. counter: collections.defaultdict[str, int]# log_handler = None# log_level = 'DEBUG'# property read_only: bool# Is the store read-only? property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. property supports_deletes: bool# Does the store support deletes? property supports_listing: bool# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. property supports_writes: bool# Does the store support writes? class zarr.storage.MemoryStore( store_dict: collections.abc.MutableMapping[str, zarr.core.buffer.Buffer] | None = None, *, read_only: bool = False, )# Bases: `zarr.abc.store.Store` Store for local memory. Parameters: store_dictdict Initial data read_onlybool Whether the store is read-only Attributes: supports_writes supports_deletes supports_listing async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool async get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. async list() -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store. Returns: AsyncIterator[str] async list_dir(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] async list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] classmethod open(*args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. async set( key: str, value: zarr.core.buffer.Buffer, byte_range: tuple[int, int] | None = None, ) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer with_read_only(read_only: bool = False) -> MemoryStore# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. property read_only: bool# Is the store read-only? property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. supports_deletes: bool = True# Does the store support deletes? supports_listing: bool = True# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. supports_writes: bool = True# Does the store support writes? class zarr.storage.ObjectStore(store: obstore.store.ObjectStore, *, read_only: bool = False)# Bases: `zarr.abc.store.Store` Store that uses obstore for fast read/write from AWS, GCP, Azure. Parameters: storeobstore.store.ObjectStore An obstore store instance that is set up with the proper credentials. read_onlybool Whether to open the store in read-only mode. Warning ObjectStore is experimental and subject to API changes without notice. Please raise an issue with any comments/concerns about the store. async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool async get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. list() -> collections.abc.AsyncGenerator[str, None]# Retrieve all keys in the store. Returns: AsyncIterator[str] list_dir(prefix: str) -> collections.abc.AsyncGenerator[str, None]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] list_prefix(prefix: str) -> collections.abc.AsyncGenerator[str, None]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] classmethod open(*args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. async set(key: str, value: zarr.core.buffer.Buffer) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer with_read_only(read_only: bool = False) -> ObjectStore# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. property read_only: bool# Is the store read-only? store: obstore.store.ObjectStore# The underlying obstore instance. property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. property supports_deletes: bool# Does the store support deletes? property supports_listing: bool# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. property supports_writes: bool# Does the store support writes? class zarr.storage.StorePath(store: zarr.abc.store.Store, path: str = '')# Path-like interface for a Store. Parameters: storeStore The store to use. pathstr The path within the store. async delete() -> None# Delete the key from the store. Raises: NotImplementedError If the store does not support deletion. async delete_dir() -> None# Delete all keys with the given prefix from the store. async exists() -> bool# Check if the key exists in the store. Returns: bool True if the key exists in the store, False otherwise. async get( prototype: zarr.core.buffer.BufferPrototype | None = None, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Read bytes from the store. Parameters: prototypeBufferPrototype, optional The buffer prototype to use when reading the bytes. byte_rangeByteRequest, optional The range of bytes to read. Returns: bufferBuffer or None The read bytes, or None if the key does not exist. async is_empty() -> bool# Check if any keys exist in the store with the given prefix. Returns: bool True if no keys exist in the store with the given prefix, False otherwise. classmethod open( store: zarr.abc.store.Store, path: str, mode: zarr.core.common.AccessModeLiteral | None = None, ) -> Self# Async: Open StorePath based on the provided mode. * If the mode is None, return an opened version of the store with no changes. * If the mode is ‘r+’, ‘w-’, ‘w’, or ‘a’ and the store is read-only, raise a ValueError. * If the mode is ‘r’ and the store is not read-only, return a copy of the store with read_only set to True. * If the mode is ‘w-’ and the store is not read-only and the StorePath contains keys, raise a FileExistsError. * If the mode is ‘w’ and the store is not read-only, delete all keys nested within the StorePath. Parameters: modeAccessModeLiteral The mode to use when initializing the store path. The accepted values are: * `'r'`: read only (must exist) * `'r+'`: read/write (must exist) * `'a'`: read/write (create if doesn’t exist) * `'w'`: read/write (overwrite if exists) * `'w-'`: read/write (create if doesn’t exist). Raises: FileExistsError If the mode is ‘w-’ and the store path already exists. ValueError If the mode is not “r” and the store is read-only, or async set(value: zarr.core.buffer.Buffer) -> None# Write bytes to the store. Parameters: valueBuffer The buffer to write. async set_if_not_exists(default: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: defaultBuffer The buffer to store if the key is not already present. path: str# property read_only: bool# store: zarr.abc.store.Store# class zarr.storage.WrapperStore(store: T_Store)# Bases: `zarr.abc.store.Store`, `Generic`[`T_Store`] Store that wraps an existing Store. By default all of the store methods are delegated to the wrapped store instance, which is accessible via the `._store` attribute of this class. Use this class to modify or extend the behavior of the other store classes. async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool async get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. list() -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store. Returns: AsyncIterator[str] list_dir(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] classmethod open(store_cls: type[T_Store], *args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. async set(key: str, value: zarr.core.buffer.Buffer) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer abstract with_read_only(read_only: bool = False) -> Store# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. property read_only: bool# Is the store read-only? property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. property supports_deletes: bool# Does the store support deletes? property supports_listing: bool# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. property supports_writes: bool# Does the store support writes? class zarr.storage.ZipStore( path: pathlib.Path | str, *, mode: ZipStoreAccessModeLiteral = 'r', read_only: bool | None = None, compression: int = zipfile.ZIP_STORED, allowZip64: bool = True, )# Bases: `zarr.abc.store.Store` Store using a ZIP file. Parameters: pathstr Location of file. modestr, optional One of ‘r’ to read an existing file, ‘w’ to truncate and write a new file, ‘a’ to append to an existing file, or ‘x’ to exclusively create and write a new file. compressionint, optional Compression method to use when writing to the archive. allowZip64bool, optional If True (the default) will create ZIP files that use the ZIP64 extensions when the zipfile is larger than 2 GiB. If False will raise an exception when the ZIP file would require ZIP64 extensions. Attributes: allowed_exceptions supports_writes supports_deletes supports_listing path compression allowZip64 async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool async get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: zarr.abc.store.ByteRequest | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. async list() -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store. Returns: AsyncIterator[str] async list_dir(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] async list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] async move(path: pathlib.Path | str) -> None# Move the store to another path. classmethod open(*args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. async set(key: str, value: zarr.core.buffer.Buffer) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer abstract with_read_only(read_only: bool = False) -> Store# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. allowZip64: bool# compression: int# path: pathlib.Path# property read_only: bool# Is the store read-only? property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. supports_deletes: bool = False# Does the store support deletes? supports_listing: bool = True# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. supports_writes: bool = True# Does the store support writes? zarr.storage.StoreLike: TypeAlias = Store | StorePath | FSMap | Path | str | dict[str, Buffer]# #### zarr.testing# ##### Submodules# ###### zarr.testing.buffer# ###### Classes# `NDBufferUsingTestNDArrayLike` Example of a custom NDBuffer that handles MyNDArrayLike `StoreExpectingTestBuffer` Example of a custom Store that expect MyBuffer for all its non-metadata `TestBuffer` Example of a custom Buffer that handles ArrayLike ###### Module Contents# class zarr.testing.buffer.NDBufferUsingTestNDArrayLike(array: zarr.core.buffer.core.NDArrayLike)# Bases: `zarr.core.buffer.cpu.NDBuffer` Example of a custom NDBuffer that handles MyNDArrayLike all_equal(other: Any, equal_nan: bool = True) -> bool# Compare to other using np.array_equal. as_ndarray_like() -> NDArrayLike# Returns the underlying array (host or device memory) of this buffer This will never copy data. Returns: The underlying array such as a NumPy or CuPy array. as_numpy_array() -> numpy.typing.NDArray[Any]# Returns the buffer as a NumPy array (host memory). Returns: NumPy array of this buffer (might be a data copy) Warning Might have to copy data, consider using .as_ndarray_like() instead. as_scalar() -> ScalarType# Returns the buffer as a scalar value astype( dtype: numpy.typing.DTypeLike, order: Literal['K', 'A', 'C', 'F'] = 'K', ) -> Self# copy() -> Self# classmethod create( *, shape: collections.abc.Iterable[int], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', fill_value: Any | None = None, ) -> Self# Overwrite NDBuffer.create to create an TestNDArrayLike instance classmethod empty( shape: tuple[int, Ellipsis], dtype: numpy.typing.DTypeLike, order: Literal['C', 'F'] = 'C', ) -> Self# Create an empty buffer with the given shape, dtype, and order. This method can be faster than `NDBuffer.create` because it doesn’t have to initialize the memory used by the underlying ndarray-like object. Parameters: shape The shape of the buffer and its underlying ndarray-like object dtype The datatype of the buffer and its underlying ndarray-like object order Whether to store multi-dimensional data in row-major (C-style) or column-major (Fortran-style) order in memory. Returns: buffer New buffer representing a new ndarray_like object with empty data. See also `NDBuffer.create` Create a new buffer with some initial fill value. fill(value: Any) -> None# classmethod from_ndarray_like(ndarray_like: NDArrayLike) -> Self# Create a new buffer of a ndarray-like object Parameters: ndarray_like ndarray-like object Returns: New buffer representing ndarray_like classmethod from_numpy_array(array_like: numpy.typing.ArrayLike) -> Self# Create a new buffer of Numpy array-like object Parameters: array_like Object that can be coerced into a Numpy array Returns: New buffer representing array_like reshape(newshape: tuple[int, Ellipsis] | Literal[-1]) -> Self# squeeze(axis: tuple[int, Ellipsis]) -> Self# transpose( axes: SupportsIndex | collections.abc.Sequence[SupportsIndex] | None, ) -> Self# property byteorder: zarr.codecs.bytes.Endian# property dtype: numpy.dtype[Any]# property shape: tuple[int, Ellipsis]# class zarr.testing.buffer.StoreExpectingTestBuffer( store_dict: collections.abc.MutableMapping[str, zarr.core.buffer.Buffer] | None = None, *, read_only: bool = False, )# Bases: `zarr.storage.MemoryStore` Example of a custom Store that expect MyBuffer for all its non-metadata We assume that keys containing “json” is metadata async clear() -> None# Clear the store. Remove all keys and values from the store. close() -> None# Close the store. async delete(key: str) -> None# Remove a key from the store Parameters: keystr async delete_dir(prefix: str) -> None# Remove all keys and prefixes in the store that begin with a given prefix. async exists(key: str) -> bool# Check if a key exists in the store. Parameters: keystr Returns: bool async get( key: str, prototype: zarr.core.buffer.BufferPrototype, byte_range: tuple[int, int | None] | None = None, ) -> zarr.core.buffer.Buffer | None# Retrieve the value associated with a given key. Parameters: keystr prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. byte_rangeByteRequest, optional ByteRequest may be one of the following. If not provided, all data associated with the key is retrieved. \- RangeByteRequest(int, int): Request a specific range of bytes in the form (start, end). The end is exclusive. If the given range is zero-length or starts after the end of the object, an error will be returned. Additionally, if the range ends after the end of the object, the entire remainder of the object will be returned. Otherwise, the exact requested range will be returned. \- OffsetByteRequest(int): Request all bytes starting from a given byte offset. This is equivalent to bytes={int}- as an HTTP header. \- SuffixByteRequest(int): Request the last int bytes. Note that here, int is the size of the request, not the byte offset. This is equivalent to bytes=-{int} as an HTTP header. Returns: Buffer async get_partial_values( prototype: zarr.core.buffer.BufferPrototype, key_ranges: collections.abc.Iterable[tuple[str, zarr.abc.store.ByteRequest | None]], ) -> list[zarr.core.buffer.Buffer | None]# Retrieve possibly partial values from given key_ranges. Parameters: prototypeBufferPrototype The prototype of the output buffer. Stores may support a default buffer prototype. key_rangesIterable[tuple[str, tuple[int | None, int | None]]] Ordered set of key, range pairs, a key may occur multiple times with different ranges Returns: list of values, in the order of the key_ranges, may contain null/none for missing keys async getsize(key: str) -> int# Return the size, in bytes, of a value in a Store. Parameters: keystr Returns: nbytesint The size of the value (in bytes). Raises: FileNotFoundError When the given key does not exist in the store. async getsize_prefix(prefix: str) -> int# Return the size, in bytes, of all values under a prefix. Parameters: prefixstr The prefix of the directory to measure. Returns: nbytesint The sum of the sizes of the values in the directory (in bytes). See also `zarr.Array.nbytes_stored` `Store.getsize` Notes `getsize_prefix` is just provided as a potentially faster alternative to listing all the keys under a prefix calling `Store.getsize()` on each. In general, `prefix` should be the path of an Array or Group in the Store. Implementations may differ on the behavior when some other `prefix` is provided. async is_empty(prefix: str) -> bool# Check if the directory is empty. Parameters: prefixstr Prefix of keys to check. Returns: bool True if the store is empty, False otherwise. async list() -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store. Returns: AsyncIterator[str] async list_dir(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys and prefixes with a given prefix and which do not contain the character “/” after the given prefix. Parameters: prefixstr Returns: AsyncIterator[str] async list_prefix(prefix: str) -> collections.abc.AsyncIterator[str]# Retrieve all keys in the store that begin with a given prefix. Keys are returned relative to the root of the store. Parameters: prefixstr Returns: AsyncIterator[str] classmethod open(*args: Any, **kwargs: Any) -> Self# Async: Create and open the store. Parameters: *argsAny Positional arguments to pass to the store constructor. **kwargsAny Keyword arguments to pass to the store constructor. Returns: Store The opened store instance. async set( key: str, value: zarr.core.buffer.Buffer, byte_range: tuple[int, int] | None = None, ) -> None# Store a (key, value) pair. Parameters: keystr valueBuffer async set_if_not_exists(key: str, value: zarr.core.buffer.Buffer) -> None# Store a key to `value` if the key is not already present. Parameters: keystr valueBuffer with_read_only(read_only: bool = False) -> MemoryStore# Return a new store with a new read_only setting. The new store points to the same location with the specified new read_only state. The returned Store is not automatically opened, and this store is not automatically closed. Parameters: read_only If True, the store will be created in read-only mode. Defaults to False. Returns: A new store of the same type with the new read only attribute. property read_only: bool# Is the store read-only? property supports_consolidated_metadata: bool# Does the store support consolidated metadata?. If it doesn’t an error will be raised on requests to consolidate the metadata. Returning False can be useful for stores which implement their own consolidation mechanism outside of the zarr-python implementation. supports_deletes: bool = True# Does the store support deletes? supports_listing: bool = True# Does the store support listing? property supports_partial_writes: Literal[False]# Does the store support partial writes? Partial writes are no longer used by Zarr, so this is always false. supports_writes: bool = True# Does the store support writes? class zarr.testing.buffer.TestBuffer(array_like: zarr.core.buffer.core.ArrayLike)# Bases: `zarr.core.buffer.cpu.Buffer` Example of a custom Buffer that handles ArrayLike as_array_like() -> ArrayLike# Returns the underlying array (host or device memory) of this buffer This will never copy data. Returns: The underlying 1d array such as a NumPy or CuPy array. as_buffer_like() -> zarr.core.common.BytesLike# Returns the buffer as an object that implements the Python buffer protocol. Returns: An object that implements the Python buffer protocol Notes Might have to copy data, since the implementation uses .as_numpy_array(). as_numpy_array() -> numpy.typing.NDArray[Any]# Returns the buffer as a NumPy array (host memory). Returns: NumPy array of this buffer (might be a data copy) Notes Might have to copy data, consider using .as_array_like() instead. classmethod create_zero_length() -> Self# Create an empty buffer with length zero Returns: New empty 0-length buffer classmethod from_array_like(array_like: ArrayLike) -> Self# Create a new buffer of an array-like object Parameters: array_like array-like object that must be 1-dim, contiguous, and byte dtype. Returns: New buffer representing array_like classmethod from_buffer(buffer: zarr.core.buffer.core.Buffer) -> Self# Create a new buffer of an existing Buffer This is useful if you want to ensure that an existing buffer is of the correct subclass of Buffer. E.g., MemoryStore uses this to return a buffer instance of the subclass specified by its BufferPrototype argument. Typically, this only copies data if the data has to be moved between memory types, such as from host to device memory. Parameters: buffer buffer object. Returns: A new buffer representing the content of the input buffer Notes Subclasses of Buffer must override this method to implement more optimal conversions that avoid copies where possible classmethod from_bytes(bytes_like: zarr.core.common.BytesLike) -> Self# Create a new buffer of a bytes-like object (host memory) Parameters: bytes_like bytes-like object Returns: New buffer representing bytes_like to_bytes() -> bytes# Returns the buffer as bytes (host memory). Returns: bytes of this buffer (data copy) Warning Will always copy data, only use this method for small buffers such as metadata buffers. If possible, use .as_numpy_array() or .as_array_like() instead. ###### zarr.testing.conftest# ###### Functions# `pytest_configure`(→ None) ###### Module Contents# zarr.testing.conftest.pytest_configure(config: pytest.Config) -> None# ###### zarr.testing.stateful# ###### Attributes# `F` `MAX_BINARY_SIZE` ###### Classes# `SyncStoreWrapper` `ZarrHierarchyStateMachine` This state machine models operations that modify a zarr store's `ZarrStoreStateMachine` ###### Functions# `split_prefix_name`(→ tuple[str, str]) `with_frequency`(→ collections.abc.Callable[[F], F]) This needs to be deterministic for hypothesis replaying ###### Module Contents# class zarr.testing.stateful.SyncStoreWrapper(store: zarr.abc.store.Store)# Bases: `zarr.core.sync.SyncMixin` clear() -> None# delete(path: str) -> None# exists(key: str) -> bool# get( key: str, prototype: zarr.core.buffer.BufferPrototype, ) -> zarr.core.buffer.Buffer | None# get_partial_values( key_ranges: list[Any], prototype: zarr.core.buffer.BufferPrototype, ) -> list[zarr.core.buffer.Buffer | None]# is_empty(prefix: str) -> bool# list() -> list[str]# abstract list_dir(prefix: str) -> None# abstract list_prefix(prefix: str) -> None# set(key: str, data_buffer: zarr.core.buffer.Buffer) -> None# property read_only: bool# store# property supports_deletes: bool# property supports_listing: bool# property supports_writes: bool# class zarr.testing.stateful.ZarrHierarchyStateMachine(store: zarr.abc.store.Store)# Bases: `zarr.core.sync.SyncMixin`, `hypothesis.stateful.RuleBasedStateMachine` This state machine models operations that modify a zarr store’s hierarchy. That is, user actions that modify arrays/groups as well as list operations. It is intended to be used by external stores, and compares their results to a MemoryStore that is assumed to be perfect. add_array( data: hypothesis.strategies.DataObject, name: str, array_and_chunks: tuple[numpy.ndarray[Any, Any], tuple[int, Ellipsis]], ) -> None# add_group(name: str, data: hypothesis.strategies.DataObject) -> None# can_add(path: str) -> bool# check_array(data: hypothesis.strategies.DataObject) -> None# check_list_dir(data: hypothesis.strategies.DataObject) -> None# check_list_prefix_from_root() -> None# clear() -> None# delete_array_using_del(data: hypothesis.strategies.DataObject) -> None# delete_chunk(data: hypothesis.strategies.DataObject) -> None# delete_dir(data: hypothesis.strategies.DataObject) -> None# delete_group_using_del(data: hypothesis.strategies.DataObject) -> None# draw_directory(data: hypothesis.strategies.DataObject) -> str# init_store() -> None# overwrite_array_basic_indexing(data: hypothesis.strategies.DataObject) -> None# overwrite_array_orthogonal_indexing( data: hypothesis.strategies.DataObject, ) -> None# resize_array(data: hypothesis.strategies.DataObject) -> None# all_arrays: set[str]# all_groups: set[str]# model# store# class zarr.testing.stateful.ZarrStoreStateMachine(store: zarr.abc.store.Store)# Bases: `hypothesis.stateful.RuleBasedStateMachine` ” Zarr store state machine > This is a subclass of a Hypothesis RuleBasedStateMachine. It is testing a framework to ensure that the state of a Zarr store matches an expected state after a set of random operations. It contains a store (currently, a Zarr MemoryStore) and a model, a simplified version of a zarr store (in this case, a dict). It also contains rules which represent actions that can be applied to a zarr store. Rules apply an action to both the store and the model, and invariants assert that the state of the model is equal to the state of the store. Hypothesis then generates sequences of rules, running invariants after each rule. It raises an error if a sequence produces discontinuity between state of the model and state of the store (ie. an invariant is violated). https://hypothesis.readthedocs.io/en/latest/stateful.html check_num_zarr_keys_equal() -> None# check_paths_equal() -> None# check_vals_equal() -> None# check_zarr_keys() -> None# clear() -> None# delete(data: hypothesis.strategies.DataObject) -> None# exists(key: str) -> None# get(key: str, data: hypothesis.strategies.DataObject) -> None# get_invalid_zarr_keys(key: str, data: hypothesis.strategies.DataObject) -> None# get_partial_values(data: hypothesis.strategies.DataObject) -> None# init_store() -> None# is_empty() -> None# set(key: str, data: bytes) -> None# model: dict[str, zarr.core.buffer.Buffer]# prototype# store# zarr.testing.stateful.split_prefix_name(path: str) -> tuple[str, str]# zarr.testing.stateful.with_frequency(frequency: float) -> collections.abc.Callable[[F], F]# This needs to be deterministic for hypothesis replaying zarr.testing.stateful.F# zarr.testing.stateful.MAX_BINARY_SIZE = 100# ###### zarr.testing.store# ###### Classes# `StoreTests` ###### Module Contents# class zarr.testing.store.StoreTests# Bases: `Generic`[`S`, `B`] abstract get(store: S, key: str) -> zarr.core.buffer.Buffer# Async: Retrieve a value from a storage backend, by key. This should not use any store methods. Bypassing the store methods allows them to be tested. open_kwargs(store_kwargs: dict[str, Any]) -> dict[str, Any]# abstract set(store: S, key: str, value: zarr.core.buffer.Buffer) -> None# Async: Insert a value into a storage backend, with a specific key. This should not use any store methods. Bypassing the store methods allows them to be tested. async store(open_kwargs: dict[str, Any]) -> zarr.abc.store.Store# abstract store_kwargs(*args: Any, **kwargs: Any) -> dict[str, Any]# Kwargs for instantiating a store async store_not_open(store_kwargs: dict[str, Any]) -> zarr.abc.store.Store# async test_clear(store: S) -> None# async test_delete(store: S) -> None# async test_delete_dir(store: S) -> None# async test_delete_nonexistent_key_does_not_raise(store: S) -> None# async test_exists(store: S) -> None# async test_get( store: S, key: str, data: bytes, byte_range: zarr.abc.store.ByteRequest, ) -> None# Ensure that data can be read from the store using the store.get method. async test_get_many(store: S) -> None# Ensure that multiple keys can be retrieved at once with the _get_many method. async test_get_not_open(store_not_open: S) -> None# Ensure that data can be read from the store that isn’t yet open using the store.get method. async test_get_partial_values( store: S, key_ranges: list[tuple[str, zarr.abc.store.ByteRequest]], ) -> None# async test_get_raises(store: S) -> None# Ensure that a ValueError is raise for invalid byte range syntax async test_getsize(store: S, key: str, data: bytes) -> None# Test the result of store.getsize(). async test_getsize_prefix(store: S) -> None# Test the result of store.getsize_prefix(). async test_getsize_raises(store: S) -> None# Test that getsize() raise a FileNotFoundError if the key doesn’t exist. async test_is_empty(store: S) -> None# async test_list(store: S) -> None# async test_list_dir(store: S) -> None# async test_list_empty_path(store: S) -> None# Verify that list and list_prefix work correctly when path is an empty string, i.e. no unwanted replacement occurs. async test_list_prefix(store: S) -> None# Test that the list_prefix method works as intended. Given a prefix, it should return all the keys in storage that start with this prefix. async test_read_only_store_raises(open_kwargs: dict[str, Any]) -> None# async test_serializable_store(store: S) -> None# async test_set(store: S, key: str, data: bytes) -> None# Ensure that data can be written to the store using the store.set method. async test_set_if_not_exists(store: S) -> None# async test_set_many(store: S) -> None# Test that a dict of key : value pairs can be inserted into the store via the _set_many method. async test_set_not_open(store_not_open: S) -> None# Ensure that data can be written to the store that’s not yet open using the store.set method. async test_store_context_manager(open_kwargs: dict[str, Any]) -> None# test_store_eq(store: S, store_kwargs: dict[str, Any]) -> None# async test_store_open_read_only(open_kwargs: dict[str, Any], read_only: bool) -> None# test_store_read_only(store: S) -> None# abstract test_store_repr(store: S) -> None# abstract test_store_supports_listing(store: S) -> None# test_store_supports_partial_writes(store: S) -> None# abstract test_store_supports_writes(store: S) -> None# test_store_type(store: S) -> None# async test_with_read_only_store(open_kwargs: dict[str, Any]) -> None# buffer_cls: type[B]# store_cls: type[S]# ###### zarr.testing.strategies# ###### Attributes# `array_names` `array_shapes` `attrs` `compressors` `node_names` `short_node_names` `stores` `zarr_formats` `zarr_key_chars` ###### Functions# `array_metadata`(...) `arrays`(, array_names, arrays, attrs, zarr_formats) `basic_indices`(→ Any) Basic indices without unsupported negative slices. `chunk_paths`(→ str) `chunk_shapes`(→ tuple[int, Ellipsis]) `clear_store`(→ zarr.abc.store.Store) `dimension_names`(→ list[None | str] | None) `dtypes`(...) `end_slices`(→ Any) A strategy that slices ranges that include the last chunk. `is_negative_slice`(→ bool) `key_ranges`(...) Function to generate key_ranges strategy for get_partial_values() `keys`(→ str) `np_array_and_chunks`() → tuple[numpy.ndarray, ...) A hypothesis strategy to generate small sized random arrays. `numpy_arrays`(→ numpy.typing.NDArray[Any]) Generate numpy arrays that can be saved in the provided Zarr format. `orthogonal_indices`(→ tuple[tuple[numpy.ndarray[Any, ...) Strategy that returns `paths`(→ str) `safe_unicode_for_dtype`(...) Generate UTF-8-safe text constrained to max_len of dtype. `shard_shapes`(→ tuple[int, Ellipsis]) `simple_arrays`(→ Any) `v2_dtypes`(...) `v3_dtypes`(...) ###### Module Contents# zarr.testing.strategies.array_metadata( *, array_shapes: collections.abc.Callable[Ellipsis, hypothesis.strategies.SearchStrategy[tuple[int, Ellipsis]]] = npst.array_shapes, zarr_formats: hypothesis.strategies.SearchStrategy[Literal[2, 3]] = zarr_formats, attributes: hypothesis.strategies.SearchStrategy[collections.abc.Mapping[str, zarr.core.common.JSON] | None] = attrs, ) -> zarr.core.metadata.ArrayV2Metadata | zarr.core.metadata.ArrayV3Metadata# zarr.testing.strategies.arrays( *, shapes: hypothesis.strategies.SearchStrategy[tuple[int, Ellipsis]] = array_shapes, compressors: hypothesis.strategies.SearchStrategy = compressors, stores: hypothesis.strategies.SearchStrategy[zarr.storage.StoreLike] = stores, paths: hypothesis.strategies.SearchStrategy[str] = paths(), array_names: hypothesis.strategies.SearchStrategy = array_names, arrays: hypothesis.strategies.SearchStrategy | None = None, attrs: hypothesis.strategies.SearchStrategy = attrs, zarr_formats: hypothesis.strategies.SearchStrategy = zarr_formats, ) -> zarr.core.array.Array# zarr.testing.strategies.basic_indices( *, shape: tuple[int, Ellipsis], min_dims: int = 0, max_dims: int | None = None, allow_newaxis: bool = False, allow_ellipsis: bool = True, ) -> Any# Basic indices without unsupported negative slices. zarr.testing.strategies.chunk_paths( ndim: int, numblocks: tuple[int, Ellipsis], subset: bool = True, ) -> str# zarr.testing.strategies.chunk_shapes(*, shape: tuple[int, Ellipsis]) -> tuple[int, Ellipsis]# zarr.testing.strategies.clear_store(x: zarr.abc.store.Store) -> zarr.abc.store.Store# zarr.testing.strategies.dimension_names(*, ndim: int | None = None) -> list[None | str] | None# zarr.testing.strategies.dtypes() -> hypothesis.strategies.SearchStrategy[numpy.dtype[Any]]# zarr.testing.strategies.end_slices(*, shape: tuple[int, Ellipsis]) -> Any# A strategy that slices ranges that include the last chunk. This is intended to stress-test handling of a possibly smaller last chunk. zarr.testing.strategies.is_negative_slice(idx: Any) -> bool# zarr.testing.strategies.key_ranges( keys: hypothesis.strategies.SearchStrategy[str] = node_names, max_size: int = sys.maxsize, ) -> hypothesis.strategies.SearchStrategy[list[tuple[str, zarr.abc.store.RangeByteRequest]]]# Function to generate key_ranges strategy for get_partial_values() returns list strategy w/ form: [(key, (range_start, range_end)), (key, (range_start, range_end)),...] zarr.testing.strategies.keys(*, max_num_nodes: int | None = None) -> str# zarr.testing.strategies.np_array_and_chunks( *, arrays: hypothesis.strategies.SearchStrategy[numpy.typing.NDArray[Any]] = numpy_arrays(), ) -> tuple[numpy.ndarray, tuple[int, Ellipsis]]# A hypothesis strategy to generate small sized random arrays. Returns: a tuple of the array and a suitable random chunking for it. zarr.testing.strategies.numpy_arrays( *, shapes: hypothesis.strategies.SearchStrategy[tuple[int, Ellipsis]] = array_shapes, dtype: numpy.dtype[Any] | None = None, ) -> numpy.typing.NDArray[Any]# Generate numpy arrays that can be saved in the provided Zarr format. zarr.testing.strategies.orthogonal_indices( *, shape: tuple[int, Ellipsis], ) -> tuple[tuple[numpy.ndarray[Any, Any], Ellipsis], tuple[numpy.ndarray[Any, Any], Ellipsis]]# Strategy that returns (1) a tuple of integer arrays used for orthogonal indexing of Zarr arrays. (2) an tuple of integer arrays that can be used for equivalent indexing of numpy arrays zarr.testing.strategies.paths(*, max_num_nodes: int | None = None) -> str# zarr.testing.strategies.safe_unicode_for_dtype( dtype: numpy.dtype[numpy.str_], ) -> hypothesis.strategies.SearchStrategy[str]# Generate UTF-8-safe text constrained to max_len of dtype. zarr.testing.strategies.shard_shapes( *, shape: tuple[int, Ellipsis], chunk_shape: tuple[int, Ellipsis], ) -> tuple[int, Ellipsis]# zarr.testing.strategies.simple_arrays( *, shapes: hypothesis.strategies.SearchStrategy[tuple[int, Ellipsis]] = array_shapes, ) -> Any# zarr.testing.strategies.v2_dtypes() -> hypothesis.strategies.SearchStrategy[numpy.dtype[Any]]# zarr.testing.strategies.v3_dtypes() -> hypothesis.strategies.SearchStrategy[numpy.dtype[Any]]# zarr.testing.strategies.array_names# zarr.testing.strategies.array_shapes# zarr.testing.strategies.attrs: hypothesis.strategies.SearchStrategy[collections.abc.Mapping[str, zarr.core.common.JSON] | None]# zarr.testing.strategies.compressors# zarr.testing.strategies.node_names# zarr.testing.strategies.short_node_names# zarr.testing.strategies.stores# zarr.testing.strategies.zarr_formats: hypothesis.strategies.SearchStrategy[zarr.core.common.ZarrFormat]# zarr.testing.strategies.zarr_key_chars# ###### zarr.testing.utils# ###### Functions# `assert_bytes_equal`(→ None) Help function to assert if two bytes-like or Buffers are equal ###### Module Contents# zarr.testing.utils.assert_bytes_equal( b1: zarr.core.buffer.Buffer | zarr.core.common.BytesLike | None, b2: zarr.core.buffer.Buffer | zarr.core.common.BytesLike | None, ) -> None# Help function to assert if two bytes-like or Buffers are equal Warning Always copies data, only use for testing and debugging ##### Functions# `assert_bytes_equal`(→ None) Help function to assert if two bytes-like or Buffers are equal ##### Package Contents# zarr.testing.assert_bytes_equal( b1: zarr.core.buffer.Buffer | zarr.core.common.BytesLike | None, b2: zarr.core.buffer.Buffer | zarr.core.common.BytesLike | None, ) -> None# Help function to assert if two bytes-like or Buffers are equal Warning Always copies data, only use for testing and debugging ### Attributes# `config` ### Classes# `Array` A Zarr array. `AsyncArray` An asynchronous array class representing a chunked array stored in a Zarr store. `AsyncGroup` Asynchronous Group object. `Group` A Zarr group. ### Functions# `array`(→ zarr.core.array.Array) Create an array filled with data. `consolidate_metadata`(→ zarr.core.group.Group) Consolidate the metadata of all nodes in a hierarchy. `copy`(→ tuple[int, int, int]) Not implemented. `copy_all`(→ tuple[int, int, int]) Not implemented. `copy_store`(→ tuple[int, int, int]) Not implemented. `create`(→ zarr.core.array.Array) Create an array. `create_array`(→ zarr.core.array.Array) Create an array. `create_group`(→ zarr.core.group.Group) Create a group. `create_hierarchy`(→ collections.abc.Iterator[tuple[str, ...) Create a complete zarr hierarchy from a collection of metadata objects. `empty`(→ zarr.core.array.Array) Create an empty array with the specified shape. The contents will be filled with the `empty_like`(→ zarr.core.array.Array) Create an empty array like another array. The contents will be filled with the `from_array`(→ zarr.core.array.Array) Create an array from an existing array or array-like. `full`(→ zarr.core.array.Array) Create an array with a default fill value. `full_like`(→ zarr.core.array.Array) Create a filled array like another array. `group`(→ zarr.core.group.Group) Create a group. `load`(...) Load data from an array or group into memory. `ones`(→ zarr.core.array.Array) Create an array with a fill value of one. `ones_like`(→ zarr.core.array.Array) Create an array of ones like another array. `open`(→ zarr.core.array.Array | zarr.core.group.Group) Open a group or array using file-mode-like semantics. `open_array`(→ zarr.core.array.Array) Open an array using file-mode-like semantics. `open_consolidated`(→ zarr.core.group.Group) Alias for `open_group()` with `use_consolidated=True`. `open_group`(→ zarr.core.group.Group) Open a group using file-mode-like semantics. `open_like`(→ zarr.core.array.Array) Open a persistent array like another array. `print_debug_info`(→ None) Print version info for use in bug reports. `save`(→ None) Save an array or group of arrays to the local file system. `save_array`(→ None) Save a NumPy array to the local file system. `save_group`(→ None) Save several NumPy arrays to the local file system. `tree`(→ Any) Provide a rich display of the hierarchy. `zeros`(→ zarr.core.array.Array) Create an array with a fill value of zero. `zeros_like`(→ zarr.core.array.Array) Create an array of zeros like another array. ### Package Contents# class zarr.Array# A Zarr array. append(data: numpy.typing.ArrayLike, axis: int = 0) -> tuple[int, Ellipsis]# Append data to axis. Parameters: dataarray-like Data to be appended. axisint Axis along which to append. Returns: new_shapetuple Notes The size of all dimensions other than axis must match between this array and data. Examples >>> import numpy as np >>> import zarr >>> a = np.arange(10000000, dtype='i4').reshape(10000, 1000) >>> z = zarr.array(a, chunks=(1000, 100)) >>> z.shape (10000, 1000) >>> z.append(a) (20000, 1000) >>> z.append(np.vstack([a, a]), axis=1) (20000, 2000) >>> z.shape (20000, 2000) classmethod create( store: zarr.storage.StoreLike, *, shape: tuple[int, Ellipsis], dtype: zarr.core.dtype.ZDTypeLike, zarr_format: zarr.core.common.ZarrFormat = 3, fill_value: Any | None = DEFAULT_FILL_VALUE, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_shape: tuple[int, Ellipsis] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncoding | tuple[Literal['default'], Literal['.', '/']] | tuple[Literal['v2'], Literal['.', '/']] | None = None, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] | None = None, dimension_names: zarr.core.common.DimensionNames = None, chunks: tuple[int, Ellipsis] | None = None, dimension_separator: Literal['.', '/'] | None = None, order: zarr.core.common.MemoryOrder | None = None, filters: list[dict[str, zarr.core.common.JSON]] | None = None, compressor: CompressorLike = 'auto', overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) -> Array# Creates a new Array instance from an initialized store. Deprecated since version 3.0.0: Deprecated in favor of `zarr.create_array()`. Parameters: storeStoreLike The array store that has already been initialized. shapetuple[int, …] The shape of the array. dtypeZDTypeLike The data type of the array. chunk_shapetuple[int, …], optional The shape of the Array’s chunks. Zarr format 3 only. Zarr format 2 arrays should use chunks instead. If not specified, default are guessed based on the shape and dtype. chunk_key_encodingChunkKeyEncodingLike, optional A specification of how the chunk keys are represented in storage. Zarr format 3 only. Zarr format 2 arrays should use dimension_separator instead. Default is `("default", "/")`. codecsSequence of Codecs or dicts, optional An iterable of Codec or dict serializations of Codecs. The elements of this collection specify the transformation from array values to stored bytes. Zarr format 3 only. Zarr format 2 arrays should use `filters` and `compressor` instead. If no codecs are provided, default codecs will be used: * For numeric arrays, the default is `BytesCodec` and `ZstdCodec`. * For Unicode strings, the default is `VLenUTF8Codec` and `ZstdCodec`. * For bytes or objects, the default is `VLenBytesCodec` and `ZstdCodec`. dimension_namesIterable[str | None], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. chunkstuple[int, …], optional The shape of the array’s chunks. Zarr format 2 only. Zarr format 3 arrays should use `chunk_shape` instead. If not specified, default are guessed based on the shape and dtype. dimension_separatorLiteral[“.”, “/”], optional The dimension separator (default is “.”). Zarr format 2 only. Zarr format 3 arrays should use `chunk_key_encoding` instead. orderLiteral[“C”, “F”], optional The memory of the array (default is “C”). If `zarr_format` is 2, this parameter sets the memory order of the array. If zarr_format` is 3, then this parameter is deprecated, because memory order is a runtime parameter for Zarr 3 arrays. The recommended way to specify the memory order for Zarr 3 arrays is via the `config` parameter, e.g. `{'order': 'C'}`. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressordict[str, JSON], optional Primary compressor to compress chunk data. Zarr format 2 only. Zarr format 3 arrays should use `codecs` instead. If no `compressor` is provided, a default compressor will be used: * For numeric arrays, the default is `ZstdCodec`. * For Unicode strings, the default is `VLenUTF8Codec`. * For bytes or objects, the default is `VLenBytesCodec`. These defaults can be changed by modifying the value of `array.v2_default_compressor` in `zarr.core.config`. overwritebool, optional Whether to raise an error if the store already exists (default is False). Returns: Array Array created from the store. classmethod from_dict( store_path: zarr.storage._common.StorePath, data: dict[str, zarr.core.common.JSON], ) -> Array# Create a Zarr array from a dictionary. Parameters: store_pathStorePath The path within the store where the array should be created. datadict A dictionary representing the array data. This dictionary should include necessary metadata for the array, such as shape, dtype, fill value, and attributes. Returns: Array The created Zarr array. Raises: ValueError If the dictionary data is invalid or missing required fields for array creation. get_basic_selection( selection: zarr.core.indexing.BasicSelection = Ellipsis, *, out: zarr.core.buffer.NDBuffer | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, fields: zarr.core.indexing.Fields | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# Retrieve data for an item or region of the array. Parameters: selectiontuple A tuple specifying the requested item or region for each dimension of the array. May be any combination of int and/or slice or ellipsis for multidimensional arrays. outNDBuffer, optional If given, load the selected data directly into this buffer. prototypeBufferPrototype, optional The prototype of the buffer to use for the output data. If not provided, the default buffer prototype is used. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to extract data for. Returns: NDArrayLikeOrScalar An array-like or scalar containing the data for the requested region. See also `set_basic_selection`, `get_mask_selection`, `set_mask_selection` `get_coordinate_selection`, `set_coordinate_selection`, `get_orthogonal_selection` `set_orthogonal_selection`, `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Slices with step > 1 are supported, but slices with negative step are not. For arrays with a structured dtype, see Zarr format 2 for examples of how to use the fields parameter. This method provides the implementation for accessing data via the square bracket notation (__getitem__). See `__getitem__()` for examples using the alternative notation. Examples Setup a 1-dimensional array: >>> import zarr >>> import numpy as np >>> data = np.arange(100, dtype="uint16") >>> z = zarr.create_array( >>> StorePath(MemoryStore(mode="w")), >>> shape=data.shape, >>> chunks=(3,), >>> dtype=data.dtype, >>> ) >>> z[:] = data Retrieve a single item: >>> z.get_basic_selection(5) 5 Retrieve a region via slicing: >>> z.get_basic_selection(slice(5)) array([0, 1, 2, 3, 4]) >>> z.get_basic_selection(slice(-5, None)) array([95, 96, 97, 98, 99]) >>> z.get_basic_selection(slice(5, 10)) array([5, 6, 7, 8, 9]) >>> z.get_basic_selection(slice(5, 10, 2)) array([5, 7, 9]) >>> z.get_basic_selection(slice(None, None, 2)) array([ 0, 2, 4, ..., 94, 96, 98]) Setup a 3-dimensional array: >>> data = np.arange(1000).reshape(10, 10, 10) >>> z = zarr.create_array( >>> StorePath(MemoryStore(mode="w")), >>> shape=data.shape, >>> chunks=(5, 5, 5), >>> dtype=data.dtype, >>> ) >>> z[:] = data Retrieve an item: >>> z.get_basic_selection((1, 2, 3)) 123 Retrieve a region via slicing and Ellipsis: >>> z.get_basic_selection((slice(1, 3), slice(1, 3), 0)) array([[110, 120], [210, 220]]) >>> z.get_basic_selection(0, (slice(1, 3), slice(None))) array([[10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [20, 21, 22, 23, 24, 25, 26, 27, 28, 29]]) >>> z.get_basic_selection((..., 5)) array([[ 2 12 22 32 42 52 62 72 82 92] [102 112 122 132 142 152 162 172 182 192] ... [802 812 822 832 842 852 862 872 882 892] [902 912 922 932 942 952 962 972 982 992]] get_block_selection( selection: zarr.core.indexing.BasicSelection, *, out: zarr.core.buffer.NDBuffer | None = None, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# Retrieve a selection of individual items, by providing the indices (coordinates) for each selected item. Parameters: selectionint or slice or tuple of int or slice An integer (coordinate) or slice for each dimension of the array. outNDBuffer, optional If given, load the selected data directly into this buffer. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to extract data for. prototypeBufferPrototype, optional The prototype of the buffer to use for the output data. If not provided, the default buffer prototype is used. Returns: NDArrayLikeOrScalar An array-like or scalar containing the data for the requested block selection. See also `get_basic_selection`, `set_basic_selection`, `get_mask_selection`, `set_mask_selection` `get_orthogonal_selection`, `set_orthogonal_selection`, `get_coordinate_selection` `set_coordinate_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Block indexing is a convenience indexing method to work on individual chunks with chunk index slicing. It has the same concept as Dask’s Array.blocks indexing. Slices are supported. However, only with a step size of one. Block index arrays may be multidimensional to index multidimensional arrays. For example: >>> z.blocks[0, 1:3] array([[ 3, 4, 5, 6, 7, 8], [13, 14, 15, 16, 17, 18], [23, 24, 25, 26, 27, 28]]) Examples Setup a 2-dimensional array: >>> import zarr >>> import numpy as np >>> data = np.arange(0, 100, dtype="uint16").reshape((10, 10)) >>> z = zarr.create_array( >>> StorePath(MemoryStore(mode="w")), >>> shape=data.shape, >>> chunks=(3, 3), >>> dtype=data.dtype, >>> ) >>> z[:] = data Retrieve items by specifying their block coordinates: >>> z.get_block_selection((1, slice(None))) array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) Which is equivalent to: >>> z[3:6, :] array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) For convenience, the block selection functionality is also available via the blocks property, e.g.: >>> z.blocks[1] array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49], [50, 51, 52, 53, 54, 55, 56, 57, 58, 59]]) get_coordinate_selection( selection: zarr.core.indexing.CoordinateSelection, *, out: zarr.core.buffer.NDBuffer | None = None, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# Retrieve a selection of individual items, by providing the indices (coordinates) for each selected item. Parameters: selectiontuple An integer (coordinate) array for each dimension of the array. outNDBuffer, optional If given, load the selected data directly into this buffer. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to extract data for. prototypeBufferPrototype, optional The prototype of the buffer to use for the output data. If not provided, the default buffer prototype is used. Returns: NDArrayLikeOrScalar An array-like or scalar containing the data for the requested coordinate selection. See also `get_basic_selection`, `set_basic_selection`, `get_mask_selection`, `set_mask_selection` `get_orthogonal_selection`, `set_orthogonal_selection`, `set_coordinate_selection` `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Coordinate indexing is also known as point selection, and is a form of vectorized or inner indexing. Slices are not supported. Coordinate arrays must be provided for all dimensions of the array. Coordinate arrays may be multidimensional, in which case the output array will also be multidimensional. Coordinate arrays are broadcast against each other before being applied. The shape of the output will be the same as the shape of each coordinate array after broadcasting. Examples Setup a 2-dimensional array: >>> import zarr >>> import numpy as np >>> data = np.arange(0, 100, dtype="uint16").reshape((10, 10)) >>> z = zarr.create_array( >>> StorePath(MemoryStore(mode="w")), >>> shape=data.shape, >>> chunks=(3, 3), >>> dtype=data.dtype, >>> ) >>> z[:] = data Retrieve items by specifying their coordinates: >>> z.get_coordinate_selection(([1, 4], [1, 4])) array([11, 44]) For convenience, the coordinate selection functionality is also available via the vindex property, e.g.: >>> z.vindex[[1, 4], [1, 4]] array([11, 44]) get_mask_selection( mask: zarr.core.indexing.MaskSelection, *, out: zarr.core.buffer.NDBuffer | None = None, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# Retrieve a selection of individual items, by providing a Boolean array of the same shape as the array against which the selection is being made, where True values indicate a selected item. Parameters: maskndarray, bool A Boolean array of the same shape as the array against which the selection is being made. outNDBuffer, optional If given, load the selected data directly into this buffer. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to extract data for. prototypeBufferPrototype, optional The prototype of the buffer to use for the output data. If not provided, the default buffer prototype is used. Returns: NDArrayLikeOrScalar An array-like or scalar containing the data for the requested selection. See also `get_basic_selection`, `set_basic_selection`, `set_mask_selection` `get_orthogonal_selection`, `set_orthogonal_selection`, `get_coordinate_selection` `set_coordinate_selection`, `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Mask indexing is a form of vectorized or inner indexing, and is equivalent to coordinate indexing. Internally the mask array is converted to coordinate arrays by calling np.nonzero. Examples Setup a 2-dimensional array: >>> import zarr >>> import numpy as np >>> data = np.arange(100).reshape(10, 10) >>> z = zarr.create_array( >>> StorePath(MemoryStore(mode="w")), >>> shape=data.shape, >>> chunks=data.shape, >>> dtype=data.dtype, >>> ) >>> z[:] = data Retrieve items by specifying a mask: >>> sel = np.zeros_like(z, dtype=bool) >>> sel[1, 1] = True >>> sel[4, 4] = True >>> z.get_mask_selection(sel) array([11, 44]) For convenience, the mask selection functionality is also available via the vindex property, e.g.: >>> z.vindex[sel] array([11, 44]) get_orthogonal_selection( selection: zarr.core.indexing.OrthogonalSelection, *, out: zarr.core.buffer.NDBuffer | None = None, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# Retrieve data by making a selection for each dimension of the array. For example, if an array has 2 dimensions, allows selecting specific rows and/or columns. The selection for each dimension can be either an integer (indexing a single item), a slice, an array of integers, or a Boolean array where True values indicate a selection. Parameters: selectiontuple A selection for each dimension of the array. May be any combination of int, slice, integer array or Boolean array. outNDBuffer, optional If given, load the selected data directly into this buffer. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to extract data for. prototypeBufferPrototype, optional The prototype of the buffer to use for the output data. If not provided, the default buffer prototype is used. Returns: NDArrayLikeOrScalar An array-like or scalar containing the data for the requested selection. See also `get_basic_selection`, `set_basic_selection`, `get_mask_selection`, `set_mask_selection` `get_coordinate_selection`, `set_coordinate_selection`, `set_orthogonal_selection` `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Orthogonal indexing is also known as outer indexing. Slices with step > 1 are supported, but slices with negative step are not. Examples Setup a 2-dimensional array: >>> import zarr >>> import numpy as np >>> data = np.arange(100).reshape(10, 10) >>> z = zarr.create_array( >>> StorePath(MemoryStore(mode="w")), >>> shape=data.shape, >>> chunks=data.shape, >>> dtype=data.dtype, >>> ) >>> z[:] = data Retrieve rows and columns via any combination of int, slice, integer array and/or Boolean array: >>> z.get_orthogonal_selection(([1, 4], slice(None))) array([[10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49]]) >>> z.get_orthogonal_selection((slice(None), [1, 4])) array([[ 1, 4], [11, 14], [21, 24], [31, 34], [41, 44], [51, 54], [61, 64], [71, 74], [81, 84], [91, 94]]) >>> z.get_orthogonal_selection(([1, 4], [1, 4])) array([[11, 14], [41, 44]]) >>> sel = np.zeros(z.shape[0], dtype=bool) >>> sel[1] = True >>> sel[4] = True >>> z.get_orthogonal_selection((sel, sel)) array([[11, 14], [41, 44]]) For convenience, the orthogonal selection functionality is also available via the oindex property, e.g.: >>> z.oindex[[1, 4], :] array([[10, 11, 12, 13, 14, 15, 16, 17, 18, 19], [40, 41, 42, 43, 44, 45, 46, 47, 48, 49]]) >>> z.oindex[:, [1, 4]] array([[ 1, 4], [11, 14], [21, 24], [31, 34], [41, 44], [51, 54], [61, 64], [71, 74], [81, 84], [91, 94]]) >>> z.oindex[[1, 4], [1, 4]] array([[11, 14], [41, 44]]) >>> sel = np.zeros(z.shape[0], dtype=bool) >>> sel[1] = True >>> sel[4] = True >>> z.oindex[sel, sel] array([[11, 14], [41, 44]]) info_complete() -> Any# Returns all the information about an array, including information from the Store. In addition to the statically known information like `name` and `zarr_format`, this includes additional information like the size of the array in bytes and the number of chunks written. Note that this method will need to read metadata from the store. Returns: ArrayInfo See also `Array.info` The statically known subset of metadata about an array. nbytes_stored() -> int# Determine the size, in bytes, of the array actually written to the store. Returns: sizeint classmethod open(store: zarr.storage.StoreLike) -> Array# Opens an existing Array from a store. Parameters: storeStoreLike Store containing the Array. Returns: Array Array opened from the store. resize(new_shape: zarr.core.common.ShapeLike) -> None# Change the shape of the array by growing or shrinking one or more dimensions. Parameters: new_shapetuple New shape of the array. Notes If one or more dimensions are shrunk, any chunks falling outside the new array shape will be deleted from the underlying store. However, it is noteworthy that the chunks partially falling inside the new array (i.e. boundary chunks) will remain intact, and therefore, the data falling outside the new array but inside the boundary chunks would be restored by a subsequent resize operation that grows the array size. Examples >>> import zarr >>> z = zarr.zeros(shape=(10000, 10000), >>> chunk_shape=(1000, 1000), >>> dtype="i4",) >>> z.shape (10000, 10000) >>> z = z.resize(20000, 1000) >>> z.shape (20000, 1000) >>> z2 = z.resize(50, 50) >>> z.shape (20000, 1000) >>> z2.shape (50, 50) set_basic_selection( selection: zarr.core.indexing.BasicSelection, value: numpy.typing.ArrayLike, *, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> None# Modify data for an item or region of the array. Parameters: selectiontuple A tuple specifying the requested item or region for each dimension of the array. May be any combination of int and/or slice or ellipsis for multidimensional arrays. valuenpt.ArrayLike An array-like containing values to be stored into the array. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to set data for. prototypeBufferPrototype, optional The prototype of the buffer used for setting the data. If not provided, the default buffer prototype is used. See also `get_basic_selection`, `get_mask_selection`, `set_mask_selection` `get_coordinate_selection`, `set_coordinate_selection`, `get_orthogonal_selection` `set_orthogonal_selection`, `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes For arrays with a structured dtype, see Zarr format 2 for examples of how to use the fields parameter. This method provides the underlying implementation for modifying data via square bracket notation, see `__setitem__()` for equivalent examples using the alternative notation. Examples Setup a 1-dimensional array: >>> import zarr >>> z = zarr.zeros( >>> shape=(100,), >>> store=StorePath(MemoryStore(mode="w")), >>> chunk_shape=(100,), >>> dtype="i4", >>> ) Set all array elements to the same scalar value: >>> z.set_basic_selection(..., 42) >>> z[...] array([42, 42, 42, ..., 42, 42, 42]) Set a portion of the array: >>> z.set_basic_selection(slice(10), np.arange(10)) >>> z.set_basic_selection(slice(-10, None), np.arange(10)[::-1]) >>> z[...] array([ 0, 1, 2, ..., 2, 1, 0]) Setup a 2-dimensional array: >>> z = zarr.zeros( >>> shape=(5, 5), >>> store=StorePath(MemoryStore(mode="w")), >>> chunk_shape=(5, 5), >>> dtype="i4", >>> ) Set all array elements to the same scalar value: >>> z.set_basic_selection(..., 42) Set a portion of the array: >>> z.set_basic_selection((0, slice(None)), np.arange(z.shape[1])) >>> z.set_basic_selection((slice(None), 0), np.arange(z.shape[0])) >>> z[...] array([[ 0, 1, 2, 3, 4], [ 1, 42, 42, 42, 42], [ 2, 42, 42, 42, 42], [ 3, 42, 42, 42, 42], [ 4, 42, 42, 42, 42]]) set_block_selection( selection: zarr.core.indexing.BasicSelection, value: numpy.typing.ArrayLike, *, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> None# Modify a selection of individual blocks, by providing the chunk indices (coordinates) for each block to be modified. Parameters: selectiontuple An integer (coordinate) or slice for each dimension of the array. valuenpt.ArrayLike An array-like containing the data to be stored in the block selection. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to set data for. prototypeBufferPrototype, optional The prototype of the buffer used for setting the data. If not provided, the default buffer prototype is used. See also `get_basic_selection`, `set_basic_selection`, `get_mask_selection`, `set_mask_selection` `get_orthogonal_selection`, `set_orthogonal_selection`, `get_coordinate_selection` `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Block indexing is a convenience indexing method to work on individual chunks with chunk index slicing. It has the same concept as Dask’s Array.blocks indexing. Slices are supported. However, only with a step size of one. Examples Set up a 2-dimensional array: >>> import zarr >>> z = zarr.zeros( >>> shape=(6, 6), >>> store=StorePath(MemoryStore(mode="w")), >>> chunk_shape=(2, 2), >>> dtype="i4", >>> ) Set data for a selection of items: >>> z.set_block_selection((1, 0), 1) >>> z[...] array([[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]]) For convenience, this functionality is also available via the blocks property. E.g.: >>> z.blocks[2, 1] = 4 >>> z[...] array([[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0], [0, 0, 4, 4, 0, 0], [0, 0, 4, 4, 0, 0]]) >>> z.blocks[:, 2] = 7 >>> z[...] array([[0, 0, 0, 0, 7, 7], [0, 0, 0, 0, 7, 7], [1, 1, 0, 0, 7, 7], [1, 1, 0, 0, 7, 7], [0, 0, 4, 4, 7, 7], [0, 0, 4, 4, 7, 7]]) set_coordinate_selection( selection: zarr.core.indexing.CoordinateSelection, value: numpy.typing.ArrayLike, *, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> None# Modify a selection of individual items, by providing the indices (coordinates) for each item to be modified. Parameters: selectiontuple An integer (coordinate) array for each dimension of the array. valuenpt.ArrayLike An array-like containing values to be stored into the array. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to set data for. See also `get_basic_selection`, `set_basic_selection`, `get_mask_selection`, `set_mask_selection` `get_orthogonal_selection`, `set_orthogonal_selection`, `get_coordinate_selection` `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Coordinate indexing is also known as point selection, and is a form of vectorized or inner indexing. Slices are not supported. Coordinate arrays must be provided for all dimensions of the array. Examples Setup a 2-dimensional array: >>> import zarr >>> z = zarr.zeros( >>> shape=(5, 5), >>> store=StorePath(MemoryStore(mode="w")), >>> chunk_shape=(5, 5), >>> dtype="i4", >>> ) Set data for a selection of items: >>> z.set_coordinate_selection(([1, 4], [1, 4]), 1) >>> z[...] array([[0, 0, 0, 0, 0], [0, 1, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 1]]) For convenience, this functionality is also available via the vindex property. E.g.: >>> z.vindex[[1, 4], [1, 4]] = 2 >>> z[...] array([[0, 0, 0, 0, 0], [0, 2, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 2]]) set_mask_selection( mask: zarr.core.indexing.MaskSelection, value: numpy.typing.ArrayLike, *, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> None# Modify a selection of individual items, by providing a Boolean array of the same shape as the array against which the selection is being made, where True values indicate a selected item. Parameters: maskndarray, bool A Boolean array of the same shape as the array against which the selection is being made. valuenpt.ArrayLike An array-like containing values to be stored into the array. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to set data for. See also `get_basic_selection`, `set_basic_selection`, `get_mask_selection` `get_orthogonal_selection`, `set_orthogonal_selection`, `get_coordinate_selection` `set_coordinate_selection`, `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Mask indexing is a form of vectorized or inner indexing, and is equivalent to coordinate indexing. Internally the mask array is converted to coordinate arrays by calling np.nonzero. Examples Setup a 2-dimensional array: >>> import zarr >>> z = zarr.zeros( >>> shape=(5, 5), >>> store=StorePath(MemoryStore(mode="w")), >>> chunk_shape=(5, 5), >>> dtype="i4", >>> ) Set data for a selection of items: >>> sel = np.zeros_like(z, dtype=bool) >>> sel[1, 1] = True >>> sel[4, 4] = True >>> z.set_mask_selection(sel, 1) >>> z[...] array([[0, 0, 0, 0, 0], [0, 1, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 1]]) For convenience, this functionality is also available via the vindex property. E.g.: >>> z.vindex[sel] = 2 >>> z[...] array([[0, 0, 0, 0, 0], [0, 2, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 2]]) set_orthogonal_selection( selection: zarr.core.indexing.OrthogonalSelection, value: numpy.typing.ArrayLike, *, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> None# Modify data via a selection for each dimension of the array. Parameters: selectiontuple A selection for each dimension of the array. May be any combination of int, slice, integer array or Boolean array. valuenpt.ArrayLike An array-like array containing the data to be stored in the array. fieldsstr or sequence of str, optional For arrays with a structured dtype, one or more fields can be specified to set data for. prototypeBufferPrototype, optional The prototype of the buffer used for setting the data. If not provided, the default buffer prototype is used. See also `get_basic_selection`, `set_basic_selection`, `get_mask_selection`, `set_mask_selection` `get_coordinate_selection`, `set_coordinate_selection`, `get_orthogonal_selection` `get_block_selection`, `set_block_selection` `vindex`, `oindex`, `blocks`, `__getitem__`, `__setitem__` Notes Orthogonal indexing is also known as outer indexing. Slices with step > 1 are supported, but slices with negative step are not. Examples Setup a 2-dimensional array: >>> import zarr >>> z = zarr.zeros( >>> shape=(5, 5), >>> store=StorePath(MemoryStore(mode="w")), >>> chunk_shape=(5, 5), >>> dtype="i4", >>> ) Set data for a selection of rows: >>> z.set_orthogonal_selection(([1, 4], slice(None)), 1) >>> z[...] array([[0, 0, 0, 0, 0], [1, 1, 1, 1, 1], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [1, 1, 1, 1, 1]]) Set data for a selection of columns: >>> z.set_orthogonal_selection((slice(None), [1, 4]), 2) >>> z[...] array([[0, 2, 0, 0, 2], [1, 2, 1, 1, 2], [0, 2, 0, 0, 2], [0, 2, 0, 0, 2], [1, 2, 1, 1, 2]]) Set data for a selection of rows and columns: >>> z.set_orthogonal_selection(([1, 4], [1, 4]), 3) >>> z[...] array([[0, 2, 0, 0, 2], [1, 3, 1, 1, 3], [0, 2, 0, 0, 2], [0, 2, 0, 0, 2], [1, 3, 1, 1, 3]]) Set data from a 2D array: >>> values = np.arange(10).reshape(2, 5) >>> z.set_orthogonal_selection(([0, 3], ...), values) >>> z[...] array([[0, 1, 2, 3, 4], [1, 3, 1, 1, 3], [0, 2, 0, 0, 2], [5, 6, 7, 8, 9], [1, 3, 1, 1, 3]]) For convenience, this functionality is also available via the oindex property. E.g.: >>> z.oindex[[1, 4], [1, 4]] = 4 >>> z[...] array([[0, 1, 2, 3, 4], [1, 4, 1, 1, 4], [0, 2, 0, 0, 2], [5, 6, 7, 8, 9], [1, 4, 1, 1, 4]]) update_attributes(new_attributes: dict[str, zarr.core.common.JSON]) -> Array# Update the array’s attributes. Parameters: new_attributesdict A dictionary of new attributes to update or add to the array. The keys represent attribute names, and the values must be JSON-compatible. Returns: Array The array with the updated attributes. Raises: ValueError If the attributes are invalid or incompatible with the array’s metadata. Notes * The updated attributes will be merged with existing attributes, and any conflicts will be overwritten by the new values. property attrs: zarr.core.attributes.Attributes# Returns a MutableMapping containing user-defined attributes. Returns: attrsMutableMapping A MutableMapping object containing user-defined attributes. Notes Note that attribute values must be JSON serializable. property basename: str# Final component of name. property blocks: zarr.core.indexing.BlockIndex# Shortcut for blocked chunked indexing, see `get_block_selection()` and `set_block_selection()` for documentation and examples. property cdata_shape: tuple[int, Ellipsis]# The shape of the chunk grid for this array. property chunks: tuple[int, Ellipsis]# Returns a tuple of integers describing the length of each dimension of a chunk of the array. If sharding is used the inner chunk shape is returned. Only defined for arrays using using RegularChunkGrid. If array doesn’t use RegularChunkGrid, NotImplementedError is raised. Returns: tuple A tuple of integers representing the length of each dimension of a chunk. property compressor: zarr.abc.numcodec.Numcodec | None# Compressor that is applied to each chunk of the array. Deprecated since version 3.0.0: array.compressor is deprecated and will be removed in a future release. Use array.compressors instead. property compressors: tuple[zarr.abc.numcodec.Numcodec, Ellipsis] | tuple[zarr.abc.codec.BytesBytesCodec, Ellipsis]# Compressors that are applied to each chunk of the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. property dtype: numpy.dtype[Any]# Returns the NumPy data type. Returns: np.dtype The NumPy data type. property fill_value: Any# property filters: tuple[zarr.abc.numcodec.Numcodec, Ellipsis] | tuple[zarr.abc.codec.ArrayArrayCodec, Ellipsis]# Filters that are applied to each chunk of the array, in order, before serializing that chunk to bytes. property info: Any# Return the statically known information for an array. Returns: ArrayInfo See also `Array.info_complete` All information about a group, including dynamic information like the number of bytes and chunks written. Examples >>> arr = zarr.create(shape=(10,), chunks=(2,), dtype="float32") >>> arr.info Type : Array Zarr format : 3 Data type : DataType.float32 Shape : (10,) Chunk shape : (2,) Order : C Read-only : False Store type : MemoryStore Codecs : [BytesCodec(endian=)] No. bytes : 40 property metadata: zarr.core.metadata.ArrayMetadata# property name: str# Array name following h5py convention. property nbytes: int# The total number of bytes that can be stored in the chunks of this array. Notes This value is calculated by multiplying the number of elements in the array and the size of each element, the latter of which is determined by the dtype of the array. For this reason, `nbytes` will likely be inaccurate for arrays with variable-length dtypes. It is not possible to determine the size of an array with variable-length elements from the shape and dtype alone. property nchunks: int# The number of chunks in this array. Note that if a sharding codec is used, then the number of chunks may exceed the number of stored objects supporting this array. property nchunks_initialized: int# Calculate the number of chunks that have been initialized in storage. This value is calculated as the product of the number of initialized shards and the number of chunks per shard. For arrays that do not use sharding, the number of chunks per shard is effectively 1, and in that case the number of chunks initialized is the same as the number of stored objects associated with an array. For a direct count of the number of initialized stored objects, see `nshards_initialized`. Returns: nchunks_initializedint The number of chunks that have been initialized. Examples >>> arr = zarr.create_array(store={}, shape=(10,), chunks=(1,), shards=(2,)) >>> arr.nchunks_initialized 0 >>> arr[:5] = 1 >>> arr.nchunks_initialized 6 property ndim: int# Returns the number of dimensions in the array. Returns: int The number of dimensions in the array. property oindex: zarr.core.indexing.OIndex# Shortcut for orthogonal (outer) indexing, see `get_orthogonal_selection()` and `set_orthogonal_selection()` for documentation and examples. property order: zarr.core.common.MemoryOrder# property path: str# Storage path. property read_only: bool# property serializer: None | zarr.abc.codec.ArrayBytesCodec# Array-to-bytes codec to use for serializing the chunks into bytes. property shape: tuple[int, Ellipsis]# Returns the shape of the array. Returns: tuple[int, …] The shape of the array. property shards: tuple[int, Ellipsis] | None# Returns a tuple of integers describing the length of each dimension of a shard of the array. Returns None if sharding is not used. Only defined for arrays using using RegularChunkGrid. If array doesn’t use RegularChunkGrid, NotImplementedError is raised. Returns: tuple | None A tuple of integers representing the length of each dimension of a shard or None if sharding is not used. property size: int# Returns the total number of elements in the array. Returns: int Total number of elements in the array. property store: zarr.abc.store.Store# property store_path: zarr.storage._common.StorePath# property vindex: zarr.core.indexing.VIndex# Shortcut for vectorized (inner) indexing, see `get_coordinate_selection()`, `set_coordinate_selection()`, `get_mask_selection()` and `set_mask_selection()` for documentation and examples. class zarr.AsyncArray( metadata: zarr.core.metadata.ArrayV2Metadata | zarr.core.metadata.ArrayV2MetadataDict, store_path: zarr.storage._common.StorePath, config: zarr.core.array_spec.ArrayConfigLike | None = None, )# class zarr.AsyncArray( metadata: zarr.core.metadata.ArrayV3Metadata | zarr.core.metadata.ArrayV3MetadataDict, store_path: zarr.storage._common.StorePath, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) Bases: `Generic`[`zarr.core.metadata.T_ArrayMetadata`] An asynchronous array class representing a chunked array stored in a Zarr store. Parameters: metadataArrayMetadata The metadata of the array. store_pathStorePath The path to the Zarr store. configArrayConfigLike, optional The runtime configuration of the array, by default None. Attributes: metadataArrayMetadata The metadata of the array. store_pathStorePath The path to the Zarr store. codec_pipelineCodecPipeline The codec pipeline used for encoding and decoding chunks. _configArrayConfig The runtime configuration of the array. async append(data: numpy.typing.ArrayLike, axis: int = 0) -> tuple[int, Ellipsis]# Append data to axis. Parameters: dataarray-like Data to be appended. axisint Axis along which to append. Returns: new_shapetuple Notes The size of all dimensions other than axis must match between this array and data. classmethod create( store: zarr.storage.StoreLike, *, shape: zarr.core.common.ShapeLike, dtype: zarr.core.dtype.ZDTypeLike, zarr_format: Literal[2], fill_value: Any | None = DEFAULT_FILL_VALUE, attributes: dict[str, zarr.core.common.JSON] | None = None, chunks: zarr.core.common.ShapeLike | None = None, dimension_separator: Literal['.', '/'] | None = None, order: zarr.core.common.MemoryOrder | None = None, filters: list[dict[str, zarr.core.common.JSON]] | None = None, compressor: zarr.core.metadata.v2.CompressorLikev2 | Literal['auto'] = 'auto', overwrite: bool = False, data: numpy.typing.ArrayLike | None = None, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) -> AsyncArray[zarr.core.metadata.ArrayV2Metadata]# classmethod create( store: zarr.storage.StoreLike, *, shape: zarr.core.common.ShapeLike, dtype: zarr.core.dtype.ZDTypeLike, zarr_format: Literal[3], fill_value: Any | None = DEFAULT_FILL_VALUE, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_shape: zarr.core.common.ShapeLike | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncoding | tuple[Literal['default'], Literal['.', '/']] | tuple[Literal['v2'], Literal['.', '/']] | None = None, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] | None = None, dimension_names: zarr.core.common.DimensionNames = None, overwrite: bool = False, data: numpy.typing.ArrayLike | None = None, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) -> AsyncArray[zarr.core.metadata.ArrayV3Metadata] classmethod create( store: zarr.storage.StoreLike, *, shape: zarr.core.common.ShapeLike, dtype: zarr.core.dtype.ZDTypeLike, zarr_format: Literal[3] = 3, fill_value: Any | None = DEFAULT_FILL_VALUE, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_shape: zarr.core.common.ShapeLike | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncoding | tuple[Literal['default'], Literal['.', '/']] | tuple[Literal['v2'], Literal['.', '/']] | None = None, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] | None = None, dimension_names: zarr.core.common.DimensionNames = None, overwrite: bool = False, data: numpy.typing.ArrayLike | None = None, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) -> AsyncArray[zarr.core.metadata.ArrayV3Metadata] classmethod create( store: zarr.storage.StoreLike, *, shape: zarr.core.common.ShapeLike, dtype: zarr.core.dtype.ZDTypeLike, zarr_format: zarr.core.common.ZarrFormat, fill_value: Any | None = DEFAULT_FILL_VALUE, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_shape: zarr.core.common.ShapeLike | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncoding | tuple[Literal['default'], Literal['.', '/']] | tuple[Literal['v2'], Literal['.', '/']] | None = None, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] | None = None, dimension_names: zarr.core.common.DimensionNames = None, chunks: zarr.core.common.ShapeLike | None = None, dimension_separator: Literal['.', '/'] | None = None, order: zarr.core.common.MemoryOrder | None = None, filters: list[dict[str, zarr.core.common.JSON]] | None = None, compressor: CompressorLike = 'auto', overwrite: bool = False, data: numpy.typing.ArrayLike | None = None, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) -> AsyncArray[zarr.core.metadata.ArrayV3Metadata] | AsyncArray[zarr.core.metadata.ArrayV2Metadata] Async: Method to create a new asynchronous array instance. Deprecated since version 3.0.0: Deprecated in favor of `zarr.api.asynchronous.create_array()`. Parameters: storeStoreLike The store where the array will be created. shapeShapeLike The shape of the array. dtypeZDTypeLike The data type of the array. zarr_formatZarrFormat, optional The Zarr format version (default is 3). fill_valueAny, optional The fill value of the array (default is None). attributesdict[str, JSON], optional The attributes of the array (default is None). chunk_shapetuple[int, …], optional The shape of the array’s chunks Zarr format 3 only. Zarr format 2 arrays should use chunks instead. If not specified, default are guessed based on the shape and dtype. chunk_key_encodingChunkKeyEncodingLike, optional A specification of how the chunk keys are represented in storage. Zarr format 3 only. Zarr format 2 arrays should use dimension_separator instead. Default is `("default", "/")`. codecsSequence of Codecs or dicts, optional An iterable of Codec or dict serializations of Codecs. The elements of this collection specify the transformation from array values to stored bytes. Zarr format 3 only. Zarr format 2 arrays should use `filters` and `compressor` instead. If no codecs are provided, default codecs will be used: dimension_namesIterable[str | None], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. chunksShapeLike, optional The shape of the array’s chunks. Zarr format 2 only. Zarr format 3 arrays should use `chunk_shape` instead. If not specified, default are guessed based on the shape and dtype. dimension_separatorLiteral[“.”, “/”], optional The dimension separator (default is “.”). Zarr format 2 only. Zarr format 3 arrays should use `chunk_key_encoding` instead. orderLiteral[“C”, “F”], optional The memory of the array (default is “C”). If `zarr_format` is 2, this parameter sets the memory order of the array. If zarr_format` is 3, then this parameter is deprecated, because memory order is a runtime parameter for Zarr 3 arrays. The recommended way to specify the memory order for Zarr 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressordict[str, JSON], optional The compressor used to compress the data (default is None). Zarr format 2 only. Zarr format 3 arrays should use `codecs` instead. If no `compressor` is provided, a default compressor will be used: * For numeric arrays, the default is `ZstdCodec`. * For Unicode strings, the default is `VLenUTF8Codec`. * For bytes or objects, the default is `VLenBytesCodec`. These defaults can be changed by modifying the value of `array.v2_default_compressor` in `zarr.core.config`. overwritebool, optional Whether to raise an error if the store already exists (default is False). datanpt.ArrayLike, optional The data to be inserted into the array (default is None). configArrayConfigLike, optional Runtime configuration for the array. Returns: AsyncArray The created asynchronous array instance. classmethod from_dict( store_path: zarr.storage._common.StorePath, data: dict[str, zarr.core.common.JSON], ) -> AsyncArray[zarr.core.metadata.ArrayV3Metadata] | AsyncArray[zarr.core.metadata.ArrayV2Metadata]# Create a Zarr array from a dictionary, with support for both Zarr format 2 and 3 metadata. Parameters: store_pathStorePath The path within the store where the array should be created. datadict A dictionary representing the array data. This dictionary should include necessary metadata for the array, such as shape, dtype, and other attributes. The format of the metadata will determine whether a Zarr format 2 or 3 array is created. Returns: AsyncArray[ArrayV3Metadata] or AsyncArray[ArrayV2Metadata] The created Zarr array, either using Zarr format 2 or 3 metadata based on the provided data. Raises: ValueError If the dictionary data is invalid or incompatible with either Zarr format 2 or 3 array creation. async get_coordinate_selection( selection: zarr.core.indexing.CoordinateSelection, *, out: zarr.core.buffer.NDBuffer | None = None, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# async get_mask_selection( mask: zarr.core.indexing.MaskSelection, *, out: zarr.core.buffer.NDBuffer | None = None, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# async get_orthogonal_selection( selection: zarr.core.indexing.OrthogonalSelection, *, out: zarr.core.buffer.NDBuffer | None = None, fields: zarr.core.indexing.Fields | None = None, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# async getitem( selection: zarr.core.indexing.BasicSelection, *, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar# Asynchronous function that retrieves a subset of the array’s data based on the provided selection. Parameters: selectionBasicSelection A selection object specifying the subset of data to retrieve. prototypeBufferPrototype, optional A buffer prototype to use for the retrieved data (default is None). Returns: NDArrayLikeOrScalar The retrieved subset of the array’s data. Examples >>> import zarr >>> store = zarr.storage.MemoryStore() >>> async_arr = await zarr.api.asynchronous.create_array( ... store=store, ... shape=(100,100), ... chunks=(10,10), ... dtype='i4', ... fill_value=0) >>> await async_arr.getitem((0,1)) array(0, dtype=int32) async info_complete() -> Any# Return all the information for an array, including dynamic information like a storage size. In addition to the static information, this provides * The count of chunks initialized * The sum of the bytes written Returns: ArrayInfo See also `AsyncArray.info` A property giving just the statically known information about an array. async nbytes_stored() -> int# async nchunks_initialized() -> int# Calculate the number of chunks that have been initialized in storage. This value is calculated as the product of the number of initialized shards and the number of chunks per shard. For arrays that do not use sharding, the number of chunks per shard is effectively 1, and in that case the number of chunks initialized is the same as the number of stored objects associated with an array. Returns: nchunks_initializedint The number of chunks that have been initialized. Notes On `AsyncArray` this is an asynchronous method, unlike the (synchronous) property `Array.nchunks_initialized`. Examples >>> arr = await zarr.api.asynchronous.create(shape=(10,), chunks=(1,), shards=(2,)) >>> await arr.nchunks_initialized() 0 >>> await arr.setitem(slice(5), 1) >>> await arr.nchunks_initialized() 6 classmethod open( store: zarr.storage.StoreLike, zarr_format: zarr.core.common.ZarrFormat | None = 3, ) -> AsyncArray[zarr.core.metadata.ArrayV3Metadata] | AsyncArray[zarr.core.metadata.ArrayV2Metadata]# Async: Async method to open an existing Zarr array from a given store. Parameters: storeStoreLike The store containing the Zarr array. zarr_formatZarrFormat | None, optional The Zarr format version (default is 3). Returns: AsyncArray The opened Zarr array. Examples >>> import zarr >>> store = zarr.storage.MemoryStore() >>> async_arr = await AsyncArray.open(store) async resize( new_shape: zarr.core.common.ShapeLike, delete_outside_chunks: bool = True, ) -> None# Asynchronously resize the array to a new shape. Parameters: new_shapetuple[int, …] The desired new shape of the array. delete_outside_chunksbool, optional If True (default), chunks that fall outside the new shape will be deleted. If False, the data in those chunks will be preserved. Returns: AsyncArray The resized array. Raises: ValueError If the new shape is incompatible with the current array’s chunking configuration. Notes * This method is asynchronous and should be awaited. async setitem( selection: zarr.core.indexing.BasicSelection, value: numpy.typing.ArrayLike, prototype: zarr.core.buffer.BufferPrototype | None = None, ) -> None# Asynchronously set values in the array using basic indexing. Parameters: selectionBasicSelection The selection defining the region of the array to set. valuenumpy.typing.ArrayLike The values to be written into the selected region of the array. prototypeBufferPrototype or None, optional A prototype buffer that defines the structure and properties of the array chunks being modified. If None, the default buffer prototype is used. Default is None. Returns: None This method does not return any value. Raises: IndexError If the selection is out of bounds for the array. ValueError If the values are not compatible with the array’s dtype or shape. Notes * This method is asynchronous and should be awaited. * Supports basic indexing, where the selection is contiguous and does not involve advanced indexing. async update_attributes(new_attributes: dict[str, zarr.core.common.JSON]) -> Self# Asynchronously update the array’s attributes. Parameters: new_attributesdict of str to JSON A dictionary of new attributes to update or add to the array. The keys represent attribute names, and the values must be JSON-compatible. Returns: AsyncArray The array with the updated attributes. Raises: ValueError If the attributes are invalid or incompatible with the array’s metadata. Notes * This method is asynchronous and should be awaited. * The updated attributes will be merged with existing attributes, and any conflicts will be overwritten by the new values. property attrs: dict[str, zarr.core.common.JSON]# Returns the attributes of the array. Returns: dict Attributes of the array property basename: str# Final component of name. Returns: str The basename or final component of the array name. property cdata_shape: tuple[int, Ellipsis]# The shape of the chunk grid for this array. Returns: tuple[int, …] The shape of the chunk grid for this array. property chunks: tuple[int, Ellipsis]# Returns the chunk shape of the Array. If sharding is used the inner chunk shape is returned. Only defined for arrays using using RegularChunkGrid. If array doesn’t use RegularChunkGrid, NotImplementedError is raised. Returns: tuple[int, …]: The chunk shape of the Array. codec_pipeline: zarr.abc.codec.CodecPipeline# property compressor: zarr.abc.numcodec.Numcodec | None# Compressor that is applied to each chunk of the array. Deprecated since version 3.0.0: array.compressor is deprecated and will be removed in a future release. Use array.compressors instead. property compressors: tuple[zarr.abc.numcodec.Numcodec, Ellipsis] | tuple[zarr.abc.codec.BytesBytesCodec, Ellipsis]# Compressors that are applied to each chunk of the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. property dtype: zarr.core.dtype.wrapper.TBaseDType# Returns the data type of the array. Returns: np.dtype Data type of the array property filters: tuple[zarr.abc.numcodec.Numcodec, Ellipsis] | tuple[zarr.abc.codec.ArrayArrayCodec, Ellipsis]# Filters that are applied to each chunk of the array, in order, before serializing that chunk to bytes. property info: Any# Return the statically known information for an array. Returns: ArrayInfo See also `AsyncArray.info_complete` All information about a group, including dynamic information like the number of bytes and chunks written. Examples >>> arr = await zarr.api.asynchronous.create( ... path="array", shape=(3, 4, 5), chunks=(2, 2, 2)) ... ) >>> arr.info Type : Array Zarr format : 3 Data type : DataType.float64 Shape : (3, 4, 5) Chunk shape : (2, 2, 2) Order : C Read-only : False Store type : MemoryStore Codecs : [{'endian': }] No. bytes : 480 metadata: zarr.core.metadata.T_ArrayMetadata# property name: str# Array name following h5py convention. Returns: str The name of the array. property nbytes: int# The total number of bytes that can be stored in the chunks of this array. Notes This value is calculated by multiplying the number of elements in the array and the size of each element, the latter of which is determined by the dtype of the array. For this reason, `nbytes` will likely be inaccurate for arrays with variable-length dtypes. It is not possible to determine the size of an array with variable-length elements from the shape and dtype alone. property nchunks: int# The number of chunks in this array. Note that if a sharding codec is used, then the number of chunks may exceed the number of stored objects supporting this array. Returns: int The total number of chunks in the array. property ndim: int# Returns the number of dimensions in the Array. Returns: int The number of dimensions in the Array. property oindex: zarr.core.indexing.AsyncOIndex[zarr.core.metadata.T_ArrayMetadata]# Shortcut for orthogonal (outer) indexing, see `get_orthogonal_selection()` and `set_orthogonal_selection()` for documentation and examples. property order: zarr.core.common.MemoryOrder# Returns the memory order of the array. Returns: bool Memory order of the array property path: str# Storage path. Returns: str The path to the array in the Zarr store. property read_only: bool# Returns True if the array is read-only. Returns: bool True if the array is read-only property serializer: zarr.abc.codec.ArrayBytesCodec | None# Array-to-bytes codec to use for serializing the chunks into bytes. property shape: tuple[int, Ellipsis]# Returns the shape of the Array. Returns: tuple The shape of the Array. property shards: tuple[int, Ellipsis] | None# Returns the shard shape of the Array. Returns None if sharding is not used. Only defined for arrays using using RegularChunkGrid. If array doesn’t use RegularChunkGrid, NotImplementedError is raised. Returns: tuple[int, …]: The shard shape of the Array. property size: int# Returns the total number of elements in the array Returns: int Total number of elements in the array property store: zarr.abc.store.Store# store_path: zarr.storage._common.StorePath# property vindex: zarr.core.indexing.AsyncVIndex[zarr.core.metadata.T_ArrayMetadata]# Shortcut for vectorized (inner) indexing, see `get_coordinate_selection()`, `set_coordinate_selection()`, `get_mask_selection()` and `set_mask_selection()` for documentation and examples. class zarr.AsyncGroup# Asynchronous Group object. async array_keys() -> collections.abc.AsyncGenerator[str, None]# Iterate over array names. async array_values() -> collections.abc.AsyncGenerator[zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata], None]# Iterate over array values. async arrays() -> collections.abc.AsyncGenerator[tuple[str, zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]], None]# Iterate over arrays. async contains(member: str) -> bool# Check if a member exists in the group. Parameters: memberstr Member name. Returns: bool async create_array( name: str, *, shape: zarr.core.common.ShapeLike | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, data: numpy.ndarray[Any, numpy.dtype[Any]] | None = None, chunks: tuple[int, Ellipsis] | Literal['auto'] = 'auto', shards: zarr.core.array.ShardsLike | None = None, filters: zarr.core.array.FiltersLike = 'auto', compressors: zarr.core.array.CompressorsLike = 'auto', compressor: zarr.core.array.CompressorLike = 'auto', serializer: zarr.core.array.SerializerLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, write_data: bool = True, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array within this group. This method lightly wraps `zarr.core.array.create_array()`. Parameters: namestr The name of the array relative to the group. If `path` is `None`, the array will be located at the root of the store. shapetuple[int, …] Shape of the array. dtypenpt.DTypeLike Data type of the array. chunkstuple[int, …], optional Chunk shape of the array. If not specified, default are guessed based on the shape and dtype. shardstuple[int, …], optional Shard shape of the array. The default value of `None` results in no sharding at all. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec], optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. If no `compressors` are provided, a default set of compressors will be used. These defaults can be changed by modifying the value of `array.v3_default_compressors` in `zarr.core.config`. Use `None` to omit default compressors. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. If no `compressor` is provided, a default compressor will be used. in `zarr.core.config`. Use `None` to omit the default compressor. compressorCodec, optional Deprecated in favor of `compressors`. serializerdict[str, JSON] | ArrayBytesCodec, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. If no `serializer` is provided, a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. fill_valueAny, optional Fill value for the array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If no `order` is provided, a default order will be used. This default can be changed by modifying the value of `array.order` in `zarr.core.config`. attributesdict, optional Attributes for the array. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. dimension_namesIterable[str], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. configArrayConfig or ArrayConfigLike, optional Runtime configuration for the array. write_databool If a pre-existing array-like object was provided to this function via the `data` parameter then `write_data` determines whether the values in that array-like object should be written to the Zarr array created by this function. If `write_data` is `False`, then the array will be left empty. Returns: AsyncArray async create_dataset( name: str, *, shape: zarr.core.common.ShapeLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array. Deprecated since version 3.0.0: The h5py compatibility methods will be removed in 3.1.0. Use AsyncGroup.create_array instead. Arrays are known as “datasets” in HDF5 terminology. For compatibility with h5py, Zarr groups also implement the `zarr.AsyncGroup.require_dataset()` method. Parameters: namestr Array name. **kwargsdict Additional arguments passed to `zarr.AsyncGroup.create_array()`. Returns: aAsyncArray async create_group( name: str, *, overwrite: bool = False, attributes: dict[str, Any] | None = None, ) -> AsyncGroup# Create a sub-group. Parameters: namestr Group name. overwritebool, optional If True, do not raise an error if the group already exists. attributesdict, optional Group attributes. Returns: gAsyncGroup async create_hierarchy( nodes: dict[str, zarr.core.metadata.ArrayV2Metadata | zarr.core.metadata.ArrayV3Metadata | GroupMetadata], *, overwrite: bool = False, ) -> collections.abc.AsyncIterator[tuple[str, AsyncGroup | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]]]# Create a hierarchy of arrays or groups rooted at this group. This function will parse its input to ensure that the hierarchy is complete. Any implicit groups will be inserted as needed. For example, an input like ``{'a/b': GroupMetadata}`` will be parsed to ``{'': GroupMetadata, 'a': GroupMetadata, 'b': Groupmetadata}``. Explicitly specifying a root group, e.g. with `nodes = {'': GroupMetadata()}` is an error because this group instance is the root group. After input parsing, this function then creates all the nodes in the hierarchy concurrently. Arrays and Groups are yielded in the order they are created. This order is not stable and should not be relied on. Parameters: nodesdict[str, GroupMetadata | ArrayV3Metadata | ArrayV2Metadata] A dictionary defining the hierarchy. The keys are the paths of the nodes in the hierarchy, relative to the path of the group. The values are instances of `GroupMetadata` or `ArrayMetadata`. Note that all values must have the same `zarr_format` as the parent group – it is an error to mix zarr versions in the same hierarchy. Leading “/” characters from keys will be removed. overwritebool Whether to overwrite existing nodes. Defaults to `False`, in which case an error is raised instead of overwriting an existing array or group. This function will not erase an existing group unless that group is explicitly named in `nodes`. If `nodes` defines implicit groups, e.g. `{`'a/b/c': GroupMetadata}`, and a group already exists at path `a`, then this function will leave the group at `a` as-is. Yields: tuple[str, AsyncArray | AsyncGroup]. async delitem(key: str) -> None# Delete a group member. Parameters: keystr Array or group name async empty( *, name: str, shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an empty array with the specified shape in this Group. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: namestr Name of the array. shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. async empty_like( *, name: str, data: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an empty sub-array like data. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: namestr Name of the array. dataarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: AsyncArray The new array. classmethod from_dict( store_path: zarr.storage.StorePath, data: dict[str, Any], ) -> AsyncGroup# classmethod from_store( store: zarr.storage.StoreLike, *, attributes: dict[str, Any] | None = None, overwrite: bool = False, zarr_format: zarr.core.common.ZarrFormat = 3, ) -> AsyncGroup# Async: async full( *, name: str, shape: tuple[int, Ellipsis], fill_value: Any | None, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array, with “fill_value” being used as the default value for uninitialized portions of the array. Parameters: namestr Name of the array. shapeint or tuple of int Shape of the empty array. fill_valuescalar Value to fill the array with. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: AsyncArray The new array. async full_like( *, name: str, data: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create a sub-array like data filled with the fill_value of data . Parameters: namestr Name of the array. dataarray-like The array to create the new array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: AsyncArray The new array. async get( key: str, default: DefaultT | None = None, ) -> zarr.core.array.AsyncArray[Any] | AsyncGroup | DefaultT | None# Obtain a group member, returning default if not found. Parameters: keystr Group member name. defaultobject Default value to return if key is not found (default: None). Returns: object Group member (AsyncArray or AsyncGroup) or default if not found. async getitem( key: str, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata] | AsyncGroup# Get a subarray or subgroup from the group. Parameters: keystr Array or group name Returns: AsyncArray or AsyncGroup async group_keys() -> collections.abc.AsyncGenerator[str, None]# Iterate over group names. async group_values() -> collections.abc.AsyncGenerator[AsyncGroup, None]# Iterate over group values. async groups() -> collections.abc.AsyncGenerator[tuple[str, AsyncGroup], None]# Iterate over subgroups. async info_complete() -> Any# Return all the information for a group. This includes dynamic information like the number of child Groups or Arrays. If this group doesn’t contain consolidated metadata then this will need to read from the backing Store. Returns: GroupInfo See also `AsyncGroup.info` async keys() -> collections.abc.AsyncGenerator[str, None]# Iterate over member names. async members( max_depth: int | None = 0, *, use_consolidated_for_children: bool = True, ) -> collections.abc.AsyncGenerator[tuple[str, zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata] | AsyncGroup], None]# Returns an AsyncGenerator over the arrays and groups contained in this group. This method requires that store_path.store supports directory listing. The results are not guaranteed to be ordered. Parameters: max_depthint, default 0 The maximum number of levels of the hierarchy to include. By default, (`max_depth=0`) only immediate children are included. Set `max_depth=None` to include all nodes, and some positive integer to consider children within that many levels of the root Group. use_consolidated_for_childrenbool, default True Whether to use the consolidated metadata of child groups loaded from the store. Note that this only affects groups loaded from the store. If the current Group already has consolidated metadata, it will always be used. Returns: path: A string giving the path to the target, relative to the Group `self`. value: AsyncArray or AsyncGroup The AsyncArray or AsyncGroup that is a child of `self`. abstract move(source: str, dest: str) -> None# Async: Move a sub-group or sub-array from one path to another. Notes Not implemented async nmembers(max_depth: int | None = 0) -> int# Count the number of members in this group. Parameters: max_depthint, default 0 The maximum number of levels of the hierarchy to include. By default, (`max_depth=0`) only immediate children are included. Set `max_depth=None` to include all nodes, and some positive integer to consider children within that many levels of the root Group. Returns: countint async ones( *, name: str, shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array, with one being used as the default value for uninitialized portions of the array. Parameters: namestr Name of the array. shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: AsyncArray The new array. async ones_like( *, name: str, data: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create a sub-array of ones like data. Parameters: namestr Name of the array. dataarray-like The array to create the new array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: AsyncArray The new array. classmethod open( store: zarr.storage.StoreLike, zarr_format: zarr.core.common.ZarrFormat | None = 3, use_consolidated: bool | str | None = None, ) -> AsyncGroup# Async: Open a new AsyncGroup Parameters: storeStoreLike zarr_format{2, 3}, optional use_consolidatedbool or str, default None Whether to use consolidated metadata. By default, consolidated metadata is used if it’s present in the store (in the `zarr.json` for Zarr format 3 and in the `.zmetadata` file for Zarr format 2) and the Store supports it. To explicitly require consolidated metadata, set `use_consolidated=True`. In this case, if the Store doesn’t support consolidation or consolidated metadata is not found, a `ValueError` exception is raised. To explicitly not use consolidated metadata, set `use_consolidated=False`, which will fall back to using the regular, non consolidated metadata. Zarr format 2 allowed configuring the key storing the consolidated metadata (`.zmetadata` by default). Specify the custom key as `use_consolidated` to load consolidated metadata from a non-default key. async require_array( name: str, *, shape: zarr.core.common.ShapeLike, dtype: numpy.typing.DTypeLike = None, exact: bool = False, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Obtain an array, creating if it doesn’t exist. Other kwargs are as per `zarr.AsyncGroup.create_dataset()`. Parameters: namestr Array name. shapeint or tuple of ints Array shape. dtypestr or dtype, optional NumPy dtype. exactbool, optional If True, require dtype to match exactly. If false, require dtype can be cast from array dtype. Returns: aAsyncArray async require_dataset( name: str, *, shape: tuple[int, Ellipsis], dtype: numpy.typing.DTypeLike = None, exact: bool = False, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Obtain an array, creating if it doesn’t exist. Deprecated since version 3.0.0: The h5py compatibility methods will be removed in 3.1.0. Use AsyncGroup.require_dataset instead. Arrays are known as “datasets” in HDF5 terminology. For compatibility with h5py, Zarr groups also implement the `zarr.AsyncGroup.create_dataset()` method. Other kwargs are as per `zarr.AsyncGroup.create_dataset()`. Parameters: namestr Array name. shapeint or tuple of ints Array shape. dtypestr or dtype, optional NumPy dtype. exactbool, optional If True, require dtype to match exactly. If false, require dtype can be cast from array dtype. Returns: aAsyncArray async require_group(name: str, overwrite: bool = False) -> AsyncGroup# Obtain a sub-group, creating one if it doesn’t exist. Parameters: namestr Group name. overwritebool, optional Overwrite any existing group with given name if present. Returns: gAsyncGroup async require_groups(*names: str) -> tuple[AsyncGroup, Ellipsis]# Convenience method to require multiple groups in a single call. Parameters: *namesstr Group names. Returns: Tuple[AsyncGroup, …] async setitem(key: str, value: Any) -> None# Fastpath for creating a new array New arrays will be created with default array settings for the array type. Parameters: keystr Array name valuearray-like Array data async tree(expand: bool | None = None, level: int | None = None) -> Any# Return a tree-like representation of a hierarchy. This requires the optional `rich` dependency. Parameters: expandbool, optional This keyword is not yet supported. A NotImplementedError is raised if it’s used. levelint, optional The maximum depth below this Group to display in the tree. Returns: TreeRepr A pretty-printable object displaying the hierarchy. async update_attributes(new_attributes: dict[str, Any]) -> AsyncGroup# Update group attributes. Parameters: new_attributesdict New attributes to set on the group. Returns: selfAsyncGroup async zeros( *, name: str, shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create an array, with zero being used as the default value for uninitialized portions of the array. Parameters: namestr Name of the array. shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: AsyncArray The new array. async zeros_like( *, name: str, data: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV2Metadata] | zarr.core.array.AsyncArray[zarr.core.metadata.ArrayV3Metadata]# Create a sub-array of zeros like data. Parameters: namestr Name of the array. dataarray-like The array to create the new array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: AsyncArray The new array. property attrs: dict[str, Any]# property basename: str# Final component of name. property info: Any# Return a visual representation of the statically known information about a group. Note that this doesn’t include dynamic information, like the number of child Groups or Arrays. Returns: GroupInfo See also `AsyncGroup.info_complete` All information about a group, including dynamic information metadata: GroupMetadata# property name: str# Group name following h5py convention. property path: str# Storage path. property read_only: bool# property store: zarr.abc.store.Store# store_path: zarr.storage.StorePath# property synchronizer: None# class zarr.Group# Bases: `zarr.core.sync.SyncMixin` A Zarr group. array( name: str, *, shape: zarr.core.common.ShapeLike, dtype: numpy.typing.DTypeLike, chunks: tuple[int, Ellipsis] | Literal['auto'] = 'auto', shards: tuple[int, Ellipsis] | Literal['auto'] | None = None, filters: zarr.core.array.FiltersLike = 'auto', compressors: zarr.core.array.CompressorsLike = 'auto', compressor: zarr.core.array.CompressorLike = None, serializer: zarr.core.array.SerializerLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, data: numpy.typing.ArrayLike | None = None, ) -> zarr.core.array.Array# Create an array within this group. Deprecated since version 3.0.0: Use Group.create_array instead. This method lightly wraps `zarr.core.array.create_array()`. Parameters: namestr The name of the array relative to the group. If `path` is `None`, the array will be located at the root of the store. shapetuple[int, …] Shape of the array. dtypenpt.DTypeLike Data type of the array. chunkstuple[int, …], optional Chunk shape of the array. If not specified, default are guessed based on the shape and dtype. shardstuple[int, …], optional Shard shape of the array. The default value of `None` results in no sharding at all. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec], optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. If no `compressors` are provided, a default set of compressors will be used. These defaults can be changed by modifying the value of `array.v3_default_compressors` in `zarr.core.config`. Use `None` to omit default compressors. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. If no `compressor` is provided, a default compressor will be used. in `zarr.core.config`. Use `None` to omit the default compressor. compressorCodec, optional Deprecated in favor of `compressors`. serializerdict[str, JSON] | ArrayBytesCodec, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. If no `serializer` is provided, a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. fill_valueAny, optional Fill value for the array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If no `order` is provided, a default order will be used. This default can be changed by modifying the value of `array.order` in `zarr.core.config`. attributesdict, optional Attributes for the array. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. dimension_namesIterable[str], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. configArrayConfig or ArrayConfigLike, optional Runtime configuration for the array. dataarray_like The data to fill the array with. Returns: AsyncArray array_keys() -> collections.abc.Generator[str, None]# Return an iterator over group member names. Examples >>> import zarr >>> group = zarr.group() >>> group.create_array("subarray", shape=(10,), chunks=(10,)) >>> for name in group.array_keys(): ... print(name) subarray array_values() -> collections.abc.Generator[zarr.core.array.Array, None]# Return an iterator over group members. Examples >>> import zarr >>> group = zarr.group() >>> group.create_array("subarray", shape=(10,), chunks=(10,)) >>> for subarray in group.array_values(): ... print(subarray) arrays() -> collections.abc.Generator[tuple[str, zarr.core.array.Array], None]# Return the sub-arrays of this group as a generator of (name, array) pairs Examples >>> import zarr >>> group = zarr.group() >>> group.create_array("subarray", shape=(10,), chunks=(10,)) >>> for name, subarray in group.arrays(): ... print(name, subarray) subarray create( name: str, *, shape: zarr.core.common.ShapeLike | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, data: numpy.ndarray[Any, numpy.dtype[Any]] | None = None, chunks: tuple[int, Ellipsis] | Literal['auto'] = 'auto', shards: zarr.core.array.ShardsLike | None = None, filters: zarr.core.array.FiltersLike = 'auto', compressors: zarr.core.array.CompressorsLike = 'auto', compressor: zarr.core.array.CompressorLike = 'auto', serializer: zarr.core.array.SerializerLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, write_data: bool = True, ) -> zarr.core.array.Array# Create an array within this group. This method lightly wraps `zarr.core.array.create_array()`. Parameters: namestr The name of the array relative to the group. If `path` is `None`, the array will be located at the root of the store. shapeShapeLike, optional Shape of the array. Must be `None` if `data` is provided. dtypenpt.DTypeLike | None Data type of the array. Must be `None` if `data` is provided. dataArray-like data to use for initializing the array. If this parameter is provided, the `shape` and `dtype` parameters must be `None`. chunkstuple[int, …], optional Chunk shape of the array. If not specified, default are guessed based on the shape and dtype. shardstuple[int, …], optional Shard shape of the array. The default value of `None` results in no sharding at all. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec], optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. If no `compressors` are provided, a default set of compressors will be used. These defaults can be changed by modifying the value of `array.v3_default_compressors` in `zarr.core.config`. Use `None` to omit default compressors. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. If no `compressor` is provided, a default compressor will be used. in `zarr.core.config`. Use `None` to omit the default compressor. compressorCodec, optional Deprecated in favor of `compressors`. serializerdict[str, JSON] | ArrayBytesCodec, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. If no `serializer` is provided, a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. fill_valueAny, optional Fill value for the array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If no `order` is provided, a default order will be used. This default can be changed by modifying the value of `array.order` in `zarr.core.config`. attributesdict, optional Attributes for the array. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. dimension_namesIterable[str], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. configArrayConfig or ArrayConfigLike, optional Runtime configuration for the array. write_databool If a pre-existing array-like object was provided to this function via the `data` parameter then `write_data` determines whether the values in that array-like object should be written to the Zarr array created by this function. If `write_data` is `False`, then the array will be left empty. Returns: AsyncArray create_array( name: str, *, shape: zarr.core.common.ShapeLike | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, data: numpy.ndarray[Any, numpy.dtype[Any]] | None = None, chunks: tuple[int, Ellipsis] | Literal['auto'] = 'auto', shards: zarr.core.array.ShardsLike | None = None, filters: zarr.core.array.FiltersLike = 'auto', compressors: zarr.core.array.CompressorsLike = 'auto', compressor: zarr.core.array.CompressorLike = 'auto', serializer: zarr.core.array.SerializerLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, write_data: bool = True, ) -> zarr.core.array.Array# Create an array within this group. This method lightly wraps `zarr.core.array.create_array()`. Parameters: namestr The name of the array relative to the group. If `path` is `None`, the array will be located at the root of the store. shapeShapeLike, optional Shape of the array. Must be `None` if `data` is provided. dtypenpt.DTypeLike | None Data type of the array. Must be `None` if `data` is provided. dataArray-like data to use for initializing the array. If this parameter is provided, the `shape` and `dtype` parameters must be `None`. chunkstuple[int, …], optional Chunk shape of the array. If not specified, default are guessed based on the shape and dtype. shardstuple[int, …], optional Shard shape of the array. The default value of `None` results in no sharding at all. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec], optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. If no `compressors` are provided, a default set of compressors will be used. These defaults can be changed by modifying the value of `array.v3_default_compressors` in `zarr.core.config`. Use `None` to omit default compressors. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. If no `compressor` is provided, a default compressor will be used. in `zarr.core.config`. Use `None` to omit the default compressor. compressorCodec, optional Deprecated in favor of `compressors`. serializerdict[str, JSON] | ArrayBytesCodec, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. If no `serializer` is provided, a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. fill_valueAny, optional Fill value for the array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If no `order` is provided, a default order will be used. This default can be changed by modifying the value of `array.order` in `zarr.core.config`. attributesdict, optional Attributes for the array. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. dimension_namesIterable[str], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. configArrayConfig or ArrayConfigLike, optional Runtime configuration for the array. write_databool If a pre-existing array-like object was provided to this function via the `data` parameter then `write_data` determines whether the values in that array-like object should be written to the Zarr array created by this function. If `write_data` is `False`, then the array will be left empty. Returns: AsyncArray create_dataset(name: str, **kwargs: Any) -> zarr.core.array.Array# Create an array. Deprecated since version 3.0.0: The h5py compatibility methods will be removed in 3.1.0. Use Group.create_array instead. Arrays are known as “datasets” in HDF5 terminology. For compatibility with h5py, Zarr groups also implement the `zarr.Group.require_dataset()` method. Parameters: namestr Array name. **kwargsdict Additional arguments passed to `zarr.Group.create_array()` Returns: aArray create_group(name: str, **kwargs: Any) -> Group# Create a sub-group. Parameters: namestr Name of the new subgroup. Returns: Group Examples >>> import zarr >>> group = zarr.group() >>> subgroup = group.create_group("subgroup") >>> subgroup create_hierarchy( nodes: dict[str, zarr.core.metadata.ArrayV2Metadata | zarr.core.metadata.ArrayV3Metadata | GroupMetadata], *, overwrite: bool = False, ) -> collections.abc.Iterator[tuple[str, Group | zarr.core.array.Array]]# Create a hierarchy of arrays or groups rooted at this group. This function will parse its input to ensure that the hierarchy is complete. Any implicit groups will be inserted as needed. For example, an input like ``{'a/b': GroupMetadata}`` will be parsed to ``{'': GroupMetadata, 'a': GroupMetadata, 'b': Groupmetadata}``. Explicitly specifying a root group, e.g. with `nodes = {'': GroupMetadata()}` is an error because this group instance is the root group. After input parsing, this function then creates all the nodes in the hierarchy concurrently. Arrays and Groups are yielded in the order they are created. This order is not stable and should not be relied on. Parameters: nodesdict[str, GroupMetadata | ArrayV3Metadata | ArrayV2Metadata] A dictionary defining the hierarchy. The keys are the paths of the nodes in the hierarchy, relative to the path of the group. The values are instances of `GroupMetadata` or `ArrayMetadata`. Note that all values must have the same `zarr_format` as the parent group – it is an error to mix zarr versions in the same hierarchy. Leading “/” characters from keys will be removed. overwritebool Whether to overwrite existing nodes. Defaults to `False`, in which case an error is raised instead of overwriting an existing array or group. This function will not erase an existing group unless that group is explicitly named in `nodes`. If `nodes` defines implicit groups, e.g. `{`'a/b/c': GroupMetadata}`, and a group already exists at path `a`, then this function will leave the group at `a` as-is. Yields: tuple[str, Array | Group]. Examples >>> import zarr >>> from zarr.core.group import GroupMetadata >>> root = zarr.create_group(store={}) >>> for key, val in root.create_hierarchy({'a/b/c': GroupMetadata()}): ... print(key, val) ... empty( *, name: str, shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.Array# Create an empty array with the specified shape in this Group. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: namestr Name of the array. shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. empty_like( *, name: str, data: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an empty sub-array like data. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: namestr Name of the array. dataarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. classmethod from_store( store: zarr.storage.StoreLike, *, attributes: dict[str, Any] | None = None, zarr_format: zarr.core.common.ZarrFormat = 3, overwrite: bool = False, ) -> Group# Instantiate a group from an initialized store. Parameters: storeStoreLike StoreLike containing the Group. attributesdict, optional A dictionary of JSON-serializable values with user-defined attributes. zarr_format{2, 3}, optional Zarr storage format version. overwritebool, optional If True, do not raise an error if the group already exists. Returns: Group Group instantiated from the store. Raises: ContainsArrayError, ContainsGroupError, ContainsArrayAndGroupError full( *, name: str, shape: tuple[int, Ellipsis], fill_value: Any | None, **kwargs: Any, ) -> zarr.core.array.Array# Create an array, with “fill_value” being used as the default value for uninitialized portions of the array. Parameters: namestr Name of the array. shapeint or tuple of int Shape of the empty array. fill_valuescalar Value to fill the array with. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. full_like( *, name: str, data: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create a sub-array like data filled with the fill_value of data . Parameters: namestr Name of the array. dataarray-like The array to create the new array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. get( path: str, default: DefaultT | None = None, ) -> zarr.core.array.Array | Group | DefaultT | None# Obtain a group member, returning default if not found. Parameters: pathstr Group member name. defaultobject Default value to return if key is not found (default: None). Returns: object Group member (Array or Group) or default if not found. Examples >>> import zarr >>> group = Group.from_store(zarr.storage.MemoryStore() >>> group.create_array(name="subarray", shape=(10,), chunks=(10,)) >>> group.create_group(name="subgroup") >>> group.get("subarray") >>> group.get("subgroup") >>> group.get("nonexistent", None) group_keys() -> collections.abc.Generator[str, None]# Return an iterator over group member names. Examples >>> import zarr >>> group = zarr.group() >>> group.create_group("subgroup") >>> for name in group.group_keys(): ... print(name) subgroup group_values() -> collections.abc.Generator[Group, None]# Return an iterator over group members. Examples >>> import zarr >>> group = zarr.group() >>> group.create_group("subgroup") >>> for subgroup in group.group_values(): ... print(subgroup) groups() -> collections.abc.Generator[tuple[str, Group], None]# Return the sub-groups of this group as a generator of (name, group) pairs. Examples >>> import zarr >>> group = zarr.group() >>> group.create_group("subgroup") >>> for name, subgroup in group.groups(): ... print(name, subgroup) subgroup info_complete() -> Any# Return information for a group. If this group doesn’t contain consolidated metadata then this will need to read from the backing Store. Returns: GroupInfo See also `Group.info` keys() -> collections.abc.Generator[str, None]# Return an iterator over group member names. Examples >>> import zarr >>> g1 = zarr.group() >>> g2 = g1.create_group('foo') >>> g3 = g1.create_group('bar') >>> d1 = g1.create_array('baz', shape=(10,), chunks=(10,)) >>> d2 = g1.create_array('quux', shape=(10,), chunks=(10,)) >>> for name in g1.keys(): ... print(name) baz bar foo quux members( max_depth: int | None = 0, *, use_consolidated_for_children: bool = True, ) -> tuple[tuple[str, zarr.core.array.Array | Group], Ellipsis]# Returns an AsyncGenerator over the arrays and groups contained in this group. This method requires that store_path.store supports directory listing. The results are not guaranteed to be ordered. Parameters: max_depthint, default 0 The maximum number of levels of the hierarchy to include. By default, (`max_depth=0`) only immediate children are included. Set `max_depth=None` to include all nodes, and some positive integer to consider children within that many levels of the root Group. use_consolidated_for_childrenbool, default True Whether to use the consolidated metadata of child groups loaded from the store. Note that this only affects groups loaded from the store. If the current Group already has consolidated metadata, it will always be used. Returns: path: A string giving the path to the target, relative to the Group `self`. value: AsyncArray or AsyncGroup The AsyncArray or AsyncGroup that is a child of `self`. move(source: str, dest: str) -> None# Move a sub-group or sub-array from one path to another. Notes Not implemented nmembers(max_depth: int | None = 0) -> int# Count the number of members in this group. Parameters: max_depthint, default 0 The maximum number of levels of the hierarchy to include. By default, (`max_depth=0`) only immediate children are included. Set `max_depth=None` to include all nodes, and some positive integer to consider children within that many levels of the root Group. Returns: countint ones( *, name: str, shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.Array# Create an array, with one being used as the default value for uninitialized portions of the array. Parameters: namestr Name of the array. shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. ones_like( *, name: str, data: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create a sub-array of ones like data. Parameters: namestr Name of the array. dataarray-like The array to create the new array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. classmethod open( store: zarr.storage.StoreLike, zarr_format: zarr.core.common.ZarrFormat | None = 3, ) -> Group# Open a group from an initialized store. Parameters: storeStoreLike Store containing the Group. zarr_format{2, 3, None}, optional Zarr storage format version. Returns: Group Group instantiated from the store. require_array( name: str, *, shape: zarr.core.common.ShapeLike, **kwargs: Any, ) -> zarr.core.array.Array# Obtain an array, creating if it doesn’t exist. Other kwargs are as per `zarr.Group.create_array()`. Parameters: namestr Array name. **kwargs See `zarr.Group.create_array()`. Returns: aArray require_dataset( name: str, *, shape: zarr.core.common.ShapeLike, **kwargs: Any, ) -> zarr.core.array.Array# Obtain an array, creating if it doesn’t exist. Deprecated since version 3.0.0: The h5py compatibility methods will be removed in 3.1.0. Use Group.require_array instead. Arrays are known as “datasets” in HDF5 terminology. For compatibility with h5py, Zarr groups also implement the `zarr.Group.create_dataset()` method. Other kwargs are as per `zarr.Group.create_dataset()`. Parameters: namestr Array name. **kwargs See `zarr.Group.create_dataset()`. Returns: aArray require_group(name: str, **kwargs: Any) -> Group# Obtain a sub-group, creating one if it doesn’t exist. Parameters: namestr Group name. Returns: gGroup require_groups(*names: str) -> tuple[Group, Ellipsis]# Convenience method to require multiple groups in a single call. Parameters: *namesstr Group names. Returns: groupstuple of Groups tree(expand: bool | None = None, level: int | None = None) -> Any# Return a tree-like representation of a hierarchy. This requires the optional `rich` dependency. Parameters: expandbool, optional This keyword is not yet supported. A NotImplementedError is raised if it’s used. levelint, optional The maximum depth below this Group to display in the tree. Returns: TreeRepr A pretty-printable object displaying the hierarchy. update_attributes(new_attributes: dict[str, Any]) -> Group# Update the attributes of this group. Examples >>> import zarr >>> group = zarr.group() >>> group.update_attributes({"foo": "bar"}) >>> group.attrs.asdict() {'foo': 'bar'} async update_attributes_async(new_attributes: dict[str, Any]) -> Group# Update the attributes of this group. Examples >>> import zarr >>> group = zarr.group() >>> await group.update_attributes_async({"foo": "bar"}) >>> group.attrs.asdict() {'foo': 'bar'} zeros( *, name: str, shape: tuple[int, Ellipsis], **kwargs: Any, ) -> zarr.core.array.Array# Create an array, with zero being used as the default value for uninitialized portions of the array. Parameters: namestr Name of the array. shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zeros_like( *, name: str, data: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create a sub-array of zeros like data. Parameters: namestr Name of the array. dataarray-like The array to create the new array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. property attrs: zarr.core.attributes.Attributes# Attributes of this Group property basename: str# Final component of name. property info: Any# Return the statically known information for a group. Returns: GroupInfo See also `Group.info_complete` All information about a group, including dynamic information like the children members. property metadata: GroupMetadata# Group metadata. property name: str# Group name following h5py convention. property path: str# Storage path. property read_only: bool# property store: zarr.abc.store.Store# property store_path: zarr.storage.StorePath# Path-like interface for the Store. property synchronizer: None# zarr.array( data: numpy.typing.ArrayLike | zarr.core.array.Array, **kwargs: Any, ) -> zarr.core.array.Array# Create an array filled with data. Parameters: dataarray_like The data to fill the array with. **kwargs Passed through to `create()`. Returns: arrayArray The new array. zarr.consolidate_metadata( store: zarr.storage.StoreLike, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, ) -> zarr.core.group.Group# Consolidate the metadata of all nodes in a hierarchy. Upon completion, the metadata of the root node in the Zarr hierarchy will be updated to include all the metadata of child nodes. For Stores that do not use consolidated metadata, this operation raises a TypeError. Parameters: storeStoreLike The store-like object whose metadata you wish to consolidate. pathstr, optional A path to a group in the store to consolidate at. Only children below that group will be consolidated. By default, the root node is used so all the metadata in the store is consolidated. zarr_format{2, 3, None}, optional The zarr format of the hierarchy. By default the zarr format is inferred. Returns: group: Group The group, with the `consolidated_metadata` field set to include the metadata of each child node. If the Store doesn’t support consolidated metadata, this function raises a TypeError. See `Store.supports_consolidated_metadata`. zarr.copy(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.copy_all(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.copy_store(*args: Any, **kwargs: Any) -> tuple[int, int, int]# Not implemented. zarr.create( shape: tuple[int, Ellipsis] | int, *, chunks: tuple[int, Ellipsis] | int | bool | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, compressor: zarr.core.array.CompressorLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, store: zarr.storage.StoreLike | None = None, synchronizer: Any | None = None, overwrite: bool = False, path: zarr.api.asynchronous.PathLike | None = None, chunk_store: zarr.storage.StoreLike | None = None, filters: collections.abc.Iterable[dict[str, zarr.core.common.JSON] | zarr.abc.numcodec.Numcodec] | None = None, cache_metadata: bool | None = None, cache_attrs: bool | None = None, read_only: bool | None = None, object_codec: zarr.abc.codec.Codec | None = None, dimension_separator: Literal['.', '/'] | None = None, write_empty_chunks: bool | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_shape: tuple[int, Ellipsis] | int | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncoding | tuple[Literal['default'], Literal['.', '/']] | tuple[Literal['v2'], Literal['.', '/']] | None = None, codecs: collections.abc.Iterable[zarr.abc.codec.Codec | dict[str, zarr.core.common.JSON]] | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, config: zarr.core.array_spec.ArrayConfigLike | None = None, **kwargs: Any, ) -> zarr.core.array.Array# Create an array. Parameters: shapeint or tuple of ints Array shape. chunksint or tuple of ints, optional Chunk shape. If True, will be guessed from `shape` and `dtype`. If False, will be set to `shape`, i.e., single chunk for the whole array. If an int, the chunk size in each dimension will be given by the value of `chunks`. Default is True. dtypestr or dtype, optional NumPy dtype. compressorCodec, optional Primary compressor to compress chunk data. Zarr format 2 only. Zarr format 3 arrays should use `codecs` instead. If neither `compressor` nor `filters` are provided, the default compressor `zarr.codecs.ZstdCodec` is used. If `compressor` is set to `None`, no compression is used. fill_valueAny, optional Fill value for the array. order{‘C’, ‘F’}, optional Deprecated in favor of the `config` keyword argument. Pass `{'order': }` to `create` instead of using this parameter. Memory layout to be used within each chunk. If not specified, the `array.order` parameter in the global config will be used. storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. synchronizerobject, optional Array synchronizer. overwritebool, optional If True, delete all pre-existing data in `store` at `path` before creating the array. pathstr, optional Path under which array is stored. chunk_storeStoreLike or None, default=None Separate storage for chunks. If not provided, `store` will be used for storage of both chunks and metadata. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. cache_metadatabool, optional If True, array configuration metadata will be cached for the lifetime of the object. If False, array metadata will be reloaded prior to all data access and modification operations (may incur overhead depending on storage and data access pattern). cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. read_onlybool, optional True if array should be protected against modification. object_codecCodec, optional A codec to encode object arrays, only needed if dtype=object. dimension_separator{‘.’, ‘/’}, optional Separator placed between the dimensions of a chunk. Zarr format 2 only. Zarr format 3 arrays should use `chunk_key_encoding` instead. write_empty_chunksbool, optional Deprecated in favor of the `config` keyword argument. Pass `{'write_empty_chunks': }` to `create` instead of using this parameter. If True, all chunks will be stored regardless of their contents. If False, each chunk is compared to the array’s fill value prior to storing. If a chunk is uniformly equal to the fill value, then that chunk is not be stored, and the store entry for that chunk’s key is deleted. zarr_format{2, 3, None}, optional The Zarr format to use when creating an array. The default is `None`, which instructs Zarr to choose the default Zarr format value defined in the runtime configuration. meta_arrayarray-like, optional Not implemented. attributesdict[str, JSON], optional A dictionary of user attributes to store with the array. chunk_shapeint or tuple of ints, optional The shape of the Array’s chunks (default is None). Zarr format 3 only. Zarr format 2 arrays should use chunks instead. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. Zarr format 3 only. Zarr format 2 arrays should use dimension_separator instead. Default is `("default", "/")`. codecsSequence of Codecs or dicts, optional An iterable of Codec or dict serializations of Codecs. Zarr V3 only. The elements of `codecs` specify the transformation from array values to stored bytes. Zarr format 3 only. Zarr format 2 arrays should use `filters` and `compressor` instead. If no codecs are provided, default codecs will be used based on the data type of the array. For most data types, the default codecs are the tuple `(BytesCodec(), ZstdCodec())`; data types that require a special `zarr.abc.codec.ArrayBytesCodec`, like variable-length strings or bytes, will use the `zarr.abc.codec.ArrayBytesCodec` required for the data type instead of `zarr.codecs.BytesCodec`. dimension_namesIterable[str | None] | None = None An iterable of dimension names. Zarr format 3 only. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. configArrayConfigLike, optional Runtime configuration of the array. If provided, will override the default values from zarr.config.array. Returns: zArray The array. zarr.create_array( store: zarr.storage.StoreLike, *, name: str | None = None, shape: zarr.core.common.ShapeLike | None = None, dtype: zarr.core.dtype.ZDTypeLike | None = None, data: numpy.ndarray[Any, numpy.dtype[Any]] | None = None, chunks: tuple[int, Ellipsis] | Literal['auto'] = 'auto', shards: zarr.core.array.ShardsLike | None = None, filters: zarr.core.array.FiltersLike = 'auto', compressors: zarr.core.array.CompressorsLike = 'auto', serializer: zarr.core.array.SerializerLike = 'auto', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, zarr_format: zarr.core.common.ZarrFormat | None = 3, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, write_data: bool = True, ) -> zarr.core.array.Array# Create an array. This function wraps `zarr.core.array.create_array()`. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. namestr or None, optional The name of the array within the store. If `name` is `None`, the array will be located at the root of the store. shapeShapeLike, optional Shape of the array. Must be `None` if `data` is provided. dtypeZDTypeLike | None Data type of the array. Must be `None` if `data` is provided. datanp.ndarray, optional Array-like data to use for initializing the array. If this parameter is provided, the `shape` and `dtype` parameters must be `None`. chunkstuple[int, …] | Literal[“auto”], default=”auto” Chunk shape of the array. If chunks is “auto”, a chunk shape is guessed based on the shape of the array and the dtype. shardstuple[int, …], optional Shard shape of the array. The default value of `None` results in no sharding at all. filtersIterable[Codec] | Literal[“auto”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"auto"` instructs Zarr to use a default used based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are empty. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters contains a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec], optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. If no `compressors` are provided, a default set of compressors will be used. These defaults can be changed by modifying the value of `array.v3_default_compressors` in `zarr.core.config`. Use `None` to omit default compressors. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. If no `compressor` is provided, a default compressor will be used. in `zarr.core.config`. Use `None` to omit the default compressor. serializerdict[str, JSON] | ArrayBytesCodec, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. If no `serializer` is provided, a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. fill_valueAny, optional Fill value for the array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If no `order` is provided, a default order will be used. This default can be changed by modifying the value of `array.order` in `zarr.core.config`. zarr_format{2, 3}, optional The zarr format to use when saving. attributesdict, optional Attributes for the array. chunk_key_encodingChunkKeyEncodingLike, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. dimension_namesIterable[str], optional The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. If `True`, all existing paths in the store will be deleted. configArrayConfigLike, optional Runtime configuration for the array. write_databool If a pre-existing array-like object was provided to this function via the `data` parameter then `write_data` determines whether the values in that array-like object should be written to the Zarr array created by this function. If `write_data` is `False`, then the array will be left empty. Returns: Array The array. Examples >>> import zarr >>> store = zarr.storage.MemoryStore() >>> arr = await zarr.create_array( >>> store=store, >>> shape=(100,100), >>> chunks=(10,10), >>> dtype='i4', >>> fill_value=0) zarr.create_group( store: zarr.storage.StoreLike, *, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, overwrite: bool = False, attributes: dict[str, Any] | None = None, storage_options: dict[str, Any] | None = None, ) -> zarr.core.group.Group# Create a group. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. pathstr, optional Group path within store. overwritebool, optional If True, pre-existing data at `path` will be deleted before creating the group. zarr_format{2, 3, None}, optional The zarr format to use when saving. If no `zarr_format` is provided, the default format will be used. This default can be changed by modifying the value of `default_zarr_format` in `zarr.core.config`. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. Returns: Group The new group. zarr.create_hierarchy( *, store: zarr.abc.store.Store, nodes: dict[str, zarr.core.group.GroupMetadata | zarr.core.metadata.ArrayV2Metadata | zarr.core.metadata.ArrayV3Metadata], overwrite: bool = False, ) -> collections.abc.Iterator[tuple[str, zarr.core.group.Group | zarr.core.array.Array]]# Create a complete zarr hierarchy from a collection of metadata objects. This function will parse its input to ensure that the hierarchy is complete. Any implicit groups will be inserted as needed. For example, an input like ``{'a/b': GroupMetadata}`` will be parsed to ``{'': GroupMetadata, 'a': GroupMetadata, 'b': Groupmetadata}`` After input parsing, this function then creates all the nodes in the hierarchy concurrently. Arrays and Groups are yielded in the order they are created. This order is not stable and should not be relied on. Parameters: storeStore The storage backend to use. nodesdict[str, GroupMetadata | ArrayV3Metadata | ArrayV2Metadata] A dictionary defining the hierarchy. The keys are the paths of the nodes in the hierarchy, relative to the root of the `Store`. The root of the store can be specified with the empty string `''`. The values are instances of `GroupMetadata` or `ArrayMetadata`. Note that all values must have the same `zarr_format` – it is an error to mix zarr versions in the same hierarchy. Leading “/” characters from keys will be removed. overwritebool Whether to overwrite existing nodes. Defaults to `False`, in which case an error is raised instead of overwriting an existing array or group. This function will not erase an existing group unless that group is explicitly named in `nodes`. If `nodes` defines implicit groups, e.g. `{`'a/b/c': GroupMetadata}`, and a group already exists at path `a`, then this function will leave the group at `a` as-is. Yields: tuple[str, Group | Array] This function yields (path, node) pairs, in the order the nodes were created. Examples >>> from zarr import create_hierarchy >>> from zarr.storage import MemoryStore >>> from zarr.core.group import GroupMetadata >>> store = MemoryStore() >>> nodes = {'a': GroupMetadata(attributes={'name': 'leaf'})} >>> nodes_created = dict(create_hierarchy(store=store, nodes=nodes)) >>> print(nodes) # {'a': GroupMetadata(attributes={'name': 'leaf'}, zarr_format=3, consolidated_metadata=None, node_type='group')} zarr.empty(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an empty array with the specified shape. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. zarr.empty_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an empty array like another array. The contents will be filled with the array’s fill value or zeros if no fill value is provided. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. Notes The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next. zarr.from_array( store: zarr.storage.StoreLike, *, data: zarr.core.array.Array | numpy.typing.ArrayLike, write_data: bool = True, name: str | None = None, chunks: Literal['auto', 'keep'] | tuple[int, Ellipsis] = 'keep', shards: zarr.core.array.ShardsLike | None | Literal['keep'] = 'keep', filters: zarr.core.array.FiltersLike | Literal['keep'] = 'keep', compressors: zarr.core.array.CompressorsLike | Literal['keep'] = 'keep', serializer: zarr.core.array.SerializerLike | Literal['keep'] = 'keep', fill_value: Any | None = DEFAULT_FILL_VALUE, order: zarr.core.common.MemoryOrder | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, chunk_key_encoding: zarr.core.chunk_key_encodings.ChunkKeyEncodingLike | None = None, dimension_names: zarr.core.common.DimensionNames = None, storage_options: dict[str, Any] | None = None, overwrite: bool = False, config: zarr.core.array_spec.ArrayConfigLike | None = None, ) -> zarr.core.array.Array# Create an array from an existing array or array-like. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. dataArray | array-like The array to copy. write_databool, default True Whether to copy the data from the input array to the new array. If `write_data` is `False`, the new array will be created with the same metadata as the input array, but without any data. namestr or None, optional The name of the array within the store. If `name` is `None`, the array will be located at the root of the store. chunkstuple[int, …] or “auto” or “keep”, optional Chunk shape of the array. Following values are supported: * “auto”: Automatically determine the chunk shape based on the array’s shape and dtype. * “keep”: Retain the chunk shape of the data array if it is a zarr Array. * tuple[int, …]: A tuple of integers representing the chunk shape. If not specified, defaults to “keep” if data is a zarr Array, otherwise “auto”. shardstuple[int, …], optional Shard shape of the array. Following values are supported: * “auto”: Automatically determine the shard shape based on the array’s shape and chunk shape. * “keep”: Retain the shard shape of the data array if it is a zarr Array. * tuple[int, …]: A tuple of integers representing the shard shape. * None: No sharding. If not specified, defaults to “keep” if data is a zarr Array, otherwise None. filtersIterable[Codec] | Literal[“auto”, “keep”], optional Iterable of filters to apply to each chunk of the array, in order, before serializing that chunk to bytes. For Zarr format 3, a “filter” is a codec that takes an array and returns an array, and these values must be instances of `zarr.abc.codec.ArrayArrayCodec`, or a dict representations of `zarr.abc.codec.ArrayArrayCodec`. For Zarr format 2, a “filter” can be any numcodecs codec; you should ensure that the the order if your filters is consistent with the behavior of each filter. The default value of `"keep"` instructs Zarr to infer `filters` from `data`. If that inference is not possible, Zarr will fall back to the behavior specified by `"auto"`, which is to choose default filters based on the data type of the array and the Zarr format specified. For all data types in Zarr V3, and most data types in Zarr V2, the default filters are the empty tuple `()`. The only cases where default filters are not empty is when the Zarr format is 2, and the data type is a variable-length data type like `zarr.dtype.VariableLengthUTF8` or `zarr.dtype.VariableLengthUTF8`. In these cases, the default filters is a tuple with a single element which is a codec specific to that particular data type. To create an array with no filters, provide an empty iterable or the value `None`. compressorsIterable[Codec] or “auto” or “keep”, optional List of compressors to apply to the array. Compressors are applied in order, and after any filters are applied (if any are specified) and the data is serialized into bytes. For Zarr format 3, a “compressor” is a codec that takes a bytestream, and returns another bytestream. Multiple compressors my be provided for Zarr format 3. For Zarr format 2, a “compressor” can be any numcodecs codec. Only a single compressor may be provided for Zarr format 2. Following values are supported: * Iterable[Codec]: List of compressors to apply to the array. * “auto”: Automatically determine the compressors based on the array’s dtype. * “keep”: Retain the compressors of the input array if it is a zarr Array. If no `compressors` are provided, defaults to “keep” if data is a zarr Array, otherwise “auto”. serializerdict[str, JSON] | ArrayBytesCodec or “auto” or “keep”, optional Array-to-bytes codec to use for encoding the array data. Zarr format 3 only. Zarr format 2 arrays use implicit array-to-bytes conversion. Following values are supported: * dict[str, JSON]: A dict representation of an `ArrayBytesCodec`. * ArrayBytesCodec: An instance of `ArrayBytesCodec`. * “auto”: a default serializer will be used. These defaults can be changed by modifying the value of `array.v3_default_serializer` in `zarr.core.config`. * “keep”: Retain the serializer of the input array if it is a zarr Array. fill_valueAny, optional Fill value for the array. If not specified, defaults to the fill value of the data array. order{“C”, “F”}, optional The memory of the array (default is “C”). For Zarr format 2, this parameter sets the memory order of the array. For Zarr format 3, this parameter is deprecated, because memory order is a runtime parameter for Zarr format 3 arrays. The recommended way to specify the memory order for Zarr format 3 arrays is via the `config` parameter, e.g. `{'config': 'C'}`. If not specified, defaults to the memory order of the data array. zarr_format{2, 3}, optional The zarr format to use when saving. If not specified, defaults to the zarr format of the data array. attributesdict, optional Attributes for the array. If not specified, defaults to the attributes of the data array. chunk_key_encodingChunkKeyEncoding, optional A specification of how the chunk keys are represented in storage. For Zarr format 3, the default is `{"name": "default", "separator": "/"}}`. For Zarr format 2, the default is `{"name": "v2", "separator": "."}}`. If not specified and the data array has the same zarr format as the target array, the chunk key encoding of the data array is used. dimension_namesIterable[str | None] | None The names of the dimensions (default is None). Zarr format 3 only. Zarr format 2 arrays should not use this parameter. If not specified, defaults to the dimension names of the data array. storage_optionsdict, optional If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. overwritebool, default False Whether to overwrite an array with the same name in the store, if one exists. configArrayConfig or ArrayConfigLike, optional Runtime configuration for the array. Returns: Array The array. Examples Create an array from an existing Array: >>> import zarr >>> store = zarr.storage.MemoryStore() >>> store2 = zarr.storage.LocalStore('example.zarr') >>> arr = zarr.create_array( >>> store=store, >>> shape=(100,100), >>> chunks=(10,10), >>> dtype='int32', >>> fill_value=0) >>> arr2 = zarr.from_array(store2, data=arr) Create an array from an existing NumPy array: >>> import numpy as np >>> arr3 = zarr.from_array( zarr.storage.MemoryStore(), >>> data=np.arange(10000, dtype='i4').reshape(100, 100), >>> ) Create an array from any array-like object: >>> arr4 = zarr.from_array( >>> zarr.storage.MemoryStore(), >>> data=[[1, 2], [3, 4]], >>> ) >>> arr4[...] array([[1, 2],[3, 4]]) Create an array from an existing Array without copying the data: >>> arr5 = zarr.from_array( >>> zarr.storage.MemoryStore(), >>> data=arr4, >>> write_data=False, >>> ) >>> arr5[...] array([[0, 0],[0, 0]]) zarr.full( shape: tuple[int, Ellipsis], fill_value: Any, **kwargs: Any, ) -> zarr.core.array.Array# Create an array with a default fill value. Parameters: shapeint or tuple of int Shape of the empty array. fill_valuescalar Fill value. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.full_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create a filled array like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.group( store: zarr.storage.StoreLike | None = None, *, overwrite: bool = False, chunk_store: zarr.storage.StoreLike | None = None, cache_attrs: bool | None = None, synchronizer: Any | None = None, path: str | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, storage_options: dict[str, Any] | None = None, ) -> zarr.core.group.Group# Create a group. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. overwritebool, optional If True, delete any pre-existing data in store at path before creating the group. chunk_storeStoreLike or None, default=None Separate storage for chunks. Not implemented. cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. synchronizerobject, optional Array synchronizer. pathstr, optional Group path within store. meta_arrayarray-like, optional An array instance to use for determining arrays to create and return to users. Use numpy.empty(()) by default. zarr_format{2, 3, None}, optional The zarr format to use when saving. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. Returns: gGroup The new group. zarr.load( store: zarr.storage.StoreLike, path: str | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, ) -> zarr.core.buffer.NDArrayLikeOrScalar | dict[str, zarr.core.buffer.NDArrayLikeOrScalar]# Load data from an array or group into memory. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. pathstr or None, optional The path within the store from which to load. Returns: out If the path contains an array, out will be a numpy array. If the path contains a group, out will be a dict-like object where keys are array names and values are numpy arrays. See also `save`, `savez` Notes If loading data from a group of arrays, data will not be immediately loaded into memory. Rather, arrays will be loaded into memory as they are requested. zarr.ones(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an array with a fill value of one. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.ones_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an array of ones like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.open( store: zarr.storage.StoreLike | None = None, *, mode: zarr.core.common.AccessModeLiteral | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> zarr.core.array.Array | zarr.core.group.Group# Open a group or array using file-mode-like semantics. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. mode{‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘}, optional Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-’ means create (fail if exists). If the store is read-only, the default is ‘r’; otherwise, it is ‘a’. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional The path within the store to open. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Additional parameters are passed through to `zarr.creation.open_array()` or `zarr.hierarchy.open_group()`. Returns: zarray or group Return type depends on what exists in the given store. zarr.open_array( store: zarr.storage.StoreLike | None = None, *, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: zarr.api.asynchronous.PathLike = '', storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> zarr.core.array.Array# Open an array using file-mode-like semantics. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. zarr_version{2, 3, None}, optional The zarr format to use when saving. Deprecated in favor of zarr_format. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr, optional Path in store to array. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Any keyword arguments to pass to `create()`. Returns: AsyncArray The opened array. zarr.open_consolidated( *args: Any, use_consolidated: Literal[True] = True, **kwargs: Any, ) -> zarr.core.group.Group# Alias for `open_group()` with `use_consolidated=True`. zarr.open_group( store: zarr.storage.StoreLike | None = None, *, mode: zarr.core.common.AccessModeLiteral = 'a', cache_attrs: bool | None = None, synchronizer: Any = None, path: str | None = None, chunk_store: zarr.storage.StoreLike | None = None, storage_options: dict[str, Any] | None = None, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, meta_array: Any | None = None, attributes: dict[str, zarr.core.common.JSON] | None = None, use_consolidated: bool | str | None = None, ) -> zarr.core.group.Group# Open a group using file-mode-like semantics. Parameters: storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. Strings are interpreted as paths on the local file system and used as the `root` argument to `zarr.storage.LocalStore`. Dictionaries are used as the `store_dict` argument in `zarr.storage.MemoryStore``. By default (`store=None`) a new `zarr.storage.MemoryStore` is created. mode{‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘}, optional Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-’ means create (fail if exists). cache_attrsbool, optional If True (default), user attributes will be cached for attribute read operations. If False, user attributes are reloaded from the store prior to all attribute read operations. synchronizerobject, optional Array synchronizer. pathstr, optional Group path within store. chunk_storeStoreLike or None, default=None Store or path to directory in file system or name of zip file. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. meta_arrayarray-like, optional An array instance to use for determining arrays to create and return to users. Use numpy.empty(()) by default. attributesdict A dictionary of JSON-serializable values with user-defined attributes. use_consolidatedbool or str, default None Whether to use consolidated metadata. By default, consolidated metadata is used if it’s present in the store (in the `zarr.json` for Zarr format 3 and in the `.zmetadata` file for Zarr format 2). To explicitly require consolidated metadata, set `use_consolidated=True`, which will raise an exception if consolidated metadata is not found. To explicitly not use consolidated metadata, set `use_consolidated=False`, which will fall back to using the regular, non consolidated metadata. Zarr format 2 allowed configuring the key storing the consolidated metadata (`.zmetadata` by default). Specify the custom key as `use_consolidated` to load consolidated metadata from a non-default key. Returns: gGroup The new group. zarr.open_like( a: zarr.api.asynchronous.ArrayLike, path: str, **kwargs: Any, ) -> zarr.core.array.Array# Open a persistent array like another array. Parameters: aArray The shape and data-type of a define these same attributes of the returned array. pathstr The path to the new array. **kwargs Any keyword arguments to pass to the array constructor. Returns: AsyncArray The opened array. zarr.print_debug_info() -> None# Print version info for use in bug reports. zarr.save( store: zarr.storage.StoreLike, *args: zarr.core.buffer.NDArrayLike, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, **kwargs: Any, ) -> None# Save an array or group of arrays to the local file system. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. *argsndarray NumPy arrays with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional The path within the group where the arrays will be saved. **kwargs NumPy arrays with data to save. zarr.save_array( store: zarr.storage.StoreLike, arr: zarr.core.buffer.NDArrayLike, *, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: Any, ) -> None# Save a NumPy array to the local file system. Follows a similar API to the NumPy save() function. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. arrndarray NumPy array with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. The default is `None`, which will use the default Zarr format defined in the global configuration object. pathstr or None, optional The path within the store where the array will be saved. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs Passed through to `create()`, e.g., compressor. zarr.save_group( store: zarr.storage.StoreLike, *args: zarr.core.buffer.NDArrayLike, zarr_version: zarr.core.common.ZarrFormat | None = None, zarr_format: zarr.core.common.ZarrFormat | None = None, path: str | None = None, storage_options: dict[str, Any] | None = None, **kwargs: zarr.core.buffer.NDArrayLike, ) -> None# Save several NumPy arrays to the local file system. Follows a similar API to the NumPy savez()/savez_compressed() functions. Parameters: storeStoreLike Store or path to directory in file system or name of zip file. *argsndarray NumPy arrays with data to save. zarr_format{2, 3, None}, optional The zarr format to use when saving. pathstr or None, optional Path within the store where the group will be saved. storage_optionsdict If using an fsspec URL to create the store, these will be passed to the backend implementation. Ignored otherwise. **kwargs NumPy arrays with data to save. zarr.tree( grp: zarr.core.group.Group, expand: bool | None = None, level: int | None = None, ) -> Any# Provide a rich display of the hierarchy. Deprecated since version 3.0.0: zarr.tree() is deprecated and will be removed in a future release. Use group.tree() instead. Parameters: grpGroup Zarr or h5py group. expandbool, optional Only relevant for HTML representation. If True, tree will be fully expanded. levelint, optional Maximum depth to descend into hierarchy. Returns: TreeRepr A pretty-printable object displaying the hierarchy. zarr.zeros(shape: tuple[int, Ellipsis], **kwargs: Any) -> zarr.core.array.Array# Create an array with a fill value of zero. Parameters: shapeint or tuple of int Shape of the empty array. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.zeros_like( a: zarr.api.asynchronous.ArrayLike, **kwargs: Any, ) -> zarr.core.array.Array# Create an array of zeros like another array. Parameters: aarray-like The array to create an empty array like. **kwargs Keyword arguments passed to `zarr.api.asynchronous.create()`. Returns: Array The new array. zarr.config# ## Release notes# ### 3.1.2 (2025-08-25)# #### Features# * Added support for async vectorized and orthogonal indexing. (#3083) * Make config param optional in init_array (#3391) #### Bugfixes# * Ensure that -0.0 is not considered equal to 0.0 when checking if all the values in a chunk are equal to an array’s fill value.``` (#3144) * Fix a bug in `create_array` caused by iterating over chunk-aligned regions instead of shard-aligned regions when writing data. Additionally, the behavior of `nchunks_initialized` has been adjusted. This function consistently reports the number of chunks present in stored objects, even when the array uses the sharding codec. (#3299) * Opening an array or group with `mode="r+"` will no longer create new arrays or groups. (#3307) * Added zarr.errors.ArrayNotFoundError, which is raised when attempting to open a zarr array that does not exist, and zarr.errors.NodeNotFoundError, which is raised when failing to open an array or a group in a context where either an array or a group was expected. (#3367) * Ensure passing config is handled properly when open`ing an existing array. (:issue:`3378) * Raise a Zarr-specific error class when a codec can’t be found by name when deserializing the given codecs. This avoids hiding this error behind a “not part of a zarr hierarchy” warning. (#3395) #### Misc# * #3098, #3288, #3318, #3368, #3371, #3372, #3374 ### 3.1.1 (2025-07-28)# #### Features# * Add lightweight implementations of .getsize() and .getsize_prefix() for ObjectStore. (#3227) #### Bugfixes# * Creating a Zarr format 2 array with the `order` keyword argument no longer raises a warning. (#3112) * Fixed the error message when passing both `config` and `write_empty_chunks` arguments to reflect the current behaviour (`write_empty_chunks` takes precedence). (#3112) * Creating a Zarr format 3 array with the `order` argument now conistently ignores this argument and raises a warning. (#3112) * When using `from_array` to copy a Zarr format 2 array to a Zarr format 3 array, if the memory order of the input array is `"F"` a warning is raised and the order ignored. This is because Zarr format 3 arrays are always stored in “C” order. (#3112) * The `config` argument to zarr.create (and functions that create arrays) is now used - previously it had no effect. (#3112) * Ensure that all abstract methods of `ZDType` raise a `NotImplementedError` when invoked. (#3251) * Register ‘gpu’ marker with pytest for downstream StoreTests. (#3258) * Expand the range of types accepted by `parse_data_type` to include strings and Sequences. * Move the functionality of `parse_data_type` to a new function called `parse_dtype`. This change ensures that nomenclature is consistent across the codebase. `parse_data_type` remains, so this change is not breaking. (#3264) * Fix a regression introduced in 3.1.0 that prevented `inf`, `-inf`, and `nan` values from being stored in `attributes`. (#3280) * Fixes Group.nmembers() ignoring depth when using consolidated metadata. (#3287) #### Improved Documentation# * Expand the data type docs to include a demonstration of the `parse_data_type` function. Expand the docstring for the `parse_data_type` function. (#3249) * Add a section on codecs to the migration guide. (#3273) #### Misc# * #3268 ### 3.1.0 (2025-07-14)# #### Features# * Ensure that invocations of `create_array` use consistent keyword arguments, with consistent defaults. `zarr.api.synchronous.create_array` now takes a `write_data` keyword argument The `Group.create_array` method takes `data` and `write_data` keyword arguments. The functions `api.asynchronous.create`, `api.asynchronous.create_array` and the methods `Group.create_array`, `Group.array`, had the default `fill_value` changed from `0` to the `DEFAULT_FILL_VALUE` value, which instructs Zarr to use the default scalar value associated with the array’s data type as the fill value. These are all functions or methods for array creation that mirror, wrap or are wrapped by, another function that already has a default `fill_value` set to `DEFAULT_FILL_VALUE`. This change is necessary to make these functions consistent across the entire codebase, but as this changes default values, new data might have a different fill value than expected after this change. For data types where 0 is meaningful, like integers or floats, the default scalar is 0, so this change should not be noticeable. For data types where 0 is ambiguous, like fixed-length unicode strings, the default fill value might be different after this change. Users who were relying on how Zarr interpreted `0` as a non-numeric scalar value should set their desired fill value explicitly after this change. * Added public API for Buffer ABCs and implementations. Use `zarr.buffer` to access buffer implementations, and `zarr.abc.buffer` for the interface to implement new buffer types. Users previously importing buffer from `zarr.core.buffer` should update their imports to use `zarr.buffer`. As a reminder, all of `zarr.core` is considered a private API that’s not covered by zarr-python’s versioning policy. (#2871) * Adds zarr-specific data type classes. This change adds a `ZDType` base class for Zarr V2 and Zarr V3 data types. Child classes are defined for each NumPy data type. Each child class defines routines for `JSON` serialization. New data types can be created and registered dynamically. Prior to this change, Zarr Python had two streams for handling data types. For Zarr V2 arrays, we used NumPy data type identifiers. For Zarr V3 arrays, we used a fixed set of string enums. Both of these systems proved hard to extend. This change is largely internal, but it does change the type of the `dtype` and `data_type` fields on the `ArrayV2Metadata` and `ArrayV3Metadata` classes. Previously, `ArrayV2Metadata.dtype` was a NumPy `dtype` object, and `ArrayV3Metadata.data_type` was an internally-defined `enum`. After this change, both `ArrayV2Metadata.dtype` and `ArrayV3Metadata.data_type` are instances of `ZDType`. A NumPy data type can be generated from a `ZDType` via the `ZDType.to_native_dtype()` method. The internally-defined Zarr V3 `enum` class is gone entirely, but the `ZDType.to_json(zarr_format=3)` method can be used to generate either a string, or dictionary that has a string `name` field, that represents the string value previously associated with that `enum`. For more on this new feature, see the documentation (#2874) * Added NDBuffer.empty method for faster ndbuffer initialization. (#3191) * The minimum version of NumPy has increased to 1.26. (#3226) * Add an alternate from_array_metadata_and_store constructor to CodecPipeline. (#3233) #### Bugfixes# * Fixes a variety of issues related to string data types. * Brings the `VariableLengthUTF8` data type Zarr V3 identifier in alignment with Zarr Python 3.0.8 * Disallows creation of 0-length fixed-length data types * Adds a regression test for the `VariableLengthUTF8` data type that checks against version 3.0.8 * Allows users to request the `VariableLengthUTF8` data type with `str`, `"str"`, or `"string"`. (#3170) * Add human readable size for No. bytes stored to info_complete (#3190) * Restores the ability to create a Zarr V2 array with a `null` fill value by introducing a new class `DefaultFillValue`, and setting the default value of the `fill_value` parameter in array creation routines to an instance of `DefaultFillValue`. For Zarr V3 arrays, `None` will act as an alias for a `DefaultFillValue` instance, thus preserving compatibility with existing code. (#3198) * Fix the type of `ArrayV2Metadata.codec` to constrain it to `numcodecs.abc.Codec | None`. Previously the type was more permissive, allowing objects that can be parsed into Codecs (e.g., the codec name). The constructor of `ArrayV2Metadata` still allows the permissive input when creating new objects. (#3232) #### Improved Documentation# * Add a self-contained example of data type extension to the `examples` directory, and expanded the documentation for data types. (#3157) * * Add a description on how to create a RemoteStore of a specific filesystem to the Remote Store section in docsuser-guidestorage.rst. * State in the docstring of FsspecStore.from_url that the filesystem type is inferred from the URL scheme. It should help a user handling the case when the type of FsspecStore doesn’t match the URL scheme. (#3212) #### Deprecations and Removals# * Removes default chunk encoding settings (filters, serializer, compressors) from the global configuration object. This removal is justified on the basis that storing chunk encoding settings in the config required a brittle, confusing, and inaccurate categorization of array data types, which was particularly unsuitable after the recent addition of new data types that didn’t fit naturally into the pre-existing categories. The default chunk encoding is the same (Zstandard compression, and the required object codecs for variable length data types), but the chunk encoding is now generated by functions that cannot be reconfigured at runtime. Users who relied on setting the default chunk encoding via the global configuration object should instead specify the desired chunk encoding explicitly when creating an array. This change also adds an extra validation step to the creation of Zarr V2 arrays, which ensures that arrays with a `VariableLengthUTF8` or `VariableLengthBytes` data type cannot be created without the correct “object codec”. (#3228) * Removes support for passing keyword-only arguments positionally to the following functions and methods: `save_array`, `open`, `group`, `open_group`, `create`, `get_basic_selection`, `set_basic_selection`, `get_orthogonal_selection`, `set_orthogonal_selection`, `get_mask_selection`, `set_mask_selection`, `get_coordinate_selection`, `set_coordinate_selection`, `get_block_selection`, `set_block_selection`, `Group.create_array`, `Group.empty`, `Group.zeroes`, `Group.ones`, `Group.empty_like`, `Group.full`, `Group.zeros_like`, `Group.ones_like`, `Group.full_like`, `Group.array`. Prior to this change, passing a keyword-only argument positionally to one of these functions or methods would raise a deprecation warning. That warning is now gone. Passing keyword-only arguments to these functions and methods positionally is now an error. ### 3.0.10 (2025-07-03)# #### Bugfixes# * Removed an unnecessary check from `_fsspec._make_async` that would raise an exception when creating a read-only store backed by a local file system with `auto_mkdir` set to `False`. (#3193) * Add missing import for AsyncFileSystemWrapper for _make_async in _fsspec.py (#3195) ### 3.0.9 (2025-06-30)# #### Features# * Add zarr.storage.FsspecStore.from_mapper() so that zarr.open() supports stores of type fsspec.mapping.FSMap. (#2774) * Implemented `move` for `LocalStore` and `ZipStore`. This allows users to move the store to a different root path. (#3021) * Added ~zarr.errors.GroupNotFoundError, which is raised when attempting to open a group that does not exist. (#3066) * Adds `fill_value` to the list of attributes displayed in the output of the `AsyncArray.info()` method. (#3081) * Use `numpy.zeros()` instead of `np.full()` for a performance speedup when creating a zarr.core.buffer.NDBuffer with fill_value=0. (#3082) * Port more stateful testing actions from Icechunk. (#3130) * Adds a with_read_only convenience method to the Store abstract base class (raises NotImplementedError) and implementations to the MemoryStore, ObjectStore, LocalStore, and FsspecStore classes. (#3138) #### Bugfixes# * Ignore stale child metadata when reconsolidating metadata. (#2921) * For Zarr format 2, allow fixed-length string arrays to be created without automatically inserting a `Vlen-UT8` codec in the array of filters. Fixed-length string arrays do not need this codec. This change fixes a regression where fixed-length string arrays created with Zarr Python 3 could not be read with Zarr Python 2.18. (#3100) * When creating arrays without explicitly specifying a chunk size using zarr.create and other array creation routines, the chunk size will now set automatically instead of defaulting to the data shape. For large arrays this will result in smaller default chunk sizes. To retain previous behaviour, explicitly set the chunk shape to the data shape. This fix matches the existing chunking behaviour of zarr.save_array and zarr.api.asynchronous.AsyncArray.create. (#3103) * When zarr.save has an argument path=some/path/ and multiple arrays in args, the path resulted in some/path/some/path due to using the path argument twice while building the array path. This is now fixed. (#3127) * Fix zarr.open default for argument mode when store is read_only (#3128) * Suppress FileNotFoundError when deleting non-existent keys in the obstore adapter. When writing empty chunks (i.e. chunks where all values are equal to the array’s fill value) to a zarr array, zarr will delete those chunks from the underlying store. For zarr arrays backed by the obstore adapter, this will potentially raise a FileNotFoundError if the chunk doesn’t already exist. Since whether or not a delete of a non-existing object raises an error depends on the behavior of the underlying store, suppressing the error in all cases results in consistent behavior across stores, and is also what zarr seems to expect from the store. (#3140) * Trying to open a StorePath/Array with `mode='r'` when the store is not read-only creates a read-only copy of the store. (#3156) ### 3.0.8 (2025-05-19)# Warning In versions 3.0.0 to 3.0.7 opening arrays or groups with `mode='a'` (the default for many builtin functions) would cause any existing paths in the store to be deleted. This is fixed in 3.0.8, and we recommend all users upgrade to avoid this bug that could cause unintentional data loss. #### Features# * Added a print_debug_info function for bug reports. (#2913) #### Bugfixes# * Fix a bug that prevented the number of initialized chunks being counted properly. (#2862) * Fixed sharding with GPU buffers. (#2978) * Fix structured dtype fill value serialization for consolidated metadata (#2998) * It is now possible to specify no compressor when creating a zarr format 2 array. This can be done by passing `compressor=None` to the various array creation routines. The default behaviour of automatically choosing a suitable default compressor remains if the compressor argument is not given. To reproduce the behaviour in previous zarr-python versions when `compressor=None` was passed, pass `compressor='auto'` instead. (#3039) * Fixed the typing of `dimension_names` arguments throughout so that it now accepts iterables that contain None alongside str. (#3045) * Using various functions to open data with `mode='a'` no longer deletes existing data in the store. (#3062) * Internally use typesize constructor parameter for `numcodecs.blosc.Blosc` to improve compression ratios back to the v2-package levels. (#2962) * Specifying the memory order of Zarr format 2 arrays using the `order` keyword argument has been fixed. (#2950) #### Misc# * #2972, #3027, #3049 ### 3.0.7 (2025-04-22)# #### Features# * Add experimental ObjectStore storage class based on obstore. (#1661) * Add `zarr.from_array` using concurrent streaming of source data (#2622) #### Bugfixes# * 0-dimensional arrays are now returning a scalar. Therefore, the return type of `__getitem__` changed to NDArrayLikeOrScalar. This change is to make the behavior of 0-dimensional arrays consistent with `numpy` scalars. (#2718) * Fix fill_value serialization for NaN in ArrayV2Metadata and add property-based testing of round-trip serialization (#2802) * Fixes ConsolidatedMetadata serialization of nan, inf, and -inf to be consistent with the behavior of ArrayMetadata. (#2996) #### Improved Documentation# * Updated the 3.0 migration guide to include the removal of “.” syntax for getting group members. (#2991, #2997) #### Misc# * Define a new versioning policy based on Effective Effort Versioning. This replaces the old Semantic Versioning-based policy. (#2924, #2910) * Make warning filters in the tests more specific, so warnings emitted by tests added in the future are more likely to be caught instead of ignored. (#2714) * Avoid an unnecessary memory copy when writing Zarr to a local file (#2944) ### 3.0.6 (2025-03-20)# #### Bugfixes# * Restore functionality of del z.attrs[‘key’] to actually delete the key. (#2908) ### 3.0.5 (2025-03-07)# #### Bugfixes# * Fixed a bug where `StorePath` creation would not apply standard path normalization to the `path` parameter, which led to the creation of arrays and groups with invalid keys. (#2850) * Prevent update_attributes calls from deleting old attributes (#2870) #### Misc# * #2796 ### 3.0.4 (2025-02-23)# #### Features# * Adds functions for concurrently creating multiple arrays and groups. (#2665) #### Bugfixes# * Fixed a bug where `ArrayV2Metadata` could save `filters` as an empty array. (#2847) * Fix a bug when setting values of a smaller last chunk. (#2851) #### Misc# * #2828 ### 3.0.3 (2025-02-14)# #### Features# * Improves performance of FsspecStore.delete_dir for remote filesystems supporting concurrent/batched deletes, e.g., s3fs. (#2661) * Added `zarr.config.enable_gpu()` to update Zarr’s configuration to use GPUs. (#2751) * Avoid reading chunks during writes where possible. #757 (#2784) * `LocalStore` learned to `delete_dir`. This makes array and group deletes more efficient. (#2804) * Add zarr.testing.strategies.array_metadata to generate ArrayV2Metadata and ArrayV3Metadata instances. (#2813) * Add arbitrary shards to Hypothesis strategy for generating arrays. (#2822) #### Bugfixes# * Fixed bug with Zarr using device memory, instead of host memory, for storing metadata when using GPUs. (#2751) * The array returned by `zarr.empty` and an empty `zarr.core.buffer.cpu.NDBuffer` will now be filled with the specified fill value, or with zeros if no fill value is provided. This fixes a bug where Zarr format 2 data with no fill value was written with un-predictable chunk sizes. (#2755) * Fix zip-store path checking for stores with directories listed as files. (#2758) * Use removeprefix rather than replace when removing filename prefixes in FsspecStore.list (#2778) * Enable automatic removal of needs release notes with labeler action (#2781) * Use the proper label config (#2785) * Alters the behavior of `create_array` to ensure that any groups implied by the array’s name are created if they do not already exist. Also simplifies the type signature for any function that takes an ArrayConfig-like object. (#2795) * Enitialise empty chunks to the default fill value during writing and add default fill values for datetime, timedelta, structured, and other (void* fixed size) data types (#2799) * Ensure utf8 compliant strings are used to construct numpy arrays in property-based tests (#2801) * Fix pickling for ZipStore (#2807) * Update numcodecs to not overwrite codec configuration ever. Closes #2800. (#2811) * Fix fancy indexing (e.g. arr[5, [0, 1]]) with the sharding codec (#2817) #### Improved Documentation# * Added new user guide on Using GPUs with Zarr. (#2751) ### 3.0.2 (2025-01-31)# #### Features# * Test `getsize()` and `getsize_prefix()` in `StoreTests`. (#2693) * Test that a `ValueError` is raised for invalid byte range syntax in `StoreTests`. (#2693) * Separate instantiating and opening a store in `StoreTests`. (#2693) * Add a test for using Stores as a context managers in `StoreTests`. (#2693) * Implemented `LogingStore.open()`. (#2693) * `LoggingStore` is now a generic class. (#2693) * Change StoreTest’s `test_store_repr`, `test_store_supports_writes`, `test_store_supports_partial_writes`, and `test_store_supports_listing` to to be implemented using `@abstractmethod`, rather raising `NotImplementedError`. (#2693) * Test the error raised for invalid buffer arguments in `StoreTests`. (#2693) * Test that data can be written to a store that’s not yet open using the store.set method in `StoreTests`. (#2693) * Adds a new function `init_array` for initializing an array in storage, and refactors `create_array` to use `init_array`. `create_array` takes two new parameters: `data`, an optional array-like object, and `write_data`, a bool which defaults to `True`. If `data` is given to `create_array`, then the `dtype` and `shape` attributes of `data` are used to define the corresponding attributes of the resulting Zarr array. Additionally, if `data` given and `write_data` is `True`, then the values in `data` will be written to the newly created array. (#2761) #### Bugfixes# * Wrap sync fsspec filesystems with `AsyncFileSystemWrapper`. (#2533) * Added backwards compatibility for Zarr format 2 structured arrays. (#2681) * Update equality for `LoggingStore` and `WrapperStore` such that ‘other’ must also be a `LoggingStore` or `WrapperStore` respectively, rather than only checking the types of the stores they wrap. (#2693) * Ensure that `ZipStore` is open before getting or setting any values. (#2693) * Use stdout rather than stderr as the default stream for `LoggingStore`. (#2693) * Match the errors raised by read only stores in `StoreTests`. (#2693) * Fixed `ZipStore` to make sure the correct attributes are saved when instances are pickled. This fixes a previous bug that prevent using `ZipStore` with a `ProcessPoolExecutor`. (#2762) * Updated the optional test dependencies to include `botocore` and `fsspec`. (#2768) * Fixed the fsspec tests to skip if `botocore` is not installed. Previously they would have failed with an import error. (#2768) * Optimize full chunk writes. (#2782) #### Improved Documentation# * Changed the machinery for creating changelog entries. Now individual entries should be added as files to the changes directory in the zarr-python repository, instead of directly to the changelog file. (#2736) #### Other# * Created a type alias `ChunkKeyEncodingLike` to model the union of `ChunkKeyEncoding` instances and the dict form of the parameters of those instances. `ChunkKeyEncodingLike` should be used by high-level functions to provide a convenient way for creating `ChunkKeyEncoding` objects. (#2763) ### 3.0.1 (Jan. 17, 2025)# * Implement `zarr.from_array` using concurrent streaming (#2622). #### Bug fixes# * Fixes `order` argument for Zarr format 2 arrays (#2679). * Fixes a bug that prevented reading Zarr format 2 data with consolidated metadata written using `zarr-python` version 2 (#2694). * Ensure that compressor=None results in no compression when writing Zarr format 2 data (#2708). * Fix for empty consolidated metadata dataset: backwards compatibility with Zarr-Python 2 (#2695). #### Documentation# * Add v3.0.0 release announcement banner (#2677). * Quickstart guide alignment with V3 API (#2697). * Fix doctest failures related to numcodecs 0.15 (#2727). #### Other# * Removed some unnecessary files from the source distribution to reduce its size. (#2686). * Enable codecov in GitHub actions (#2682). * Speed up hypothesis tests (#2650). * Remove multiple imports for an import name (#2723). ### 3.0.0 (Jan. 9, 2025)# 3.0.0 is a new major release of Zarr-Python, with many breaking changes. See the 3.0 Migration Guide for a listing of what’s changed. Normal release note service will resume with further releases in the 3.0.0 series. Release notes for the zarr-python 2.x and 1.x releases can be found here: https://zarr.readthedocs.io/en/support-v2/release.html ## Developer’s Guide# ### Contributing to Zarr# Zarr is a community maintained project. We welcome contributions in the form of bug reports, bug fixes, documentation, enhancement proposals and more. This page provides information on how best to contribute. #### Asking for help# If you have a question about how to use Zarr, please post your question on StackOverflow using the “zarr” tag. If you don’t get a response within a day or two, feel free to raise a GitHub issue including a link to your StackOverflow question. We will try to respond to questions as quickly as possible, but please bear in mind that there may be periods where we have limited time to answer questions due to other commitments. #### Bug reports# If you find a bug, please raise a GitHub issue. Please include the following items in a bug report: 1. A minimal, self-contained snippet of Python code reproducing the problem. You can format the code nicely using markdown, e.g.: ```python import zarr g = zarr.group() # etc. ``` 2. An explanation of why the current behaviour is wrong/not desired, and what you expect instead. 3. Information about the version of Zarr, along with versions of dependencies and the Python interpreter, and installation information. The version of Zarr can be obtained from the `zarr.__version__` property. Please also state how Zarr was installed, e.g., “installed via pip into a virtual environment”, or “installed using conda”. Information about other packages installed can be obtained by executing `pip freeze` (if using pip to install packages) or `conda env export` (if using conda to install packages) from the operating system command prompt. The version of the Python interpreter can be obtained by running a Python interactive session, e.g.: $ python Python 3.12.7 | packaged by conda-forge | (main, Oct 4 2024, 15:57:01) [Clang 17.0.6 ] on darwin #### Enhancement proposals# If you have an idea about a new feature or some other improvement to Zarr, please raise a GitHub issue first to discuss. We very much welcome ideas and suggestions for how to improve Zarr, but please bear in mind that we are likely to be conservative in accepting proposals for new features. The reasons for this are that we would like to keep the Zarr code base lean and focused on a core set of functionalities, and available time for development, review and maintenance of new features is limited. But if you have a great idea, please don’t let that stop you from posting it on GitHub, just please don’t be offended if we respond cautiously. #### Contributing code and/or documentation# ##### Forking the repository# The Zarr source code is hosted on GitHub at the following location: * zarr-developers/zarr-python You will need your own fork to work on the code. Go to the link above and hit the “Fork” button. Then clone your fork to your local machine: $ git clone git@github.com:your-user-name/zarr-python.git $ cd zarr-python $ git remote add upstream git@github.com:zarr-developers/zarr-python.git ##### Creating a development environment# To work with the Zarr source code, it is recommended to use hatch to create and manage development environments. Hatch will automatically install all Zarr dependencies using the same versions as are used by the core developers and continuous integration services. Assuming you have a Python 3 interpreter already installed, and you have cloned the Zarr source code and your current working directory is the root of the repository, you can do something like the following: $ pip install hatch $ hatch env show # list all available environments To verify that your development environment is working, you can run the unit tests for one of the test environments, e.g.: $ hatch env run --env test.py3.12-2.2-optional run-pytest ##### Creating a branch# Before you do any new work or submit a pull request, please open an issue on GitHub to report the bug or propose the feature you’d like to add. It’s best to synchronize your fork with the upstream repository, then create a new, separate branch for each piece of work you want to do. E.g.: git checkout main git fetch upstream git checkout -b shiny-new-feature upstream/main git push -u origin shiny-new-feature This changes your working directory to the ‘shiny-new-feature’ branch. Keep any changes in this branch specific to one bug or feature so it is clear what the branch brings to Zarr. To update this branch with latest code from Zarr, you can retrieve the changes from the main branch and perform a rebase: git fetch upstream git rebase upstream/main This will replay your commits on top of the latest Zarr git main. If this leads to merge conflicts, these need to be resolved before submitting a pull request. Alternatively, you can merge the changes in from upstream/main instead of rebasing, which can be simpler: git pull upstream main Again, any conflicts need to be resolved before submitting a pull request. ##### Running the test suite# Zarr includes a suite of unit tests. The simplest way to run the unit tests is to activate your development environment (see creating a development environment above) and invoke: $ hatch env run --env test.py3.12-2.2-optional run-pytest All tests are automatically run via GitHub Actions for every pull request and must pass before code can be accepted. Test coverage is also collected automatically via the Codecov service. Note Previous versions of Zarr-Python made extensive use of doctests. These tests were not maintained during the 3.0 refactor but may be brought back in the future. See #2614 for more details. ##### Code standards - using pre-commit# All code must conform to the PEP8 standard. Regarding line length, lines up to 100 characters are allowed, although please try to keep under 90 wherever possible. `Zarr` uses a set of `pre-commit` hooks and the `pre-commit` bot to format, type-check, and prettify the codebase. `pre-commit` can be installed locally by running: $ python -m pip install pre-commit The hooks can be installed locally by running: $ pre-commit install This would run the checks every time a commit is created locally. These checks will also run on every commit pushed to an open PR, resulting in some automatic styling fixes by the `pre-commit` bot. The checks will by default only run on the files modified by a commit, but the checks can be triggered for all the files by running: $ pre-commit run --all-files If you would like to skip the failing checks and push the code for further discussion, use the `--no-verify` option with `git commit`. ##### Test coverage# Note Test coverage for Zarr-Python 3 is currently not at 100%. This is a known issue and help is welcome to bring test coverage back to 100%. See #2613 for more details. Zarr strives to maintain 100% test coverage under the latest Python stable release Both unit tests and docstring doctests are included when computing coverage. Running: $ hatch env run --env test.py3.12-2.2-optional run-coverage will automatically run the test suite with coverage and produce a XML coverage report. This should be 100% before code can be accepted into the main code base. You can also generate an HTML coverage report by running: $ hatch env run --env test.py3.12-2.2-optional run-coverage-html When submitting a pull request, coverage will also be collected across all supported Python versions via the Codecov service, and will be reported back within the pull request. Codecov coverage must also be 100% before code can be accepted. ##### Documentation# Docstrings for user-facing classes and functions should follow the numpydoc standard, including sections for Parameters and Examples. All examples should run and pass as doctests under Python 3.11. Zarr uses Sphinx for documentation, hosted on readthedocs.org. Documentation is written in the RestructuredText markup language (.rst files) in the `docs` folder. The documentation consists both of prose and API documentation. All user-facing classes and functions are included in the API documentation, under the `docs/api` folder using the autodoc extension to sphinx. Any new features or important usage information should be included in the user-guide (`docs/user-guide`). Any changes should also be included as a new file in the `changes` directory. The documentation can be built locally by running: $ hatch --env docs run build The resulting built documentation will be available in the `docs/_build/html` folder. Hatch can also be used to serve continuously updating version of the documentation during development at http://0.0.0.0:8000/. This can be done by running: $ hatch --env docs run serve ##### Changelog# zarr-python uses towncrier to manage release notes. Most pull requests should include at least one news fragment describing the changes. To add a release note, you’ll need the GitHub issue or pull request number and the type of your change (`feature`, `bugfix`, `doc`, `removal`, `misc`). With that, run ``towncrier create`` with your development environment, which will prompt you for the issue number, change type, and the news text: towncrier create Alternatively, you can manually create the files in the `changes` directory using the naming convention `{issue-number}.{change-type}.rst`. See the towncrier docs for more. The following information is mainly for core developers, but may also be of interest to contributors. #### Merging pull requests# Pull requests submitted by an external contributor should be reviewed and approved by at least one core developer before being merged. Ideally, pull requests submitted by a core developer should be reviewed and approved by at least one other core developer before being merged. Pull requests should not be merged until all CI checks have passed (GitHub Actions, Codecov) against code that has had the latest main merged in. Before merging the milestone must be set either to decide whether a PR will be in the next patch, minor, or major release. The next section explains which types of changes go in each release. #### Compatibility and versioning policies# ##### Versioning# Versions of this library are identified by a triplet of integers with the form `..`, for example `3.0.4`. A release of `zarr-python` is associated with a new version identifier. That new identifier is generated by incrementing exactly one of the components of the previous version identifier by 1. When incrementing the `major` component of the version identifier, the `minor` and `patch` components is reset to 0. When incrementing the minor component, the patch component is reset to 0. Releases are classified by the library changes contained in that release. This classification determines which component of the version identifier is incremented on release. * `major` releases (for example, `2.18.0` -> `3.0.0`) are for changes that will require extensive adaptation efforts from many users and downstream projects. For example, breaking changes to widely-used user-facing APIs should only be applied in a major release. Users and downstream projects should carefully consider the impact of a major release before adopting it. In advance of a major release, developers should communicate the scope of the upcoming changes, and help users prepare for them. * `minor` releases (or example, `3.0.0` -> `3.1.0`) are for changes that do not require significant effort from most users or downstream downstream projects to respond to. API changes are possible in minor releases if the burden on users imposed by those changes is sufficiently small. For example, a recently released API may need fixes or refinements that are breaking, but low impact due to the recency of the feature. Such API changes are permitted in a minor release. Minor releases are safe for most users and downstream projects to adopt. * `patch` releases (for example, `3.1.0` -> `3.1.1`) are for changes that contain no breaking or behaviour changes for downstream projects or users. Examples of changes suitable for a patch release are bugfixes and documentation improvements. Users should always feel safe upgrading to a the latest patch release. Note that this versioning scheme is not consistent with Semantic Versioning. Contrary to SemVer, the Zarr library may release breaking changes in `minor` releases, or even `patch` releases under exceptional circumstances. But we should strive to avoid doing so. A better model for our versioning scheme is Intended Effort Versioning, or “EffVer”. The guiding principle off EffVer is to categorize releases based on the expected effort required to upgrade to that release. Zarr developers should make changes as smooth as possible for users. This means making backwards-compatible changes wherever possible. When a backwards-incompatible change is necessary, users should be notified well in advance, e.g. via informative deprecation warnings. ###### Data format compatibility# The Zarr library is an implementation of a file format standard defined externally – see the Zarr specifications website for the list of Zarr file format specifications. If an existing Zarr format version changes, or a new version of the Zarr format is released, then the Zarr library will generally require changes. It is very likely that a new Zarr format will require extensive breaking changes to the Zarr library, and so support for a new Zarr format in the Zarr library will almost certainly come in new `major` release. When the Zarr library adds support for a new Zarr format, there may be a period of accelerated changes as developers refine newly added APIs and deprecate old APIs. In such a transitional phase breaking changes may be more frequent than usual. #### Release procedure# Open an issue on GitHub announcing the release using the release checklist template: zarr-developers/zarr-python#new. The release checklist includes all steps necessary for the release. ### Roadmap# * Status: active * Author: Joe Hamman * Created On: October 31, 2023 * Input from: * Davis Bennett / @d-v-b * Norman Rzepka / @normanrz * Deepak Cherian @dcherian * Brian Davis / @monodeldiablo * Oliver McCormack / @olimcc * Ryan Abernathey / @rabernat * Jack Kelly / @JackKelly * Martin Durrant / @martindurant Note This document was written in the early stages of the 3.0 refactor. Some aspects of the design have changed since this was originally written. Questions and discussion about the contents of this document should be directed to this GitHub Discussion. #### Introduction# This document lays out a design proposal for version 3.0 of the Zarr-Python package. A specific focus of the design is to bring Zarr-Python’s API up to date with the Zarr V3 specification, with the hope of enabling the development of the many features and extensions that motivated the V3 Spec. The ideas presented here are expected to result in a major release of Zarr-Python (version 3.0) including significant a number of breaking API changes. For clarity, “V3” will be used to describe the version of the Zarr specification and “3.0” will be used to describe the release tag of the Zarr-Python project. ##### Current status of V3 in Zarr-Python# During the development of the V3 Specification, a prototype implementation was added to the Zarr-Python library. Since that implementation, the V3 spec evolved in significant ways and as a result, the Zarr-Python library is now out of sync with the approved spec. Downstream libraries (e.g. Xarray) have added support for this implementation and will need to migrate to the accepted spec when its available in Zarr-Python. #### Goals# * Provide a complete implementation of Zarr V3 through the Zarr-Python API * Clear the way for exciting extensions / ZEPs (i.e. sharding, variable chunking, etc.) * Provide a developer API that can be used to implement and register V3 extensions * Improve the performance of Zarr-Python by streamlining the interface between the Store layer and higher level APIs (e.g. Groups and Arrays) * Clean up the internal and user facing APIs * Improve code quality and robustness (e.g. achieve 100% type hint coverage) * Align the Zarr-Python array API with the array API Standard #### Examples of what 3.0 will enable?# 1. Reading and writing V3 spec-compliant groups and arrays 2. V3 extensions including sharding and variable chunking. 3. Improved performance by leveraging concurrency when creating/reading/writing to stores (imagine a `create_hierarchy(zarr_objects)` function). 4. User-developed extensions (e.g. storage-transformers) can be registered with Zarr-Python at runtime #### Non-goals (of this document)# * Implementation of any unaccepted Zarr V3 extensions * Major revisions to the Zarr V3 spec #### Requirements# 1. Read and write spec compliant V2 and V3 data 2. Limit unnecessary traffic to/from the store 3. Cleanly define the Array/Group/Store abstractions 4. Cleanly define how V2 will be supported going forward 5. Provide a clear roadmap to help users upgrade to 3.0 6. Developer tools / hooks for registering extensions #### Design# ##### Async API# Zarr-Python is an IO library. As such, supporting concurrent action against the storage layer is critical to achieving acceptable performance. The Zarr-Python 2 was not designed with asynchronous computation in mind and as a result has struggled to effectively leverage the benefits of concurrency. At one point, `getitems` and `setitems` support was added to the Zarr store model but that is only used for operating on a set of chunks in a single variable. With Zarr-Python 3.0, we have the opportunity to revisit this design. The proposal here is as follows: 1. The `Store` interface will be entirely async. 2. On top of the async `Store` interface, we will provide an `AsyncArray` and `AsyncGroup` interface. 3. Finally, the primary user facing API will be synchronous `Array` and `Group` classes that wrap the async equivalents. Examples * Store class Store: ... async def get(self, key: str) -> bytes: ... async def get_partial_values(self, key_ranges: List[Tuple[str, Tuple[int, Optional[int]]]]) -> bytes: ... # (no sync interface here) * Array class AsyncArray: ... async def getitem(self, selection: Selection) -> np.ndarray: # the core logic for getitem goes here class Array: _async_array: AsyncArray def __getitem__(self, selection: Selection) -> np.ndarray: return sync(self._async_array.getitem(selection)) * Group class AsyncGroup: ... async def create_group(self, path: str, **kwargs) -> AsyncGroup: # the core logic for create_group goes here class Group: _async_group: AsyncGroup def create_group(self, path: str, **kwargs) -> Group: return sync(self._async_group.create_group(path, **kwargs)) Internal Synchronization API With the `Store` and core `AsyncArray`/ `AsyncGroup` classes being predominantly async, Zarr-Python will need an internal API to provide a synchronous API. The proposal here is to use the approach in fsspec to provide a high-level `sync` function that takes an `awaitable` and runs it in its managed IO Loop / thread. FAQ 1\. Why two levels of Arrays/groups? a. First, this is an intentional decision and departure from the current Zarrita implementation b. The idea is that users rarely want to mix interfaces. Either they are working within an async context (currently quite rare) or they are in a typical synchronous context. c. Splitting the two will allow us to clearly define behavior on the `AsyncObj` and simply wrap it in the `SyncObj`. 2. What if a store is only has a synchronous backend? a. First off, this is expected to be a fairly rare occurrence. Most storage backends have async interfaces. b. But in the event a storage backend doesn’t have a async interface, there is nothing wrong with putting synchronous code in `async` methods. There are approaches to enabling concurrent action through wrappers like AsyncIO’s `loop.run_in_executor` (ref 1, ref 2, ref 3, ref 4. 3\. Will Zarr help manage the async contexts encouraged by some libraries (e.g. AioBotoCore)? a. Many async IO libraries require entering an async context before interacting with the API. We expect some experimentation to be needed here but the initial design will follow something close to what fsspec does (example in s3fs). 4\. Why not provide a synchronous Store interface? a. We could but this design is simpler. It would mean supporting it in the `AsyncGroup` and `AsyncArray` classes which, may be more trouble than its worth. Storage backends that do not have an async API will be encouraged to wrap blocking calls in an async wrapper (e.g. `loop.run_in_executor`). ##### Store API# The `Store` API is specified directly in the V3 specification. All V3 stores should implement this abstract API, omitting Write and List support as needed. As described above, all stores will be expected to expose the required methods as async methods. Example class ReadWriteStore: ... async def get(self, key: str) -> bytes: ... async def get_partial_values(self, key_ranges: List[Tuple[str, int, int]) -> bytes: ... async def set(self, key: str, value: Union[bytes, bytearray, memoryview]) -> None: ... # required for writable stores async def set_partial_values(self, key_start_values: List[Tuple[str, int, Union[bytes, bytearray, memoryview]]]) -> None: ... # required for writable stores async def list(self) -> List[str]: ... # required for listable stores async def list_prefix(self, prefix: str) -> List[str]: ... # required for listable stores async def list_dir(self, prefix: str) -> List[str]: ... # required for listable stores # additional (optional methods) async def getsize(self, prefix: str) -> int: ... async def rename(self, src: str, dest: str) -> None ... Recognizing that there are many Zarr applications today that rely on the `MutableMapping` interface supported by Zarr-Python 2, a wrapper store will be developed to allow existing stores to plug directly into this API. ##### Array API# The user facing array interface will implement a subset of the Array API Standard. Most of the computational parts of the Array API Standard don’t fit into Zarr right now. That’s okay. What matters most is that we ensure we can give downstream applications a compliant API. Note, Zarr already does most of this so this is more about formalizing the relationship than a substantial change in API. Included Not Included Unknown / Maybe Possible Attributes `dtype` `mT` `device` `ndim` `T` `shape` `size` Methods `__getitem__` `__array_namespace__` `to_device` `__setitem__` `__abs__` `__bool__` `__eq__` `__add__` `__complex__` `__bool__` `__and__` `__dlpack__` `__floordiv__` `__dlpack_device__` `__ge__` `__float__` `__gt__` `__index__` `__invert__` `__int__` `__le__` `__lshift__` `__lt__` `__matmul__` `__mod__` `__mul__` `__ne__` `__neg__` `__or__` `__pos__` `__pow__` `__rshift__` `__sub__` `__truediv__` `__xor__` Creation functions (`zarr.creation`) `zeros` `arange` `zeros_like` `asarray` `ones` `eye` `ones_like` `from_dlpack` `full` `linspace` `full_like` `meshgrid` `empty` `tril` `empty_like` `triu` In addition to the core array API defined above, the Array class should have the following Zarr specific properties: * `.metadata` (see Metadata Interface below) * `.attrs` \- (pulled from metadata object) * `.info` \- (repolicated from existing property †) † In Zarr-Python 2, the info property listed the store to identify initialized chunks. By default this will be turned off in 3.0 but will be configurable. Indexing Zarr-Python currently supports `__getitem__` style indexing and the special `oindex` and `vindex` indexers. These are not part of the current Array API standard (see data-apis/array-api#669) but they have been proposed as a NEP. Zarr-Python will maintain these in 3.0. We are also exploring a new high-level indexing API that will enabled optimized batch/concurrent loading of many chunks. We expect this to be important to enable performant loading of data in the context of sharding. See this discussion for more detail. Concurrent indexing across multiple arrays will be possible using the AsyncArray API. Async and Sync Array APIs Most the logic to support Zarr Arrays will live in the `AsyncArray` class. There are a few notable differences that should be called out. Sync Method Async Method `__getitem__` `getitem` `__setitem__` `setitem` `__eq__` `equals` Metadata interface Zarr-Python 2.* closely mirrors the V2 spec metadata schema in the Array and Group classes. In 3.0, we plan to move the underlying metadata representation to a separate interface (e.g. `Array.metadata`). This interface will return either a `V2ArrayMetadata` or `V3ArrayMetadata` object (both will inherit from a parent `ArrayMetadataABC` class. The `V2ArrayMetadata` and `V3ArrayMetadata` classes will be responsible for producing valid JSON representations of their metadata, and yielding a consistent view to the `Array` or `Group` class. ##### Group API# The main question is how closely we should follow the existing Zarr-Python implementation / `MutableMapping` interface. The table below shows the primary `Group` methods in Zarr-Python 2 and attempts to identify if and how they would be implemented in 3.0. V2 Group Methods `AsyncGroup` `Group` `h5py_compat.Group` `__len__` `length` `__len__` `__len__` `__iter__` `__aiter__` `__iter__` `__iter__` `__contains__` `contains` `__contains__` `__contains__` `__getitem__` `getitem` `__getitem__` `__getitem__` `__enter__` N/A N/A `__enter__` `__exit__` N/A N/A `__exit__` `group_keys` `group_keys` `group_keys` N/A `groups` `groups` `groups` N/A `array_keys` `array_key` `array_keys` N/A `arrays` `arrays` `arrays` N/A `visit` ? ? `visit` `visitkeys` ? ? ? `visitvalues` ? ? ? `visititems` ? ? `visititems` `tree` `tree` `tree` `Both` `create_group` `create_group` `create_group` `create_group` `require_group` N/A N/A `require_group` `create_groups` ? ? N/A `require_groups` ? ? ? `create_dataset` N/A N/A `create_dataset` `require_dataset` N/A N/A `require_dataset` `create` `create_array` `create_array` N/A `empty` `empty` `empty` N/A `zeros` `zeros` `zeros` N/A `ones` `ones` `ones` N/A `full` `full` `full` N/A `array` `create_array` `create_array` N/A `empty_like` `empty_like` `empty_like` N/A `zeros_like` `zeros_like` `zeros_like` N/A `ones_like` `ones_like` `ones_like` N/A `full_like` `full_like` `full_like` N/A `move` `move` `move` `move` ``zarr.h5compat.Group`` – Zarr-Python 2.* made an attempt to align its API with that of h5py. With 3.0, we will relax this alignment in favor of providing an explicit compatibility module (`zarr.h5py_compat`). This module will expose the `Group` and `Dataset` APIs that map to Zarr-Python’s `Group` and `Array` objects. ##### Creation API# Zarr-Python 2.* bundles together the creation and serialization of Zarr objects. Zarr-Python 3.* will make it possible to create objects in memory separate from serializing them. This will specifically enable writing hierarchies of Zarr objects in a single batch step. For example: arr1 = Array(shape=(10, 10), path="foo/bar", dtype="i4", store=store) arr2 = Array(shape=(10, 10), path="foo/spam", dtype="f8", store=store) arr1.save() arr2.save() # or equivalently zarr.save_many([arr1 ,arr2]) Note: this batch creation API likely needs additional design effort prior to implementation. ##### Plugin API# Zarr V3 was designed to be extensible at multiple layers. Zarr-Python will support these extensions through a combination of Abstract Base Classes (ABCs) and Entrypoints. ABCs Zarr V3 will expose Abstract base classes for the following objects: * `Store`, `ReadStore`, `ReadWriteStore`, `ReadListStore`, and `ReadWriteListStore` * `BaseArray`, `SynchronousArray`, and `AsynchronousArray` * `BaseGroup`, `SynchronousGroup`, and `AsynchronousGroup` * `Codec`, `ArrayArrayCodec`, `ArrayBytesCodec`, `BytesBytesCodec` Entrypoints Lots more thinking here but the idea here is to provide entrypoints for `data type`, `chunk grid`, `chunk key encoding`, `codecs`, `storage_transformers` and `stores`. These might look something like: entry_points=""" [zarr.codecs] blosc_codec=codec_plugin:make_blosc_codec zlib_codec=codec_plugin:make_zlib_codec """ ##### Python type hints and static analysis# Target 100% Mypy coverage in 3.0 source. ##### Observability# A persistent problem in Zarr-Python is diagnosing problems that span many parts of the stack. To address this in 3.0, we will add a basic logging framework that can be used to debug behavior at various levels of the stack. We propose to add the separate loggers for the following namespaces: * `array` * `group` * `store` * `codec` These should be documented such that users know how to activate them and developers know how to use them when developing extensions. ##### Dependencies# Today, Zarr-Python has the following required dependencies: dependencies = [ 'asciitree', 'numpy>=1.20,!=1.21.0', 'fasteners', 'numcodecs>=0.10.0', ] What other dependencies should be considered? 1. Attrs - Zarrita makes extensive use of the Attrs library 2. Fsspec - Zarrita has a hard dependency on Fsspec. This could be easily relaxed though. #### Breaking changes relative to Zarr-Python 2.*# 1. H5py compat moved to a stand alone module? 2. `Group.__getitem__` support moved to `Group.members.__getitem__`? 3. Others? #### Open questions# 1. How to treat V2 1. Note: Zarrita currently implements a separate `V2Array` and `V3Array` classes. This feels less than ideal. 2. We could easily convert metadata from v2 to the V3 Array, but what about writing? 3. Ideally, we don’t have completely separate code paths. But if its too complicated to support both within one interface, its probably better. 2. How and when to remove the current implementation of V3. 1. It’s hidden behind a hard-to-use feature flag so we probably don’t need to do anything. 3. How to model runtime configuration? 4. Which extensions belong in Zarr-Python and which belong in separate packages? 1. We don’t need to take a strong position on this here. It’s likely that someone will want to put Sharding in. That will be useful to develop in parallel because it will give us a good test case for the plugin interface. #### Testing# Zarr-python 3.0 adds a major new dimension to Zarr: Async support. This also comes with a compatibility risk, we will need to thoroughly test support in key execution environments. Testing plan: - Reuse the existing test suite for testing the `v3` API. - `xfail` tests that expose breaking changes with `3.0 - breaking change` description. This will help identify additional and/or unintentional breaking changes - Rework tests that were only testing internal APIs. - Add a set of functional / integration tests targeting real-world workflows in various contexts (e.g. w/ Dask) #### Development process# Zarr-Python 3.0 will introduce a number of new APIs and breaking changes to existing APIs. In order to facilitate ongoing support for Zarr-Python 2.*, we will take on the following development process: * Create a `v3` branch that can be use for developing the core functionality apart from the `main` branch. This will allow us to support ongoing work and bug fixes on the `main` branch. * Put the `3.0` APIs inside a `zarr.v3` module. Imports from this namespace will all be new APIs that users can develop and test against once the `v3` branch is merged to `main`. * Kickstart the process by pulling in the current state of `zarrita` \- which has many of the features described in this design. * Release a series of 2.* releases with the `v3` namespace * When `v3` is complete, move contents of `v3` to the package root Milestones Below are a set of specific milestones leading toward the completion of this process. As work begins, we expect this list to grow in specificity. 1. Port current version of Zarrita to Zarr-Python 2. Formalize Async interface by splitting `Array` and `Group` objects into Sync and Async versions 3. Implement “fancy” indexing operations on the `AsyncArray` 4. Implement an abstract base class for the `Store` interface and a wrapper `Store` to make use of existing `MutableMapping` stores. 5. Rework the existing unit test suite to use the `v3` namespace. 6. Develop a plugin interface for extensions 7. Develop a set of functional and integration tests 8. Work with downstream libraries (Xarray, Dask, etc.) to test new APIs #### TODOs# The following subjects are not covered in detail above but perhaps should be. Including them here so they are not forgotten. 1. [Store] Should Zarr provide an API for caching objects after first read/list/etc. Read only stores? 2. [Array] buffer protocol support 3. [Array] `meta_array` support 4. [Extensions] Define how Zarr-Python will consume the various plugin types 5. [Misc] H5py compatibility requires a bit more work and a champion to drive it forward. 6. [Misc] Define `chunk_store` API in 3.0 7. [Misc] Define `synchronizer` API in 3.0 #### References# 1. Zarr-Python repository 2. Zarr core specification (version 3.0) — Zarr specs documentation 3. Zarrita repository 4. Async-Zarr 5. Zarr-Python Discussion Topic ## About# Zarr is a format for the storage of chunked, compressed, N-dimensional arrays inspired by HDF5, h5py and bcolz. These documents describe the Zarr-Python implementation. More information about the Zarr format can be found on the main website. ### Projects using Zarr# If you are using Zarr-Python, we would love to hear about it. ### Funding# The project is fiscally sponsored by NumFOCUS, a US 501(c)(3) public charity, and development is supported by the MRC Centre for Genomics and Global Health and the Chan Zuckerberg Initiative. Version: 3.1.3 Useful links: Source Repository | Issue Tracker | Developer Chat | Zarr specifications Zarr-Python is a Python library for reading and writing Zarr groups and arrays. Highlights include: * Specification support for both Zarr format 2 and 3. * Create and read from N-dimensional arrays using NumPy-like semantics. * Flexible storage enables reading and writing from local, cloud and in-memory stores. * High performance: Enables fast I/O with support for asynchronous I/O and multi-threading. * Extensible: Customizable with user-defined codecs and stores. Quick Start New to Zarr? Check out the quick start guide. It contains a brief introduction to Zarr’s main concepts and links to additional tutorials. To the Quick Start Guide A detailed guide for how to use Zarr-Python. To the user guide API Reference The reference guide contains a detailed description of the functions, modules, and objects included in Zarr. The reference describes how the methods work and which parameters can be used. It assumes that you have an understanding of the key concepts. To the API reference Contributor’s Guide Want to contribute to Zarr? We welcome contributions in the form of bug reports, bug fixes, documentation, enhancement proposals and more. The contributing guidelines will guide you through the process of improving Zarr. To the contributor’s guide Download documentation: PDF/Zipped HTML On this page * Quickstart * User guide * API reference * Release notes * Developer’s Guide * About © Copyright 2025, Zarr Developers. Created using Sphinx 8.1.3. Built with the PyData Sphinx Theme 0.16.1.