sparx.datasets

Neuromorphic datasets as dense, binned spike counts.

shd reads the Spiking Heidelberg Digits (Cramer et al., “The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks”, IEEE TNNLS 2020): spoken digits 0-9 in English and German, 20 classes, rendered as spikes on 700 cochlear channels. Each record becomes a [steps, channels] array of spike counts, batch-major as dew’s loaders expect; sparx.encode.EventsEncoder() moves the time axis to the front. SHD is the same data as a dew dataset spec, which a run’s data holds (--data.channels 140 on the recipe’s command line). write_synthetic_shd writes small files in SHD’s layout, which smoke runs and tests read in its place.

mnist reads MNIST (LeCun et al. 1998) or Fashion-MNIST (Xiao et al. 2017) as uint8 images and labels.

holdout splits records into a part to train on and a part to validate on.

Reading the files needs h5py (pip install "sparxml[datasets]").

Contents

Name
MNIST_URLS	Where mnist fetches each IDX file: the MNIST mirror and Zalando’s Fashion-MNIST repository.
SHD	The Spiking Heidelberg Digits as a dew dataset: the train split to train on, the test split to validate on (SHD has no separate validation split), binned by shd.
SHD_URL
Binning	How bin_events cuts time into steps.
bin_events	Count one record’s spikes into [steps, channels] uint8 bins.
holdout	Split records (columns of equal length) into a random 1 - fraction and fraction.
mnist	MNIST split, or Fashion-MNIST’s with fashion=True: uint8 images [N, 28, 28] under "image" and int32 labels [N] under "label".
shd	The SHD split as {"spikes": uint8 [N, steps, channels], "label": int32 [N]}.
write_synthetic_shd	Write shd_train.h5 and shd_test.h5 in SHD’s layout into directory, and return it.

MNIST_URLS

MNIST_URLS

sparx.datasets on GitHub

Where mnist fetches each IDX file: the MNIST mirror and Zalando’s Fashion-MNIST repository.

SHD

class SHD(DatasetSpec)

sparx.datasets on GitHub

The Spiking Heidelberg Digits as a dew dataset: the train split to train on, the test split to validate on (SHD has no separate validation split), binned by shd.

Records are {"spikes": uint8 [steps, channels], "label": int32}; sparx.encode.EventsEncoder() turns a batch into the network’s time-major input. Both splits are held in memory (about 700 MB at 100 steps over 700 channels), shuffled from seed every epoch, and each process reads its share of every batch.

Field	Type	Default
steps	int	100
max_time	float	1.4
channels	int	700
cache	str | None	None
binning	Binning	'grid'

SHD.load

def load(batch: int, tokenize: Tokenize | None = None) -> Dataset

SHD_URL

SHD_URL = 'https://zenkelab.org/datasets/shd/{split}.h5.gz'

sparx.datasets on GitHub

Binning

Binning

sparx.datasets on GitHub

How bin_events cuts time into steps.

• grid: equal bins from time 0, the usual binning.
• events: SpikingJelly’s SHD frames by duration, in the releases SNN-delays (Hammouamri et al., ICLR 2024) trained on: each step opens at the first event not yet counted and holds every event within one step’s duration of it. Silences longer than a step are dropped, so a recording is shorter than on the grid and its timing is compressed; times are scaled to milliseconds in the file’s own float16, as theirs are.

bin_events

def bin_events(times: ArrayLike, units: ArrayLike, steps: int, max_time: float, channels: int, source_channels: int = _CHANNELS, binning: Binning = 'grid') -> np.ndarray

sparx.datasets on GitHub

Count one record’s spikes into [steps, channels] uint8 bins.

A step lasts max_time / steps. On the grid, time [0, max_time) is cut into steps equal bins and events at or after max_time are dropped; with binning="events" steps open at events (Binning) and steps past the steps-th are dropped. channels must divide source_channels; adjacent source channels are pooled into each output channel. Counts saturate at 255.

holdout

def holdout(records: Mapping[str, np.ndarray], fraction: float, seed: int = 0) -> tuple[dict[str, np.ndarray], dict[str, np.ndarray]]

sparx.datasets on GitHub

Split records (columns of equal length) into a random 1 - fraction and fraction.

SHD has no validation split, and SNN-delays selects its epochs on the test set; holding out part of the training set gives a validation set to select on, so the test accuracy stays an estimate.

mnist

def mnist(split: Literal['train', 'test'], fashion: bool = False, cache: str | Path | None = None) -> dict[str, np.ndarray]

sparx.datasets on GitHub

MNIST split, or Fashion-MNIST’s with fashion=True: uint8 images [N, 28, 28] under "image" and int32 labels [N] under "label".

The gzipped IDX files download once to cache, ~/.cache/sparx by default, Fashion-MNIST’s under fashion/.

shd

def shd(split: Literal['train', 'test'], steps: int = 100, max_time: float = 1.4, channels: int = 700, cache: str | Path | None = None, path: str | Path | None = None, binning: Binning = 'grid') -> dict[str, np.ndarray]

sparx.datasets on GitHub

The SHD split as {"spikes": uint8 [N, steps, channels], "label": int32 [N]}.

steps bins over the first max_time seconds. Every spike of the train split falls before 1.37 s, and 100 steps of 14 ms is the binning of Zenke’s SpyTorch SHD tutorial. binning="events" is SNN-delays’ binning (Binning); at 10 ms its recordings last at most 124 steps (train) and 105 (test), so steps=124, max_time=1.24 keeps every event. The file is read from path when given, otherwise downloaded once into cache (default ~/.cache/sparx; 131 MB for train, 38 MB for test) and decompressed beside it.

write_synthetic_shd

def write_synthetic_shd(directory: str | Path, records: int = 64, seed: int = 0) -> Path

sparx.datasets on GitHub

Write shd_train.h5 and shd_test.h5 in SHD’s layout into directory, and return it.

Each split holds records recordings of 60 spikes over 1.4 s, labelled 0 or 1 by which half of the 700 channels fires, so a network learns them in a few steps. shd(split, cache=directory) and SHD(cache=directory) read them as they read SHD’s own files, which lets a smoke run or a test exercise the whole path without the 169 MB download.