Sparse and connectivity subsystem¶
Design note for the PyTorch-first sparse / connectivity system. Part 1 is the audit of the implementation this work started from (kept as a record; the classes it describes no longer exist). Parts 2 onwards describe what replaced it. For usage see the sparse connectivity guide.
1. Audit of the starting point¶
1.1 Relevant code¶
| Path | Names |
|---|---|
btorch/models/linear.py |
Linear |
btorch/models/constrain.py |
HasConstraint, constrain_net (runs constrain() under torch.no_grad) |
btorch/connectome/connection.py |
make_sparse_mat, make_constraint_by_neuron_type, make_hetersynapse_conn, make_hetersynapse_constraint, make_hetersynapse_constrained_conn, stack_hetersynapse, expand_conn_for_delays |
btorch/models/synapse.py |
BasePSC and subclasses call self.linear(z); HeterSynapsePSC, DelayedPSC, BilinearMixingSynapse consume the physically expanded receptor / delay axes |
btorch/models/history.py |
SpikeHistory produces the [..., n_neuron * n_delay_bins] input of a delay-expanded matrix |
Tests: tests/models/test_linear.py, tests/models/test_hetersynapse_conn.py,
tests/connectome/ (test_hetersynapse_conn_modes.py, test_delay_expansion.py,
test_connection_validation.py), tests/models/rnn/test_sparse_rnn.py,
tests/models/test_hetero_rnn.py, tests/models/test_cudagraph_layers.py,
tests/models/test_mem_load_save.py, tests/models/test_numpy_parity.py,
tests/test_pipeline_e2e.py. Baseline before any change: 131 passed, 2 xfailed
for these files.
1.2 Behaviour, checked against the code¶
All eight characteristics listed in the task specification hold:
BaseSparseConntakes a SciPy sparse array (legacyspmatrixalso works because only.tocoo(),.Tand.sum_duplicates()are used). A torch sparse tensor is rejected with anAttributeError.- The matrix is converted to COO, transposed, and
sum_duplicates()is applied, which also sorts. Theindicesbuffer therefore holds[dst, src], sorted by destination then source. forwardflattens all leading dimensions to one batch axis and computesx @ Aas(Aᵀ @ xᵀ)ᵀ; a 1-D input is supported.nativerebuilds atorch.sparse_coo_tensorfromindicesand the current effective values on every call and usestorch.sparse.mm.torch_sparsecallstorch_sparse.spmm(indices, values, n_dst, n_src, xᵀ).SparseConstrainedConnstoresinitial_weight[e], a group index per edge (_constraint_scatter_indices) and onemagnitudeper group; the effective weight isinitial_weight[e] * magnitude[group[e]].- The edge-to-group map is built at construction by a pandas merge on
(row, col); group ids in the constraint matrix are 1-based. make_hetersynapse_connexpands columns topost * n_receptor + receptorand, withdelay_col, rows topre * n_delay_bins + delay_bin.
1.3 Orientation and shape conventions¶
- User-facing connectivity matrices are
(n_src, n_dst): rows are source (pre-synaptic) neurons, columns are destinations. The layer computesy = x @ Awithx[..., n_src],y[..., n_dst]. - Internally the layer stores the transpose, so the stored COO is the standard
linear-algebra operator
M = Aᵀof shape(n_dst, n_src)withy = M @ x, sorted row-major. The stored order is exactly CSR order ofM. BaseSparseConn.shapeis(in_features, out_features), i.e. the user orientation, although its docstring says(num_dst, num_src).get_sparse_matrix()swaps the index rows back and returns the user orientation(n_src, n_dst). It passesis_coalesced=Truealthough the swapped indices are no longer sorted.Linear(weight=W)follows PyTorch directly:Whas shape(out, in)and is stored unchanged innn.Linear.weight.
1.4 Autograd path¶
magnitude → (gather through the group index for the constrained layer) →
values of a freshly built sparse tensor → torch.sparse.mm / torch_sparse.spmm
→ output. Gradients reach both the values and the dense input. constrain()
mutates magnitude.data (Dale's law) and is called from constrain_net under
torch.no_grad().
1.5 torch.compile¶
Measured with torch 2.11:
| Backend | torch.compile(conn) |
torch.compile(conn, fullgraph=True) |
|---|---|---|
native |
runs, but Dynamo graph-breaks on torch.sparse_coo_tensor |
fails: Unsupported: Attempted to wrap sparse Tensor |
torch_sparse |
runs | runs (one graph, no breaks) |
The existing test only passes for native because compile_or_skip does not
request fullgraph.
1.6 Checkpoint and device-move risks¶
SparseConn.initial_signis a non-persistent buffer derived from the construction-time matrix. Loading a checkpoint into a layer built from a different matrix keeps the old signs; the nextconstrain()zeroes every weight whose sign differs. Reproduced: loading weights[-1, 2]into a layer built from[1, -2]gives[0, -0]after oneconstrain().SparseConstrainedConn.initial_weightis non-persistent unlesspersist_initial_weight=True, and_constraint_scatter_indicesis never persistent. A checkpoint therefore only reproduces the layer if it is loaded into a layer built from the same matrices.constraint_info(a DataFrame) is a plain attribute and is not saved.forwardsilently casts the effective values to the input's device and dtype on every call (float64input withfloat32weights yieldsfloat64).- The stale cached sparse tensor the specification mentions has already been
removed (the COO tensor is rebuilt per call); the regression test
test_sparse_conn_state_dict_roundtrip_loads_new_patterncovers it. The price is a COO construction, and insidetorch.sparse.mma COO→CSR conversion, on every forward. torch.sparse.mmbackward with respect to the sparse values does not scale: at 100k neurons and 10M edges thenativebackend fails with an out-of-memory error asking for 37 GiB (N × Nfloats).
1.7 Pain points¶
- SciPy-only input; no torch sparse input; conversion logic lives in the layer.
- The
nativepath is notfullgraph-compilable and rebuilds a sparse tensor per call;torch_sparseis the de-facto required backend for large graphs. - Backend names appear in every constructor (
sparse_backend=). - Grouped weights are a separate matrix class rather than a weight parameterisation; Dale's law is implemented twice.
- Receptors and delays exist only as physically expanded matrix dimensions.
- No network batch (ensembles), no shared-pattern batched values.
- No sparse-spike execution path; cost is independent of spike density.
1.8 Baseline measurements¶
The legacy layers were measured before the refactor and are kept as a frozen
copy in benchmarks/sparse_conn/_legacy_baseline.py, so they can be timed
side by side with the new path. Two properties of the old implementation
matter for the comparison:
- Latency did not depend on spike density (1% and 50% gave the same numbers).
- The
nativebackend could not train at scale: at 100k neurons and 10M edges its backward failed with an out-of-memory error asking for 37 GiB, andtorch_sparseneeded 5.1 GB of peak memory at batch 32.
Current numbers for both paths are in
benchmarks/sparse_conn/RESULTS.md.
1.9 Prior kernel work consulted¶
These are references for kernel algorithms and measurements only.
- Task-packed Triton SpMSpV (PR 70,
_sparse_triton.py): source-major blocked layout, on-device pack → persistent workers with float atomics, a per-worker hash table for duplicate destinations. Forward only in Triton; backward is denseindex_add_with a[B, E]temporary. 8.6× over the per-call COO path at 0.1% activity, break-even near 10%. Its preprocessing is a Python loop over sources. - Pull CSR kernels (megakernel /
glif_net): one warp per row, deterministic, bandwidth-equal to cuSPARSE at large N; the gains at small N come from fusing with the neuron update. One batched Tritontriton_opexists (btorch::csr_spmv). - Push kernels (
connectome_dataset, Praxist): per-active-source atomic scatter over source-major CSR. On the 5090, push beats pull below 30–60% spike density on connectomes up to 140k neurons, by 14–27× at 1%. All measurements are single-vector; no batched push kernel exists, and no kernel has a custom backward. - Sort-based, hash-based (VDHA) and tensor-core SpMM kernels never won in the connectome regime and are not planned.
Consequences for this design: the runtime keeps a destination-major CSR (pull,
batched SpMM, deterministic) and a source-major CSR of the same edges (push,
and the transpose product for the input gradient) linked by an edge
permutation, so one value array serves both. Gradients with respect to edge
values are a gather-multiply (SDDMM) and never build an N × N intermediate.
1.10 Contradictions with the specification or earlier decisions¶
- Compatibility. The specification asks to preserve behaviour through
legacy wrappers and deprecation paths. The project decision is the opposite:
no backward compatibility.
SparseConn,SparseConstrainedConn,BaseSparseConnand thesparse_backend=argument are removed and every caller moves toSparseConnection. Numerical behaviour (orientation, duplicate summing, Dale's law, grouped weights) is preserved and pinned by the contract tests; names andstate_dictkeys are not. - Package layout. The specification suggests
btorch/nn/connectionandbtorch/sparse_runtime. The repository has nobtorch.nn; modules live inbtorch.models. The new code is placed inbtorch/sparse/(numerical),btorch/sparse/runtime/(planner, caches, registered ops) andbtorch/models/connection/(connection semantics). - Name clash.
btorch.models.synapse.Synapseis an existing protocol for PSC modules. The new synapse description isbtorch.models.connection.Synapse. connection_conversion.pyis referenced by an earlier decision but does not exist on this branch; nothing here depends on it.
2. Architecture¶
Three layers, kept separate on purpose:
btorch.models.connection modelling semantics
Projection, ConnectionRule, Synapse, Weight
Connection: SparseConnection | StructuredConnection | ImplicitConnection | HybridConnection
btorch.sparse numerical objects
Sparse: COO | CSR | CSC LinearOperator: Structured | Implicit | Composite
btorch.sparse.runtime execution
Planner, RepresentationCache, BackendRegistry, KernelCache, registered ops
- A
ConnectionRulesays which pairs are connected. It is not a sparse array and does not choose a storage format. - A
Synapsesays what the edges do (weight, receptor, delay). These are orthogonal edge attributes, not matrix formats and not subclasses. Theplasticityfield is reserved: any value other thanNoneraisesNotImplementedError. - A
Sparseis a format-agnostic numerical array;COO,CSR,CSCare representations of it. Procedural operators areLinearOperators, notSparse. - The runtime picks representation, algorithm and backend independently. None of them appears in a model definition.
torch.compile(model) is the only compilation step a user performs. There is
no conn.compile() and no public execution-plan object.
3. Numerical API (btorch.sparse)¶
- Shape model
[*batch, *sparse, *dense]withbatch_shape,sparse_shape,dense_shape. Stored values are[*value_batch, nnz, *dense]. COOsupports any number of sparse dimensions;CSR/CSCexactly two. BSR / BSC are read and written through COO and are not native formats.tocsr()/tocoo()/tocsc()convert;as_csr()/as_coo()/as_csc()return the object only if it already has that format and raise otherwise.as_sparse(aliasasarray) is the single conversion entry point for btorch, PyTorch and SciPy sparse objects. Everything that accepts "a sparse matrix" calls it. Conversions never transpose and never densify; dense input is rejected.A @ xfollowstorch.matmul.matvec(A, x)appliesAalong the last axis ofx [..., N]. Both are standard orientation:A.shape == (M, N)maps length-Nto length-M. For a squareAand a square batch the two accept the same input and return different results. The function formsmatvec,matmulandrmatvecaccept linear operators as well as sparse arrays.from_torch(t, batch_dim=k)andas_sparse(t, batch_dim=k)read the firstksparse dimensions of a PyTorch COO tensor as batch coordinates (PyTorch COO has no notion of a batch). An uncoalesced COO tensor that does not require grad keeps its stored order and duplicates; one that requires grad is coalesced, because PyTorch exposes its values to autograd only then.- Representation changes that reorder or merge entries return an
EdgeMap. Every edge-aligned array (weights, group ids, receptors, delays) is moved with the same map: weights are summed over merged duplicates, categorical metadata must agree or the merge is refused.
Batches¶
Two kinds of batch exist and are not the same thing.
| Meaning | Where it lives | |
|---|---|---|
| Sample batch | independent inputs / trajectories of one network | leading dimensions of x |
| Network batch | different networks | batch_shape of the sparse array |
A network batch with a shared pattern stores the indices once and values
[G, nnz]. A batch of different patterns (sparse.stack) is a COO whose
first index row is the network id; members may have different nnz and
nothing is padded. A connection executes it as one block-diagonal operator.
The network dimensions of x are its leading dimensions and broadcast against
batch_shape: A [G, M, N] with x [G, B, N] gives [G, B, M], and
x.unsqueeze(0) shares B samples across all networks. An input without the
network dimensions is an error; networks are never combined with samples
implicitly.
4. Orientation¶
Two strings name the two orientations everywhere: "post_pre" (the operator)
and "pre_post" (the adjacency matrix of connectome tables).
| Orientation | Shape | Product | Objects |
|---|---|---|---|
"post_pre" |
(n_post, n_pre) |
y = A @ x |
btorch.sparse arrays, SparseConnection(A), from_adjacency(A, orientation="post_pre"), FromSparse(A, orientation="post_pre") |
"pre_post" |
(n_pre, n_post) |
y = x @ W |
connectome matrices, hetersynapse helpers, from_adjacency(W) and FromSparse(W) (their default), from_hetersynapse |
SparseConnection.from_adjacencyandFromSparseare the places where a transpose is expressed, and both default to"pre_post". The bare constructorSparseConnection(A)takes the operator.- Functions that take index tensors or populations take the source first:
from_edges(pre, post, ...),FromEdges(pre, post),find_edges(pre, post),Projection(pre, post, ...),candidate(pre, post).set_edges_(slots, *, pre, post)is keyword-only. - The canonical edge buffer
indicesholds[post, pre]per edge;conn.preandconn.postare the named accessors. - A connection remembers the orientation it was built from in
conn.orientation;conn.to_sparse(orientation=None)defaults to it and validates its argument. - For a square matrix a wrong orientation is a valid input. Nothing can detect it.
5. Connections¶
SparseConnection state:
- Persistent (in the
state_dict):indices [2, E],layout([n_post, n_pre, n_receptor, n_delay], followed by the batch shape of a batch of different patterns), optionalreceptor [E]anddelay [E], an optionalbias [out_features], and the weight module's state (weight.valueand, with Dale's law,weight.sign; orweight.base,weight.group,weight.scale).load_state_dictcompareslayoutwith the module, checks that the ids lie inside the populations, and refuses a checkpoint withindicesbut withoutlayout(written by the removed layers) with a migration message. These checks raise also withstrict=False. - Derived (non-persistent buffers of
conn.cache): destination-major CSR (crow,col,perm) and source-major CSR (t_crow,t_col,t_perm) of the same edges. They are rebuilt from the persistent buffers afterload_state_dictand after rewiring, into the existing tensors when the number of edges is unchanged. Nothing derived can outlive the state it was derived from.
Weights are modules returning one effective value per edge: EdgeWeight
(per edge), ConstantWeight, ConstrainedWeight (w[e] = base[e] *
scale[group[e]]). Dale's law is a constraint of the weight module, applied by
constrain_net under torch.no_grad(); its reference sign is persistent.
Weights are bound by identity, not by copy:
- A
Weightmodule passed inSynapse(weight=...)isconn.weightitself. It holds the parameters of one connection; binding it a second time raises. - An
nn.Parameteris adopted asconn.weight.value(same object) when the connection keeps the edge order, that is when the input entries are already sorted by target and then source and nothing is merged. Otherwise the values are copied into a new parameter and a warning is emitted. - A callable
f(n_edge)or atorch.distributions.Distributioncreates trainable per-edge weights once the edges are known. Synapse(weight=<number>, dale=True)raises: one fixed number cannot change sign.- Every weight module exposes
shape,dtypeanddeviceof its effective weights.edge_table()["weight"]is a detached snapshot.
bias=True creates a zero-initialised [out_features] bias; a tensor is
validated against that shape at construction.
Receptors and delays are per-edge attributes. The executed operator is the
expanded reference lowering (n_post * n_receptor, n_pre * n_delay), which is
the layout HeterSynapsePSC and SpikeHistory already use; the lowering is
derived state, the semantic edge list is the model. The hetersynapse helpers
are unchanged and SparseConnection.from_hetersynapse(matrix, synapse,
n_receptor=None, n_delay=1, receptor_type_index=None) decodes their expanded
matrices into edge attributes; the number of receptor channels is given
directly or as the receptor index table of the helper. from_edges validates
that the ids lie inside the populations.
Projection forwards n_delay, n_receptor, weight, edge_table() and
to_sparse() to the connection it built.
6. Spikes during training¶
The spike tensor produced by a neuron model stays an ordinary dense tensor
[..., N]. Its forward values are sparse, its surrogate gradient is not, so
the logical object and the autograd object are dense. Sparse execution is
derived from it inside the operator: spike_propagate finds the non-zero
entries (on the device with the Triton backend, on the host with the
reference backend), visits only the out-edges of active sources, and returns
the same dense gradient as the destination-driven product, including for
silent neurons. Genuinely sparse input (event data) can be passed as a Sparse
array to SparseConnection.propagate_events.
7. torch.compile boundary¶
SparseConnection.forward Python, traced by Dynamo
weight() ordinary tensor ops
torch.ops.btorch.csr_propagate one opaque node: raw buffers in, tensor out
kernel from BackendRegistry (ATen reference, Triton, ...)
The registered operators (btorch::csr_propagate, btorch::spike_propagate,
and the non-differentiable building blocks btorch::csr_matvec,
btorch::csr_edge_grad) have fake implementations and an explicit backward:
the value gradient is a sampled product over the edges, the input gradient a
destination-driven product over the source-major CSR. No dense M x N
intermediate exists in either direction. PyTorch sparse tensors never reach
the tracer, and forward contains no planning: the plan is fixed when the
connection is built or its hints change.
8. Dynamic topology¶
Structural plasticity is an update policy, not a format. A connection has a
fixed number of edge slots; set_edges_ changes which edge a slot represents
without changing any tensor shape or parameter identity, increments
topology_version, and rebuilds the derived layouts in place. Value updates
do not touch topology-derived state. HardDeepR is the provided policy: it
runs from an optimizer post-step hook, samples new positions uniformly among
unconnected pairs without building a dense mask, applies an explicit policy to
the optimizer state of rewired slots, never raises when a layer is full
(unplaced slots wait at zero weight) and has its own checkpoint state. It
accepts a SparseConnection or a Projection realised as one, and calls
candidate(pre, post) to restrict new positions.
set_edges_(slots, *, pre, post, receptor=None, delay=None) takes its index
arguments by keyword only.
Four counters describe what changed. None of them is read in forward, so
rewiring does not retrace a module compiled with the default mode.
| Counter | Changes when | Consumer |
|---|---|---|
topology_version |
set_edges_, load_state_dict |
derived layouts |
routing_version |
set_edges_ with receptors or delays, load_state_dict |
derived layouts |
value_version |
a weight tensor is written in place (optimizer step, constrain, checkpoint load); derived from the tensors' in-place version counters |
none yet |
capture_version |
a CUDA graph captured around the connection became invalid: set_edges_, load_state_dict, .to(), a set_hints that changes the plan, a reallocation of backend layouts. Not on weight updates. |
CudaGraphRunner |
CUDA graphs¶
A CUDA graph replays recorded kernel launches and does not run Python again, so the wiring, the execution path and the device buffers of a connection are frozen at capture time. Weights are read at replay time and may change.
conn.capture_versionchanges whenever a captured graph is no longer valid (table above).conn.capture_incompatibility()returns the reason why a connection cannot be captured at all, orNone; the one reason today is the host-side packing of"adaptive-push".btorch.models.cudagraphdefines the protocol for any submodule:capture_versions(module)andcapture_incompatibilities(module)collect the two over a module tree.-
RecurrentNN(cudagraph=True)reads the versions on every call and captures again when one changed, and refuses a module that reports an incompatibility. -
torch.compile(mode="reduce-overhead")uses CUDA graphs internally and has no such hook. It must not be combined with rewiring or with loading a different pattern, and a connection should be called once in eager mode before it is compiled in that mode.
9. Runtime backends¶
| Kernel | ATen (reference, all devices) | Triton (CUDA) |
|---|---|---|
csr_matvec (pull, also the transposed product of the input gradient) |
torch.sparse CSR product; gather / index_add for dtypes without one |
row-block tiles over (rows, samples), hub rows cut into segments, one writer per output: deterministic |
edge_grad (value gradient) |
chunked gather-multiply | one launch over entry blocks, no [B, E] temporary |
spike_push (packed events) |
expand active sources into out-edges, index_add |
thread per delivered edge with a binary search over the prefix sum of active degrees, float atomics |
spike_push_dense (device compaction) |
– | fixed grid of 64-source tiles, silent tiles exit after one load; long out-edge lists as static chunks; no host sync |
The Triton kernels reuse ideas from the earlier experiments listed in 1.9
(row-block and CSR-vector pull, nnz-balanced handling of hubs, tiled and
per-edge atomic push). Sort-based and hash-based push variants were not ported
because they never won in the measured regime. Push results depend on the
arrival order of float atomics (relative error up to about 4e-7); pull
results are bitwise reproducible.
The planner chooses one of three algorithms when a connection is built,
moved or given new hints. Without an expected_density hint, with a hint
above the measured crossover (2% on CUDA, 0.2% on CPU), and for a
shared-pattern batch of weights it plans "pull". With a hint at or below
the crossover it plans a source-driven algorithm, and which one depends on
the backend:
| Algorithm | Backend | Behaviour |
|---|---|---|
"pull" |
all | destination-driven product over every edge; deterministic |
"push" |
device-compacting (spike_push_dense, Triton on CUDA) |
taken for every call once planned, whatever the density of an individual input; one launch, no host synchronisation; float atomics, so not bitwise reproducible; can be captured in a CUDA graph |
"adaptive-push" |
reference (spike_push, ATen) |
packs the active sources on the host; each call falls back to pull when its input is denser than the limit; cannot be captured in a CUDA graph |
Two consequences of "push" differ from "pull". Under
torch.use_deterministic_algorithms(True) the planner plans "pull" instead
of "push" (the flag is read when the plan is made). And "push" never
visits the edges of a silent source, so a non-finite weight on such an edge
does not reach the output, whereas "pull" propagates it (inf * 0).
Hints validates its fields: expected_density must lie in [0, 1], and the
reserved expected_calls and expected_batch must be positive integers.
The optional torch_sparse backend remains selectable with
runtime.use_backend("torch_sparse") and nothing depends on it.
btorch.sparse.runtime.__all__ lists the registry, planner, cache and
operator names only; the kernel modules (ops, kernels_aten,
kernels_triton, kernels_triton_push) are internal. Importing
kernels_aten no longer installs process-wide warning filters.
Eager and compiled execution take different routes to the same kernels:
compiled code calls the registered operators, eager code a plain
autograd.Function, because the operator dispatch costs tens of microseconds
per call and dominates for networks of a few thousand neurons.
Measured behaviour¶
Full tables are in benchmarks/sparse_conn/RESULTS.md, together with the
recorded state of the GPU. The numbers below are from an RTX 5090 that had no
other compute process when the run was launched, synthetic graphs with
uniform in-degree, medians per call.
| Neurons (edges) | Batch | Forward (ms) | Forward + backward (ms) | Legacy native forward / forward + backward |
|---|---|---|---|---|
| 4,096 (0.4M) | 1 | 0.016 | 0.17 | 0.068 / 1.4 |
| 4,096 (0.4M) | 32 | 0.033 | 0.19-0.23 | 0.093 / 0.55 |
| 100,000 (10M) | 1 | 0.105 | 0.42 | 0.235 / cannot run |
| 100,000 (10M) | 32 | 0.28 | 1.03 | 3.04 / cannot run |
| 1,000,000 (50M) | 1 | 0.54 | 2.4 | 1.39 / cannot run |
| 1,000,000 (50M) | 32 | 2.1 | 9.8 | cannot run |
- A dense layer is still faster at 4,096 neurons (0.13-0.14 ms forward + backward).
- With a density hint at 1 % spikes, the push path brings the forward to 0.030 ms at 100,000 neurons and 0.031 ms at one million (batch 1), and to 0.10 ms and 1.29 ms at batch 32. It makes no useful difference at 4,096 neurons.
torch.compileof a single connection equals eager from 100,000 neurons up and is slower below (0.052 against 0.016 ms forward at 4,096 neurons): the registered-operator boundary costs more per call than the eager autograd node.- In a 200-step recurrent network of 4,096 neurons the connection is not the bottleneck: the new path, dense and legacy are within 0.23-0.29 ms per inference step. At 100,000 neurons and batch 32 a step takes 0.47 ms (inference) and 1.37 ms (training) against 3.26 ms and "cannot run".
- The comparison against the legacy
torch_sparsepath exists only from a shared GPU: at 100,000 neurons and batch 32, 1.06 ms and 0.76 GB against 17.4 ms and 5.1 GB for forward + backward.
10. Migration¶
| Removed | Replacement |
|---|---|
BaseSparseConn |
SparseConnection over btorch.sparse.Sparse |
SparseConn(conn, bias=b, enforce_dale=E) |
SparseConnection.from_adjacency(conn, Synapse(dale=E), bias=b); Dale's law is now off by default |
SparseConstrainedConn(conn, constraint, enforce_dale=E) |
SparseConnection.from_adjacency(conn, Synapse(weight=ConstrainedWeight(group=constraint, dale=E))) |
SparseConstrainedConn.from_hetersynapse, constraint_info, persist_initial_weight |
SparseConnection.from_hetersynapse(conn, synapse, n_receptor=, n_delay=, receptor_type_index=); base weights and groups are always persistent |
| SciPy-only constructor input | anything btorch.sparse.as_sparse accepts (btorch, PyTorch, SciPy) |
magnitude, initial_sign, initial_weight, _constraint_scatter_indices |
weight.value, weight.sign, weight.base, weight.group (weight.scale for group scales) |
get_sparse_matrix() |
to_sparse() after from_adjacency (the orientation of the input), or to_sparse("pre_post") explicitly |
get_group_info, set_group_magnitude, get_weights_by_group |
weight.group_info, weight.set_scale, weight.weights_by_group |
sparse_backend=, available_sparse_backends(), SparseBackend |
internal backend selection; btorch.sparse.runtime.use_backend(...) for experts |
make_hetersynapse_conn, make_hetersynapse_constraint, make_hetersynapse_constrained_conn, stack_hetersynapse, expand_conn_for_delays |
unchanged; their expanded matrices are one lowering, decoded by from_hetersynapse |
state_dict keys magnitude, indices |
indices, layout, weight.* (and receptor, delay); old checkpoints are refused with a migration message, also with strict=False |
Linear follows the standard PyTorch interface; its constrain() does not write through
.data.
Names that changed during the development of this subsystem (no aliases exist):
| Earlier | Now |
|---|---|
orientation="src_dst" / "dst_src" |
"pre_post" / "post_pre" |
FromSparse(A) defaulting to "post_pre" |
defaults to "pre_post", like from_adjacency |
conn.to_sparse() defaulting to the operator |
defaults to conn.orientation |
set_edges_(slots, post, pre) |
set_edges_(slots, *, pre, post, receptor=None, delay=None) |
candidate(post, pre) |
candidate(pre, post) |
weights passed in a Synapse were copied |
a Weight module is conn.weight; an nn.Parameter is adopted when the edge order is kept |
planner algorithm "adaptive-push" on every backend |
"push" on a device-compacting backend, "adaptive-push" on the reference backend |
11. Known limitations¶
- N-D sparse. COO carries N-D metadata and
sparse.einsumhandles one sparse operand against dense operands with a dense output. There is no CSF format, no sparse-sparse contraction, no sparse output and no ellipsis. See the einsum notes for what a Scorch/TACO-style compiler would add. - BSR / BSC are imported and exported through COO; they are not native formats and there is no block kernel.
- Different-pattern network batches are a reference fallback: stored as batched COO and executed as one block-diagonal operator. There is no ragged CSR, and they cannot be combined with a shared-pattern value batch, with receptor/delay routing, or with rewiring.
- Receptors and delays execute through the expanded reference lowering. There is no routed kernel, delay ring buffer or event queue.
- Source-driven propagation is opt-in through a density hint and is not
used for value-batched weights.
"push"(Triton on CUDA) is not bitwise deterministic, is used for every input once planned even when that input is dense, and does not propagate a non-finite weight of a silent source."adaptive-push"(CPU, and CUDA without Triton) packs spikes on the host, which synchronises and cannot be captured in a CUDA graph. - CUDA graphs. A graph captured around a connection is invalid after
set_edges_,load_state_dictwith different edges,.to()and aset_hintsthat changes the plan; replaying it computes with the old structure and raises nothing.conn.capture_versionreports these events andRecurrentNN(cudagraph=True)acts on it.torch.compile(mode= "reduce-overhead")has no such hook and must not be combined with rewiring or with loading a different pattern. Modules compiled with the default mode are unaffected. - Triton kernels are
float32only. For any other dtype they call the reference backend themselves, without a message, andexplain()still namestriton. - A shared-pattern batch on the reference backend is executed as a Python loop over the batch members.
A @ xandA.matvec(x)on aSparseuse the gather reference path ofbtorch.sparseand never a backend kernel. OnlySparseConnectionreaches the kernels.torch.functransforms (vmap,func.grad, forward-mode differentiation) are not supported through a connection.- Index memory. All index buffers are
int64and both CSR layouts are always built, which costs 52 bytes per edge on CPU in addition to the weights. - Import time. Importing
btorch.modelsalso importstorch._dynamothrough the registration of the custom operators, which takes roughly 2 s. - Kernel inputs are trusted. The registered operators check the shapes of what a connection passes them, not arbitrary hand-built buffers: an index permutation or column index outside its range is undefined behaviour on the Triton backend where the ATen reference raises.
- Sparse event input (
propagate_events) uses the ATen kernel directly: it is not behind a registered operator, has no CUDA kernel, and does not support network batches. - Double backward through the propagation operators is not supported and raises.
- Complex dtypes are rejected by the propagation operators.
A @ xinsidetorch.compilefails in PyTorch 2.11 because Dynamo does not dispatch@to user objects.A.matvec(x)andsparse.matmul(A, x)compile, as does every connection module.- Checkpoints require the same number of edge slots as the module they are loaded into. A stochastic rule with a different seed can therefore produce a module that refuses the checkpoint; build with the same seed or with a rule of fixed edge count.
- Soft Deep R is not implemented. Exact soft rewiring keeps a latent parameter for every candidate edge, which is dense memory; a pooled approximation would need a distinct name.
HardDeepRsupports only an unbatched, trainableEdgeWeight.ConstantWeight,ConstrainedWeight, fixed weights and network batches are refused.- Parameter adoption depends on the edge order. An
nn.Parameterpassed as weight stays the trained object only when the input entries are already sorted by target and then source. A CSR adjacency matrix and the edges of a random rule are not, so the common case is a copy with a warning. Synapse(plasticity=...)is reserved; any value other thanNoneraisesNotImplementedError.value_versionis derived from the in-place version counters of the weight tensors, so it changes on optimizer steps, but nothing consumes it yet.- Hints
expected_callsandexpected_batchare accepted and not yet used by the planner. - Distributed / offloaded execution is not implemented. Storage, the logical sparse object and connection semantics are separate layers so that placement can be added below the connection.
12. Test status¶
pytest tests/sparse tests/models tests/connectome tests/test_pipeline_e2e.py
passes with four expected failures, all the Dynamo @ limitation listed
above. What the new tests cover:
| Area | Files |
|---|---|
Conversion round trips (SciPy COO/CSR/CSC/BSR/LIL/DOK/DIA, torch COO/CSR/CSC/BSR/BSC), as_sparse, EdgeMap |
tests/sparse/test_conversion.py, test_as_sparse.py, test_edge_map.py |
Products against dense references, batches, gradients, closures under torch.compile |
tests/sparse/test_matmul.py, test_batch.py, test_grad.py |
einsum, linear operators |
tests/sparse/test_einsum.py, test_operator.py |
Registered operators (opcheck, gradcheck), caches, backends, eager/compiled parity |
tests/sparse/test_runtime_ops.py, test_runtime_propagate.py |
| Triton kernels against the ATen reference | tests/sparse/test_kernels_triton_pull.py, test_kernels_triton_push.py |
SparseConnection: construction, orientation, weights, Dale's law, gradients, fullgraph compile, checkpoints, batches, receptor/delay routing against the hetersynapse helpers |
tests/models/connection/test_sparse_connection*.py, test_connection_*.py |
Rules, Projection, HardDeepR |
tests/models/connection/test_rule.py, test_projection.py, test_rewire.py |
| Regression tests for both code reviews | tests/models/connection/test_review_regressions.py |
The Triton tests run only where CUDA and Triton are available; they pass on
an RTX 5090 and on a small local GPU with the current kernels. Tests that
need the optional torch_sparse package are skipped where it is missing.