Technical Reports 1 and 2 found that unbounded ledger-based replay outperforms bounded or absent replay on task-incremental continual learning (binary decisions evaluated one task at a time). This report tests a harder, more realistic setting — class-incremental learning, where a single shared classifier must correctly categorize among all classes introduced so far, not just the current task. Using the same real image dataset as Technical Report 2, extended to a 10-class digit classification problem introduced two classes at a time across five sequential tasks, we find that none of the three replay conditions — including unbounded ledger replay — prevents severe accuracy collapse (all three conditions fall to 9.4–19.4% final accuracy, only marginally above the 10% chance level for ten classes). We report this negative result in full, discuss the likely mechanism, and identify it as an open problem for the ledger-replay architecture rather than obscuring it.
Task-incremental learning, evaluated in our earlier reports, is the easier of the standard continual learning scenarios: the model is told which task it is being evaluated on, and only needs to distinguish between a small number of classes within that task. Class-incremental learning is widely considered the hardest scenario in the field, because the model must distinguish among all classes learned so far using a single shared output layer, with no information at evaluation time about which task a given example belongs to. A method that only succeeds in the task-incremental setting has narrower practical value than one that also succeeds here, so this test is a necessary honesty check on our earlier results.
We used the same 8×8 grayscale digit image dataset as Technical Report 2, this time using all ten digit classes. Five sequential tasks each introduced two new classes: (0,1), (2,3), (4,5), (6,7), (8,9). The network was extended to a 10-unit softmax output layer trained with cross-entropy loss, evaluated after each task on a cumulative test set containing all classes introduced so far — the standard protocol for measuring class-incremental forgetting. The same three replay conditions were tested, with replay again drawing 3 examples per training step from either no memory, a 50-example bounded buffer, or the full unbounded history.
An initial implementation of the softmax output layer exhibited numerical overflow (unbounded logits causing floating-point overflow in the exponential function), which produced a degenerate all-conditions-near-chance result on the first run. We identified this as a numerical stability bug — not a finding about the architecture — and corrected it with logit clipping and gradient clipping before the run reported below. We flag this explicitly because without additional mechanisms such as output masking, class-balanced replay sampling, or per-class output isolation — is known to be insufficient in this setting. Our result is a specific, measured instance of that general difficulty as it applies to the ledger-replay architecture, not a novel discovery of the difficulty itself.
| Condition | Final 10‑class accuracy | Chance level |
|---|---|---|
| No replay | 9.4% | 10% |
| Bounded RAM (n=50) | 12.8% | |
| Unbounded ledger | 19.4% |
Table 1 — Final accuracy on the 10-class cumulative test set after sequential training.
The failure mode is consistent with a known theoretical result in continual learning: replay-based methods that sample uniformly from memory without class-balancing or task-specific output heads suffer from severe bias toward recently seen classes. As the number of classes grows, the shared classifier's decision boundary is increasingly dominated by the most recent classes, and earlier classes are effectively forgotten — even when replayed.
Unbounded ledger replay retains the data, but the naive sampling strategy (3 examples per step, uniformly at random) does not ensure that the model receives a balanced signal across all classes. This is a structural limitation of the replay mechanism itself, not a failure of memory capacity.
This result narrows the honest scope of applicability for the current implementation. The architecture, as currently implemented, is well suited to binary or few-way, task-segmented decisions (for example: is this transaction fraudulent, should this loan application be flagged) evaluated one decision type at a time. It is not, in its current form, suited to a single model performing open-ended multi-category classification across many learned categories simultaneously. We consider this an important scope boundary to state plainly rather than discover in front of a customer or reviewer.
Unbounded ledger replay does not solve class-incremental learning in its current, naive implementation. We report this as a genuine limitation rather than omit it, consistent with our commitment to reporting what experiments actually show. The architecture's demonstrated strength remains task-incremental, binary or few-way decision settings — which happens to describe the primary target application (fraud and compliance decisions) accurately.