# When Do Shared Training Trajectories Help a New Model?

A low-rank trajectory controller saves online work on parity models, fails zero-shot on nested majority, then recovers a smaller gain after multi-family calibration.

Author: yoyo
Publication date: 2026-10-18
Version: 1.0

## Takeaway

A trajectory denoiser cut accounted online work 8.55% on held-out parity and 2.02% after nested-majority meta-training, while its frozen parity-only transfer failed.

## The idea, in a minute

### The question

Can successful training histories predict safe updates for a new model, and does that prediction transfer across task families?

### What we found

The v4 parity controller passed its held-out gate; frozen zero-shot transfer worsened both compute and success. Adding nested-majority trajectories restored a modest gain.

### Why it matters

A learned optimizer shortcut may depend on what its example trajectories represent, and its preparation cost can dominate savings.

### Keep in mind

These are small aligned models sharing an exact initialization. Offline costs remain substantial and the outcomes do not show transfer to language models or unrelated weights.

## A concrete example: The nested-majority transfer gate

This recorded experiment is the final multi-family evaluation on 90 untouched nested-majority episodes, after a separate parity-only zero-shot test failed.

### Learn

Fit a low-rank update predictor on 60 parity and 60 nested-majority training trajectories from a shared initialization.

### Guard

At step 150, propose one update only when predicted remaining time is at least 90 steps; test validation loss and a metamorphic residual before accepting.

### Account

Charge observer, validation, and controller forward work as online example-equivalents; report the 120-trajectory offline corpus separately.

Multi-family representation changed a failed zero-shot transfer into a small measured online saving, at a high amortization threshold.

## Reading the main result

+2.57% — zero-shot transfer overhead

On 90 nested-majority tasks, the frozen parity controller added 2.57% accounted online work and lowered success by 1.11 percentage points; adding target-family trajectories later produced a 2.02% online reduction.

The multi-family model zoo cost about 3.07 million training-example equivalents, corresponding to roughly 6,405 comparable runs for online savings to offset it.

## What this suggests

- Shared-base trajectory structure can support useful updates on represented task families.
- A safety gate protects quality but cannot guarantee positive compute savings.
- Training diversity matters: adding target-family trajectories mattered more than simply increasing the controller’s confidence.

## What remains unknown

- Whether these gains hold for larger networks, unseen task families, or asynchronous/distributed training.
- How much the reported offline cost could be reduced through reuse of compatible preexisting checkpoints.

## Abstract

A training history can look predictive inside one task family and still point in the wrong direction elsewhere. This study follows a shared-base, low-rank trajectory denoiser from a frozen parity evaluation through a negative transfer test and a calibrated repair. On 90 held-out parity episodes, version 4 preserved 100% success and reduced fully accounted online work by 8.55%, with paired savings of 1,167.7 example-equivalents (95% interval 633.8–1,810.3). The parity-only controller then failed zero-shot on nested majority: compute rose 2.57%, and success fell from 63.3% to 62.2%. After adding nested-majority examples to meta-training, a frozen 90-episode evaluation cut online work 2.02% with unchanged 98.89% success. The results make family coverage and offline amortization central to judging a learned training shortcut.

## Research question

Can a low-rank model of shared-base training trajectories accelerate held-out parity runs, and what happens when that controller is transferred to a different task family?

## Experiment setup

The system trains small two-hidden-layer parity MLPs with 16 input bits, width 64, two output classes, AdamW learning rate 0.01, weight decay 1e-4, batch size 64, and a 400-step maximum horizon. Model-zoo trajectories share the exact initialization at seed 12345. A rank-32 SVD basis represents centered final parameter changes; ridge heads predict both remaining displacement and time. Candidate weight jumps are scaled, norm-clipped, and gated by validation loss and counterfactual geometry. Results charge observer, validation, and inference work to online example-equivalents, and report model-zoo training separately.

Frozen v4 used 60 successful parity trajectories (orders 2 and 3), a rank-16 time gate, and a single proposal at step 100 if at least 125 steps were predicted to remain. It evaluated 90 new episodes at offset 800. The nested-majority zero-shot gate reused this parity-only controller without retraining and evaluated 90 tasks at offset 1000. The final multi-family evaluation added 60 nested-majority trajectories to the 60 parity trajectories, calibrated separately, then froze a rank-64 displacement model and rank-32 time gate. It proposed at step 150 only when at least 90 steps remained, used 32 labeled validation examples plus 16 metamorphic probes as guards, required no validation-loss increase and at most 0.10 residual increase, and used episodes 3000–3089.

The publication date is 2026-10-18; experiments are documented in local reports from earlier dates. No trained model checkpoint files were located in the scoped source folders; result JSON files contain metrics and denoiser diagnostics, not deployable weights.

The drivers require Python, PyTorch, and NumPy. Example commands from the repository root are `python projects/geometric-meta-optimization/experiments/geometric_trajectory_denoiser.py --model-zoo-episodes 60 --evaluation-episodes-per-family 45 --evaluation-offset 800 --input-bits 16 --width 64 --base-seed 12345 --learning-rate 0.01 --train-size 2048 --validation-size 128 --horizon 400 --interval 50 --proposal-step 100 --minimum-predicted-remaining-steps 125 --rank 32 --gate-rank 16 --prediction-scale 0.40 --output /tmp/parity-v4.json` for the parity driver, and `python projects/geometric-meta-optimization/experiments/geometric_nested_majority_transfer.py --model-zoo-episodes 60 --majority-model-zoo-episodes 60 --evaluation-episodes 90 --evaluation-offset 3000 --input-bits 16 --width 64 --base-seed 12345 --learning-rate 0.01 --train-size 1024 --validation-size 128 --horizon 400 --interval 50 --proposal-step 150 --rank 64 --gate-rank 32 --minimum-predicted-remaining-steps 90 --prediction-scale 0.30 --maximum-validation-loss-ratio 1.0 --guard-validation-size 32 --guard-probe-size 16 --output /tmp/multifamily.json` for multi-family evaluation. The zero-shot report used the transfer driver’s parity-only configuration and episode offset 1000, as specified in the protocol. Commands rerun the experiments; none were run for this article.

## Results

On held-out parity, both baseline and v4 controller had 100% success at the 90% accuracy threshold. Mean test accuracy was 0.9999783 versus 0.9999837; the 95% paired bootstrap interval for the difference was 0 to 0.0000163. Mean optimizer steps fell from 185.0 to 166.7, while accounted online work fell from 13,658.3 to 12,490.6 (8.55%). Paired savings averaged 1,167.7, interval 633.8–1,810.3. The controller attempted 22 jumps and accepted 20. Its parity model zoo cost 1,536,000 training-example equivalents, implying about 1,315 online runs to amortize at the measured saving.

The frozen zero-shot nested-majority evaluation had success of 63.3% baseline versus 62.2% controller, mean accuracy 0.94414 versus 0.94372, and accounted work 19,047.8 versus 19,536.9 example-equivalents. The controller attempted 87.8% of jumps and accepted only 4.4%; savings were negative 489.1 examples per run (95% interval −745.8 to −88.3). Fewer optimizer steps did not offset the controller overhead.

After adding 60 nested-majority trajectories, the final 90 episodes had equal 98.89% success and mean accuracy 0.99747 baseline versus 0.99761 controller. Mean steps fell from 314.44 to 303.89; accounted work fell from 23,746.84 to 23,267.22 (2.02%). Paired savings were 479.62 examples per run (95% interval 173.83–807.20). Jump attempt/accept rates were 84.44%/42.22%. The 120-trajectory model zoo cost about 3.07 million training examples, requiring roughly 6,405 comparable runs to amortize.

## Discussion

The parity result is a useful gain for models sharing architecture and initialization. The zero-shot failure demonstrates that a guard can protect individual candidate updates without making the overall controller worthwhile: frequent probes and rejected proposals still cost compute, and parity-derived directions were mismatched to nested-majority trajectories.

The multi-family result supports a narrower transfer claim. Including examples of the target structure recovered a positive online saving, but the 2.02% gain is smaller than the parity-only gain and far from paying back the offline corpus over a small number of uses. Optimizer-state clearing after accepted updates and a post-jump training interval were part of the calibrated repair, not evidence that the original zero-shot controller worked.

## Limitations

These experiments use synthetic tasks, a small MLP, and exact shared-base initialization. Weight-space alignment is a major assumption; arbitrary independently initialized or downloaded networks can have functionally equivalent but differently arranged parameters.

Example-equivalents are an accounting proxy, not measured latency or energy. Amortization assumes future runs achieve the same saving and can reuse the compatible model zoo. The final nested-majority cohort passed its predefined rule, but the bootstrap interval and single frozen cohort do not guarantee broader transfer. No trained weights were found in the inspected source folders; diagnostic JSON records results rather than model checkpoints.

## Summary results

| Evaluation | Success (%) | Online work reduction (%) |
| --- | --- | --- |
| Parity v4 | 100 | 8.55 |
| Nested majority zero-shot | 62.22 | -2.57 |
| Nested majority multi-family | 98.89 | 2.02 |

Success is percent of episodes at or above 90% test accuracy. The last column is work reduction against the same-task baseline; positive values mean savings, negative values mean added compute.

## Available materials

The frozen parity v4, zero-shot failure, and final multi-family evidence are included with protocols and code dependencies. No model weight/checkpoint files were found in the scoped paths; trajectory corpus quantities and diagnostic JSON are reports/data, not downloadable trained checkpoints. Reproduction commands and dependency details are included in the study setup.

The evidence archive preserves selected reports and result records. Source paths and SHA-256 hashes are listed in provenance.json. This publication rechecked recorded outputs and archive integrity; no research experiment was rerun.

## References

- [Source reports and measured results](/downloads/trajectory-denoising-and-transfer/evidence.zip)