Contiguous Layer-Range Fragmentation and Reassembly in SmolLM2-135M
The updated experiment keeps the cleaned local pipeline and current results, but restores the earlier visual presentation: a figure-led paper shell, sidebar summaries, and the darker atmospheric layout.
The repository now tests a narrower and more defensible variant of the original idea. Each fragment is an active local Python program that stages a contiguous range of model layers into a sandbox directory. Fragment 0 also carries the shared non-layer weights required to instantiate a valid checkpoint. After each fragment arrives, the system rebuilds the longest contiguous prefix of the transformer stack and records the resulting output. This turns “fragmentation” into a measurable reconstruction and degradation experiment rather than a byte-level novelty demo.
The implemented path remains local-only. Fragments stage payloads only into artifacts/quorum/, reconstruction happens inside artifacts/reassembled/, and the experiment measures deterministic model behavior rather than propagation or persistence.
1. Introduction
The repository originally mixed two different prototypes: a byte-wrapper path and a layer-based reconstruction path. The revised implementation keeps the experimentally useful part: layer-aware fragmentation. That choice matters because a quality-degradation claim only makes sense when a checkpoint corresponds to an ordered prefix of the transformer stack.
Partial reassembly only maps cleanly to quality degradation when fragments represent contiguous transformer layer ranges. Arbitrary byte chunks or random layer subsets can be reassembled mechanically, but they do not produce a defensible degradation curve.
2. Methodology
- Model:
HuggingFaceTB/SmolLM2-135M - Prompt:
The capital of France is - Generation mode: deterministic
do_sample=Falsewith24max new tokens - Fragment strategy: 4 contiguous layer ranges across 30 decoder layers
- Reassembly rule: rebuild the longest contiguous prefix present after each staged agent payload
2.1 Fragment Plan
| Index | Name | Layer Range | Layers | Includes Base |
|---|---|---|---|---|
| 0 | fragment_00 | 0-7 | 8 | Yes |
| 1 | fragment_01 | 8-15 | 8 | No |
| 2 | fragment_02 | 16-22 | 7 | No |
| 3 | fragment_03 | 23-29 | 7 | No |
3. Experimental Results
The latest run produced a full deterministic match at the final checkpoint while partial checkpoints showed strong collapse behavior before the late layers were restored.
3.1 Baseline
| Metric | Recorded Result |
|---|---|
| Baseline generation time | 3.23 seconds |
| Baseline generated tokens | 24 |
| Baseline completion | the capital of the country. |
| Final checkpoint matches baseline | Yes |
3.2 Checkpoint Outputs
| Checkpoint | Active Layers | Layer Ranges | Seconds | Label | Observed Completion |
|---|---|---|---|---|---|
| 1/4 | 8 | 0-7 | 0.87 | Repetition loop | , and and, and, and, and, and, and, and, and, and, and, |
| 2/4 | 16 | 0-7, 8-15 | 1.59 | Whitespace collapse | [whitespace only] |
| 3/4 | 23 | 0-7, 8-15, 16-22 | 1.26 | Repetition loop | . |
| 4/4 | 30 | 0-7, 8-15, 16-22, 23-29 | 1.46 | Coherent | the capital of the country. |
4. Discussion & Conclusion
The strongest result here is structural. Full reconstruction now has a precise criterion: the last snapshot must both load successfully and reproduce the deterministic baseline output. That happened in the latest run.
The partial checkpoints are also more interpretable than before. At 8 layers the model fell into a comma-heavy repetition loop, at 16 layers it collapsed to whitespace, and at 23 layers it still failed to regain coherent completion. Only the full 30-layer reconstruction recovered the baseline behavior.
- Compare 2-fragment, 4-fragment, and 6-fragment splits while keeping contiguous layer ranges.
- Add repeated trials per checkpoint to estimate variance in generation time.
- Add a lightweight text-quality metric beside the current human-readable labels.
5. References
[1] Hugging Face model identifier: HuggingFaceTB/SmolLM2-135M
[2] Primary experiment artifact: artifacts/results/latest_experiment.json
[3] Reassembled checkpoint snapshots: artifacts/reassembled/