ArborLM · open research
Trainable hierarchical memory for language models.
Can a model learn to navigate a very large store of trainable parameters while reading only a small, local working set for each token? Can a small student acquire and combine capabilities from several teacher models through that memory? ArborLM tests these questions with preregistered experiments, and reports the negative outcomes as clearly as the positive ones.
The questions
The target architecture is a list of requirements, and each one is a separate question with its own studies. Being clear about what is open matters more here than sounding finished.
Does external memory help prediction?
Measured against the same backbone without memory, against shuffled memory, and against a dense adapter with the same number of parameters.
Can addressing be learned reliably?
Whether the model finds the right slot as the number of stored items and the load per slot grow.
Does a hierarchy earn its keep?
Whether a tree of memory beats a flat memory on quality, work per token and storage locality, or only looks like it should.
Can memory live on an SSD?
Whether learned addressing gives access patterns that make storage-backed inference practical. Not evaluable yet.
Can a student learn from teachers?
Whether one model can acquire, retain and combine capabilities from several open teacher models. Untested.
Does it hold on a real model?
Everything so far uses small backbones trained from scratch. Attaching the memory to a pretrained model is still ahead.
Status: the up-to-date scorecard, with every registered outcome quoted from its record, is in the ArborLM README. This page does not restate results, so it cannot drift from them.
How the work is done
Preregistered
Each study fixes its design, budget, seeds and pass/fail criteria in writing before it runs. Labels are assigned by the criteria, not after the fact.
Negative results stay published
Failed hypotheses are recorded with the same care as successes, and later work never rewrites an earlier outcome.
Science and engineering kept apart
ArborLM holds the research record. The ArborML Lab builds the runtime, telemetry and memory explorer; passing engineering tests is never presented as a scientific result.
Independent review
Studies and code are audited by a reviewer other than the author, and the review is recorded, including when it was not performed.
Where the record lives
Everything is in the public repository, under the Apache-2.0 licence for code. Start with these:
Scorecard
Each requirement of the target architecture, its registered evidence and what would move it. Read the scorecard.
Research handoff
The origin, the precise problem, the first architecture, hypotheses and prior art. Read the handoff.
Experiment protocol
Baselines, metrics, controls and reproducibility rules every study follows. Read the protocol.
Building a model from this research happens separately, at arborml.ai. Engineering milestones there are never presented as research results here.
Get in touch
Researchers working on model memory, retrieval, sparse updates or distillation: review, critique and collaboration are welcome.
Or write directly to
You will get a reply from a person, usually within a few days.