ArborLM

ArborLM · open research

Trainable hierarchical memory for language models.

Can a model learn to navigate a very large store of trainable parameters while reading only a small, local working set for each token? Can a small student acquire and combine capabilities from several teacher models through that memory? ArborLM tests these questions with preregistered experiments, and reports the negative outcomes as clearly as the positive ones.

The questions

The target architecture is a list of requirements, and each one is a separate question with its own studies. Being clear about what is open matters more here than sounding finished.

Does external memory help prediction?

Measured against the same backbone without memory, against shuffled memory, and against a dense adapter with the same number of parameters.

Can addressing be learned reliably?

Whether the model finds the right slot as the number of stored items and the load per slot grow.

Does a hierarchy earn its keep?

Whether a tree of memory beats a flat memory on quality, work per token and storage locality, or only looks like it should.

Can memory live on an SSD?

Whether learned addressing gives access patterns that make storage-backed inference practical. Not evaluable yet.

Can a student learn from teachers?

Whether one model can acquire, retain and combine capabilities from several open teacher models. Untested.

Does it hold on a real model?

Everything so far uses small backbones trained from scratch. Attaching the memory to a pretrained model is still ahead.

Status: the up-to-date scorecard, with every registered outcome quoted from its record, is in the ArborLM README. This page does not restate results, so it cannot drift from them.

How the work is done

  1. Preregistered

    Each study fixes its design, budget, seeds and pass/fail criteria in writing before it runs. Labels are assigned by the criteria, not after the fact.

  2. Negative results stay published

    Failed hypotheses are recorded with the same care as successes, and later work never rewrites an earlier outcome.

  3. Science and engineering kept apart

    ArborLM holds the research record. The ArborML Lab builds the runtime, telemetry and memory explorer; passing engineering tests is never presented as a scientific result.

  4. Independent review

    Studies and code are audited by a reviewer other than the author, and the review is recorded, including when it was not performed.

Where the record lives

Everything is in the public repository, under the Apache-2.0 licence for code. Start with these:

Status

Scorecard

Each requirement of the target architecture, its registered evidence and what would move it. Read the scorecard.

Context

Research handoff

The origin, the precise problem, the first architecture, hypotheses and prior art. Read the handoff.

Method

Experiment protocol

Baselines, metrics, controls and reproducibility rules every study follows. Read the protocol.

History

Research log

The chronology of decisions and outcomes, negative ones included. Read the log.

Building a model from this research happens separately, at arborml.ai. Engineering milestones there are never presented as research results here.

Get in touch

Researchers working on model memory, retrieval, sparse updates or distillation: review, critique and collaboration are welcome.

Or write directly to

hello@arborlm.ai

You will get a reply from a person, usually within a few days.

Your message is emailed to the project owner and used only to reply to you.