# ModelBrain public benchmark pre-registration

Written 2026-10-14, before any held-out test split has been scored by ModelBrain.
Status: DRAFT until the config hash below is filled in. After the hash is filled in, this file is append-only.

## 1. Goal

Match or beat the published retrieval numbers of totalmemory (TAM) on LoCoMo, LongMemEval and BEAM, using local models only and zero paid API spend.
Target for LoCoMo: 0.729 recall_any@5 (the number their own code reproduces on our hardware, not the 0.607 in their README).

## 2. Data and splits (fixed now)

- LoCoMo: conversations 0-2 are dev. Conversations 3-9 are test. Category 5 (adversarial) excluded from scoring.
- LongMemEval: the 100 questions with the smallest sha256(question_id) are dev. The rest are test. Abstention questions excluded.
- BEAM: fnv1a32(conv_id) % 100 < 40 is the test split. The rest is dev.
- Text formats copy the TAM runners at pinned commit 16d8c82.
- Test splits are not scored by ModelBrain until section 6.

## 3. Metrics

Primary: recall_any@5 on the product view (ranked_product) (the first 5 distinct source records in the order the shipping recall path returns them).
Secondary: recall_all@5, NDCG@5, MRR, p50 and p95 latency per query.
Intervals: unit-clustered bootstrap, 10,000 resamples, 95 percent. Deltas between configs use paired unit-clustered bootstrap.
BEAM is reported twice: raw, and with length normalisation only. No other BEAM adjustment.

## 4. Tuning rules (dev only)

Revised 2026-10-14 after an independent review of the first draft. The first draft had 17 configs and a mean-of-two-corpora rule; the review showed that LoCoMo dev has only 3 conversations, so its intervals cannot carry a decision, and that 17 uncorrected comparisons will always produce a winner.

- Metric: ranked_product recall_any@5 only. NDCG and MRR are not used for decisions (the engine re-sorts the returned page by vector distance, so they do not reflect fused order).
- Grid, fixed now: base, base2 (exact duplicate of base, a determinism control, its delta must be 0), dw6, dw7, dw8, lp10, dw7_lp10. After the reranker lands, one further family is added in the same commit as the reranker and is pre-declared in the table at the end of this section before it is run.
- Primary tuning corpus: BEAM 100K dev (13 conversations). LoCoMo dev (3 conversations) is a no-harm check only: a config is rejected if its LoCoMo dev delta vs base is below -0.01.
- Multiplicity: Holm correction across the comparisons vs base within a family. A config is a candidate only if its paired unit-clustered delta CI excludes zero after Holm.
- Confirmation: the single best candidate is then run once on held-out data never used in the grid: LongMemEval dev (100 units) and the BEAM units not used for tuning (BEAM 100K units are split by fnv1a32(conv_id) % 2, with even = tune, odd = confirm). It is adopted only if it does not lose on either.
- Implemented as compare.py --half tune (even) and --half confirm (odd) on BEAM dev units.
- If no candidate survives, the shipping config (base) is the frozen config and we say so.
- Every config tried is reported, not only the winner. Dev gains are described as optimistic.
- The date boost is not part of the product grid. Its parser matches only the dataset stamp formats and does nothing for real memories, so any result with it is a labelled benchmark-specific extra, never presented as a product improvement.
- The length penalty acts as a role proxy on BEAM (gold records are mostly short user turns). It is reported with that caveat.
- Comparison to TAM during tuning uses TAM recomputed on the dev units only, not the all-conversation numbers, so no test-split information steers the tuning.

## 5. Frozen configuration

Frozen before the test run and recorded here:

- engine commit: (fill in)
- embedder: nomic-embed-text-v1.5 Q8_0, sha256 (fill in)
- reranker: bge-reranker-base-q8_0.gguf, sha256 (fill in)
- RecallTuning values: (fill in)
- reranker settings (pool, neighbour chars, rank-fusion weight): (fill in)
- units files sha256: (fill in)
- CONFIG HASH: (sha256 of the lines above, fill in)

## 6. Test run

One run per benchmark with the frozen config. No reruns after seeing results, except for a crash that produced no scores, in which case the rerun is logged here with the reason.
Full-set numbers and held-out numbers are both published.

## 7. Predictions (written before the matrix runs)

- LoCoMo test recall_any@5 lands between 0.68 and 0.76. Reranker plus neighbour context is expected to close most of the single-hop gap (dev 0.645 vs TAM 0.776).
- BEAM 100K lands within 0.03 of 0.603 either way.
- LongMemEval stays at or above 0.95.
- Median latency stays under 100 ms without the reranker and under 1 s with it.
- Honest failure case: if LoCoMo test is below 0.729 we publish that and say we did not match it.

## 8. Publication

- Benchmarks-only page with this file, the result files and the scripts linked.
- Disclosure of what was tuned, on which split, and how many configs.
- 14 day window for the TAM maintainers to review before wider promotion.
- No claim of "beats" unless the paired CI of the delta excludes zero or the number is plainly higher and we say the interval overlaps.

## 4a. Reranker family declaration and amendment log

Added 2026-10-14 20:55 local, after the first reranker dev run and after an independent review. Disclosure: the section 4 promise to declare the reranker grid before running it was not kept for the first three configs. They were run first and are reported as such.

Already run (declared late): rr30_nb0, rr30_nb400, rr50_nb400. Results: LoCoMo dev 0.749 / 0.789 / 0.783 against base 0.713. BEAM 100K dev 0.581 / 0.590 / 0.594 against base 0.585, none significant.

Diagnosis from the review (counts on BEAM 100K dev): 69 percent of candidate pairs have a body over 512 tokens and are cut to their head, the neighbour-shrink loop fires on 72 percent so nb400 and nb0 are nearly the same on BEAM, and the retrieval rank fed to the fusion was the engine's distance-sorted page order rather than fused order (top 5 in that order hit 0.550 against 0.585 for the fused product view). The first two are defects in the runner, not corpus tricks.

Declared now, before running (new configs):
1. rr30_fo: rr 30, rw 2, nb 400, retrieval rank in fused order (fo 1).
2. rr30_fo_rw1: as 1 with reranker weight 1.
3. rr30_fo_win: as 1 with windowed scoring of over-long passages, max over windows (win 1).
4. rr30_fo_win_rw1: windowed scoring and weight 1.
5. rr50_fo_win_rw1: as 4 with pool 50.

Family: all 8 reranker configs form one Holm family against base. Selection uses BEAM 100K dev, tune half only (compare.py --half tune). The confirm half has already been seen by the first three configs, so it is no longer a clean holdout; LongMemEval dev (100 units) is the clean confirmation. Role filtering and length priors are excluded: BEAM gold is mostly user turns, so they would be corpus fits.

Expected outcome, written now: nothing passes Holm on BEAM tune half (about 123 queries, a real effect needs about 5 points). If so, the shipping config stays the frozen config and the reranker is described as an optional mode with its LoCoMo dev gain and BEAM null result.

Product-improvement claims: windowed scoring and fused-order input fix real defects and can be claimed as such whatever the numbers. Weight and pool size are fitted to these corpora and are not claimed as product improvements.

Notes: compare.py uses 2,000 bootstrap resamples (section 3 says 10,000; the code value is what was run, update section 3 to match unless the code changes). The reranker views are ranked_rr in the bench runner, not the shipping recall path, and are not called product view until the reranker ships inside recall.

## 4b. Outcome log: reranker family 2 (2026-10-15, dev only, append-only)

Full BEAM 100K dev (13 conversations), recall_any@5, base 0.585: rr30_nb0 0.581, rr30_nb400 0.590, rr50_nb400 0.594, rr30_fo 0.581, rr30_fo_rw1 0.576, rr30_fo_win 0.563, rr30_fo_win_rw1 0.572, rr50_fo_win_rw1 0.568. None significant (Holm p 1.0). The two defect fixes (fused-order input, windowed scoring) did not help; windowed scoring was the worst variant.

Tune half (7 conversations): rr30_nb400 0.626 (+0.041, Holm p 0.008) and rr50_nb400 0.634 (+0.049, Holm p 0.008) pass the Holm rule. Confirm half (6 conversations): the same two configs score 0.547 (-0.038, CI -0.085 to +0.010). The sign flips between halves. By the section 4 rule a candidate is adopted only if it does not lose on the confirm half, and these lose, so they are not adopted. The confirm half was already seen by the first three configs, so it is a weak check, and LongMemEval dev has not been run for any reranker config. The tune-half result is reported as a split artefact, not as evidence of a gain.

LoCoMo dev (3 conversations, no-harm check only): base 0.713, rr30_nb400 0.789, rr50_nb400 0.783, rr30_fo 0.762, rr30_fo_win 0.762, rr30_fo_rw1 0.736, rr50_fo_win_rw1 0.747. Gains are large and consistent but come from 3 conversations.

Latency: reranker time per query on the benchmark machine is p50 7.98 s (p95 8.93 s) for rr30_fo, p50 15.1 s (p95 21.8 s) for rr30_fo_win and p50 9.9 s for rr50_fo_win_rw1. The section 7 prediction of under 1 s with the reranker is false. The weight-1 variants show 0 ms because they reuse cached scores.

Status of rules: no reranker config qualifies for the default configuration. Frozen default stays at shipping behaviour unless amended below.

## 4c. Correction (2026-10-15)

The comparison target was mis-stated in earlier sections. TAM's reproduced LoCoMo 0.729 comes from its default pipeline, which includes a cross-encoder reranker (ms-marco-MiniLM-L-6-v2, 50 candidates, 400 chars of neighbour context, rank weight 2.0). The README figure of 0.607 and TAM's own pre-reranker result files (R@5 0.527, p50 9 ms) are the no-reranker tier. Comparisons are therefore made per tier: no-reranker vs no-reranker, and reranker vs reranker. Comparing ModelBrain without a reranker to TAM with one, as done in earlier notes, was wrong.

TAM recomputed on exactly our dev units (LoCoMo conversations 0-2, and the BEAM dev chats) is required before any dev-level claim against TAM. Until it is run, dev comparisons to the all-conversation 0.729 are indicative only.

The section 7 latency prediction (under 1 s with the reranker) is retained as a target. The measured 8 s came from the runner's implementation (CPU 2 threads, a 278M-parameter reranker, one inference context per pair), not from the approach.

## 4d. Overnight run declaration (2026-10-15)

Dev splits only; no test split is read. Declared before running.
- Reranker latency and accuracy re-measure with the batched, 12-thread scorer on LoCoMo dev and BEAM 100K dev. Configs: rr30_nb400, rr30_fo, rr50_nb400 (all already declared in 4a) and rr30_fo_nb200 (new: pool 30, fused-order input, 200 neighbour chars).
- LongMemEval dev (100 units), base vs rr30_nb400 and rr30_fo: the clean confirmation set for the reranker. Adoption still requires no loss here.
- BEAM 500K dev, first 5 units, base only: scouting for Phase 3, not a decision input.
- TAM on LoCoMo conversations 0-2 with its cross-encoder off (MEMORY_CROSS_RERANK=off): the no-reranker tier on our exact slice.
- Engine unit tests and clippy run once at the end.

## 4e. Small-reranker family declaration (2026-10-15, before any run)

New models, both downloaded with approval, Apache-2.0: jina-reranker-v1-turbo-en (F16 GGUF, sha256 71abc010...1fca) and ms-marco-MiniLM-L6-v2 (Q8_0 GGUF, sha256 abd99810...f7b7). Step 0 is a score-parity check against fastembed (parity_rerank.py); a model is eligible only if Spearman is at least 0.98 on the 30 probe pairs.

Configs, per eligible model, each with context 2048 for jina and 512 for MiniLM, 12 threads, fused-order input:
- rr20_nb400, rr30_nb400, rr30_nb200.
Datasets: LoCoMo dev, BEAM 100K dev, LongMemEval dev. The family size is 3 configs x 2 models = 6 per dataset, Holm within dataset.

Falsifier, written now: jina turbo rr30_nb400 below 0.742 on LoCoMo dev, or losing to base on BEAM dev, ends the jina path; MiniLM then gets the same test once.
Latency target: p50 at or below 0.9 s per query (TAM 0.92 s on the same slice) with the reranker on.
Run mechanics: run_overnight2.sh stops run_overnight.sh when it reaches its LongMemEval stage (LoCoMo and BEAM 100K with the big reranker are finished by then), runs the parity gate, then the small-model grid above on LoCoMo, BEAM 100K and LongMemEval dev. The big reranker is not run on LongMemEval; the shipped candidate is a small model. Holm is applied within each model and dataset (3 configs); the cross-model comparison is read as exploratory.

### 4f. Run 3 declaration: judge second stage (declared 2026-10-15, before any result)

Family 3. A relevance judge (Qwen3-4B-Instruct Q4_K_M, the model ModelBrain already loads, one forward pass per
candidate, score = logit(" A") - logit(" B") for "does the passage contain the information needed to answer the
question") is applied to the top 5 or 8 records after the small-reranker ordering, then rank-fused with that ordering
(1/(60+i) + jw/(60+judge_rank)). Configs: jv8_w1, jv8_w2, jv5_w1, each on top of rr30_nb400 (rw 2, nb 400, fo 1).
Datasets: LoCoMo dev and BEAM 100K dev, for each small reranker that passed the run 2 parity gate. No config is added
after seeing results. Same statistics as section 4 (paired, clustered bootstrap, Holm over the family).
Falsifier: no jv config beats rr30_nb400 of the same model on BEAM 100K dev (paired delta CI includes zero or is negative
on both halves), in which case the judge stage is dropped from the product plan. Latency target for any shipped
config: p50 total (rerank + judge) <= 0.9 s, otherwise the judge is offered only as an optional deeper mode.
Code lives in engine-r3 (a copy of the engine tree), so run 2 builds are unaffected. Run script: run_overnight3.sh.

### 4g. Corrections after independent review (2026-10-15, before any run 2 or run 3 result)

- Reranker threads in runs 2 and 3 are hw.ncpu (12 on the test machine), as 4e intended; 4e's "12 threads" is that value.
- Holm correction is applied per output file (per model and dataset), over the non-base configs in that file. The 4f
  comparison (jv configs against rr30_nb400 of the same model) uses compare.py --base rr30_nb400, with its own Holm
  over the three jv configs. The comparison against base is reported alongside it.
- Script fixes only, no change to configs or datasets: bash 3.2 compatible model tables, build guards, a parity time
  limit, and run 3 waiting strictly for run 2 to print ALL DONE.

### 4h. Run 4 declaration: bge pool sizes for latency (declared 2026-10-15, before any result)

Run 2's parity gate failed for both small GGUF models (Spearman 0.77 and 0.58): the GGUF conversions drop the BERT
pooler dense layer, so their scores are not the models' scores. The small-model grid and run 3 did not run. Family 4:
bge-reranker-base Q8_0 (the model that passed in family 1) at pool 10, 15 and 20 (rw 2, nb 400, no fused-order), LoCoMo
dev and BEAM 100K dev, one process per pool so the score cache cannot flatter latency. Target: p50 rerank <= 0.9 s with
LoCoMo dev recall_any@5 >= 0.742. Run script: run_bge_pool.sh.

### 4i. MiniLM with pooler restored (declared 2026-10-15, before any result)

The ms-marco-MiniLM-L6-v2 GGUF (sinjab) lacks BERT's pooler dense layer. A patched copy adds cls.weight [384,384] and
cls.bias [384] from the official cross-encoder/ms-marco-MiniLM-L6-v2 weights (sha256 821d1aa6...0aeb), and the pair
layout is the BERT single-separator template (RerankerConfig.single_sep). token_type_ids stay 0 for the passage (a
llama.cpp limitation). Gate unchanged: Spearman >= 0.98 against fastembed on the 30 probe pairs; if it misses, the
model is not used and the miss is reported. If it passes: pools 15, 20, 30 (rw 2, nb 400) on LoCoMo dev and BEAM 100K
dev, one process per pool. Pass line: LoCoMo dev recall_any@5 >= 0.742 with rerank p50 <= 0.9 s. Script: run_minilm_fix.sh.

### 4j. Amendment to 4i: segment-B emulation (declared 2026-10-15, after the first MiniLM parity attempt, before any accuracy result)

First attempt with the restored pooler scored Spearman 0.684 against fastembed (recorded in results/minilm_fix/parity.json
before this amendment). A NumPy reference forward pass (models/work/np_minilm.py) reproduces fastembed exactly with segment
ids, and reproduces the GGUF's scores to quantization noise with segment ids forced to 0 (Spearman between the two 0.677),
so the missing segment-B embedding is the sole remaining cause. llama.cpp always applies token type 0, so the GGUF now
carries a second copy of the word-embedding table equal to word + (type1 - type0), the vocab is extended to 61,044
entries, and passage tokens plus the closing separator are fed as id + 30522 (RerankerConfig.passage_id_offset). This is
mathematically identical to segment ids. The 0.98 gate is unchanged. No accuracy run happened before this amendment.

### 4k. CPU and GPU rows (declared 2026-10-15, before any GPU run)

Publication shows both. CPU (as shipped: embedder and reranker pinned to the CPU) is the headline and the only row compared
with TAM, which runs on CPU. GPU rows are labelled ModelBrain fast mode and carry the hardware and thread count. The GPU
switch is benchmark-only (--gpu rerank[,embed]); no shipped code path enables it. Gates: (1) CPU vs GPU reranker scores on the
30 probe pairs, Spearman >= 0.999 for every model used (gpu_parity.py); (2) GPU recall_any@5 within the CPU row's bootstrap
interval on the same dev units. Embedding on the GPU can change stored vectors slightly, so the embed-on-GPU row is
reported as an ingest-speed and recall measurement only. Rows: MiniLM-L6 (segment-fixed) and bge, pools 15 and 30, LoCoMo
and BEAM 100K dev, plus one embed+rerank GPU row (MiniLM pool 15). A GPU row that misses a gate is not published.
Script: run_gpu_compare.sh.

### 4l. Corrections after Opus review of 4i/4j/4k (2026-10-15)
- 4j correction: the 0.684 first-attempt record in results/minilm_fix/parity.json was overwritten when parity_rerank.py was repointed from the -pooler GGUF to the -seg GGUF under the same key. The number is restored by hand from the terminal log into results/minilm_fix/parity_attempt1.json. Parity results now go to per-attempt files.
- 4k gate 2 is defined as: for each GPU row, the paired per-query delta (GPU minus CPU) in recall_any@5 has a unit-clustered bootstrap CI containing 0. Score parity (Spearman >= 0.999 and max abs diff <= 0.05 on 30 probe pairs, copied to results/gpu/probe_pairs.jsonl) is gate 1.
- 4k label: the GPU rows are called "GPU, benchmark build, not shipped". There is no fast mode feature.
- CPU references for GPU rows: bge pool 30 CPU rows (run_bge_pool30.sh) are declared here and run before the GPU comparison. MiniLM BEAM 100K pools 15/20/30 CPU rows come from run_minilm_fix.sh. No GPU row is published without its CPU row.
- MiniLM no-harm line: MiniLM is adopted as a reranker only if its BEAM 100K dev delta vs base is not significantly negative. The decision corpus is BEAM, not LoCoMo alone.
- The 0.762 LoCoMo interval comes from 3 conversations and is reported as a CI, not a p-value.

### 4m. Clean-machine rerun of the latency rows (run 2026-10-15 14:50 to 17:06, results/clean1)
Cause found before this run: the ModelBrain daemon's file connector (root ~/Documents/ModelBrain, gitignore_aware false) re-embedded the repo, models, results and logs continuously, holding about two cores and contaminating every earlier latency row. Accuracy rows are deterministic and unchanged. The connector was paused for this run.
Conditions: script run_clean1.sh, guard requires measured CPU idle of at least 80 percent (declared here; the 90 percent first draft was never met by macOS background services) and daemon CPU of 5 percent or less before each row, 12 threads, display asleep, Brave and Telegram quit, a 15 s process sampler in results/clean1/watch.log. Load checks are in results/clean1/load.log.
Results (dev, recall_any@5, rerank p50 / p95 ms): MiniLM LoCoMo pool 15 0.762 109/246 (pass A) and 238/434 (pass B); pool 20 0.775 145/317 and 144/374; pool 30 0.789 200/228 and 386/510. MiniLM BEAM 100K pool 15 0.568 499/742; pool 20 0.568 423/1044; pool 30 0.572 757/1380. bge pool 30 LoCoMo 0.789 1059/1557; BEAM 0.590 3151/5561.
Status: recall_any@5 identical across passes and to the earlier contaminated runs. Latency repeat limit (max/min p50 at most 1.10, p95 at most 1.25) met only by MiniLM LoCoMo pool 20. Pool 15 and 30 pass B were about 2x slower than pass A; cause not yet identified. BEAM rows ran once. No latency row is publishable until three repeats meet the limit. Earlier latency figures (MiniLM 362/295/511 ms, bge 2735 ms) are withdrawn as contaminated.

### 4n. Missing rows: TAM on our BEAM and LongMemEval dev slices, MiniLM on LongMemEval (declared 2026-10-15, before any of these runs)

Rows added, all on dev splits only, all on a clean machine (CPU idle at least 80 percent, ModelBrain daemon at most 5 percent, display asleep):
- TAM (pinned commit) on the 13 BEAM 100K dev chats, cross-encoder on and off (`run_tam_slices.sh`, slice built by `tam_slice_beam.py`).
- TAM on the 100 LongMemEval dev questions, store mode with cross-encoder on and off, plus hybrid mode once as the algorithm-only figure (`tam_slice_lme.py`).
- MiniLM-L6 segment-fixed reranker on LongMemEval dev at pools 15, 20 and 30 (`run_lme_minilm.sh`).

Metric: recall_any@5. Same unit-clustered bootstrap and Holm rules as earlier sections.

Expected ranges (written before running, from the settle plan): TAM reranked on the 13 BEAM chats at 0.63 or higher would change the plan; TAM no-rerank on BEAM at 0.585 or higher would change the plan; MiniLM on LongMemEval below 0.96 at any pool means MiniLM is not the default reranker there. LongMemEval is expected to tie near ceiling.

Reporting rule: every row is reported whatever it shows. TAM latency from these runs is the only TAM latency used in any comparison. No ModelBrain calls are made while a timed row runs.

#### 4n addendum (2026-10-15, after independent review, still before any run)

- Slice identity: BEAM = the 13 conversation ids with split dev in units/beam100K.jsonl (unit beam100K_<conversation_id>); LME = the 100 question ids with split dev in units/lme.jsonl (unit lme_<question_id>, full id, no abstention). A scored row must show n 229 (BEAM, abstention and empty-gold probes excluded) and n 100 (LME) or it is void and re-run.
- TAM environment: pinned commit 16d8c82, fastembed and onnxruntime versions recorded in machine.txt, MEMORY_CROSS_RERANK forced to on or off (the default auto scores early queries without the reranker while the model loads, so earlier TAM rerank rows that used auto are treated as possibly understated and are re-run here), cross window 50, weight 2.0, context 400, BEAM top_k 10 scored at 5, LME limit 5. TAM threads are its defaults; ours are 12 explicit; both are as shipped. Ollama must not be running.
- Latency definitions: TAM = Recall.search wall clock per query, all queries. ModelBrain = ms plus rr_ms per query for reranker rows, ms alone for base rows. The first TAM query of an on run includes model load; it is kept in p50 and p95 and excluded from any mean.
- Repeats: these TAM rows are single runs and are labelled single run until repeated; ModelBrain rows follow the 4m repeat limit.
- Statistics: BEAM TAM rows are scored per question from the trace file written by tam_beam_trace.py (the pinned code is not edited), with our gold, k 5 and the same clustered bootstrap. LME TAM rows use the records in its JSON.
- Sanity rule: reranker on and off must differ in both recall_any@5 and p50, else the reranker did not run and the row is re-run.
- Hybrid mode is algorithm-only, never in the comparison table; if sentence-transformers is absent the row is reported as not run.
- Our BEAM reranker rows are the bench view ranked_rr, not Store::recall, until plan 4.4 lands; the table says so.
- MiniLM LME rows: configs rr15_nb400, rr20_nb400, rr30_nb400 (rr:p, rw:2, nb:400), model ms-marco-MiniLM-L6-v2-Q8_0-seg.gguf, ctx 512, single-sep, passage offset 30522, 12 threads, scored field ranked_rr, base from the same run. Pools may be capped by the roughly 50 records per LME unit.
- If TAM lands at or above the falsifier lines (BEAM reranked 0.63, BEAM no-rerank 0.585), the trace is joined to ours per question and by ability to find which questions TAM wins and why.

### 4o. Overnight queue (declared 2026-10-15 before the run; script run_overnight1.sh)

Purpose: finish the missing rows and the repeat protocol of 4m on a clean machine, existing release binary, dev splits only, nothing tuned from the results of this run.
Jobs in priority order: MiniLM on LongMemEval dev at pools 15/20/30 in one ingest pass (run_lme_minilm_joint.sh; supersedes run_lme_minilm.sh, which re-ingested per pool and was stopped after pool 15 reached 35 of 100 units); MiniLM LoCoMo pools 15/20/30 third repeat; MiniLM BEAM 100K pools 15/20/30 repeats 2 and 3; TAM LoCoMo conversations 0-2 with MEMORY_CROSS_RERANK forced on and off, three repeats (earlier TAM LoCoMo rows used the default auto mode and are superseded); TAM BEAM 13-chat and LongMemEval 100-question rows, repeats 2 and 3; MiniLM pool 20 LoCoMo thread sweep at 4, 6, 8, 10 threads; bge pool 30 LoCoMo and BEAM repeats 2 and 3.
Rules: a job that finds the machine busy for 20 minutes is skipped and retried in a second pass; a row recorded while the sampler (watch.log) shows a non-benchmark process above 25 percent CPU for the row's duration is flagged and re-run, not averaged. Latency for ModelBrain rows is total (recall plus rerank); for TAM it is Recall.search wall clock. Accuracy rows are deterministic and are reported once; only latency uses repeats. The 4m limits (max/min p50 <= 1.10, p95 <= 1.25) decide which latency rows are publishable. Every row is reported whatever it shows.

### 4p. Corrections and the re-run queue (declared 2026-10-16 before the re-run; script run_overnight4.sh, output results/overnight2/)

1. Flagged rows from the overnight1 run. The process sampler shows a non-benchmark process (Spotlight indexing, WindowServer) above 25 percent CPU for the whole row on: MiniLM LoCoMo repeat C (pools 15/20/30), MiniLM BEAM repeat B pools 15 and 20, and the LongMemEval joint run (display awake). Under 4o these are flagged: their accuracy is unaffected and stays reported, their latency is not used for any publishable figure. The clean-looking agreement between BEAM repeats B and C is therefore not accepted as the repeat protocol; repeats are re-taken under the post-row check below.
2. Shipped thread count. The product reranker is capped at 2 threads (src/reranker.rs RERANKER_MAX_THREADS = 2). Bench rows use 12 threads explicitly. The statement in 4n that both sides are "as shipped" is withdrawn for ModelBrain: the bench setting is a declared bench setting. The 8-thread row is informational and was chosen after seeing the sweep on dev (pool 20, one run); it is never swapped into a headline and needs a declared product default before it is called shipped.
3. TAM repeat 2 is results/settle/tam_slices_r2 (it was written to a misnamed folder by a script bug; all four rows are complete and verified n 229 and 100). TAM LoCoMo overnight1 outputs were lost to a filename collision and are re-run.
4. New rules in the queue: every job is verified (file counts, n, reranker on/off sanity) before it counts as done; a row is flagged and re-run when a non-benchmark process is at or above 25 percent CPU in 10 percent or more of its samples (at least 2); large logs go to a .noindex folder; ModelBrain latency is judged on total (recall plus rerank) as 4o says.

### 4q. Declaration before the BEAM 500K confirmation run (declared 2026-10-17 evening, before any ModelBrain or TAM score on BEAM 500K exists)

1. Exploration disclosed. On 2026-10-17, on dev splits only, these switches were tried and are reported whether or not they helped: sd, fz, sg, rl, v2, rr50, fs (event-ordering frame stripping), an offline spread rule for enumerating queries, and ub (a bonus added to the fused score of records tagged as user turns). Results are in ModelBrain memory and in claude/modelbrain-beat-tam-progress-log-2026-10-17.md. Only ub moved BEAM dev by more than one question: rr30_ssds 145 of 229, with ub 0.004 149, 0.008 153, 0.012 156 (gained 15, lost 4, sign-test p 0.019); LoCoMo dev unchanged (control: 0 top-5 differences). LongMemEval and LoCoMo records carry no role tags, so ub cannot act there.
2. Reversal for ub only. Section 4a excluded role priors as corpus fits. ub is a role prior, and its strength was chosen by replaying all 13 BEAM 100K dev chats, so those chats cannot confirm it. This amendment withdraws the exclusion for ub alone and replaces it with the confirmation rule in item 3. All other role or length priors stay excluded.
3. Confirmation rule. BEAM 500K dev is the confirmation set (21 chats, 377 scored questions, not tuned on; the first five dev units, beam500K_1, 3, 4, 5 and 9, were ingested once on 2026-10-15 and 2026-10-16 in a base-only scouting row, base recall_any@5 0.449 on ranked_product, and no setting was chosen from it). Configs: base, base_ss, rr30_ssds, rr30_ssds_ub12 (ub fixed at 0.012, no retuning, same reranker settings as rr30_ssds). ub is adopted as a declared Tier B candidate only if the paired, unit-clustered bootstrap 95 percent interval of recall_any@5 on ranked_rr (top 5 after reranking and slots, not the section 3 product view) for rr30_ssds_ub12 minus rr30_ssds on BEAM 500K dev excludes zero in the positive direction. Bootstrap: 2,000 resamples, seed 7, chats resampled with replacement, per-question mean of (hit with ub12 minus hit without) recomputed on each resample, 2.5 and 97.5 percentiles (adapter/compare.py, --base rr30_ssds --field ranked_rr --k 5). This is one pre-specified contrast with no Holm correction; the other rows (base, base_ss, rr30_ssds) are descriptive and any Holm column compare.py prints for them is ignored. Transfer to Tier B: Tier B is frozen at rr20_ssds, so an adopted ub is re-run at rr20 on BEAM 500K dev and LoCoMo dev before the freeze, or Tier B becomes rr30_ssds, whichever the freeze entry (4r) declares; ub is never added to a tier that has not been run with it on dev. Adoption additionally requires wiring ub into the engine as a real role field on records (set by the benchmark formatter from the [user] tag and by the memory tool from the caller) before the configuration is frozen; the runner-only text-prefix version never counts as a headline view (section 3). If the interval includes zero, ub stays a labelled, benchmark-specific extra, is excluded from product claims and from the Holm family, and is reported next to the fact that TAM ships an inactive user-turn boost of the same kind.
4. TAM on the same slice. TAM (pinned commit 16d8c82) is run with the cross-encoder on and off on exactly the same 21 BEAM 500K dev chats. A row whose scored question count is not 377 is void.
5. BEAM test composition. The locked BEAM test is the test split of BEAM 100K and the test split of BEAM 500K (section 2 split rule for both), pooled, with the unit-clustered bootstrap. BEAM 1M is not used. This is declared before any 500K test question has been scored by ModelBrain or TAM. The 500K test split has not been ingested, queried, scored or inspected in this work; its 14 test units exist in units/beam500K.jsonl as generated by the normalizer and have not been read.
6. Test-split exposure to date. Two exploratory scripts on 2026-10-17 read all 20 BEAM 100K units, including the 7 test chats, before filtering to dev, and printed aggregate counts (phrase-match shares for instruction_following and preference_following gold turns) and four sample gold turns; no tuning decision used them. The finish-plan session counted test-split totals and categories. The LongMemEval units file was regenerated to carry query dates; its records are byte-identical.
7. Not evidence. Offline replays and the 100K dev results above are not confirmation of anything; only item 3 can promote ub.
8. Run-quality rule. The sampler-based flagging rule of 4p item 4 (a non-benchmark process at or above 25 percent CPU in 10 percent or more of a row's samples, re-run) applies to latency rows only and is unchanged; the accuracy rows in this section are unaffected by machine load. Any replacement of that rule is declared in the freeze entry (4r) before the locked run.

### 4r. Engine role field (declared 2026-10-17 night, before any engine-bonus run)

1. The user-turn bonus is reimplemented inside the engine as RecallTuning.user_bonus, applied in the engine's rank fusion to memories carrying the exact tag role:user. The engine never reads a role from text; the benchmark ingest sets the tag from the [user] prefix. The engine default is off. The earlier runner-only ub (4q) is kept for reproducibility and is bench-only.
2. Step 1: values 0.004, 0.008 and 0.012 are run on BEAM 100K dev with base and standing-slot rows. The value X with the highest base-row recall_any@5 count on the product view wins; ties go to the smaller value. The script writes X before any 500K run starts.
3. Step 2: X alone is run on BEAM 500K dev as base, base_ss and rr20_ssds, each with and without the bonus. Adoption into a tier needs the paired, chat-clustered bootstrap interval (2,000 resamples, seed 7) of recall_any@5 on that tier's view to exclude zero in the positive direction. The cross-encoder tier uses the engine bonus (eub), never the runner bonus.
4. LoCoMo dev must show zero top-5 differences with the bonus on (records carry no role tag). LongMemEval records carry no role tag and are not re-run.
5. The default stays off unless this entry is amended, dated, before the locked test run. The cross-encoder is bench-only (section 3 defines the headline as the shipping recall path), so any cross-encoder row is a labelled, bench-measured extra.

## 4s. Shipped cross-encoder tier (declared 2026-10-18, before any score of it exists)

Amends 4r item 5 (the cross-encoder was bench-only). Reason: with the engine user-turn bonus the shipped path beats TAM on BEAM, but on LoCoMo dev the shipped base (0.713) is below TAM reranker-on (0.744), and the bonus applied before the cross-encoder lowers the bench row (0.594 against 0.663 when applied after).

1. Tier A is the daemon with no reranker file present. Tier B is the daemon with bge-reranker-base Q8_0 present. Both are measured on the section 3 product view through the same store recall call. Every result row records the reranker sha256, threads and whether the reranker was active, and the runner refuses a mismatch.
2. Fixed Tier B settings, no retuning: pool 20, reranker weight 2.0, neighbour characters 400, protect-1 off, user bonus 0.012. The bonus is added after the cross-encoder fusion and never before it.
3. Parity gate before any new score is read: on LoCoMo dev and three BEAM 100K dev chats, the engine path and the legacy runner path must give identical ranked lists (0 differences) with the bonus off, and the engine path must equal the existing ub12 rows with the bonus on.
4. Dev runs: LoCoMo dev, BEAM 100K dev, BEAM 500K dev, LongMemEval dev. Tier A numbers are reported as they are.
5. Adoption of Tier B as the bundled default needs all three: the paired chat-clustered bootstrap (2,000 resamples, seed 7) of B minus A on BEAM 500K dev excludes zero positively; the LoCoMo dev delta is at least -0.01; the LongMemEval dev delta is at least -0.01. Otherwise the model file is not bundled and Tier B is reported as a labelled extra.
6. Product thread count for the reranker is declared before latency is measured and the latency table states it.
7. TAM comparison: Tier A and Tier B are each compared with both TAM tiers on the same questions with paired intervals.

### 4s correction (2026-10-18, before any run of the engine reranker)

Two statements in 4s were wrong and are replaced. (a) The Tier B model is the MiniLM-L6 segment-fixed reranker used in every rr20 and rr30 row so far (ms-marco-MiniLM-L6-v2-Q8_0-seg.gguf, context 512, single separator, passage offset 30522), not bge-reranker-base. (b) The frozen rr20_ssds settings use protect-1 on (p1:1); item 2 of 4s says off, which is replaced by on. Pool 20, weight 2.0, neighbour characters 400 and the user bonus 0.012 after fusion stand.

Parity gate (item 3) is made precise because the legacy runner path scores the matched chunk with in-record context (ck:1) while the engine scores the whole memory with its neighbours: on LoCoMo dev the engine path and the legacy path (rr20_ssds and rr20_ssds_ub12) must have identical top-5 lists on at least 97 percent of questions and recall_any@5 within 0.01 of each other. If the gate fails the dev runs stop and the difference is diagnosed before any score is read as a result. Dev runs after a passed gate, accuracy only: BEAM 100K dev, BEAM 500K dev, LongMemEval dev, each with base_eub12 (Tier A with the bonus) and the engine reranker row rr20_ssds_eub12 (Tier B). The adoption rule in item 5 uses Tier B ranked_rr against Tier A ranked_product on BEAM 500K dev.

## 4t. Freeze declaration for the locked test run (declared 2026-10-18, before any test-split score of these configurations)

1. Adopted: the engine user-turn bonus 0.012 (4r, adopted on BEAM 100K and 500K dev, control passed) and the engine cross-encoder tier (4s, adopted: BEAM 500K dev B minus A +0.032 with interval above zero, LoCoMo dev +0.063, LongMemEval dev 0.000). No other change is carried into the test run. Nothing is retuned after this entry.
2. Two configurations are run on every test split and both are reported, each compared with both TAM tiers on the same questions: Tier A, `A_base_eub12` (shipped recall, no reranker file, bonus 0.012, no slots); Tier B, `B_rr20_ssds_eub12` (MiniLM-L6 segment-fixed reranker, pool 20, weight 2.0, 400 neighbour characters, retrieval first place protected, standing and date slots on, bonus 0.012 added after the cross-encoder fusion; scored on ranked_rr). Reranker threads 8 for accuracy rows. Latency is not measured in this run; it is measured separately under the 4o protocol and reported separately.
3. Test splits: LoCoMo conversations 3 to 9 (7 units), BEAM 100K 7 test chats, BEAM 500K 14 test chats, LongMemEval 370 test questions. TAM (pinned commit 16d8c82) is run once on the same splits with its cross-encoder forced on and forced off (`run_tam_test.sh`). ModelBrain is run once (`run_test_locked.sh`), which refuses a dirty tree.
4. Claims: "beats" a TAM tier only where the paired, chat-clustered bootstrap interval (2,000 resamples, seed 7) of recall_any@5 excludes zero in our favour; an interval that includes zero is reported as "no significant difference". No Holm correction across benchmarks; each benchmark and each tier pair is reported in full. Aggregate numbers are given next to every interval.
5. Disclosures that travel with every result: (a) the user-turn bonus is a role prior (the engine reads a `role:user` tag supplied by the caller; the benchmark formatter sets it from the dataset's user turns; TAM ships an inactive user-turn boost of the same kind); LongMemEval and LoCoMo records carry no role tags so it does nothing there. (b) The date slot fires only because the benchmark supplies the date each question is asked on; the daemon does not yet take a question date, so the date slot is not claimed for the shipped product. (c) Earlier peeks at test data are the aggregate figures disclosed in 4a to 4q (TAM's full-set LoCoMo reproduction among them); none was used to choose a setting. (d) Every configuration tried on dev, including the ones that lost, is listed on the publication page. (e) The bench rows run the engine reranker through `--engine-rr`, shown to match the legacy runner path on LoCoMo dev with zero differences in the top five.
6. Predictions, written now: LoCoMo test Tier B recall_any@5 between 0.74 and 0.80 (Tier A between 0.68 and 0.74); BEAM 100K test Tier B between 0.60 and 0.70; BEAM 500K test Tier B between 0.60 and 0.69; LongMemEval test both tiers at or above 0.95 and within 0.02 of TAM off. Probability that BEAM Tier B beats TAM reranker-on with an interval excluding zero on 100K test: about 55 percent; on 500K test: about 80 percent. Honest failure cases: if LoCoMo test Tier B falls below TAM reranker-on, we publish that.
7. Reruns: only for a crash that produced no scores, logged here with the reason. A row with the wrong row count is void and re-run, and the void is logged. Section 5 is filled with the commit hash, model hashes and units hashes taken from `results/locked/mb/LOCK.txt` before TAM or ModelBrain test scores are read.

## 4u. Amendments to the 4t freeze after independent review (declared 2026-10-18, before any test-split score and before any test row exists)

These amendments supersede the 4t text where they differ. Nothing here changes a tuned setting.

1. Three ModelBrain rows per test split, not two. The daemon builds its recall settings from `RecallTuning::default()` plus the reranker defaults, so standing and date slots are off in the shipped product. The row that represents the shipped reranker mode is therefore `B_rr20_eub12` (MiniLM-L6 segment-fixed reranker, pool 20, weight 2.0, 400 neighbour characters, retrieval first place protected, bonus 0.012 added after the cross-encoder fusion, slots off). It was not run on dev before this entry; it is declared now and run for the first time on the test splits, so it carries no dev claim. The 4t row `B_rr20_ssds_eub12` stays as a bench-only row labelled "with standing and date slots" because the +0.032 BEAM 500K dev gain over Tier A was measured with slots on and mixes the reranker and the slots. Tier A is `A_base_eub12` as in 4t. Expected row counts: units times 3.
2. Wording of 4t item 1. The bonus value 0.012 was chosen by the 4r rule on BEAM 100K dev and confirmed, not retuned, on BEAM 500K dev. 4r step 3 results for the shipped base path: BEAM 500K dev 0.549 to 0.631 (+0.082, 95 percent interval +0.053 to +0.114); BEAM 100K dev 0.585 to 0.655 (+0.070, interval +0.048 to +0.091); LoCoMo dev control 0 differences in 383. For the reranker tier on BEAM 500K dev the runner version of the bonus moved rr30_ssds from 0.5597 to 0.6631 (interval +0.071 to +0.135, 4q). Without the bonus the reranker tier on BEAM is 0.620 (100K dev) and 0.565 (500K dev), so the BEAM gains against TAM are driven by the role prior.
3. Disclosure of the role tag. 4q item 3 asked for the memory tool to set `role:user` from the caller. It does not yet: the tag is set only by the benchmark formatter and by the store code that reads it. In the shipped product the bonus is therefore inert until a caller passes the tag. The BEAM results are benchmark-formatted results, not a claim about default product behaviour for untagged callers.
4. Claims rule, replacing 4t item 4. Two contrasts per benchmark are the pre-named headline contrasts: `A_base_eub12` minus TAM reranker-off, and `B_rr20_eub12` minus TAM reranker-on. "Beats" is claimed only where the headline interval (2,000 resamples, seed 7, chats or conversations resampled with replacement, LongMemEval questions resampled) excludes zero in our favour. Four benchmarks times 3 rows times 2 TAM arms print 24 intervals; with no multiple-comparison correction, every interval that is not a named headline is descriptive and is labelled that way on the page. A headline interval that includes zero is reported as "no significant difference".
5. Section 3 note. Section 3 still says 10,000 resamples and the ranked_product view. The code value, 2,000 resamples, is what was run and is final. Reranker rows are scored on `ranked_rr` (top 5 after reranking), not the product view, until the reranker is the default recall path. Rows are paired to TAM per question by `adapter/score_pair.py`, which refuses to score if any arm is missing a question.
6. Product inconsistency, disclosed. With a reranker loaded the bonus is deferred to the cross-encoder fusion only for pages that will be reranked (unfiltered, non-collapsed recall). Filtered and collapsed recall take the bonus in the plain fused path, as Tier A does. If the reranker fails on a call, that call falls back to the plain page without the bonus, is counted in `rerank_errors`, and any test row with a nonzero `rerank_errors` is void under 7.
7. Threads, replacing the open part of 4s item 6. Accuracy rows use 8 reranker threads. The daemon uses min(cores/2, 4) threads (bypassing the library default cap of 2). Dev p50 was about 130 ms at 8 threads; the shipped latency is measured separately under the 4o protocol at the daemon's thread count and reported as such.
8. Other: `DEFAULT_USER_BONUS` 0.012 applies to every store opened by the engine library, including other bench binaries, and is harmless on stores without role tags. An unparsable `MODELBRAIN_USER_BONUS` falls back to 0.012; "0" disables it. The run scripts unset both environment variables.
9. Guards added to the run scripts before any test row: only `modelbrain-engine` is a git repository (`bench-public` is not under version control), so `run_test_locked.sh` refuses an engine tree with tracked or untracked changes (build output, `.bak` and `_to_delete` excluded) and records in LOCK.txt the engine commit plus the sha256 of the run scripts, the scorer, the TAM trace scripts and the slice builder, the units files, the reranker and each embedder file, and the binary. It expects units times 3 rows, voids and moves aside a stage whose rows show `rerank_errors` above zero or whose B rows did not have a reranker active in the store (recorded from the store, not from the config), and a voided stage is never accepted on resume. `run_tam_test.sh` also voids and moves aside LoCoMo runs that do not cover 7 conversations and LongMemEval runs that do not hold 370 records, and BEAM runs with the wrong chat count. The scorer was validated on dev rows against the recorded dev numbers (BEAM 100K dev TAM on 0.5895, off 0.5546; LongMemEval dev TAM on 0.970, off 0.990) and, for LoCoMo, by a dev TAM trace run (`validate_locomo_scorer.sh`) that must reproduce TAM on 0.744 and off 0.619 before any test row is scored.

### 4u commit record (2026-10-18)
Frozen engine commit: e945b16c3a1d78e167dd96e95bc0141fbc952d04 (modelbrain-engine, committed before any test row exists). Section 5 hashes for models, units, scripts and the binary come from `results/locked/mb/LOCK.txt`, written by `run_test_locked.sh` at its start.

### 4u incident log (2026-10-18, locked run)
The LoCoMo stage was voided by `run_test_locked.sh` because its check read `rerank_active_installed`, a field written after the recall tuning was reset, so it said false although every Tier B query carried a reranked top five and `rerank_errors` was 0. This was a harness fault found by structure only; no score was read. The check now requires `ranked_rr` on every query of every B row, none on A rows, and `rerank_errors` equal to 0. The original rows were restored unchanged rather than rerun, and the new script hash is appended to LOCK.txt. This is the only deviation so far.

### 4u results record (2026-10-19, scored after all rows existed; full output in results/locked/SCORES.txt)
Headline contrasts (recall_any@5, paired, 95 percent interval): LoCoMo A minus TAM off +0.059 [+0.033, +0.089], B_rr20_eub12 minus TAM on +0.014 [-0.009, +0.036] (no significant difference). BEAM 100K A minus off +0.135 [+0.040, +0.230], B minus on +0.079 [+0.032, +0.119]. BEAM 500K A minus off +0.151 [+0.111, +0.187], B minus on +0.079 [+0.052, +0.107]. LongMemEval A minus off +0.014 [-0.008, +0.035] (tie), B minus on +0.030 [+0.014, +0.049]. Descriptive: LoCoMo A minus TAM on -0.089 [-0.106, -0.066], a loss for the base tier against the reranked tier. Predictions in 4t item 6 missed low for LoCoMo Tier A (0.665 against 0.68 to 0.74) and BEAM 500K Tier B (0.587 against 0.60 to 0.69); all others fell in range.

### 4v. Latency rows for the locked test splits (declared 2026-10-19, before any repeat run; script run_latency.sh)
Already on file before this entry, single run, 8 reranker threads, overnight on a quiet machine, read from the locked rows (not yet a published claim): ModelBrain per-query p50 and p95 in ms, base tier (recall) LoCoMo 14.9 and 18.4, BEAM 100K 19.4 and 29.0, BEAM 500K 21.3 and 30.6, LongMemEval 16.6 and 29.0; reranker mode (recall plus rerank) LoCoMo 125.6 and 147.0, BEAM 100K 367.6 and 423.7, BEAM 500K 354.2 and 424.0, LongMemEval 547.8 and 583.5. TAM from the locked run, one run: reranker on LoCoMo p50 956 and p95 1,084, BEAM 100K 3,126 and 3,181, LongMemEval 3,011 and 3,199; reranker off BEAM 500K 39 and 110.
Protocol for the repeat run: the shipped thread count (the daemon uses min(cores/2, 4) = 4 on this 12-core Mac), configs `A_base_eub12` and `B_rr20_eub12`, LoCoMo test and BEAM 100K test three times and BEAM 500K test once; TAM reranker on and off on LoCoMo test and BEAM 100K test, repeats 2 and 3 added to the locked repeat 1. A row is taken only while the machine is at least 80 percent idle at its start; the 4m limits (max/min p50 at most 1.10, p95 at most 1.25 across repeats) decide which rows are called repeatable, and every row is reported whatever it shows. Latency definitions: ModelBrain base = the wide pool recall time in the runner (an upper bound for the shipped depth-5 call), reranker mode = the engine recall with the reranker at the product depth; TAM = Recall.search wall clock per query. LongMemEval and BEAM 500K for TAM are single-run figures and are labelled so. The top-five lists of the repeat rows must equal the locked rows for the same questions, or the differences are counted and reported. No claim of a speed advantage is made from rows that fail the repeat limits.

### 4w. Corrections and sourcing after the page review (2026-10-19, no new run)
1. TAM's user-turn boost. 4q item 3 and 4t item 5(a) say TAM ships an inactive user-turn boost. At the pinned commit 16d8c82, `MEMORY_RECALL_USER_TURN_BOOST` defaults to 1.3 (src/config.py `DEFAULT_USER_TURN_BOOST`, docs/configuration.md) and multiplies the fused score of user-turn records for advice-shaped queries only ("recommend", "how should I"). It is on by default and fires on few benchmark questions. The earlier wording was wrong and is superseded.
2. Dev figures behind the bonus table on the benchmarks page, all ranked_rr or ranked_product as noted, engine reranker rows (`engrr`): BEAM 100K dev (229 questions): reranker with slots 0.607 without the bonus, 0.659 with; base tier 0.585 to 0.655. BEAM 500K dev (377 questions): reranker with slots 0.565 without, 0.663 with; base tier 0.549 to 0.631. LoCoMo dev: base tier 0.713 with and without (the data carries no role tags). The 0.5597 to 0.6631 figure in 4u item 2 is the earlier runner-only rr30_ssds row and is a different configuration.
3. LoCoMo dev TAM reranker-off is 0.6214 under our normalized gold (238 of 383) and 0.6188 in TAM's own scorer (237 of 383); the one question differs because its evidence is stored as the single string `D8:6; D9:17`.
4. The embedder is nomic-embed-text-v1.5 Q8_0 (section 5). The embedder line in LOCK.txt lists the folder of local reasoning models, not the embedder, and is wrong; the embedder is the bundled model file at /Applications/ModelBrain.app/Contents/Resources/model.gguf (MODELBRAIN_MODEL_PATH was unset, so the runner used its default path), sha256 3e24342164b3d94991ba9692fdc0dd08e3fd7362e0aacc396a9a5c54a544c3b7, hashed on 2026-10-19 after the run.

### 4v results (2026-10-19, run_latency.sh, report in results/latency/REPORT.txt)
All repeated rows met the 4m limits. ModelBrain at 4 threads, p50/p95 ms over three repeats: LoCoMo base 14.9/18.4, reranker mode 174/204; BEAM 100K base 19.4/29.0, reranker mode about 552/645; BEAM 500K (one run) base 21.5/31.1, reranker mode 537/645. TAM p50/p95 over three runs: LoCoMo reranker on 955/1,084, off 29.9/44.8; BEAM 100K on 3,126/3,206, off 32.5/81.4; single runs: BEAM 500K on 3,133/3,209, off 39/110; LongMemEval on 3,011/3,199, off 37/77. Top-five lists matched the locked rows on all 8,312 repeat rows. The load sampler recorded an idle fraction of at least 80 percent at the start of every row.