Articles

Five questions to ask any AI memory tool before you trust it with your work

Every memory product, including this one, will tell you it remembers well. Almost none will tell you how they checked. Here's what to actually ask before you hand one your work.

A magnifying glass inspecting a small connected node network

1. What happens when you change your mind?

Ask it to remember something, then tell it something that contradicts it a week later, then ask again. A tool that returns both facts with equal confidence, or worse, the stale one, hasn't been tested against the most common real case: people revise decisions constantly, and "latest" isn't the same as "most confidently phrased."

2. Can it tell two similar things apart?

If you run two projects with the same shape (two clients, two quarters, two versions of a plan), ask it about one specifically. A system that quietly blends similar-sounding memories will hand you an answer that's plausible and wrong, which is worse than no answer, because you won't think to double-check it.

3. Does deletion actually delete?

Ask it to forget something, then ask about it again a different way. "I've removed that" is a claim, not a result. The only way to know is to check whether the fact resurfaces under a different phrasing, a different session, or after the next sync.

4. What does it do with silence?

Ask about something it was never told. A tool under pressure to seem capable will sometimes generate a confident-sounding non-answer instead of saying it doesn't know. Noise robustness, meaning it refuses to invent, is a real, testable property, not a personality trait.

5. Where's the evidence?

Not a demo video. A methodology: what was tested, on what data, against what threshold, and whether the results are something you can reproduce or only take on faith. If a vendor won't show you the harness, that's itself an answer.

We hold ourselves to the same five questions. See the recall test for our own methodology and current status. We'd rather publish "not yet tested at scale" than a number we can't reproduce.