Case study / AI infrastructure
Cognitive Mesh
An independently built AI memory system focused on persistent state, typed relationships, retrieval, token accounting, governance, and the failure modes that appear after a demo becomes a running system.
01 / Problem
A vector search result is not the same thing as dependable memory.
Long-running AI state has to handle identity, time, conflicting information, relationships between memories, account boundaries, ingestion, cost, and what happens when retrieval is uncertain.
Cognitive Mesh grew from that operating problem rather than from a need to demonstrate a chatbot interface.
02 / Architecture
- Boundaries
Domain, application, infrastructure, and interface responsibilities are separated so persistence and orchestration do not silently collapse into one layer.
- State
Typed memory nodes, relationships, episodic records, document ingestion, user and organization scoping, and event-level token records live in Postgres.
- Retrieval
Embedding-based search, cross-session memory regions, ranking, confidence, and entropy-related scoring determine which stored state becomes usable context.
- Governance
Architectural laws, an automated judge, and an audit trail exist to reduce drift across independently changed code paths.
- Evaluation
The evaluation work tracks precision at one, recall at five, and mean reciprocal rank while exposing contamination and account-state problems.
03 / What the audit exposed
The system could rank the right memory first and still answer as if it did not exist.
- Retrieval threshold
A correct top-ranked result could fall below a rigid acceptance threshold and be discarded. Ranking quality and acceptance policy were being treated as the same decision.
- False-negative persistence
A failed recall answer could be stored as new memory, creating a feedback loop in which the system learned to repeat its own denial.
- Eval contamination
Repeated probes against accumulated account state could collapse metrics without a corresponding code regression.
The write path, authentication, streaming, and data integrity performed better than recall. The known fixes include confidence-aware retrieval before dropping candidates, filtering failure responses before persistence, and isolating evaluation namespaces.
04 / What this proves
The useful proof is not that I built an AI product alone. It is that I can map where an apparently working system lies.
The project demonstrates production deployment across Fly.io, Neon Postgres, and Vercel; domain-driven backend structure; multi-tenant data; embeddings; authentication boundaries; token accounting; eval design; and the willingness to audit an architecture against its actual behavior.
I do not present the system as finished or failure-free. The retrieval failures are material, documented, and central to what I learned from the work.