Memory Service

The Memory Service is a standalone FastAPI microservice that gives the platform long-term, per-user memory. It extracts durable facts from conversation transcripts with an LLM, stores them with vector embeddings, and injects the relevant ones back into later turns. It owns the lis.memory_facts table end to end.

Purpose and port

The service is defined in component_services/memory_service/main.py, which builds a module-level app (FastAPI title Memory Service, v0.1.0) via build_memory_app(enable_db=APP_CONFIG.enable_db_on_startup). It is launched by run/run.sh on port 8052 (MEMORY_PORT in run/env.sh).

It does two things:

• Injection — return the facts most relevant to a user_id / avatar_id pair, packed into a text block that the conversation engine prepends to the prompt.
• Extraction — read a conversation transcript, distil it into candidate facts, deduplicate against what is already stored, and persist the new or updated facts.

The service is its own ASGI application. Its three endpoints are served directly on port 8052 and are not mounted behind the gateway /api_lis prefix. They are consumed server-to-server by the conversation engine's MemoryClient (conversation_engine/services/memory.py) using ENGINE_CONFIG.memory_url (default http://localhost:8052).

note

The /api_lis prefix applies only to the separate, gateway-owned memory CRUD router (GET /api_lis/memory, DELETE /api_lis/memory/{fact_id}). That is a different subsystem — see HTTP Endpoints. The Memory Service's own routes below are never behind the gateway.

Endpoints

Method	Path	Description
GET	/health	Liveness probe. Returns {"service": "memory_service", "status": "ok"}. No DB or upstream call.
GET	/get_injection_context	Return the relevant facts for a user_id / avatar_id pair as a memory block.
POST	/extract	Extract and store facts from a transcript.

GET /get_injection_context

Query parameters:

Parameter	Type	Description
user_id	UUID	The user whose facts to fetch.
avatar_id	UUID	The avatar the facts are scoped to.

Returns a MemoryInjectionBlock:

{
  "text": "## User Memory\n- [preference] favorite_topic: astronomy\n\nUse these facts to personalize your responses where relevant.\n",
  "facts_count": 1,
  "applied": true
}

When the user has no facts, the service returns an empty block (text: "", facts_count: 0, applied: false). Called by MemoryClient.inject().

POST /extract

Request body (ExtractRequest):

Field	Type	Description
transcript	string	Conversation text to extract facts from.
session_id	UUID	Source session, stored as source_session_id.
user_id	UUID	Owner of the resulting facts.
avatar_id	UUID	Avatar the facts are scoped to.

The response is a bare JSON array of the saved facts (each MemoryFact.model_dump(mode="json")), not an object wrapper. An empty or whitespace-only transcript returns HTTP 422. Called by MemoryClient.extract() as fire-and-forget.

How it works

Injection

Injector (application/injection.py) fetches the user's non-deleted facts, ranks them, and greedily packs them into a character-budgeted block:

1. Fetch facts via MemoryRepository.get_active_facts (SQL filter status <> 'deleted', so both active and superseded rows are eligible).
2. Sort in Python by (confidence, updated_at) descending — embeddings play no role in injection ordering.
3. Build the block with fixed _HEADER (## User Memory) and _FOOTER templates. The available budget is injection_char_budget minus the header and footer lengths. Facts are appended line by line until the next line would exceed the remaining budget, at which point packing stops.
4. Stamp last_seen_at = NOW() on every included fact via touch_last_seen.

note

The header and footer are always emitted. If zero facts fit the budget, the returned block is still non-empty text with facts_count: 0.

Extraction

Extractor (application/extraction/extraction.py) and Deduplicator (application/extraction/deduplication.py) run this pipeline (orchestrated in MemoryService.extract):

1. Reject an empty transcript (422).
2. LLM extract — truncate the transcript to max_transcript_chars and call the extraction LLM. The response is parsed as JSON; any parse error is swallowed and yields no facts ([]).
3. Validate each candidate: drop anything below min_confidence; normalize fact_key / fact_value to lowercase, whitespace-collapsed text; require the key to be 3–64 chars matching ^[a-z][a-z0-9_]*$; require a non-empty value of at most 200 chars; trim source_excerpt to 200 chars.
4. Secret hygiene — drop any candidate whose key, value, or excerpt matches password / token / api-key / bearer keywords, long base64-like strings (40+ chars), or credit-card-like digit groups.
5. Enrich each surviving fact with a SHA-256 dedup hash (sha256("{avatar_id}:{fact_key}:{fact_value}")) and a vector embedding computed over the text "{fact_key}: {fact_value}".
6. Deduplicate per fact (see below), then insert new facts or supersede stale ones.

Deduplication and superseding

For each enriched fact, the service applies three tiers in order:

1. Exact hash — if exists_by_hash(dedup_hash) is true, the fact is a byte-for-byte duplicate and is skipped entirely.
2. Vector similarity — search_by_embedding runs a pgvector cosine-distance query (embedding <=> %s::vector < threshold, ordered ascending, LIMIT dedup_embedding_limit). Only the top-1 result is used.
3. Exact key — if similarity finds nothing, fall back to get_by_key (exact fact_key match).

If tier 2 or 3 finds an existing fact, a second consolidation LLM call merges the old and new fact. The merged fact's hash and embedding are recomputed, then supersede_fact runs in a single transaction: the old row's status is set to 'superseded' and the merged row is inserted. If no existing fact is found, the new fact is inserted fresh.

note

The consolidation call is best-effort: if its JSON output cannot be parsed, the service falls back to the incoming new fact unchanged.

Failure behavior

The service itself fails closed. RequestValidationError maps to HTTP 422, UpstreamError to HTTP 503, and any other exception is wrapped as UpstreamError (503).

Its caller — the conversation engine turn pipeline — treats memory as best-effort. Injection runs concurrently with RAG and, on any error, emits a recoverable TurnError (code='MEMORY_ERROR') and proceeds with an empty block. Extraction is dispatched as a fire-and-forget background task whose errors are swallowed. See Conversation Pipeline.

Data ownership

The Memory Service is the sole owner of the full lifecycle of lis.memory_facts — it is the only table this service touches. Key columns:

Column	Notes
fact_id	UUID PK, generated app-side (uuid4).
user_id, avatar_id	Scope of the fact.
category, fact_key, fact_value	Normalized fact content.
confidence	Float; used for injection ranking.
status	active, superseded, or deleted.
source_session_id, source_excerpt	Provenance of the fact.
dedup_hash	SHA-256 hex for exact-duplicate detection.
embedding	pgvector vector for similarity search.
created_at, updated_at, last_seen_at	DB-defaulted; returned via RETURNING, never written by the service.
deleted_at	Never written by this service.

The service never sets status = 'deleted' and never writes deleted_at — soft-delete is performed only by the gateway. See Database.

warning

The gateway reads and soft-deletes lis.memory_facts directly (it does not call this service). Its GET /api_lis/memory lists facts and DELETE /api_lis/memory/{fact_id} sets status = 'deleted'. See HTTP Endpoints.

Because the service reads embeddings as native vectors, its connection pool registers pgvector on every connection (register_vector_async). The pool is opened at startup only when enable_db is true, with min_size=0 so the service holds no connections at idle (the Neon -pooler host is expected).

Configuration

All settings come from the repo-root config.py (MemoryConfig, re-exported as MEMORY_CONFIG). The service code has no port field of its own — the port is bound externally by the run script.

LLM and embeddings

Setting	Env var	Default	Purpose
llm_endpoint	MEMORY_LLM_ENDPOINT (fallback LLM_ENDPOINT)	https://core-llm.nezlamna-online.education/api/generate	Extraction / consolidation LLM generate endpoint.
llm_api_token	MEMORY_LLM_API_TOKEN (fallback LLM_API_TOKEN)	change-me-in-production	Bearer token sent by the LLM client.
llm_timeout_sec	MEMORY_LLM_TIMEOUT_SEC (fallback LLM_TIMEOUT_SEC)	30.0	httpx timeout for LLM calls.
llm_temperature	MEMORY_LLM_TEMPERATURE	0.1	Temperature for the extraction call.
llm_max_tokens	MEMORY_LLM_MAX_TOKENS	1024	Max tokens for the extraction call.
consolidation_temperature	MEMORY_CONSOLIDATION_TEMPERATURE	0.1	Temperature for the merge call.
consolidation_max_tokens	MEMORY_CONSOLIDATION_MAX_TOKENS	256	Max tokens for the merge call.
embed_url	MEMORY_EMBED_URL (fallback EMBED_URL)	http://localhost:8016	Base URL of the embeddings (TEI) service; the client POSTs /embed.
embedding_timeout_sec	MEMORY_EMBEDDING_TIMEOUT_SEC (fallback EMBEDDING_TIMEOUT_SEC)	10.0	httpx timeout for embedding calls.

Extraction and injection budgets

Setting	Env var	Default	Purpose
max_transcript_chars	MEMORY_MAX_TRANSCRIPT_CHARS	12000	Transcript is truncated to this many chars before extraction.
min_confidence	MEMORY_MIN_CONFIDENCE	0.6	Candidate facts below this confidence are dropped.
injection_char_budget	MEMORY_INJECTION_CHAR_BUDGET	2000	Total chars for the injection block (header + footer subtracted first).
dedup_embedding_limit	MEMORY_DEDUP_EMBEDDING_LIMIT	5	LIMIT for the similarity search (only top-1 is used).
dedup_embedding_threshold	MEMORY_DEDUP_EMBEDDING_THRESHOLD	0.15	Max cosine distance for a stored fact to count as a near-duplicate.

Application and database

Setting	Env var	Default	Purpose
enable_db_on_startup	ENABLE_DB_ON_STARTUP	True	Whether the lifespan opens/closes the Postgres pool (disabled in tests).
log_level / log_format	LOG_LEVEL / LOG_FORMAT	INFO / standard format	Logging configuration set in main.py.
DB host	PGHOST	localhost	Postgres host (use the Neon -pooler host).
DB name	PGDATABASE	lis	Database name.
DB user	PGUSER	lis	DB user.
DB password	PGPASSWORD	""	DB password.
SSL mode	PGSSLMODE	require	psycopg sslmode.
Channel binding	PGCHANNELBINDING	require	psycopg channel_binding.
Pool max size	DB_POOL_MAX	10	AsyncConnectionPool max size (min_size is hardcoded to 0).
Port	MEMORY_PORT	8052	Port uvicorn binds in the run script.

Architecture

The service follows a clean/hexagonal layout wired by a DI container (container.py):

Layer	Path	Role
API	api/	FastAPI app factory, routes, CORS, DB-pool lifespan, exception-to-HTTP mapping.
Application	application/	MemoryService orchestrator, Injector, Extractor, Deduplicator, prompts.
Ports	application/ports.py	Protocol interfaces for the repository, LLM client, and embedding client.
Outbound adapters	services/	httpx-based LLMClient and EmbeddingClient.
Persistence	db/	psycopg async pool, MemoryFact model, MemoryRepository SQL over lis.memory_facts.

For a system-wide view, see the Architecture Overview.