Architecture Overview

AMADEQ LIS 3 is a set of eight independent FastAPI services that together deliver a live, multimodal chat experience: text chat, speech-to-text, text-to-speech, retrieval-augmented generation, long-term memory, and image analysis. A single gateway is the only public entry point; every other service is internal and reached over HTTP or WebSocket.

Design principles

The codebase follows a clean / hexagonal architecture. Each service isolates its business rules from transport and from external dependencies, which are injected through ports (Python Protocol interfaces) and satisfied by adapters wired in a container.py composition root.

Every service is organized into the same layers:

Layer	Directory	Responsibility
Transport	api/	FastAPI app assembly, routes, WebSocket handlers, auth, request/response mapping.
Application	application/	Transport-agnostic business rules and orchestration; depends only on ports.py.
Adapters	services/	Outbound clients to external systems (HTTP/WebSocket to other services and models).
Persistence	db/	Async psycopg pool, Pydantic row models, and repositories issuing raw parameterized SQL.

Key consequences of this structure:

• The application layer never imports FastAPI, psycopg, or httpx directly — it depends on Protocol ports, so adapters (and DB) can be swapped for fakes in tests.
• Dependencies are constructed once in container.py and injected, which is also how test suites run with enable_db=False and stub clients.
• Configuration is centralized in a single root config.py that every service re-exports. See Configuration.

note

Not every service uses all four layers. stt_service and tts_omnivoice are single-file services with no db/ layer (they hold state only in memory), and rag_service / image_analysis_service are stateless and touch no relational database.

System topology

                         ┌──────────────────────────────────────────────┐
   client (HTTP + WS)    │                 gateway :8040                 │
  ───────────────────────▶            all routes under /api_lis          │
                         │  REST: sessions, memory  |  WS: relays         │
                         └───────┬───────────┬───────────────┬───────────┘
                                 │ WS relay  │ WS relay       │ WS relay
                                 ▼           ▼                ▼
                    conversation_engine   stt_service    tts_service (ElevenLabs)
                          :8050              :8010              :8014
                     /message_pipeline      /ws/stt            /ws/tts
                          │
      ┌───────────────────┼──────────────────────┬─────────────┬──────────────┐
      │ HTTP              │ HTTP                  │ HTTP        │ WS (primary   │ WS (fallback)
      ▼                   ▼                       ▼             │  → fallback)  ▼
 memory_service       rag_service        image_analysis    tts_service   tts_omnivoice
    :8052                :8051              :8012 ─▶ vLLM      :8014           :8015
      │                   │                        :8013
      │                   │                                   (also: guard :4915, LLM,
      ▼                   ▼                                    LoRA check — external)
   Postgres           Qdrant :6333
 (lis.* + ml.avatars) embeddings + reranker (TEI)

• The client speaks only to the gateway on port 8040. All gateway HTTP and WebSocket routes mount under the prefix /api_lis (the sole exception is the unauthenticated GET /health, mounted at root).
• The gateway is a thin authenticated relay for the three WebSocket flows (chat, transcribe, voiceover). It owns no chat/TTS/STT logic of its own.
• The conversation engine is the orchestration hub: a single turn fans out to memory, RAG, image analysis, the LLM, and TTS.
• The primary → fallback TTS relationship (ElevenLabs :8014 → OmniVoice :8015) is owned entirely by the conversation engine, not the gateway. The gateway's voiceover relay targets ElevenLabs only.

Service inventory

The eight services, as launched by run/run.sh (ports from run/env.sh):

Name	Port	Module entrypoint	Purpose
gateway	8040	gateway_service.main:app	Public HTTP + WebSocket entry point; auth, session/memory REST, WS relays.
conversation_engine	8050	conversation_engine.main:app	Live text-chat pipeline over WS /message_pipeline; orchestrates all turn stages.
stt_service	8010	component_services.stt_service:app	Streaming speech-to-text (faster-whisper) over WS /ws/stt.
image_analysis_service	8012	component_services.image_analysis_service.main:app	HTTP adapter fronting a vLLM VLM backend (:8013); question-agnostic image analysis.
tts_service (ElevenLabs)	8014	component_services.tts_service.main:app	Cloud TTS adapter over WS /ws/tts; primary voice backend.
tts_omnivoice (OmniVoice)	8015	component_services.tts_omnivoice:app	Local GPU TTS (k2-fsa/OmniVoice) over WS /ws/tts; fallback voice backend.
rag_service	8051	component_services.rag_service.main:app	Retrieve-only RAG pipeline over HTTP POST /retrieve.
memory_service	8052	component_services.memory_service.main:app	Long-term user memory: injection and LLM-driven extraction over HTTP.

note

The image analysis service is a two-tier setup: the adapter on port 8012 fronts a separately-served vLLM multimodal backend on port 8013. External model servers (embeddings/reranker via TEI, Qdrant, the safety guard, and the core LLM) are managed under run/external/ and are not counted among the eight LIS services.

Request flow

A typical chat turn flows through three tiers — gateway relay, engine pipeline, and component services:

1. Client opens a WebSocket to WS /api_lis/sessions/{session_id}/message on the gateway, passing ?user_id= and ?api_key=. The gateway authenticates the query params, then verifies session ownership and loads the avatar from the database.
2. The gateway relays the connection to the conversation engine at ws://localhost:8050/message_pipeline, sending a session.start event whose payload carries the resolved session/avatar config (system prompt, voice-clone fields, collection name, language hint, and the client's memory/voice preferences).
3. On each message.text, the engine runs its turn pipeline: safety guard → persist user message → (image analysis for image turns) → parallel memory + RAG fetch → prompt assembly → streaming LLM response → persist assistant message → best-effort TTS. Domain events are translated to wire envelopes and streamed back down the same relay to the client.
4. The engine reaches component services as needed: memory_service (:8052) and rag_service (:8051) over HTTP, image_analysis_service (:8012) over HTTP, and the TTS backends (:8014 → :8015) over WebSocket.

The full stage ordering, error recoverability, and event names are documented in Conversation Pipeline.

Per-service layer notes

gateway

Transport (api/) mounts three routers under /api_lis: REST sessions, REST memory (/memory), and the WebSocket conversation router. Application services (SessionService, MemoryService, RelayService) hold ownership and relay rules; adapters (services/) are the WebSocket clients to the engine, STT, and TTS; persistence (db/) covers lis.sessions (owned), read-only ml.avatars and lis.conversation_history, and read + soft-delete lis.memory_facts. See Gateway.

conversation_engine

Transport (api/) is the FastAPI app, the WS /message_pipeline router, and the per-connection ConnectionHandler. Application (application/) holds the Pipeline turn orchestrator, the in-memory LiveService session store, and prompt assembly. Adapters (services/) implement the guard, LLM, LoRA, memory, RAG, image-analysis, and TTS ports. Persistence (db/) writes lis.conversation_history and owns lis.conversation_attachments. See Conversation Engine.

stt_service

A single-file service: one FastAPI app exposing /health, /status, and WS /ws/stt, backed by an in-memory session manager and a faster-whisper model manager. No database. See Speech-to-Text.

tts_service and tts_omnivoice

Two independent services sharing one WebSocket envelope protocol. tts_service (ElevenLabs, :8014) uses the layered api/ → application/ → services/ structure; tts_omnivoice (OmniVoice, :8015) is a single-file local-GPU service. Neither persists to Postgres; OmniVoice reads a JSON voice-profile store from disk. See Text-to-Speech.

rag_service

Stateless, layered api/ → application/ → services/. Reads only Qdrant collections plus external embeddings, reranker, and query-rewrite LLM; no relational database. See RAG Service.

memory_service

Layered api/ → application/ → services//db/. Owns the full lifecycle of lis.memory_facts, including pgvector similarity search, three-tier deduplication, and LLM consolidation. See Memory Service.

image_analysis_service

Layered api/ → application/ → services/. A stateless HTTP adapter that validates images and calls a vLLM backend; it persists nothing (the returned description is stored by the conversation engine). See Image Analysis.

Further reading

• Conversation Pipeline — the ordered turn stages, events, and error handling in the conversation engine.
• Database — the shared Postgres schema, table ownership, and the psycopg data-access layer.
• Extending the System — how the ports/adapters structure lets you add or replace a dependency.