Running the Services

LIS 3 ships a single bash-based process launcher under run/. One entrypoint — run/run.sh — starts, stops, health-checks, and reports status for every process: the FastAPI services (local uvicorn processes) and the external dependencies (Docker containers plus local vLLM processes). The scripts spawn processes and manage containers only; they never touch a database directly.

The run/run.sh entrypoint

Invoke the launcher with an action and a target:

bash run/run.sh <action> <target>

Argument	Values
<action>	start | stop | health | status
<target>	a group name or an individual service/external name

Examples:

bash run/run.sh start all         # bring up externals, then services
bash run/run.sh status all        # report status of everything
bash run/run.sh health gateway    # probe one service's /health
bash run/run.sh stop rag          # stop one service
bash run/run.sh help              # print usage

If <target> is omitted it defaults to all; if <action> is omitted it defaults to help.

Group and individual targets

The dispatcher defines four layer groups plus the aggregate all. Each group maps to a fixed list of names.

Target	Members	Notes
all	all externals, then all services	Full bring-up / tear-down
services	stt tts_elevenlabs tts_omnivoice image_analysis memory rag chat_engine gateway	FastAPI uvicorn processes
externals	qdrant embeddings reranker guard vlm	Docker + local vLLM
infra	qdrant	Data stores only
ml	embeddings reranker guard vlm	Model servers

Individual FastAPI targets: gateway, chat_engine, rag, memory, image_analysis, stt, tts_elevenlabs, tts_omnivoice.

Individual external targets: qdrant, embeddings, reranker, guard, vlm.

Each FastAPI target name maps to a uvicorn module:

Target	Module
gateway	gateway_service.main:app
chat_engine	conversation_engine.main:app
rag	component_services.rag_service.main:app
memory	component_services.memory_service.main:app
image_analysis	component_services.image_analysis_service.main:app
stt	component_services.stt_service:app
tts_elevenlabs	component_services.tts_service.main:app
tts_omnivoice	component_services.tts_omnivoice:app

note

The infra group contains Qdrant only. There is no run/external/postgres.sh, and Postgres is not a member of any group. The run/ scripts never start, migrate, or connect to Postgres — the launched Python services own their own tables.

Start/stop ordering

start all is dependency-aware: externals start first (in the order qdrant, embeddings, reranker, guard, vlm), then services (in SVCS order, stt first through gateway last). This inline loop runs under set -euo pipefail and is not wrapped in || true, so the first failure aborts the whole bring-up.

stop all reverses the layers — services first, then externals — but every call is wrapped in || true, so tear-down is best-effort and never aborts. health all and status all are likewise best-effort.

tip

The group targets services, externals, infra, and ml tolerate individual failures (each item is wrapped in || true). Only start all is strict. If a single dependency is flaky, run the layer groups individually instead of all.

run/env.sh — shared configuration

run/env.sh is sourced by every script (never executed directly). It:

• Computes PROJECT_ROOT and sources $PROJECT_ROOT/.env if present, so any default below can be overridden there.
• Picks the Python interpreter: $PROJECT_ROOT/.venv/bin/python if present, else python3 (override with $PYTHON).
• Creates run/logs/ and run/pids/.
• Defines every port, Docker image/container name, model id, and GPU fraction as an overridable default.
• Defines the shared shell helpers.

Key defaults (all overridable via environment or .env):

Variable	Default	Purpose
GATEWAY_PORT	8040	uvicorn port for gateway
CHAT_ENGINE_PORT	8050	uvicorn port for chat_engine
RAG_PORT	8051	uvicorn port for rag
MEMORY_PORT	8052	uvicorn port for memory
IMAGE_ANALYSIS_PORT	8012	uvicorn port for image_analysis
STT_PORT	8010	uvicorn port for stt
TTS_ELEVENLABS_PORT	8014	uvicorn port for tts_elevenlabs
TTS_OMNIVOICE_PORT	8015	uvicorn port for tts_omnivoice
EMBED_PORT	8080	Host port for the TEI embeddings container
RERANKER_PORT	8081	Host port for the TEI reranker container
GUARD_PORT	4915	Port for the guard vLLM process
VLM_PORT	8013	Port for the LFM2.5-VL vLLM process
QDRANT_PORT	6333	Host port for the Qdrant container
POSTGRES_PORT	4052	Declared but unused by run/
TEI_IMAGE	ghcr.io/huggingface/text-embeddings-inference:120-1.9	Image for both TEI containers
EMBEDDINGS_MODEL	intfloat/multilingual-e5-large	TEI embeddings model id
RERANKER_MODEL	BAAI/bge-reranker-v2-m3	TEI reranker model id
TEI_CUDA_FRACTION	0.05	CUDA_MEMORY_FRACTION for both TEI containers
EMBEDDINGS_CONTAINER	lis3-embeddings	Embeddings container name
RERANKER_CONTAINER	lis3-reranker	Reranker container name
QDRANT_CONTAINER	lis3-qdrant	Qdrant container name
QDRANT_DATA_DIR	$PROJECT_ROOT/data/qdrant	Host volume for /qdrant/storage
VLLM_BIN	$PROJECT_ROOT/.venv/bin/vllm	vLLM executable for guard and vlm
GUARD_MODEL	Qwen/Qwen3Guard-Gen-0.6B	Safety guard model
GUARD_GPU_UTIL	0.1	--gpu-memory-utilization for guard
GUARD_MAX_LEN	2048	--max-model-len for guard
VLM_MODEL	LiquidAI/LFM2.5-VL-450M	Vision model for image analysis
VLM_GPU_UTIL	0.25	--gpu-memory-utilization for vlm
VLM_MAX_LEN	8192	--max-model-len for vlm
VLM_MAX_IMAGES	4	--limit-mm-per-prompt image cap for vlm
HF_TOKEN	(empty)	HuggingFace token for gated models
PYTHON	.venv/bin/python or python3	Interpreter for uvicorn and probes

warning

env.sh also declares Postgres variables (POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD, APP_POSTGRES_USER, POSTGRES_CONTAINER, POSTGRES_INIT). None are consumed by any run/ script, and POSTGRES_INIT's default target ($PROJECT_ROOT/init.sh) does not exist in the repo. Treat these as inert placeholders. See Configuration for the variables the Python services actually read.

Shell helpers

env.sh defines four probe/utility helpers used across the scripts:

Helper	Behavior
http_ok <url>	2 s HTTP probe; success on status 200–299, failure on any URLError
tcp_ok <port>	2 s TCP connect probe against 127.0.0.1:<port>
wait_for_http <name> <url> [timeout]	Poll http_ok every 1 s until healthy or timeout (default 30 s)
pid_on_port <port>	lsof lookup of the listening PID on a port

run/services/ — FastAPI service scripts

Three small scripts manage one uvicorn service each. run.sh calls them with the resolved name, port, and module.

Script	Invocation	What it does
start.sh	start.sh NAME PORT MODULE	Start one uvicorn service
stop.sh	stop.sh NAME	Stop one service by pidfile
smoke.sh	smoke.sh NAME URL	Probe a health URL; print ok or DOWN

start.sh is idempotent and defensive:

• If the pidfile exists and the process is alive, it prints already running and exits 0. A stale pidfile is removed.
• If the port is already LISTENing (pid_on_port), it refuses to start and exits 1.
• It truncates the logfile, cds to PROJECT_ROOT, then launches nohup python -m uvicorn <module> --host 127.0.0.1 --port <port>, records $! to the pidfile, and after 0.3 s checks the process is still alive (else it tails the last 20 log lines and exits 1).
• It then waits up to 30 s for GET /health. On timeout it tails the log, kills the process, removes the pidfile, and exits 1.

warning

start.sh always launches uvicorn with --host 127.0.0.1, so services launched through run/ bind to loopback only — even the gateway, whose configured gateway_host default is 0.0.0.0. The configured host is ignored by the launcher.

stop.sh sends a single SIGTERM (plain kill) — no SIGKILL escalation and no wait for exit — then always removes the pidfile. A missing pidfile prints no pidfile, not running and exits 0.

smoke.sh runs http_ok against the given URL, prints <name> ok (<url>) on success, or prints <name> DOWN to stderr and exits 1 on failure. run.sh maps the health action for a service to smoke.sh against http://127.0.0.1:<port>/health.

run/external/ — Docker and vLLM helpers

Each external dependency has its own script accepting start, stop, health, or status.

Script	Type	One-line summary
qdrant.sh	Docker	Manage the Qdrant vector-store container; health via /healthz
embeddings.sh	Docker	Run the TEI embeddings container serving $EMBEDDINGS_MODEL on 8080 (--pooling mean)
reranker.sh	Docker	Run the TEI reranker container serving $RERANKER_MODEL on 8081
guard.sh	Local vLLM	Run the Qwen3Guard safety model as a local process on 4915
vlm.sh	Local vLLM	Run the LFM2.5-VL-450M vision model as a local process on 8013

note

Only qdrant, embeddings, and reranker are Docker-managed. guard and vlm run as local vLLM processes via $VLLM_BIN, not containers, and are the only targets that additionally write a .pgid file so the whole vLLM worker tree can be killed together.

Behavioral details:

• qdrant.sh start is fail-open toward an existing instance: if the port already answers /healthz, it prints already running (external — not managed here) and does nothing. Otherwise it docker starts the existing container, or docker runs qdrant/qdrant with the data volume, then waits up to 30 s.
• embeddings.sh / reranker.sh start skip if /health already answers; otherwise they docker start the existing container or docker run $TEI_IMAGE with --gpus all, the CUDA fraction, the HF cache volume, and the model id, then wait up to 120 s.
• guard.sh / vlm.sh start launch nohup vllm serve <model> --port <p> --max-model-len <n> --gpu-memory-utilization <f>. vlm.sh also passes --limit-mm-per-prompt with the image cap and exports HUGGING_FACE_HUB_TOKEN when HF_TOKEN is set. wait_for_http timeouts are 180 s (guard) and 240 s (vlm).
• External stop for the Docker helpers runs docker stop <container> — it never removes the container, so a later start reuses it.

run/logs/ and run/pids/

• run/logs/<name>.log — per-target stdout/stderr, truncated on each start.
• run/pids/<name>.pid — per-target pidfile. guard and vlm additionally write run/pids/<name>.pgid for process-group kills.

Ports and health endpoints

All services bind 127.0.0.1. Every FastAPI service and both TEI containers expose GET /health; Qdrant is the exception and uses GET /healthz.

Service	Port	Health URL
gateway	8040	http://127.0.0.1:8040/health
chat_engine (conversation_engine)	8050	http://127.0.0.1:8050/health
rag	8051	http://127.0.0.1:8051/health
memory	8052	http://127.0.0.1:8052/health
image_analysis	8012	http://127.0.0.1:8012/health
stt	8010	http://127.0.0.1:8010/health
tts_elevenlabs	8014	http://127.0.0.1:8014/health
tts_omnivoice	8015	http://127.0.0.1:8015/health
qdrant	6333	http://127.0.0.1:6333/healthz
embeddings (TEI)	8080	http://127.0.0.1:8080/health
reranker (TEI)	8081	http://127.0.0.1:8081/health
guard (vLLM)	4915	http://127.0.0.1:4915/health
vlm (LFM2.5-VL)	8013	http://127.0.0.1:8013/health

note

image_analysis spans two ports: the FastAPI service on 8012 (target image_analysis) and its LFM2.5-VL vLLM backend on 8013 (target vlm). Postgres (4052) has defaults in env.sh but no managing script and no health probe here.

Smoke test / health check

To check everything at once:

bash run/run.sh health all

This probes every external and service best-effort and prints ok or DOWN per target. To probe a single service, name it directly:

bash run/run.sh health gateway

For a process-level view (pidfile + kill -0 for services; docker inspect state plus HTTP for the Docker externals; pid/pgid plus HTTP for the vLLM externals), use status:

bash run/run.sh status all

Related pages

• Installation — prerequisites and one-time setup.
• Configuration — the environment variables the Python services read at runtime.