STT Service

The STT service is a standalone FastAPI microservice that turns streaming microphone audio into text over a single WebSocket. It is backed by faster-whisper (WhisperModel, default large-v3) and holds all session state in memory — it never touches a database.

Purpose and port

The service is defined in component_services/stt_service.py as a module-level app (AMADEQ STT Service, v0.2.0) and launched by run/run.sh via uvicorn on port 8010 (STT_PORT in run/env.sh).

It is its own ASGI application and is not mounted behind the gateway /api_lis prefix. Clients almost never connect to it directly. Instead they reach it through the gateway relay endpoint WS /api_lis/transcribe, which forwards to ws://localhost:8010/ws/stt (GatewayConfig.stt_ws_url, env STT_WS_URL). See the transcribe flow in WebSocket Protocol.

note

Recognition language is hard-forced to English (STT_FORCED_LANGUAGE, default en). Every session is created with language_mode="forced_en", and every partial and final result reports language: "en". Any client-supplied language_hint is ignored, and the STT_DEFAULT_LANGUAGE / STT_ALLOW_AUTO_LANGUAGE settings are read but never applied — auto-language detection is not wired in.

Endpoints

The HTTP and WebSocket routes below are served directly on the STT app at port 8010 with no path prefix.

Method	Path	Description
GET	/health	Service and Whisper model readiness.
GET	/status	Runtime metrics and effective timing/VAD config.
WS	/ws/stt	Streaming speech-to-text WebSocket.

GET /health

Returns model readiness plus the active session count. status is "ok" when the model is loaded, otherwise "degraded".

{
  "service": "stt_service",
  "status": "ok",
  "model_ready": true,
  "model_name": "large-v3",
  "device": "cuda",
  "device_index": 0,
  "active_sessions": 0,
  "error": null
}

GET /status

Returns process-wide metrics and a snapshot of the effective timing/VAD configuration.

{
  "service": "stt_service",
  "model_ready": true,
  "metrics": {
    "uptime_sec": 0.0,
    "active_sessions": 0,
    "partial_events": 0,
    "final_events": 0,
    "empty_final_events": 0,
    "decode_failures": 0,
    "total_utterances": 0,
    "finalize_reason_counts": {
      "audio_end": 0,
      "silence_timeout": 0,
      "max_utterance_reached": 0,
      "disconnect_finalize": 0,
      "force_finalize": 0,
      "timeout_cancel": 0
    },
    "avg_decode_latency_sec": 0.0
  },
  "config": { "sample_rate": 16000, "silence_timeout_sec": 0.65, "...": "..." }
}

WS /ws/stt

Bidirectional streaming socket. Each inbound message is a JSON envelope; the service replies with the same envelope shape.

Inbound events: session.start, audio.chunk, audio.end, session.cancel, session.ping.

Outbound events: session.started, transcript.partial, transcript.final, transcript.empty_final, session.pong, session.cancelled, error.

Every envelope must contain event, session_id, and payload:

{
  "event": "audio.chunk",
  "session_id": "sess-123",
  "timestamp": "2026-07-08T12:00:00Z",
  "payload": { "chunk_index": 0, "audio": "<base64 pcm16le>" }
}

Outbound envelopes add a timestamp and a metadata block (model name, processing time, buffered audio). A single socket is bound to one session_id after session.start; sending audio.chunk / audio.end / session.cancel / session.ping for any other session_id raises SESSION_NOT_FOUND.

How it works

Session lifecycle

session.start validates the requested audio_format (must be pcm16le) and sample_rate (must be 16000), enforces STT_MAX_CONCURRENT_SESSIONS, and creates an in-memory SessionContext. The reply session.started echoes a read-only config snapshot (model, timeouts, decode/partial intervals, energy VAD threshold, preroll, max utterance, and the client-supplied vad_enabled, default false).

A session.start while the model is not ready raises MODEL_NOT_READY (recoverable=false). Exceeding the concurrency cap raises GPU_OVERLOADED (recoverable=false).

The audio input contract is strict. Each audio.chunk is validated for matching audio_format (pcm16le), matching sample_rate (16000), mono (channels == 1), a strictly increasing chunk_index (else CHUNK_OUT_OF_ORDER), even PCM byte length, and decodable base64. Empty chunks are rejected. audio.chunk only ingests audio and returns no immediate response — results are emitted by the background loop described below.

session.ping refreshes the session's activity timestamp and replies session.pong. session.cancel drops the current utterance, sends session.cancelled, then closes the socket.

Background decode loop

Each WebSocket connection runs one background task that ticks every STT_DECODE_INTERVAL_MS (default 60 ms). On each tick it:

1. Cancels the session if it has been idle for STT_IDLE_TIMEOUT_MS (default 60 s), emitting session.cancelled with reason: "timeout" and closing the socket.
2. Finalizes the current utterance if a finalize condition is met (see below), emitting transcript.final or transcript.empty_final.
3. Otherwise emits a throttled transcript.partial.

note

Partial and final results are produced by this loop, not synchronously in response to audio.chunk. Because a single threading.Lock guards the shared WhisperModel, only one transcription runs at a time across all sessions even though decode calls are dispatched via asyncio.to_thread.

Voice activity detection and endpointing

Each chunk is gated by an energy-based (RMS) VAD: when energy is at or above STT_ENERGY_VAD_THRESHOLD (default 0.02) the frame is marked voiced and starts or extends an utterance. A STT_PREROLL_MS (default 300 ms) lookback buffer is prepended when speech starts so leading syllables are not clipped, and trailing silence is retained to detect the endpoint. faster-whisper's own model VAD is additionally applied inside transcribe via vad_filter (STT_USE_MODEL_VAD, default true).

An utterance is finalized when:

Reason	Trigger
max_utterance	Buffered speech reaches STT_MAX_UTTERANCE_MS (default 30 s).
silence_timeout	At least STT_MIN_DECODE_AUDIO_MS of audio and no voiced frame for STT_ENDPOINT_SILENCE_MS (default 650 ms).
audio_end	Client sends audio.end, forcing an immediate finalize.

Partial results are throttled by STT_PARTIAL_INTERVAL_MS, decoded over only the trailing STT_PARTIAL_WINDOW_MS of audio, then stabilized against the previous partial with a longest-common-prefix merge. Identical or empty stabilized partials are suppressed (no event).

Finalization is resilient. transcript.final text falls back to the last partial text if the full decode returns empty; when there is no utterance or the result is still empty, transcript.empty_final is emitted with detail no_audio_detected or empty_transcript.

tip

On disconnect the handler best-effort finalizes any in-flight utterance with reason disconnect, cancels the decode task, and removes the session from memory.

Errors

Expected problems raise a ServiceError with a code, message, and recoverable flag, serialized into an error envelope. Validation, format, and ordering errors are recoverable=true; MODEL_NOT_READY and GPU_OVERLOADED are recoverable=false. Decode failures increment decode_failures, set the session to FAILED, and raise TRANSCRIPTION_FAILED (recoverable=true) while keeping the connection open. Invalid JSON yields INVALID_PAYLOAD; unknown event names yield INVALID_EVENT; unexpected exceptions yield INTERNAL_ERROR.

Model loading is fail-open: if the Whisper model fails to load, the service stays up and /health reports status: "degraded", model_ready: false.

Events

transcript.partial

Emitted while the user is still speaking. Confidence is derived per segment as exp(min(0, avg_logprob)), averaged, and reported here as stability.

Field	Type	Description
text	string	Stabilized partial transcript.
stability	number	Averaged segment confidence.
language	string	Always "en".
start_ms	number	Always 0.
end_ms	number	Buffered utterance length in ms.
chunk_index	number	Index of the last ingested chunk.
is_final	boolean	Always false.

{
  "text": "hello there",
  "stability": 0.91,
  "language": "en",
  "start_ms": 0,
  "end_ms": 1200,
  "chunk_index": 20,
  "is_final": false
}

transcript.final

Emitted when an utterance ends with non-empty text.

Field	Type	Description
text	string	Final transcript.
language	string	Always "en".
confidence	number	Averaged segment confidence.
start_ms	number	Always 0.
end_ms	number	Utterance length in ms.
utterance_id	string	Utterance identifier (e.g. utt-1).
is_final	boolean	Always true.
finalize_reason	string	Wire reason: audio_end, silence_timeout, max_utterance_reached, disconnect_finalize, or force_finalize.

{
  "text": "hello there how are you",
  "language": "en",
  "confidence": 0.94,
  "start_ms": 0,
  "end_ms": 2400,
  "utterance_id": "utt-1",
  "is_final": true,
  "finalize_reason": "silence_timeout"
}

transcript.empty_final

Emitted instead of transcript.final when the utterance has no audio or produces empty text. Payload carries utterance_id, finalize_reason, is_final: true, and a detail of no_audio_detected or empty_transcript.

Configuration

All settings are STT_* environment variables, read into the frozen STTConfig in config.py. Timing settings accept a millisecond variable with a seconds fallback, and are clamped to a minimum.

Setting	Env var	Default	Purpose
model_name	STT_MODEL_NAME	large-v3	faster-whisper checkpoint to load.
device	STT_DEVICE	cuda	Inference device.
device_index	STT_DEVICE_INDEX	0	GPU index.
compute_type	STT_COMPUTE_TYPE	float16	Compute precision.
forced_language	STT_FORCED_LANGUAGE	en	The only language applied; forced on every decode and reported in every transcript.
default_language	STT_DEFAULT_LANGUAGE	en	Read but unused (dead config).
allow_auto_language	STT_ALLOW_AUTO_LANGUAGE	False	Read but unused; auto-detect not wired in.
sample_rate	STT_SAMPLE_RATE	16000	Required input PCM sample rate; mismatches rejected.
channels	STT_CHANNELS	1	Only mono accepted.
audio_format	STT_AUDIO_FORMAT	pcm16le	Required audio format; mismatches rejected.
codec	STT_CODEC	pcm	Advertised codec stored on the session.
decode_interval_sec	STT_DECODE_INTERVAL_MS (fallback STT_PARTIAL_REFRESH_INTERVAL)	0.06 (min 0.04)	Background decode-loop tick interval.
partial_interval_sec	STT_PARTIAL_INTERVAL_MS (fallback STT_PARTIAL_INTERVAL_SEC)	0.25 (min 0.04)	Minimum gap between partial decodes.
min_decode_audio_sec	STT_MIN_DECODE_AUDIO_MS (fallback STT_MIN_DECODE_AUDIO_SEC)	0.3 (min 0.1)	Minimum buffered speech before a partial or silence-timeout final.
silence_timeout_sec	STT_ENDPOINT_SILENCE_MS (fallback STT_SILENCE_TIMEOUT)	0.65 (min 0.2)	Trailing-silence duration that ends an utterance.
min_speech_sec	STT_MIN_SPEECH_MS (fallback STT_MIN_SPEECH_SEC)	0.15 (min 0.05)	Reported in /status and session.started; not used in endpointing.
max_utterance_sec	STT_MAX_UTTERANCE_MS (fallback STT_MAX_UTTERANCE_SEC)	30.0 (min 5.0)	Hard cap on utterance length; also caps the buffer and forces finalize.
partial_window_sec	STT_PARTIAL_WINDOW_MS (fallback STT_PARTIAL_WINDOW_SEC)	3.0 (min 0.5)	Trailing audio window decoded for partials.
idle_session_timeout_sec	STT_IDLE_TIMEOUT_MS (fallback STT_IDLE_TIMEOUT_SEC)	60.0 (min 5.0)	Idle time after which the loop cancels and closes the session.
max_concurrent_sessions	STT_MAX_CONCURRENT_SESSIONS	4	Cap on simultaneous in-memory sessions; over-cap raises GPU_OVERLOADED.
preroll_sec	STT_PREROLL_MS (fallback STT_PREROLL_SEC)	0.30 (min 0.2)	Lookback audio retained so speech onset is not clipped.
energy_vad_threshold	STT_ENERGY_VAD_THRESHOLD	0.02	RMS energy threshold for the per-chunk energy VAD.
use_model_vad	STT_USE_MODEL_VAD	True	Passed as vad_filter to faster-whisper.
beam_size	STT_BEAM_SIZE	5	Beam size for final decodes (partials use 1).
best_of	STT_BEST_OF	5	best_of sampling parameter.
temperature	STT_TEMPERATURE	0.0	Decoding temperature.
word_timestamps	STT_WORD_TIMESTAMPS	False	Whether word-level timestamps are returned.

warning

STT_DEVICE defaults to cuda with float16 compute — the service expects a CUDA GPU. If the model fails to load, the service stays up in a degraded state and rejects session.start with MODEL_NOT_READY.