Domain Matching

Every avatar belongs to exactly one domain. The domain decides which base dataset seeds the training mix, which base system prompt is rewritten into the avatar's runtime prompt, which held-out questions evaluate the finished adapter, and which catalog name, description, and tag the avatar row inherits. domain/service.py owns that decision.

The catalog

state/config/domains.json is a flat list of domain objects. It currently holds 26 domains: cooking, cybersecurity, datascience, devops, digital_marketing, english_language_teaching, fitness, fitness_coaching, graphic_design, home_appliance_repair, ielts_essay, legal_consulting, math, mental, ml, nutrition_dietetics, parenting, photography, physiotherapy, programming, real_estate_consulting, relationships, science, sleep_stress, travel, web_development.

{
  "domain_id": "cooking",
  "name": "Cooking and Culinary Arts",
  "descriptions": [
    "Culinary expert providing recipes, cooking techniques, and meal preparation tips.",
    "Chef explaining flavor profiles, ingredient pairings, and kitchen safety.",
    "Specialist in various cuisines, baking, and efficient home cooking strategies."
  ],
  "metadata": {
    "vertical": "Lifestyle",
    "language_support": ["en", "es", "uk", "pl", "az", "tr"]
  }
}

Loading is strict and fails the whole catalog on the first problem: the file must be a non-empty list, every entry must be an object with a non-empty domain_id and name, ids must be unique, and each domain needs at least three descriptions. Missing metadata is defaulted to vertical: "General" and language_support: ["en"].

The parsed catalog is cached with lru_cache keyed by the resolved absolute path, so a running process picks up catalog edits only after a restart.

Derived avatar fields

build_avatar_catalog_fields turns a domain into the three catalog fields written to ml.avatars: name is the domain's display name, description is its first description, and tags is a single-element list holding the vertical (empty when no vertical is set). These are applied at run creation and again on domain confirmation, and are backfilled during schema initialization for active avatars that are still missing them.

Matching

DomainCore.suggest_domain(specialty_text, override) produces a DomainDecision.

Override path

A non-empty domain_id_override skips embedding entirely: the id is validated against the catalog and its dataset file must exist, then the decision is returned with confidence: 1.0, margin: 1.0, needs_confirmation: false, and a single top match.

Embedding path

1. Sanitize. The input is NFKC-normalized, control characters become spaces, whitespace is collapsed, and the result is lowercased. Empty input raises; input longer than DOMAIN_MAX_INPUT_LENGTH (256) raises.
2. Embed the descriptions once. Every description of every domain is embedded as passage: <description> and L2-normalized, then cached on the instance behind a lock. This happens lazily on the first match.
3. Embed the query as query: <normalized text> and normalize it.
4. Score. Each domain's score is the maximum dot product across its own descriptions — a domain wins on its single best-matching description, not on their average.
5. Rank by descending score, breaking ties on domain_id.
6. Gate. Auto-assignment requires both score >= DOMAIN_THRESHOLD and score - second_score >= DOMAIN_MARGIN_THRESHOLD. If either fails, domain_id is null, needs_confirmation is true, and the top three matches are returned for the creator to choose from.

The query:/passage: prefixes are the E5 asymmetric convention; the same model must be used for both sides.

Embedding transport

DomainCore._embed_texts batches by EMBEDDER_MAX_BATCH_SIZE (32) and tries two endpoint shapes per batch, in order:

1. POST {EMBEDDER_BASE_URL}/embeddings with {"model": …, "input": [...]} — OpenAI-compatible;
2. POST {EMBEDDER_BASE_URL}/embed with {"inputs": [...]} — native TEI.

Three response shapes are accepted: {"data": [{"embedding": …}]}, {"embeddings": [...]}, or a bare list. A vector-count mismatch is an error, and if no endpoint yields vectors the last error is raised as RuntimeError. There is no offline fallback — with the embeddings server down, run creation fails.

note

domain/matcher.py holds a second, module-level embed_texts() used for deduplication and style scoring. It differs in three ways: it prefixes all inputs with passage: for E5 models, truncates each input to EMBEDDER_MAX_INPUT_CHARS (900), and validates that vectors are non-empty, equal-length, and numeric. It also adapts its endpoint order to whether EMBEDDER_BASE_URL ends in /v1.

Avatar assignments

confirm_assignment records the decision in state/config/avatar_domain_mapping.json:

{
  "avatars": {
    "<avatar_id>": {
      "domain_id": "math",
      "dataset_file": "math_base.json",
      "source": "auto",
      "assigned_at": "2026-06-29T11:45:23.452978+00:00",
      "sample_count": 41
    }
  }
}

source is auto or manual, and nothing else is accepted. The stored dataset_file is the bare filename, never an absolute path, which is what keeps the mapping portable between machines. sample_count is recomputed from the dataset at write time as len(standalone) + len(rag_aware).

Writes are serialized by a module-level lock and are atomic: the payload goes to a per-process, per-thread temporary file that is then replace()d onto the target. Concurrent confirmations for different avatars are safe and covered by a test.

Configuration

Variable	Default	Purpose
DOMAIN_CATALOG_FILE	${RUNTIME_ROOT}/config/domains.json	Catalog location.
DOMAIN_DATASET_DIR	${RUNTIME_ROOT}/config/domain_datasets	Where {domain_id}_base.json files live.
AVATAR_MAP_FILE	${RUNTIME_ROOT}/config/avatar_domain_mapping.json	Avatar → domain assignments.
LORA_EVAL_QUESTIONS_DIR	${RUNTIME_ROOT}/config/lora_eval_questions	Held-out evaluation questions, one {domain_id}.json per domain.
DOMAIN_MODEL_NAME	falls back to EMBEDDER_MODEL	Model name sent in the OpenAI-compatible embedding request.
DOMAIN_THRESHOLD	0.85	Minimum top score for auto-assignment.
DOMAIN_MARGIN_THRESHOLD	0.03 in code; .env.example sets 0.01	Minimum gap to the runner-up.
DOMAIN_MAX_INPUT_LENGTH	256	Maximum sanitized specialty_text length.
EMBEDDER_BASE_URL	—	TEI base URL, e.g. http://127.0.0.1:8017/v1.
EMBEDDER_MODEL	—	intfloat/multilingual-e5-large in the shipped profile.
EMBEDDER_TIMEOUT_SEC	60	Per-request timeout.
EMBEDDER_MAX_BATCH_SIZE	32	Texts per embedding request.
EMBEDDER_MAX_INPUT_CHARS	900	Truncation limit in domain/matcher.py only; 0 disables it.

warning

Lowering DOMAIN_MARGIN_THRESHOLD to 0.01, as .env.example does, makes auto-assignment much more willing to pick between two near-identical domains — fitness vs fitness_coaching, or ml vs datascience. Raise it if creators report landing in an adjacent domain.

Configuration integrity

python -m domain.integrity --config-root state/config validates the whole configuration tree as one unit and prints one ERROR: line per problem, exiting non-zero if there are any.

Checks performed:

• domains.json parses as a list, and every entry has a unique non-empty domain_id;
• every catalogued domain has a {domain_id}_base.json dataset and a {domain_id}.json evaluation file;
• each dataset's internal domain_id matches its filename, and its declared sample_count matches the actual len(standalone) + len(rag_aware);
• each evaluation file has a non-empty questions list;
• no orphans in either direction — a dataset or evaluation file with no catalog entry is an error;
• avatar_domain_mapping.json contains an avatars object whose every record references a catalogued domain and stores the portable dataset_file name;
• bootstrap_checksums.json matches a freshly computed manifest on all five keys.

bootstrap_checksums.json records the SHA-256 of domains.json, of every dataset file, and of every evaluation file, plus both file counts. The mutable avatar mapping is deliberately excluded — it changes on every run.

Migration mode

--fix runs two normalizations before validating:

• normalize_avatar_mapping rewrites selected_domain_id to domain_id, replaces machine-specific dataset paths with bare filenames, and refreshes stored sample counts. It raises on a mapping that references an unknown domain or a missing dataset.
• write_checksum_manifest regenerates bootstrap_checksums.json.

Both write atomically through a temporary file. Use --fix after importing an old mapping file or after editing any domain asset.