Architecture
15 videos and 427 segments. CPU media processing and remote inference share the original Video RAG interface. Text indexes, model identity and dimensions stay versioned; multimodal research views below require compatible historical indexes.
Runtime topology
Two supervised processes. Next.js exposes port 7860; FastAPI stays on loopback. Inference uses authorized remote HTTPS providers.
Build pipeline
Browser import creates a persisted job. One bounded CPU worker prepares media; remote transcription/embedding failures are visible and retryable.
Historical research · five embedding views
Historical multimodal design: only valid for a verified shared embedding space. Current remote text adapters cannot reproduce these views. The UI enables only actually loaded indexes.
All inputs share one 8,192-token window: ~30 frames × 140 + ~750 audio tokens + transcript fits. Media gets no task prefix; text uses title: … | text: ….
Query → answer
Search retrieves timestamped transcript evidence. Ask sends transcripts and optional bounded keyframes to a configured remote model; citations open playable clips.
model-specific input preparationMatryoshka truncation
Historical MRL illustration. Truncation is enabled only when the model/index manifest explicitly declares supported dimensions. Remote text defaults use their full dimension.
Re-normalising after slicing is required: a truncated unit vector is no longer unit length and silently skews cosine scores (vrag/index.py: truncate).
Evaluation loop
Historical evaluation workflow. Preserve model labels and source provenance; synthetic sample scores demonstrate UI only and are not measured quality.
Stack & files
Two Python environments keep vLLM's pinned torch/transformers away from the embedding stack.
| Embedding | google/embeddinggemma-2 · sentence-transformers 6.1 · torchcodec |
| Generator | google/gemma-4-12B-it-qat-w4a16-ct · vLLM 0.23 · MTP |
| ASR | Gemma 4 12B (default, ~100 s for 3.9 h) · Whisper large-v3-turbo (alt, ~35 min) |
| Index | numpy float32 · brute-force cosine |
| API | FastAPI · uvicorn · range-served media |
| Web | Next.js 16 · React 19 · Tailwind 4 · Recharts |
| Corpus | 15 videos · … · 427 segments |
vrag/ pipeline + API jobs.py bounded upload → CPU proxy → remote STT → index providers.py remote vision / detection / STT / embeddings mentor_api.py live sessions, consent and authorized routing settings.py atomic profile history and rollback backup/ encrypted snapshots and isolated restore index.py cosine, MRL truncate, RRF remote.py evidence → remote LLM → cited answer evalset.py synthetic spoken/visual queries experiments.py view × dim metrics grid api.py FastAPI for the site serve/ historical offline research; not cloud runtime web/ this site scripts/dev.sh CPU web + API only; no inference services