Record model decision (Qwen3-32B FP8) and vLLM SM120 caveat

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-09-03 14:13:11 +02:00
co-authored by Claude Opus 4.8
parent be6efa7b34
commit 9c38c60783
2 changed files with 24 additions and 8 deletions
+2 -1
View File
@@ -26,7 +26,8 @@ Server is delivered; base setup + NVIDIA driver are done (see `SETUP.md`).
- [ ] Role `llm_stack` — docker compose: vLLM (fixed `--gpu-memory-utilization`),
embedding server, Open WebUI + pgvector, reverse proxy, oikb timer.
Tag compose tasks with `compose` so `just compose` works.
- [ ] Pin versions at install time (vLLM / Open WebUI / oikb / model) per §5
- [ ] Smoke-test a vLLM container on the server (official images may not support SM120)
— pin the working image, then build `llm_stack`. Start model: Qwen3-32B FP8
- [ ] OCR pipeline for ~166k scanned PDFs (§2.1) — own work block, competes with vLLM for the GPU
- [ ] TLS: currently plain HTTP on `chat.phytron.local`; retrofit an internal CA
certificate (AD passwords travel in clear text until then)