Record model decision (Qwen3-32B FP8) and vLLM SM120 caveat
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -26,7 +26,8 @@ Server is delivered; base setup + NVIDIA driver are done (see `SETUP.md`).
|
||||
- [ ] Role `llm_stack` — docker compose: vLLM (fixed `--gpu-memory-utilization`),
|
||||
embedding server, Open WebUI + pgvector, reverse proxy, oikb timer.
|
||||
Tag compose tasks with `compose` so `just compose` works.
|
||||
- [ ] Pin versions at install time (vLLM / Open WebUI / oikb / model) per §5
|
||||
- [ ] Smoke-test a vLLM container on the server (official images may not support SM120)
|
||||
— pin the working image, then build `llm_stack`. Start model: Qwen3-32B FP8
|
||||
- [ ] OCR pipeline for ~166k scanned PDFs (§2.1) — own work block, competes with vLLM for the GPU
|
||||
- [ ] TLS: currently plain HTTP on `chat.phytron.local`; retrofit an internal CA
|
||||
certificate (AD passwords travel in clear text until then)
|
||||
|
||||
Reference in New Issue
Block a user