CubelaPetarandClaude Opus 4.8
b6b242196d
Follows the homelab pattern: ironicbadger.docker_compose_generator v2 renders services/<host>/NN-<stack>/compose.yml templates into ~/docker/compose.yaml on the host. - 01-vllm: chat model, fixed --gpu-memory-utilization - 02-embeddings: second vLLM instance (--task embed) rather than a separate toolchain, so SM120 support only has to be solved once - 03-openwebui: Open WebUI + pgvector (not chroma — corpus size) - 99-network: shared bridge; leading comment keeps networks: top-level - pin docker_compose_generator to 2.0.1 — galaxy tags mix v1/v2 formats - group_vars: stack config incl. LDAP placeholders still to be filled The role only writes the compose file; starting the stack stays manual. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
40 lines
1.5 KiB
Markdown
40 lines
1.5 KiB
Markdown
# Compose stacks for phy-srv-gpu01
|
|
|
|
Rendered by `ironicbadger.docker_compose_generator` (pinned to 2.0.1) into
|
|
`{{ docker_compose_generator_output_path }}/compose.yaml` on the host.
|
|
The fragments here are **Jinja2 templates**, not plain compose files — Ansible
|
|
variables are substituted at render time.
|
|
|
|
| Stack | Contains |
|
|
| --- | --- |
|
|
| `01-vllm` | vLLM, serves the chat model on port 8000 (OpenAI API) |
|
|
| `02-embeddings` | second vLLM instance, embedding model on port 8001 |
|
|
| `03-openwebui` | Open WebUI (port 80) + pgvector Postgres |
|
|
| `99-network` | the shared `llmnet` bridge — must sort last |
|
|
|
|
Ordering comes from the `NN-` prefix; `99-network` keeps `networks:` at the
|
|
bottom of the generated file. The leading comment in `99-network/compose.yml`
|
|
is load-bearing: the role indents only the first line of each fragment, and the
|
|
comment absorbs that indentation so `networks:` stays top-level.
|
|
|
|
## Deploy
|
|
|
|
```sh
|
|
just compose phy_srv_gpu01 # renders ~/docker/compose.yaml on the host
|
|
```
|
|
|
|
The role only **writes** the file. Bringing the stack up is deliberate:
|
|
|
|
```sh
|
|
ssh sbxadmin@192.168.66.69 'cd ~/docker && docker compose up -d'
|
|
```
|
|
|
|
## Before the first run
|
|
|
|
- Secrets must exist in `group_vars/secrets.yml` (`just vault edit`):
|
|
`vault_owui_secret_key`, `vault_owui_db_password`, `vault_ldap_bind_password`
|
|
- LDAP parameters in `group_vars/phy_srv_gpu01.yml` need the real bind DN and
|
|
base DN from the customer
|
|
- `vllm_image` must point at a build that works on SM120 (Blackwell) — verify
|
|
before deploying, the official image is not guaranteed to work
|