Rename phy-z-srv-gpu01 to phy-srv-gpu01; fix share analysis scripts

- rename host/group/folder everywhere to match the server's actual
  hostname and physical label
- share-analysis.ps1: sanitize the output filename prefix — '-Paths "D:"'
  produced 'D:-file-types.csv' and Export-Csv failed with 'path format
  not supported', so no CSVs were written
- share-analysis.ps1: new -FolderDepth so folders can be aggregated at
  D:\Abteilungen\<Share> level, which matches the share layout
- list-shares.ps1: -WithSize walks local paths instead of UNC when run on
  the server itself (UNC was orders of magnitude slower and looked stuck)
  and prints progress every 50k files

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-09-03 14:47:07 +02:00
co-authored by Claude Opus 4.8
parent 9c38c60783
commit 75296ce58c
16 changed files with 81 additions and 34 deletions
@@ -1,11 +1,11 @@
# phy-z-srv-gpu01
# phy-srv-gpu01
GPU server for AI/ML workloads. Hardware is ordered/assessed; OS setup and configuration are the next step.
| | |
| - | - |
| IP | 192.168.66.69 (planned) |
| Ansible group | `phy_z_srv_gpu01` |
| Ansible group | `phy_srv_gpu01` |
| Status | planned — not yet configured |
## Hardware
@@ -1,4 +1,4 @@
# Projektplan — Setup phy-z-srv-gpu01 (LLM-Server mit SMB-Wissensbasis)
# Projektplan — Setup phy-srv-gpu01 (LLM-Server mit SMB-Wissensbasis)
Date: 2026-07-10 (Stand-Update: 2026-09-03)
Status: **In Umsetzung** — Server geliefert, Basis-Setup + GPU-Treiber fertig
@@ -144,7 +144,7 @@ und Indexgröße (§2.6).
### 2.4 Ansible vorbereiten (im Repo, testbar ohne GPU)
- `run.yml`: Play für `phy_z_srv_gpu01` ergänzen
- `run.yml`: Play für `phy_srv_gpu01` ergänzen
- Rollen-Skelett unter `ansible/roles/` (Details erst bei Umsetzung):
- `nvidia_gpu` — Treiber (≥ 580), Container Toolkit, optional MIG, DCGM
- `cifs_mounts` — ro-Mounts, Credentials aus `group_vars/secrets.yml`
@@ -152,7 +152,7 @@ und Indexgröße (§2.6).
Embedding-Server, Open WebUI + pgvector, Reverse Proxy (TLS), oikb-Timer
- Bestehendes nachnutzen: `geerlingguy.security` (Basis-Härtung),
`geerlingguy.docker`
- `group_vars/phy_z_srv_gpu01.yml`: nur Overrides (Modellname, VRAM-Quote,
- `group_vars/phy_srv_gpu01.yml`: nur Overrides (Modellname, VRAM-Quote,
Share-Liste, LDAP-Parameter)
- Alles außer GPU-Rolle ist vorab in einer Wegwerf-VM testbar (`just run`-Pfad)
@@ -210,7 +210,7 @@ wenn der Vollindex schlechte Treffer liefert.
| Phase | Inhalt | Ergebnis/Abnahme |
|---|---|---|
| **1. Basis** (Woche 1) | Rack/Strom (600-W-GPU!), iLO, Firmware, RAID, Ubuntu 24.04 LTS, Eintrag in Ansible-Basis-Setup (Security, Pakete, Nutzer) | `just run phy_z_srv_gpu01` läuft grün |
| **1. Basis** (Woche 1) | Rack/Strom (600-W-GPU!), iLO, Firmware, RAID, Ubuntu 24.04 LTS, Eintrag in Ansible-Basis-Setup (Security, Pakete, Nutzer) | `just run phy_srv_gpu01` läuft grün |
| **2. GPU-Stack** (Woche 12) | Rolle `nvidia_gpu`: Treiber, Container Toolkit, DCGM; **Burn-in unter Dauerlast** (SM120-Risiken, Deep Dive §1.6); MIG erst mal **aus** | `nvidia-smi` ok, 48 h-Lasttest ohne Reset |
| **3. Inference** (Woche 2) | vLLM-Container (NVIDIA-Build) mit gewähltem Modell, feste VRAM-Quote, Embedding-Server daneben; Benchmark mit Eval-Set | deutsche Antworten ok, Ziel-Parallelität erreicht |
| **4. UI + Auth** (Woche 23) | Open WebUI + pgvector, LDAP-Login, Reverse Proxy + TLS, vorkonfigurierter „Phytron-Assistent" | Login mit AD-Konto, Chat läuft |
@@ -16,8 +16,20 @@
.\list-shares.ps1
.EXAMPLE
# remote, and with size per share (slow on large shares)
.\list-shares.ps1 -ComputerName Z-FILESERVER -WithSize
# with size per share — run this ON the file server, it then walks the local
# paths instead of the UNC paths (far faster). Still minutes to hours on
# millions of files; progress is printed every 50k files.
.\list-shares.ps1 -WithSize
.NOTES
Faster alternative when all shares live under one drive (as on Z-FILESERVER,
where everything sits under D:): run share-analysis.ps1 once with a matching
folder depth instead of measuring every share separately, e.g.
.\share-analysis.ps1 -Paths "D:" -FolderDepth 2
That produces size/count per D:\Abteilungen\<Share> in a single pass over
the disk, which is what the share list here maps onto.
#>
param(
[string]$ComputerName = $env:COMPUTERNAME,
@@ -55,13 +67,28 @@ $result = foreach ($sh in $shares) {
$sizeGB = $null; $fileCount = $null
if ($WithSize) {
Write-Host " measuring size ..." -ForegroundColor DarkGray
try {
$m = Get-ChildItem -LiteralPath "\\$ComputerName\$($sh.Name)" -Recurse -File -Force -ErrorAction SilentlyContinue |
Measure-Object -Property Length -Sum
$sizeGB = [Math]::Round($m.Sum / 1GB, 2)
$fileCount = $m.Count
} catch { }
# Walk the LOCAL path when we are on the server itself — going through
# the UNC path (\\server\share) routes every single file through the SMB
# stack and is slower by orders of magnitude on large shares.
$scanPath = if ($ComputerName -eq $env:COMPUTERNAME -and $sh.Path) {
$sh.Path
} else {
"\\$ComputerName\$($sh.Name)"
}
$sw = [System.Diagnostics.Stopwatch]::StartNew()
$n = 0L; $bytes = 0L
Get-ChildItem -LiteralPath $scanPath -Recurse -File -Force -ErrorAction SilentlyContinue |
ForEach-Object {
$n++; $bytes += $_.Length
if ($n % 50000 -eq 0) {
Write-Host (" ... {0:N0} files, {1:N1} GB ({2:N0}s)" -f $n, ($bytes / 1GB), $sw.Elapsed.TotalSeconds) -ForegroundColor DarkGray
}
}
$sw.Stop()
$sizeGB = [Math]::Round($bytes / 1GB, 2)
$fileCount = $n
Write-Host (" {0:N0} files, {1:N1} GB in {2:N0}s" -f $n, ($bytes / 1GB), $sw.Elapsed.TotalSeconds) -ForegroundColor DarkGray
}
[PSCustomObject]@{
@@ -26,7 +26,11 @@ param(
[string]$OutDir = (Join-Path (Get-Location) ("share-analysis-" + (Get-Date -Format "yyyyMMdd-HHmmss"))),
# PDFs sampled per share for the text-layer check
[int]$PdfSampleSize = 200
[int]$PdfSampleSize = 200,
# How many folder levels to aggregate in the *-toplevel-folders.csv.
# 1 = D:\Abteilungen, 2 = D:\Abteilungen\AA (matches the share layout here).
[int]$FolderDepth = 1
)
$ErrorActionPreference = 'Continue'
@@ -80,7 +84,17 @@ function Test-PdfTextLayer {
# --- per-share pass ----------------------------------------------------------
foreach ($root in $Paths) {
# Label used for the output filenames. Must not contain characters that are
# illegal in a filename — e.g. -Paths "D:" would otherwise produce
# "D:-file-types.csv" and Export-Csv fails with "path format not supported".
$shareName = ($root.TrimEnd('\') -split '[\\/]')[-1]
if (-not $shareName) { $shareName = 'root' }
foreach ($c in [System.IO.Path]::GetInvalidFileNameChars()) {
$shareName = $shareName.Replace($c, '_')
}
$shareName = $shareName.TrimEnd('_', '.', ' ')
if (-not $shareName) { $shareName = 'root' }
Write-Host "=== Analyzing '$root' ..." -ForegroundColor Cyan
if (-not (Test-Path -LiteralPath $root)) {
@@ -118,9 +132,15 @@ foreach ($root in $Paths) {
$extStats[$ext].Count++
$extStats[$ext].Bytes += $_.Length
# top-level folder relative to the share root
# folder relative to the share root, aggregated at -FolderDepth levels
$rel = $_.FullName.Substring($rootLen).TrimStart('\')
$top = if ($rel.Contains('\')) { $rel.Split('\')[0] } else { '(root)' }
$parts = $rel.Split('\')
if ($parts.Count -le 1) {
$top = '(root)'
} else {
$n = [Math]::Min($FolderDepth, $parts.Count - 1)
$top = ($parts[0..($n - 1)] -join '\')
}
if (-not $topStats.ContainsKey($top)) { $topStats[$top] = @{ Count = 0L; Bytes = 0L } }
$topStats[$top].Count++
$topStats[$top].Bytes += $_.Length