Mission#

Install Ollama on the home server, configure it as a systemd user unit so it auto-starts on boot, pull a small set of starter models that fit comfortably in the RTX 4060's 8GB VRAM.

Steps#

1. Verify GPU is usable: nvidia-smi returns sensible output (driver + CUDA version). If not, fail loud and surface what's needed.
2. Install Ollama (one-line script: curl -fsSL https://ollama.com/install.sh | sh).
3. Move it under systemd user control:
   • Disable any default system-level service if present (it should not run as root).
   • Create ~/.config/systemd/user/ollama.service that ExecStart's ollama serve on 127.0.0.1:11434, environment includes OLLAMA_HOST=127.0.0.1:11434 and OLLAMA_KEEP_ALIVE=10m.
   • systemctl --user enable --now ollama.service.
4. Pull starter models:
   • qwen2.5:7b-instruct-q4_K_M
   • qwen2.5-coder:7b-q4_K_M
   • nomic-embed-text
5. Smoke test: ollama run qwen2.5:7b-instruct-q4_K_M 'reply with the single word: ok' returns ok in under 5s.
6. Verify GPU is actually being used: during the smoke run, nvidia-smi shows the model in VRAM. Document the VRAM footprint observed for each model.

Acceptance criteria#

• ollama.service is active (running) under systemctl --user
• Survives a reboot (test with sudo systemctl reboot is fine — schedule it for a quiet moment, or simulate via systemctl --user restart)
• All 3 starter models pulled and listed in ollama list
• Smoke test against qwen2.5:7b-instruct-q4_K_M passes
• nvidia-smi confirms GPU offload during inference
• VRAM footprint of each model documented in MEMORY.md

Files / locations#

• Service unit: ~/.config/systemd/user/ollama.service
• Models: ~/.ollama/models/
• Logs: via journalctl --user -u ollama.service

Notes#

• Llama3.1:8b is intentionally NOT in the starter set; we'll add it later if Qwen disappoints on something.
• Don't change kernel/driver. If nvidia-smi is missing, stop and ask Maxime — driver install is a different conversation.