commit 6a103adac4a5b7709df4bd44b957ca1593de2c43 Author: Hans-Christian Payer Date: Thu May 21 10:36:16 2026 +0200 Erste lauffähige Version diff --git a/README.md b/README.md new file mode 100644 index 0000000..17bb9e1 --- /dev/null +++ b/README.md @@ -0,0 +1,547 @@ +# Serienbrief – DOCX‑Mailmerge mit Django, Celery & LibreOffice + +Web‑Anwendung zur Erstellung personalisierter Serienbriefe aus DOCX‑Vorlagen und CSV‑Empfängerlisten. Ergebnis ist ein zusammengeführtes PDF mit einem Brief pro Empfänger:in. + +Zielumgebung: internes LAN, hinter zentralem Reverse‑Proxy (TLS‑Terminierung extern). Produktionsnah konzipiert: Hardening, Resource‑Limits, Healthchecks, Backup‑Job. + +--- + +## Inhaltsverzeichnis + +1. [Funktionsumfang](#funktionsumfang) +2. [Architekturübersicht](#architekturübersicht) +3. [Tech‑Stack](#techstack) +4. [Komponenten im Detail](#komponenten-im-detail) +5. [Verzeichnisstruktur](#verzeichnisstruktur) +6. [Datenfluss eines Serienbrief‑Jobs](#datenfluss-eines-serienbriefjobs) +7. [Datenmodell](#datenmodell) +8. [Sicherheitskonzept](#sicherheitskonzept) +9. [Netzwerk & Ports](#netzwerk--ports) +10. [Volumes & Persistenz](#volumes--persistenz) +11. [Konfiguration](#konfiguration) +12. [Inbetriebnahme](#inbetriebnahme) +13. [Entwicklung in VS Code](#entwicklung-in-vs-code) +14. [Betrieb & Wartung](#betrieb--wartung) +15. [Datenschutz & Compliance](#datenschutz--compliance) +16. [Bekannte Einschränkungen & Roadmap](#bekannte-einschränkungen--roadmap) + +--- + +## Funktionsumfang + +- DOCX‑Vorlagen mit Jinja‑artigen Platzhaltern (`{{ feldname }}`) via **docxtpl** +- CSV‑Upload als Empfängerliste (UTF‑8, Komma‑Trenner) +- Asynchrone Verarbeitung über Celery — UI bleibt responsiv, Status‑Polling per HTMX +- DOCX → PDF Konversion mit headless **LibreOffice** +- Zusammenführung aller Einzel‑PDFs via **pypdf** +- Authentifizierung über Djangos Auth‑System, Brute‑Force‑Schutz mit **django‑axes** +- Geschütztes Ausliefern generierter PDFs via **X‑Accel‑Redirect** (Nginx) +- Audit‑Log pro Job (Status‑Wechsel, Fehler, abgearbeitete Zeilen) +- Tägliches Backup (pg_dump + Media‑Tar, 14 Tage Retention) + +--- + +## Architekturübersicht + +``` + ┌──────────────────────────────────────────────┐ + │ Externer LAN‑Proxy (Nginx, TLS‑Terminierung)│ + │ https://serienbrief.lan │ + └────────────────────┬─────────────────────────┘ + │ HTTP (intern) + ▼ + ┌───────────────────────────────────────────────────────────┐ + │ Compose‑Stack »serienbrief« │ + │ │ + │ ┌───────────┐ ┌──────────────┐ ┌──────────────────┐ │ + │ │ nginx │──▶│ web (Django) │──▶│ db (Postgres 16) │ │ + │ │ App‑Proxy │ │ Gunicorn │ │ │ │ + │ └─────┬─────┘ └──────┬───────┘ └──────────────────┘ │ + │ │ X‑Accel │ enqueue │ + │ │ static/media ▼ │ + │ │ ┌──────────────┐ ┌──────────────────┐ │ + │ └────────▶│ redis (Broker)│◀─│ worker (Celery) │ │ + │ └──────────────┘ │ LibreOffice+pypdf│ │ + │ └──────────────────┘ │ + │ ┌──────────────┐ ┌──────────────────┐ │ + │ │ beat (Cron) │ │ backup (pg_dump) │ │ + │ └──────────────┘ └──────────────────┘ │ + └───────────────────────────────────────────────────────────┘ +``` + +Zwei Bridge‑Netze trennen Verantwortlichkeiten: + +- **frontend** – nur `nginx` ↔ `web`. Einziges Netz mit Port‑Bind nach außen. +- **backend** – `web`, `worker`, `beat`, `db`, `redis`, `backup`. Kein direktes Internet‑/LAN‑Exposing. + +--- + +## Tech‑Stack + +| Schicht | Komponente | Version | +|---|---|---| +| Host‑OS | Ubuntu Server LTS | 22.04 / 24.04 | +| Container Runtime | Docker Engine | 27.x+ | +| Orchestrierung | Docker Compose v2 | — | +| App‑Framework | Django | 5.1.4 | +| WSGI (Prod) | Gunicorn | — | +| Task Queue | Celery | 5.4 | +| Broker / Result Backend | Redis | 7‑alpine | +| Datenbank | PostgreSQL | 16‑alpine | +| DOCX‑Templating | docxtpl | 0.19 | +| DOCX→PDF | LibreOffice (headless) | bundled in Image | +| PDF‑Merge | pypdf | 5.1 | +| App‑Proxy | Nginx | 1.27‑alpine | +| Auth‑Hardening | django‑axes | aktuell | +| UI‑Interaktion | django‑htmx | aktuell | +| DB‑Adapter | psycopg | 3.2.3 (binary) | +| Python | CPython | 3.12 (slim‑bookworm) | + +--- + +## Komponenten im Detail + +### `nginx` – App‑interner Reverse‑Proxy +- Terminiert **kein** TLS (übernimmt der externe Proxy) +- Liefert Static‑ und Media‑Dateien direkt aus den Volumes aus +- Setzt `X‑Accel‑Redirect` für Auth‑geschützte PDFs um: Django prüft die Berechtigung, Nginx streamt die Datei ohne Python im Pfad +- Hört auf `${APP_BIND_IP:-127.0.0.1}:${APP_BIND_PORT:-8080}` (Default Loopback) +- Healthcheck gegen `/healthz` + +### `web` – Django + Gunicorn +- Rolle `web` (per `ROLE`‑Env unterschieden, gleiches Image wie worker/beat) +- Stateless, read‑only Root‑FS, `/tmp` als tmpfs +- Settings‑Split: `base.py`, `dev.py`, `production.py` +- Entrypoint wartet auf DB‑Readiness, führt `migrate` und (in Prod) `collectstatic` aus +- Healthcheck via `/healthz` + +### `worker` – Celery +- Verarbeitet `process_mailmerge_job`‑Tasks +- LibreOffice‑User‑Profil im tmpfs (`/tmp/lo_profile_`) → keine Leaks zwischen Jobs +- `--concurrency=2`, `--max-tasks-per-child=50` (RAM‑Leak‑Schutz bei LibreOffice) + +### `beat` – Celery Scheduler +- Nutzt `DatabaseScheduler` von django‑celery‑beat +- Geplant für: Retention‑Cleanup (alte Jobs/PDFs), Audit‑Log‑Rotation +- *Hinweis: Konkrete Periodic Tasks aktuell noch nicht registriert (siehe Roadmap)* + +### `db` – PostgreSQL 16 +- Eigenes Volume `postgres_data` +- Passwort via Docker‑Secret (`/run/secrets/postgres_password`) +- Init‑Skripte unter `./postgres/init` (z.B. zusätzliche Extensions) +- `shm_size: 256mb` für komplexere Queries + +### `redis` – Broker & Result Backend +- Passwortgeschützt (`REDIS_PASSWORD` aus `.env`) +- AOF persistence + RDB snapshot (`save 900 1`) +- `maxmemory 256mb` mit `allkeys‑lru` + +### `backup` – nightly pg_dump + media tar +- Custom‑Loop mit `sleep 86400` (kein systemd nötig) +- pg_dump im Custom‑Format (`-Fc`) → `pg_restore`‑kompatibel +- Media‑Tar aus `media_files`‑Volume (read‑only Mount) +- Retention 14 Tage (konfigurierbar) +- Schreibt nach `./backups` auf dem Host + +--- + +## Verzeichnisstruktur + +``` +serienbrief/ +├── app/ # Django‑Projekt +│ ├── Dockerfile # Multi‑Stage: builder → runtime‑base → runtime / dev +│ ├── entrypoint.sh # wait_for_db → migrate → collectstatic → exec CMD +│ ├── manage.py +│ ├── requirements.txt +│ ├── requirements-dev.txt # debugpy, ipython, ruff, pytest, debug-toolbar +│ ├── config/ # Project package +│ │ ├── celery.py +│ │ ├── urls.py +│ │ ├── wsgi.py +│ │ └── settings/ +│ │ ├── base.py +│ │ ├── dev.py +│ │ └── production.py +│ ├── mailmerge/ # Hauptapp +│ │ ├── models.py # LetterTemplate, MailMergeJob, JobLogEntry +│ │ ├── views.py +│ │ ├── admin.py +│ │ ├── forms.py +│ │ ├── urls.py +│ │ ├── tasks.py # Celery tasks +│ │ ├── services/ +│ │ │ ├── docx_renderer.py # docxtpl + LibreOffice +│ │ │ └── pdf_merge.py # pypdf +│ │ └── management/commands/ +│ │ └── wait_for_db.py +│ └── templates/ +│ ├── base.html +│ ├── registration/login.html +│ └── mailmerge/… +├── nginx/ +│ ├── nginx.conf +│ └── conf.d/serienbrief.conf +├── postgres/init/ # SQL init scripts +├── secrets/ +│ └── postgres_password.txt # 0600, NICHT in Git +├── backups/ # nightly dumps + media tars +├── docker-compose.yml # Produktion +├── docker-compose.override.yml # Dev (automatisch gemerged) +├── .env # NICHT in Git +├── .env.example +├── .devcontainer/ # VS Code Dev Container +├── .vscode/ # launch / tasks / settings / extensions +└── README.md +``` + +--- + +## Datenfluss eines Serienbrief‑Jobs + +1. **Upload Vorlage:** `LetterTemplate` mit DOCX‑Datei wird angelegt (Admin oder UI) +2. **Job‑Erstellung:** User wählt Vorlage + lädt CSV hoch → `MailMergeJob(status=queued)` +3. **Enqueue:** View löst `process_mailmerge_job.delay(job.pk)` aus +4. **Worker holt Task:** + - Liest CSV (UTF‑8, validiert Header gegen Platzhalter) + - Iteriert Zeilen → docxtpl rendert pro Zeile eine temporäre DOCX in `/tmp` + - LibreOffice (`soffice --headless --convert-to pdf …`) erzeugt pro DOCX ein PDF + - pypdf merged alle PDFs in eine Ausgabedatei unter `media_files` + - `JobLogEntry`s pro Schritt; Status `running → done` bzw. `failed` +5. **UI‑Polling:** HTMX fragt `/jobs//status/` alle ~2 s ab → Fortschrittsbalken +6. **Download:** Nutzer klickt Download → View prüft Auth → Antwort enthält `X‑Accel‑Redirect: /protected/…` → Nginx streamt das PDF + +--- + +## Datenmodell + +```text +LetterTemplate +├── id, name (unique), description +├── docx_file (FileField → media/templates/) +├── created_by (FK User), created_at, updated_at +└── is_active + +MailMergeJob +├── id, template (FK LetterTemplate) +├── csv_file (FileField → media/jobs//input.csv) +├── output_pdf (FileField → media/jobs//output.pdf) +├── status: queued | running | done | failed +├── total_rows, processed_rows +├── created_by, created_at, started_at, finished_at +└── error_message + +JobLogEntry +├── id, job (FK MailMergeJob, related_name="log_entries") +├── level: info | warning | error +├── message, created_at +``` + +--- + +## Sicherheitskonzept + +Im Sinne der Space‑Vorgabe »Security‑by‑Default«: + +### Container‑Hardening +- `no-new-privileges: true` bei allen Services +- `read_only: true` für `web`, `worker`, `beat` (writable nur via tmpfs `/tmp`) +- Non‑root‑User `app` (UID/GID 1000, anpassbar per Build‑Args) +- Minimales Image (`python:3.12-slim-bookworm`), Multi‑Stage‑Build trennt Build‑ und Laufzeit‑Dependencies +- Ressourcen‑Limits (memory + cpus) auf allen Services +- `tmpfs`‑Größen explizit limitiert (`size=256M`, etc.) + +### Netzwerk +- Zwei isolierte Bridges (`frontend`, `backend`) +- Nur `nginx` bindet nach außen, default auf **Loopback** +- DB und Redis ausschließlich auf `backend` +- Externer Proxy übernimmt TLS, HSTS, ggf. IP‑Allowlist + +### Secrets +- Postgres‑Passwort via Docker‑Secret (`./secrets/postgres_password.txt`, Mode 0600) +- Redis‑Passwort + Django‑Secret‑Key in `.env` (nicht in Git) +- `.env` und `secrets/` sind in `.gitignore` ausgenommen + +### Anwendungs‑Sicherheit +- django‑axes: Lockout nach n Fehlversuchen +- CSRF aktiv, Trusted Origins explizit gepflegt +- `SECURE_PROXY_SSL_HEADER` gesetzt, damit Django HTTPS hinter dem externen Proxy korrekt erkennt +- `USE_X_FORWARDED_HOST=True` +- X‑Accel‑Redirect statt direktem File‑Download → Nginx liefert nur, was Django freigibt +- Session‑Cookies `HttpOnly`, `SameSite=Lax`, `Secure` (in Prod) + +### Defense‑in‑Depth (Empfehlungen) +- AppArmor‑Profile pro Container (z.B. via `--security-opt apparmor=...`) – aktuell nicht aktiv +- seccomp‑Profile (Default reicht, custom optional) +- Image‑Scanning (`trivy`, `grype`) in CI integrieren +- Renovate/Dependabot für Base‑Image und Python‑Deps + +--- + +## Netzwerk & Ports + +| Port (Host) | Service | Sichtbarkeit | Zweck | +|---|---|---|---| +| 8080 | nginx | `${APP_BIND_IP}` | HTTP für externen Proxy | +| 8000 | web | nur Dev | Django runserver | +| 5678 | web (Dev) | nur Dev | debugpy | +| 5432 | db | nur Dev | Postgres (Tools wie pgAdmin) | +| 6379 | redis | nur Dev | Redis‑CLI | + +In Produktion bleiben **nur** Port 8080 (intern an externen Proxy) erreichbar. + +--- + +## Volumes & Persistenz + +| Volume | Inhalt | Backup? | +|---|---|---| +| `postgres_data` | Postgres‑Daten | pg_dump nightly | +| `redis_data` | AOF + RDB | nein (Broker‑State, regenerierbar) | +| `static_files` | Generierte Static‑Files | nein (regenerierbar via `collectstatic`) | +| `media_files` | Hochgeladene Templates, CSVs, generierte PDFs | tar nightly | +| `nginx_cache` | Nginx Cache | nein | +| `nginx_run` | Nginx PID/Socket | nein | + +Bind‑Mount im Dev: `./app:/app` für Live‑Reload. + +--- + +## Konfiguration + +`.env` (aus `.env.example` abgeleitet) — Auszug: + +```env +# App +APP_VERSION=1.0.0 +APP_BIND_IP=127.0.0.1 +APP_BIND_PORT=8080 +APP_UID=1000 +APP_GID=1000 + +# Django +DJANGO_SECRET_KEY= +DJANGO_DEBUG=False +DJANGO_ALLOWED_HOSTS=serienbrief.lan,127.0.0.1,localhost +CSRF_TRUSTED_ORIGINS=https://serienbrief.lan +SECURE_PROXY_SSL_HEADER=HTTP_X_FORWARDED_PROTO,https +USE_X_FORWARDED_HOST=True + +# Postgres +POSTGRES_DB=serienbrief +POSTGRES_USER=serienbrief +DATABASE_URL=postgres://serienbrief:@db:5432/serienbrief + +# Redis +REDIS_PASSWORD= +CELERY_BROKER_URL=redis://:@redis:6379/0 +CELERY_RESULT_BACKEND=redis://:@redis:6379/1 + +# Business +JOB_RETENTION_DAYS=30 +``` + +**Wichtig:** +- `secrets/postgres_password.txt` muss exakt denselben Wert enthalten wie in `DATABASE_URL` +- Datei darf **kein** abschließendes Newline enthalten — Generierung: `openssl rand -hex 32 | tr -d '\n' > secrets/postgres_password.txt && chmod 600 secrets/postgres_password.txt` +- Hex statt Base64, weil Base64‑`=` URL‑Parsing in `DATABASE_URL` bricht + +--- + +## Inbetriebnahme + +### Voraussetzungen +- Ubuntu 22.04/24.04 LTS, aktueller Patch‑Stand +- Docker Engine 27.x+ und Compose v2 (apt‑Pakete `docker-ce` + `docker-compose-plugin`, nicht Snap) +- Host‑User in `docker`‑Gruppe (`sudo usermod -aG docker $USER`, dann Re‑Login) + +### Erststart + +```bash +# 1. Repo holen / entpacken in z.B. ~/projekte/serienbrief +cd ~/projekte/serienbrief + +# 2. Konfiguration aus Template ableiten +cp .env.example .env +$EDITOR .env # Werte ausfüllen, siehe oben + +# 3. Postgres‑Passwort als Datei +openssl rand -hex 32 | tr -d '\n' > secrets/postgres_password.txt +chmod 600 secrets/postgres_password.txt +# → diesen Wert auch in DATABASE_URL eintragen + +# 4. Image bauen +docker compose build + +# 5. Hochfahren +docker compose up -d + +# 6. Status & Logs +docker compose ps -a +docker compose logs -f web + +# 7. Superuser anlegen +docker compose exec web python manage.py createsuperuser +``` + +### Externen Proxy konfigurieren (Beispiel) + +```nginx +server { + listen 443 ssl http2; + server_name serienbrief.lan; + + ssl_certificate /etc/ssl/lan/serienbrief.crt; + ssl_certificate_key /etc/ssl/lan/serienbrief.key; + + location / { + proxy_pass http://:8080; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto https; + proxy_redirect off; + client_max_body_size 50M; # für CSV/DOCX‑Uploads + } +} +``` + +--- + +## Entwicklung in VS Code + +- `.devcontainer/devcontainer.json` referenziert den Compose‑Stack mit Dev‑Override +- `.vscode/launch.json` enthält Configs: + - **Django: runserver (attach)** → debugpy auf 5678 + - **Django: pytest** → führt Tests im Container aus +- `.vscode/tasks.json` für `makemigrations`, `migrate`, `shell`, `collectstatic` +- `.vscode/extensions.json` empfiehlt: Python, Pylance, Django, Docker, Even Better TOML, Ruff + +### Workflow + +```bash +# Stack hochfahren (Override greift automatisch) +docker compose up -d + +# Code‑Änderungen werden via Bind‑Mount sofort sichtbar +# runserver reloaded automatisch + +# Debugging: F5 in VS Code → attach an debugpy:5678 +``` + +### Wichtige Unterschiede Dev ↔ Prod + +| Aspekt | Dev | Prod | +|---|---|---| +| WSGI | `runserver` | Gunicorn | +| Code | Bind‑Mount `./app:/app` | im Image | +| Healthcheck | deaktiviert | aktiv (`/healthz`) | +| `read_only` | aus | an | +| `DJANGO_DEBUG` | True | False | +| `collectstatic` | auto im Entrypoint | im Entrypoint | +| `target` | `dev` (mit debugpy, ipython, pytest) | `runtime` | + +--- + +## Betrieb & Wartung + +### Häufige Operationen + +```bash +# Logs eines einzelnen Services +docker compose logs -f worker + +# In den Container schauen +docker compose exec web bash +docker compose exec db psql -U serienbrief + +# Migration nach Modelländerung +docker compose exec web python manage.py makemigrations +docker compose exec web python manage.py migrate + +# Statics neu generieren +docker compose exec web python manage.py collectstatic --noinput + +# Celery‑Queue Status +docker compose exec worker celery -A config inspect active +docker compose exec worker celery -A config inspect reserved +``` + +### Updates + +```bash +# Code aktualisieren +git pull + +# Image neu bauen +docker compose build --pull + +# Mit Zero‑Downtime nicht möglich (single‑host) → kurzer Restart +docker compose up -d + +# DB‑Migrationen +docker compose exec web python manage.py migrate +``` + +### Backup & Restore + +```bash +# Manuelles Backup erzwingen +docker compose restart backup +docker compose logs --tail=50 backup + +# Restore (Beispiel) +docker compose exec -T db pg_restore -U serienbrief -d serienbrief --clean < backups/db_20260521_030000.dump +``` + +### Monitoring (Empfehlung, nicht enthalten) +- `docker stats` für ad‑hoc Container‑Ressourcen +- Healthcheck‑Status: `docker compose ps` +- Container‑Logs via zentralem syslog/journald (Docker `json-file` rotiert 5×10 MB) +- Optional: Prometheus + cAdvisor + node_exporter, Loki für Logs + +--- + +## Datenschutz & Compliance + +- **Rechtsgrundlage:** Verarbeitung personenbezogener Adress‑/Empfängerdaten je nach Anwendungsfall Art. 6 Abs. 1 lit. b/c/e DSGVO; bei besonderen Datenkategorien (Art. 9 DSGVO) gesonderte Rechtsgrundlage erforderlich +- **Speicherort:** Innerbetrieblicher Server, kein Cloud‑Transfer +- **Zugriffssteuerung:** Nur authentifizierte User; Trennung Templates/Jobs aktuell flach — Mandanten‑/Rollenmodell ggf. ergänzen (siehe Roadmap) +- **Aufbewahrung:** Jobs + CSVs werden nach `JOB_RETENTION_DAYS` (default 30) gelöscht — Cleanup‑Task noch zu implementieren (siehe Roadmap) +- **Audit:** Jeder Job hat ein `JobLogEntry`‑Protokoll (wer, wann, was, Fehler) +- **Backup‑Verschlüsselung:** Bind‑Mount `./backups` liegt aktuell unverschlüsselt — für produktiven Einsatz LUKS/Filesystem‑Encryption am Host **dringend empfohlen** +- **DSFA‑Relevanz:** Bei Verarbeitung besonderer Datenkategorien oder umfangreicher Profilbildung ist eine Datenschutz‑Folgenabschätzung gemäß Art. 35 DSGVO durchzuführen, bevor produktiver Echtbetrieb startet + +--- + +## Bekannte Einschränkungen & Roadmap + +### Tech‑Debt +- Postgres‑Passwort doppelt gehalten (`DATABASE_URL` + Secret‑Datei) → refactor zu Single Source via Entrypoint, der `DATABASE_URL` aus `*_FILE`‑Env zusammenbaut +- Keine LDAP/AD‑Anbindung — aktuell lokale Django‑User +- Keine Periodic Tasks im Beat registriert (Retention‑Cleanup, Log‑Rotation) +- CSV‑Header‑Validierung ist warn‑only, kein Abbruch bei fehlenden Pflichtfeldern +- Keine Vorschau (Single‑Letter‑Preview) vor dem Komplett‑Render +- Image enthält LibreOffice (~400 MB) — könnte in separates Worker‑Image ausgelagert werden + +### Roadmap (kurzfristig) +- [ ] Single‑Letter‑Preview im Job‑Erstellungs‑Flow +- [ ] Retention‑Cleanup als periodic task +- [ ] CSV‑Validierung strict mit Pflichtfeldliste pro Template +- [ ] Mandanten‑/Berechtigungsmodell (Abteilung X sieht nur eigene Templates) +- [ ] LDAP‑Auth über `django‑auth‑ldap` (AD‑Integration GB) + +### Roadmap (mittelfristig) +- [ ] AppArmor‑Profile pro Container +- [ ] Image‑Scanning in CI (Trivy) +- [ ] Separate Worker‑Image ohne Web‑Dependencies +- [ ] AES‑Verschlüsselung der hochgeladenen CSVs at rest +- [ ] Optional: Konvertierung über Gotenberg statt LibreOffice‑im‑Image (cleaneres Separation of Concerns) + +--- + +## Lizenz & Verantwortlichkeit + +Internes Projekt. Verantwortlich für Konzept & Betrieb: **Datenschutzkoordination / IT‑Systemadministration**. + +Keine externe Lizenz festgelegt — Verteilung ausschließlich innerbetrieblich. diff --git a/app/Dockerfile b/app/Dockerfile new file mode 100644 index 0000000..e07eb0a --- /dev/null +++ b/app/Dockerfile @@ -0,0 +1,86 @@ +# ============================================================================= +# Multi-Stage: builder → dev → runtime +# Dev-Image enthält zusätzlich debugpy, ipython, django-debug-toolbar. +# ============================================================================= + +# ---------- Stage 1: Build ---------------------------------------------------- +FROM python:3.12-slim-bookworm AS builder + +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 \ + PIP_NO_CACHE_DIR=1 \ + PIP_DISABLE_PIP_VERSION_CHECK=1 + +RUN apt-get update && apt-get install -y --no-install-recommends \ + build-essential libpq-dev gcc \ + && rm -rf /var/lib/apt/lists/* + +WORKDIR /build +COPY requirements.txt requirements-dev.txt ./ +RUN pip wheel --wheel-dir=/wheels -r requirements.txt -r requirements-dev.txt + + +# ---------- Common Runtime Base ---------------------------------------------- +FROM python:3.12-slim-bookworm AS runtime-base + +ARG APP_UID=1000 +ARG APP_GID=1000 + +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 \ + PIP_NO_CACHE_DIR=1 \ + PYTHONPATH=/app + +RUN apt-get update && apt-get install -y --no-install-recommends \ + libpq5 \ + libreoffice libreoffice-writer \ + fonts-liberation fonts-dejavu \ + tini curl \ + && apt-get clean \ + && rm -rf /var/lib/apt/lists/* /var/cache/apt/* /tmp/* + +RUN groupadd -g ${APP_GID} app && \ + useradd -u ${APP_UID} -g ${APP_GID} -m -s /bin/bash app + +WORKDIR /app +COPY --from=builder /wheels /wheels + + +# ---------- Stage 2a: Runtime (Production) ------------------------------------ +FROM runtime-base AS runtime + +COPY requirements.txt . +RUN pip install --no-index --find-links=/wheels -r requirements.txt && \ + rm -rf /wheels + +COPY --chown=app:app . /app/ +RUN mkdir -p /app/staticfiles /app/media && chown -R app:app /app +USER app +EXPOSE 8000 +ENTRYPOINT ["/usr/bin/tini", "--", "/app/entrypoint.sh"] +CMD ["gunicorn", "config.wsgi:application", \ + "--bind", "0.0.0.0:8000", \ + "--workers", "3", "--threads", "2", \ + "--worker-class", "gthread", "--worker-tmp-dir", "/tmp", \ + "--access-logfile", "-", "--error-logfile", "-", \ + "--timeout", "120"] + + +# ---------- Stage 2b: Dev ----------------------------------------------------- +FROM runtime-base AS dev + +# Dev-Tools für Container & VS Code +RUN apt-get update && apt-get install -y --no-install-recommends \ + git bash-completion vim less procps iputils-ping \ + && apt-get clean && rm -rf /var/lib/apt/lists/* + +COPY requirements.txt requirements-dev.txt ./ +RUN pip install --no-index --find-links=/wheels -r requirements.txt -r requirements-dev.txt && \ + rm -rf /wheels + +# Code wird im Dev via Volume gemountet; nichts kopieren. +RUN mkdir -p /app/staticfiles /app/media && chown -R app:app /app +USER app +EXPOSE 8000 5678 +ENTRYPOINT ["/usr/bin/tini", "--", "/app/entrypoint.sh"] +CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"] diff --git a/app/config/__init__.py b/app/config/__init__.py new file mode 100644 index 0000000..370372a --- /dev/null +++ b/app/config/__init__.py @@ -0,0 +1,3 @@ +from .celery import app as celery_app + +__all__ = ["celery_app"] diff --git a/app/config/asgi.py b/app/config/asgi.py new file mode 100644 index 0000000..5096b23 --- /dev/null +++ b/app/config/asgi.py @@ -0,0 +1,6 @@ +import os + +from django.core.asgi import get_asgi_application + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production") +application = get_asgi_application() diff --git a/app/config/celery.py b/app/config/celery.py new file mode 100644 index 0000000..020f2a5 --- /dev/null +++ b/app/config/celery.py @@ -0,0 +1,9 @@ +import os + +from celery import Celery + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production") + +app = Celery("serienbrief") +app.config_from_object("django.conf:settings", namespace="CELERY") +app.autodiscover_tasks() diff --git a/app/config/settings/__init__.py b/app/config/settings/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/config/settings/base.py b/app/config/settings/base.py new file mode 100644 index 0000000..f10dce3 --- /dev/null +++ b/app/config/settings/base.py @@ -0,0 +1,144 @@ +""" +Basis-Settings. Werden von dev.py und production.py erweitert. +""" +from pathlib import Path + +import environ + +BASE_DIR = Path(__file__).resolve().parent.parent.parent + +env = environ.Env( + DJANGO_DEBUG=(bool, False), + USE_X_FORWARDED_HOST=(bool, True), + JOB_RETENTION_DAYS=(int, 30), +) + +# --- Core -------------------------------------------------------------------- +SECRET_KEY = env("DJANGO_SECRET_KEY", default="dev-insecure-change-me") +DEBUG = env("DJANGO_DEBUG") +ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=["*"]) +CSRF_TRUSTED_ORIGINS = env.list("CSRF_TRUSTED_ORIGINS", default=[]) + +# Hinter dem äußeren Reverse-Proxy +USE_X_FORWARDED_HOST = env("USE_X_FORWARDED_HOST") +_proxy_header = env("SECURE_PROXY_SSL_HEADER", default="") +if _proxy_header: + name, value = _proxy_header.split(",", 1) + SECURE_PROXY_SSL_HEADER = (name.strip(), value.strip()) + +# --- Apps -------------------------------------------------------------------- +INSTALLED_APPS = [ + "django.contrib.admin", + "django.contrib.auth", + "django.contrib.contenttypes", + "django.contrib.sessions", + "django.contrib.messages", + "django.contrib.staticfiles", + # 3rd party + "django_celery_beat", + "django_celery_results", + "django_htmx", + "axes", + # local + "mailmerge", +] + +MIDDLEWARE = [ + "django.middleware.security.SecurityMiddleware", + "django.contrib.sessions.middleware.SessionMiddleware", + "django.middleware.common.CommonMiddleware", + "django.middleware.csrf.CsrfViewMiddleware", + "django.contrib.auth.middleware.AuthenticationMiddleware", + "django.contrib.messages.middleware.MessageMiddleware", + "django.middleware.clickjacking.XFrameOptionsMiddleware", + "django_htmx.middleware.HtmxMiddleware", + "axes.middleware.AxesMiddleware", +] + +ROOT_URLCONF = "config.urls" + +TEMPLATES = [ + { + "BACKEND": "django.template.backends.django.DjangoTemplates", + "DIRS": [BASE_DIR / "templates"], + "APP_DIRS": True, + "OPTIONS": { + "context_processors": [ + "django.template.context_processors.debug", + "django.template.context_processors.request", + "django.contrib.auth.context_processors.auth", + "django.contrib.messages.context_processors.messages", + ], + }, + }, +] + +WSGI_APPLICATION = "config.wsgi.application" + +# --- Database ---------------------------------------------------------------- +DATABASES = { + "default": env.db_url( + "DATABASE_URL", default="sqlite:///" + str(BASE_DIR / "db.sqlite3") + ), +} + +# --- Auth -------------------------------------------------------------------- +AUTH_PASSWORD_VALIDATORS = [ + {"NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator"}, + {"NAME": "django.contrib.auth.password_validation.MinimumLengthValidator", + "OPTIONS": {"min_length": 12}}, + {"NAME": "django.contrib.auth.password_validation.CommonPasswordValidator"}, + {"NAME": "django.contrib.auth.password_validation.NumericPasswordValidator"}, +] +PASSWORD_HASHERS = [ + "django.contrib.auth.hashers.Argon2PasswordHasher", + "django.contrib.auth.hashers.PBKDF2PasswordHasher", +] + +AUTHENTICATION_BACKENDS = [ + "axes.backends.AxesStandaloneBackend", + "django.contrib.auth.backends.ModelBackend", +] + +LOGIN_URL = "/accounts/login/" +LOGIN_REDIRECT_URL = "/" +LOGOUT_REDIRECT_URL = "/accounts/login/" + +# django-axes – Brute-Force-Schutz +AXES_FAILURE_LIMIT = 5 +AXES_COOLOFF_TIME = 1 # Stunde +AXES_LOCKOUT_PARAMETERS = ["username", "ip_address"] + +# --- I18N / TZ --------------------------------------------------------------- +LANGUAGE_CODE = "de-at" +TIME_ZONE = "Europe/Vienna" +USE_I18N = True +USE_TZ = True + +# --- Static / Media ---------------------------------------------------------- +STATIC_URL = "/static/" +STATIC_ROOT = BASE_DIR / "staticfiles" +MEDIA_URL = "/media/" +MEDIA_ROOT = BASE_DIR / "media" + +# --- Celery ------------------------------------------------------------------ +CELERY_BROKER_URL = env("CELERY_BROKER_URL", default="redis://redis:6379/0") +CELERY_RESULT_BACKEND = env("CELERY_RESULT_BACKEND", default="django-db") +CELERY_TASK_TRACK_STARTED = True +CELERY_TASK_TIME_LIMIT = 600 # 10 Minuten Hard-Timeout +CELERY_TASK_SOFT_TIME_LIMIT = 540 +CELERY_WORKER_PREFETCH_MULTIPLIER = 1 +CELERY_TIMEZONE = TIME_ZONE + +# --- App --------------------------------------------------------------------- +JOB_RETENTION_DAYS = env("JOB_RETENTION_DAYS") +DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField" + +# --- Security Defaults ------------------------------------------------------- +SESSION_COOKIE_HTTPONLY = True +SESSION_COOKIE_SAMESITE = "Lax" +CSRF_COOKIE_HTTPONLY = False # Bleibt false, damit JS/HTMX-Forms funktionieren +CSRF_COOKIE_SAMESITE = "Lax" +X_FRAME_OPTIONS = "DENY" +SECURE_CONTENT_TYPE_NOSNIFF = True +SECURE_REFERRER_POLICY = "strict-origin-when-cross-origin" diff --git a/app/config/settings/dev.py b/app/config/settings/dev.py new file mode 100644 index 0000000..b42785a --- /dev/null +++ b/app/config/settings/dev.py @@ -0,0 +1,16 @@ +from .base import * # noqa: F401,F403 +from .base import INSTALLED_APPS, MIDDLEWARE + +DEBUG = True +ALLOWED_HOSTS = ["*"] + +# Debug-Toolbar nur lokal +INSTALLED_APPS = INSTALLED_APPS + ["debug_toolbar"] +MIDDLEWARE = ["debug_toolbar.middleware.DebugToolbarMiddleware", *MIDDLEWARE] +INTERNAL_IPS = ["127.0.0.1"] + +# Im Dev keine Auto-Lockouts beim Testen +AXES_ENABLED = False + +# E-Mails nur in die Konsole +EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend" diff --git a/app/config/settings/production.py b/app/config/settings/production.py new file mode 100644 index 0000000..6fa51d1 --- /dev/null +++ b/app/config/settings/production.py @@ -0,0 +1,32 @@ +from .base import * # noqa: F401,F403 + +DEBUG = False + +# Strikte Security-Defaults – TLS macht der äußere Proxy. +SESSION_COOKIE_SECURE = True +CSRF_COOKIE_SECURE = True +SECURE_HSTS_SECONDS = 0 # HSTS setzt der äußere Proxy +SECURE_HSTS_INCLUDE_SUBDOMAINS = False +SECURE_HSTS_PRELOAD = False + +LOGGING = { + "version": 1, + "disable_existing_loggers": False, + "formatters": { + "verbose": { + "format": "{asctime} {levelname} {name} {message}", + "style": "{", + }, + }, + "handlers": { + "console": { + "class": "logging.StreamHandler", + "formatter": "verbose", + }, + }, + "root": {"handlers": ["console"], "level": "INFO"}, + "loggers": { + "django.security": {"handlers": ["console"], "level": "WARNING", "propagate": False}, + "mailmerge": {"handlers": ["console"], "level": "INFO", "propagate": False}, + }, +} diff --git a/app/config/urls.py b/app/config/urls.py new file mode 100644 index 0000000..f44ac97 --- /dev/null +++ b/app/config/urls.py @@ -0,0 +1,27 @@ +from django.conf import settings +from django.conf.urls.static import static +from django.contrib import admin +from django.contrib.auth import views as auth_views +from django.http import HttpResponse +from django.urls import include, path + + +def healthz(_request): + return HttpResponse("ok", content_type="text/plain") + + +urlpatterns = [ + path("healthz", healthz), + path("admin/", admin.site.urls), + path("accounts/login/", auth_views.LoginView.as_view(), name="login"), + path("accounts/logout/", auth_views.LogoutView.as_view(), name="logout"), + path("", include("mailmerge.urls")), +] + +if settings.DEBUG: + urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) + try: + import debug_toolbar + urlpatterns = [path("__debug__/", include(debug_toolbar.urls)), *urlpatterns] + except ImportError: + pass diff --git a/app/config/wsgi.py b/app/config/wsgi.py new file mode 100644 index 0000000..949d08d --- /dev/null +++ b/app/config/wsgi.py @@ -0,0 +1,6 @@ +import os + +from django.core.wsgi import get_wsgi_application + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production") +application = get_wsgi_application() diff --git a/app/entrypoint.sh b/app/entrypoint.sh new file mode 100755 index 0000000..2c30784 --- /dev/null +++ b/app/entrypoint.sh @@ -0,0 +1,19 @@ +#!/bin/bash +set -euo pipefail +ROLE="${ROLE:-web}" + +if [[ "$ROLE" == "web" ]]; then + echo "[entrypoint] waiting for database..." + python manage.py wait_for_db --timeout 60 || true + + echo "[entrypoint] running migrations..." + python manage.py migrate --noinput + + if [[ "${DJANGO_DEBUG:-False}" != "True" ]]; then + echo "[entrypoint] collecting static files..." + python manage.py collectstatic --noinput --clear + fi +fi + +echo "[entrypoint] starting role=$ROLE: $*" +exec "$@" diff --git a/app/mailmerge/__init__.py b/app/mailmerge/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/mailmerge/admin.py b/app/mailmerge/admin.py new file mode 100644 index 0000000..5587a21 --- /dev/null +++ b/app/mailmerge/admin.py @@ -0,0 +1,25 @@ +from django.contrib import admin + +from .models import JobLogEntry, LetterTemplate, MailMergeJob + + +@admin.register(LetterTemplate) +class LetterTemplateAdmin(admin.ModelAdmin): + list_display = ("name", "created_by", "created_at") + search_fields = ("name",) + readonly_fields = ("id", "placeholders", "created_at", "updated_at") + + +@admin.register(MailMergeJob) +class MailMergeJobAdmin(admin.ModelAdmin): + list_display = ("id", "template", "status", "processed_rows", "total_rows", + "created_by", "created_at") + list_filter = ("status",) + readonly_fields = ("id", "created_at", "started_at", "finished_at", + "processed_rows", "total_rows", "error_message") + + +@admin.register(JobLogEntry) +class JobLogEntryAdmin(admin.ModelAdmin): + list_display = ("job", "level", "timestamp") + list_filter = ("level",) diff --git a/app/mailmerge/apps.py b/app/mailmerge/apps.py new file mode 100644 index 0000000..5e5874e --- /dev/null +++ b/app/mailmerge/apps.py @@ -0,0 +1,7 @@ +from django.apps import AppConfig + + +class MailmergeConfig(AppConfig): + default_auto_field = "django.db.models.BigAutoField" + name = "mailmerge" + verbose_name = "Serienbrief" diff --git a/app/mailmerge/forms.py b/app/mailmerge/forms.py new file mode 100644 index 0000000..c966b51 --- /dev/null +++ b/app/mailmerge/forms.py @@ -0,0 +1,34 @@ +from django import forms + +from .models import LetterTemplate, MailMergeJob + + +class LetterTemplateForm(forms.ModelForm): + class Meta: + model = LetterTemplate + fields = ["name", "description", "file"] + widgets = { + "description": forms.Textarea(attrs={"rows": 3}), + } + + def clean_file(self): + f = self.cleaned_data["file"] + if not f.name.lower().endswith(".docx"): + raise forms.ValidationError("Nur .docx-Dateien sind erlaubt.") + if f.size > 10 * 1024 * 1024: + raise forms.ValidationError("Datei zu groß (max. 10 MB).") + return f + + +class MailMergeJobForm(forms.ModelForm): + class Meta: + model = MailMergeJob + fields = ["template", "recipients_csv"] + + def clean_recipients_csv(self): + f = self.cleaned_data["recipients_csv"] + if not f.name.lower().endswith(".csv"): + raise forms.ValidationError("Nur .csv-Dateien sind erlaubt.") + if f.size > 20 * 1024 * 1024: + raise forms.ValidationError("Datei zu groß (max. 20 MB).") + return f diff --git a/app/mailmerge/management/__init__.py b/app/mailmerge/management/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/mailmerge/management/commands/__init__.py b/app/mailmerge/management/commands/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/mailmerge/management/commands/wait_for_db.py b/app/mailmerge/management/commands/wait_for_db.py new file mode 100644 index 0000000..e6bddd1 --- /dev/null +++ b/app/mailmerge/management/commands/wait_for_db.py @@ -0,0 +1,25 @@ +"""Wartet, bis die DB Connections annimmt.""" +import time + +from django.core.management.base import BaseCommand +from django.db import OperationalError, connections + + +class Command(BaseCommand): + help = "Wartet, bis die Standard-DB verfügbar ist." + + def add_arguments(self, parser): + parser.add_argument("--timeout", type=int, default=60) + + def handle(self, *args, **opts): + deadline = time.time() + opts["timeout"] + while time.time() < deadline: + try: + connections["default"].ensure_connection() + self.stdout.write(self.style.SUCCESS("DB ist bereit.")) + return + except OperationalError: + self.stdout.write("warte auf DB...") + time.sleep(2) + self.stderr.write(self.style.ERROR("DB nach Timeout nicht erreichbar.")) + raise SystemExit(1) diff --git a/app/mailmerge/migrations/0001_initial.py b/app/mailmerge/migrations/0001_initial.py new file mode 100644 index 0000000..ca1a252 --- /dev/null +++ b/app/mailmerge/migrations/0001_initial.py @@ -0,0 +1,68 @@ +# Generated by Django 5.1.4 on 2026-05-21 07:24 + +import django.db.models.deletion +import mailmerge.models +import uuid +from django.conf import settings +from django.db import migrations, models + + +class Migration(migrations.Migration): + + initial = True + + dependencies = [ + migrations.swappable_dependency(settings.AUTH_USER_MODEL), + ] + + operations = [ + migrations.CreateModel( + name='LetterTemplate', + fields=[ + ('id', models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True, serialize=False)), + ('name', models.CharField(max_length=200)), + ('description', models.TextField(blank=True)), + ('file', models.FileField(upload_to=mailmerge.models.template_upload_path)), + ('placeholders', models.JSONField(blank=True, default=list, help_text='Aus dem DOCX extrahierte Variablen.')), + ('created_at', models.DateTimeField(auto_now_add=True)), + ('updated_at', models.DateTimeField(auto_now=True)), + ('created_by', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, related_name='templates', to=settings.AUTH_USER_MODEL)), + ], + options={ + 'ordering': ['-created_at'], + }, + ), + migrations.CreateModel( + name='MailMergeJob', + fields=[ + ('id', models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True, serialize=False)), + ('recipients_csv', models.FileField(upload_to=mailmerge.models.csv_upload_path)), + ('status', models.CharField(choices=[('pending', 'Wartet'), ('running', 'Läuft'), ('done', 'Fertig'), ('failed', 'Fehlgeschlagen')], default='pending', max_length=20)), + ('result_pdf', models.FileField(blank=True, null=True, upload_to=mailmerge.models.result_upload_path)), + ('total_rows', models.PositiveIntegerField(default=0)), + ('processed_rows', models.PositiveIntegerField(default=0)), + ('error_message', models.TextField(blank=True)), + ('created_at', models.DateTimeField(auto_now_add=True)), + ('started_at', models.DateTimeField(blank=True, null=True)), + ('finished_at', models.DateTimeField(blank=True, null=True)), + ('created_by', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, related_name='jobs', to=settings.AUTH_USER_MODEL)), + ('template', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, related_name='jobs', to='mailmerge.lettertemplate')), + ], + options={ + 'ordering': ['-created_at'], + }, + ), + migrations.CreateModel( + name='JobLogEntry', + fields=[ + ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('level', models.CharField(choices=[('info', 'Info'), ('warning', 'Warnung'), ('error', 'Fehler')], default='info', max_length=10)), + ('message', models.TextField()), + ('timestamp', models.DateTimeField(auto_now_add=True)), + ('job', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='logs', to='mailmerge.mailmergejob')), + ], + options={ + 'ordering': ['timestamp'], + }, + ), + ] diff --git a/app/mailmerge/migrations/__init__.py b/app/mailmerge/migrations/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/mailmerge/models.py b/app/mailmerge/models.py new file mode 100644 index 0000000..33d4160 --- /dev/null +++ b/app/mailmerge/models.py @@ -0,0 +1,86 @@ +""" +Datenmodell – Templates, Jobs, Log-Einträge. +""" +import uuid +from pathlib import Path + +from django.conf import settings +from django.db import models + + +def template_upload_path(instance, filename: str) -> str: + return f"templates/{instance.id}/{Path(filename).name}" + + +def csv_upload_path(instance, filename: str) -> str: + return f"jobs/{instance.id}/recipients/{Path(filename).name}" + + +def result_upload_path(instance, filename: str) -> str: + return f"jobs/{instance.id}/result/{Path(filename).name}" + + +class LetterTemplate(models.Model): + """DOCX-Vorlage mit Jinja-Platzhaltern.""" + + id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) + name = models.CharField(max_length=200) + description = models.TextField(blank=True) + file = models.FileField(upload_to=template_upload_path) + placeholders = models.JSONField(default=list, blank=True, + help_text="Aus dem DOCX extrahierte Variablen.") + created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT, + related_name="templates") + created_at = models.DateTimeField(auto_now_add=True) + updated_at = models.DateTimeField(auto_now=True) + + class Meta: + ordering = ["-created_at"] + + def __str__(self) -> str: + return self.name + + +class MailMergeJob(models.Model): + class Status(models.TextChoices): + PENDING = "pending", "Wartet" + RUNNING = "running", "Läuft" + DONE = "done", "Fertig" + FAILED = "failed", "Fehlgeschlagen" + + id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) + template = models.ForeignKey(LetterTemplate, on_delete=models.PROTECT, + related_name="jobs") + recipients_csv = models.FileField(upload_to=csv_upload_path) + status = models.CharField(max_length=20, choices=Status.choices, + default=Status.PENDING) + result_pdf = models.FileField(upload_to=result_upload_path, null=True, blank=True) + total_rows = models.PositiveIntegerField(default=0) + processed_rows = models.PositiveIntegerField(default=0) + error_message = models.TextField(blank=True) + created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT, + related_name="jobs") + created_at = models.DateTimeField(auto_now_add=True) + started_at = models.DateTimeField(null=True, blank=True) + finished_at = models.DateTimeField(null=True, blank=True) + + class Meta: + ordering = ["-created_at"] + + def __str__(self) -> str: + return f"Job {self.id} ({self.template.name})" + + +class JobLogEntry(models.Model): + class Level(models.TextChoices): + INFO = "info", "Info" + WARNING = "warning", "Warnung" + ERROR = "error", "Fehler" + + job = models.ForeignKey(MailMergeJob, on_delete=models.CASCADE, related_name="logs") + level = models.CharField(max_length=10, choices=Level.choices, default=Level.INFO) + message = models.TextField() + timestamp = models.DateTimeField(auto_now_add=True) + + class Meta: + ordering = ["timestamp"] diff --git a/app/mailmerge/services/__init__.py b/app/mailmerge/services/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/mailmerge/services/docx_renderer.py b/app/mailmerge/services/docx_renderer.py new file mode 100644 index 0000000..8e5a034 --- /dev/null +++ b/app/mailmerge/services/docx_renderer.py @@ -0,0 +1,74 @@ +""" +DOCX-Rendering und PDF-Konvertierung. +""" +from __future__ import annotations + +import logging +import re +import subprocess +import tempfile +from pathlib import Path + +from docx import Document +from docxtpl import DocxTemplate + +logger = logging.getLogger(__name__) + +PLACEHOLDER_RE = re.compile(r"\{\{\s*([A-Za-z_][A-Za-z0-9_]*)") + + +def extract_placeholders(docx_path: Path) -> list[str]: + """Liest die Jinja-Platzhalter aus einem DOCX und gibt sie sortiert zurück.""" + doc = Document(str(docx_path)) + found: set[str] = set() + for para in doc.paragraphs: + for m in PLACEHOLDER_RE.finditer(para.text): + found.add(m.group(1)) + for table in doc.tables: + for row in table.rows: + for cell in row.cells: + for para in cell.paragraphs: + for m in PLACEHOLDER_RE.finditer(para.text): + found.add(m.group(1)) + return sorted(found) + + +def render_docx(template_path: Path, context: dict, out_path: Path) -> Path: + """Füllt das DOCX-Template mit Kontext und schreibt das Ergebnis.""" + tpl = DocxTemplate(str(template_path)) + tpl.render(context) + tpl.save(str(out_path)) + return out_path + + +def docx_to_pdf(docx_path: Path, out_dir: Path) -> Path: + """Konvertiert DOCX nach PDF mit LibreOffice headless. + + LibreOffice braucht ein eigenes Profilverzeichnis, sonst kollidieren + parallele Worker. + """ + out_dir.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix="lo-profile-") as profile_dir: + cmd = [ + "soffice", + "--headless", + "--nologo", + "--norestore", + "--nolockcheck", + f"-env:UserInstallation=file://{profile_dir}", + "--convert-to", "pdf", + "--outdir", str(out_dir), + str(docx_path), + ] + logger.info("LibreOffice convert: %s", " ".join(cmd)) + result = subprocess.run( # noqa: S603 + cmd, capture_output=True, text=True, timeout=120, check=False + ) + if result.returncode != 0: + raise RuntimeError( + f"LibreOffice-Konvertierung fehlgeschlagen: {result.stderr}" + ) + pdf_path = out_dir / (docx_path.stem + ".pdf") + if not pdf_path.exists(): + raise FileNotFoundError(f"PDF nicht gefunden: {pdf_path}") + return pdf_path diff --git a/app/mailmerge/services/pdf_merge.py b/app/mailmerge/services/pdf_merge.py new file mode 100644 index 0000000..89d9294 --- /dev/null +++ b/app/mailmerge/services/pdf_merge.py @@ -0,0 +1,14 @@ +from pathlib import Path + +from pypdf import PdfWriter + + +def merge_pdfs(pdfs: list[Path], out_path: Path) -> Path: + writer = PdfWriter() + for pdf in pdfs: + writer.append(str(pdf)) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("wb") as f: + writer.write(f) + writer.close() + return out_path diff --git a/app/mailmerge/tasks.py b/app/mailmerge/tasks.py new file mode 100644 index 0000000..32ce487 --- /dev/null +++ b/app/mailmerge/tasks.py @@ -0,0 +1,87 @@ +""" +Celery-Tasks für die PDF-Erzeugung. +""" +from __future__ import annotations + +import csv +import io +import logging +import tempfile +from pathlib import Path + +from celery import shared_task +from django.core.files import File +from django.utils import timezone + +from .models import JobLogEntry, MailMergeJob +from .services.docx_renderer import docx_to_pdf, render_docx +from .services.pdf_merge import merge_pdfs + +logger = logging.getLogger(__name__) + + +def _log(job: MailMergeJob, level: str, msg: str) -> None: + JobLogEntry.objects.create(job=job, level=level, message=msg) + logger.log(getattr(logging, level.upper(), logging.INFO), "[job %s] %s", job.id, msg) + + +@shared_task(bind=True) +def run_mailmerge(self, job_id: str) -> str: + job = MailMergeJob.objects.select_related("template").get(pk=job_id) + job.status = MailMergeJob.Status.RUNNING + job.started_at = timezone.now() + job.save(update_fields=["status", "started_at"]) + _log(job, "info", f"Job gestartet (task={self.request.id}).") + + try: + # CSV einlesen + raw = job.recipients_csv.read().decode("utf-8-sig") + reader = csv.DictReader(io.StringIO(raw)) + rows = list(reader) + job.total_rows = len(rows) + job.save(update_fields=["total_rows"]) + _log(job, "info", f"{len(rows)} Empfänger gefunden.") + + if not rows: + raise ValueError("CSV enthält keine Datenzeilen.") + + # CSV-Felder vs. Template-Platzhalter prüfen + csv_fields = set(reader.fieldnames or []) + placeholders = set(job.template.placeholders or []) + missing = placeholders - csv_fields + if missing: + _log(job, "warning", + f"CSV fehlen Spalten: {', '.join(sorted(missing))}") + + # Jeden Brief rendern, alle zu einem PDF zusammenführen + with tempfile.TemporaryDirectory(prefix="mailmerge-") as tmpdir: + tmp = Path(tmpdir) + pdfs: list[Path] = [] + template_path = Path(job.template.file.path) + + for idx, row in enumerate(rows, start=1): + docx_out = tmp / f"letter_{idx:05d}.docx" + render_docx(template_path, row, docx_out) + pdf = docx_to_pdf(docx_out, tmp / "pdf") + pdfs.append(pdf) + job.processed_rows = idx + job.save(update_fields=["processed_rows"]) + + merged = tmp / f"serienbrief_{job.id}.pdf" + merge_pdfs(pdfs, merged) + with merged.open("rb") as f: + job.result_pdf.save(merged.name, File(f), save=False) + + job.status = MailMergeJob.Status.DONE + job.finished_at = timezone.now() + job.save() + _log(job, "info", "Job erfolgreich abgeschlossen.") + return str(job.id) + + except Exception as exc: # noqa: BLE001 + job.status = MailMergeJob.Status.FAILED + job.error_message = str(exc) + job.finished_at = timezone.now() + job.save() + _log(job, "error", f"Job fehlgeschlagen: {exc}") + raise diff --git a/app/mailmerge/urls.py b/app/mailmerge/urls.py new file mode 100644 index 0000000..72a9acf --- /dev/null +++ b/app/mailmerge/urls.py @@ -0,0 +1,12 @@ +from django.urls import path + +from . import views + +urlpatterns = [ + path("", views.dashboard, name="dashboard"), + path("templates/new/", views.template_upload, name="template-upload"), + path("templates//", views.template_detail, name="template-detail"), + path("jobs/new/", views.job_create, name="job-create"), + path("jobs//", views.job_detail, name="job-detail"), + path("jobs//download/", views.job_download, name="job-download"), +] diff --git a/app/mailmerge/views.py b/app/mailmerge/views.py new file mode 100644 index 0000000..e6c548c --- /dev/null +++ b/app/mailmerge/views.py @@ -0,0 +1,94 @@ +from pathlib import Path + +from django.contrib.auth.decorators import login_required +from django.http import FileResponse, Http404, HttpResponse +from django.shortcuts import get_object_or_404, redirect, render +from django.urls import reverse +from django.views.decorators.http import require_http_methods + +from .forms import LetterTemplateForm, MailMergeJobForm +from .models import LetterTemplate, MailMergeJob +from .services.docx_renderer import extract_placeholders +from .tasks import run_mailmerge + + +@login_required +def dashboard(request): + templates = LetterTemplate.objects.all()[:20] + jobs = MailMergeJob.objects.select_related("template")[:20] + return render(request, "mailmerge/dashboard.html", + {"templates": templates, "jobs": jobs}) + + +@login_required +@require_http_methods(["GET", "POST"]) +def template_upload(request): + form = LetterTemplateForm(request.POST or None, request.FILES or None) + if request.method == "POST" and form.is_valid(): + tpl = form.save(commit=False) + tpl.created_by = request.user + tpl.save() + # Platzhalter extrahieren – Datei liegt jetzt auf Disk + tpl.placeholders = extract_placeholders(Path(tpl.file.path)) + tpl.save(update_fields=["placeholders"]) + return redirect(reverse("template-detail", args=[tpl.id])) + return render(request, "mailmerge/template_form.html", {"form": form}) + + +@login_required +def template_detail(request, pk): + tpl = get_object_or_404(LetterTemplate, pk=pk) + return render(request, "mailmerge/template_detail.html", {"template": tpl}) + + +@login_required +@require_http_methods(["GET", "POST"]) +def job_create(request): + form = MailMergeJobForm(request.POST or None, request.FILES or None) + if request.method == "POST" and form.is_valid(): + job = form.save(commit=False) + job.created_by = request.user + job.save() + run_mailmerge.delay(str(job.id)) + return redirect(reverse("job-detail", args=[job.id])) + return render(request, "mailmerge/job_form.html", {"form": form}) + + +@login_required +def job_detail(request, pk): + job = get_object_or_404( + MailMergeJob.objects.select_related("template"), pk=pk + ) + logs = job.logs.all() + # HTMX partials: nur das Status-Fragment ausliefern + if request.headers.get("HX-Request"): + return render(request, "mailmerge/_job_status.html", + {"job": job, "logs": logs}) + return render(request, "mailmerge/job_detail.html", + {"job": job, "logs": logs}) + + +@login_required +def job_download(request, pk): + """PDF-Download – via X-Accel-Redirect in Production, direkter Stream im Dev.""" + job = get_object_or_404(MailMergeJob, pk=pk) + if not job.result_pdf: + raise Http404("Kein Ergebnis-PDF vorhanden.") + + # In Dev (settings.DEBUG) direkt streamen + from django.conf import settings + if settings.DEBUG: + return FileResponse(job.result_pdf.open("rb"), + as_attachment=True, + filename=Path(job.result_pdf.name).name) + + # In Production: Nginx serviert die Datei via internem Mount. + response = HttpResponse() + response["Content-Type"] = "application/pdf" + response["Content-Disposition"] = ( + f'attachment; filename="{Path(job.result_pdf.name).name}"' + ) + # Pfad relativ zu MEDIA_ROOT, gemappt auf /protected-media/ + relative = job.result_pdf.name + response["X-Accel-Redirect"] = f"/protected-media/{relative}" + return response diff --git a/app/manage.py b/app/manage.py new file mode 100644 index 0000000..fef1f23 --- /dev/null +++ b/app/manage.py @@ -0,0 +1,19 @@ +#!/usr/bin/env python +"""Django management entry point.""" +import os +import sys + + +def main() -> None: + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.dev") + try: + from django.core.management import execute_from_command_line + except ImportError as exc: + raise ImportError( + "Django konnte nicht importiert werden. Ist die venv aktiv?" + ) from exc + execute_from_command_line(sys.argv) + + +if __name__ == "__main__": + main() diff --git a/app/pyproject.toml b/app/pyproject.toml new file mode 100644 index 0000000..87a87b3 --- /dev/null +++ b/app/pyproject.toml @@ -0,0 +1,17 @@ +[tool.ruff] +line-length = 100 +target-version = "py312" +extend-exclude = ["migrations", "staticfiles", "media"] + +[tool.ruff.lint] +select = ["E", "F", "W", "I", "UP", "B", "DJ", "S"] +ignore = ["E501", "S101"] + +[tool.ruff.lint.per-file-ignores] +"**/tests/**" = ["S"] +"**/settings/**" = ["S105", "S106"] + +[tool.pytest.ini_options] +DJANGO_SETTINGS_MODULE = "config.settings.dev" +python_files = ["test_*.py", "*_test.py", "tests.py"] +addopts = "--reuse-db -ra" diff --git a/app/requirements-dev.txt b/app/requirements-dev.txt new file mode 100644 index 0000000..014fada --- /dev/null +++ b/app/requirements-dev.txt @@ -0,0 +1,8 @@ +# Nur im dev-Build installieren +debugpy==1.8.7 +django-debug-toolbar==4.4.6 +ipython==8.29.0 +ruff==0.7.4 +pytest==8.3.3 +pytest-django==4.9.0 +factory-boy==3.3.1 diff --git a/app/requirements.txt b/app/requirements.txt new file mode 100644 index 0000000..25827b4 --- /dev/null +++ b/app/requirements.txt @@ -0,0 +1,15 @@ +Django==5.1.4 +django-environ==0.11.2 +django-celery-beat==2.7.0 +django-celery-results==2.5.1 +psycopg[binary]==3.2.3 +gunicorn==23.0.0 +celery[redis]==5.4.0 +redis==5.2.0 +docxtpl==0.19.0 +python-docx==1.1.2 +pypdf==5.1.0 +argon2-cffi==23.1.0 +django-axes==7.0.1 +django-htmx==1.21.0 +python-dateutil==2.9.0.post0 diff --git a/app/templates/base.html b/app/templates/base.html new file mode 100644 index 0000000..55a413c --- /dev/null +++ b/app/templates/base.html @@ -0,0 +1,41 @@ + + + + + + {% block title %}Serienbrief{% endblock %} + + + + + {% if messages %}
    {% for m in messages %}
  • {{ m }}
    • {% endfor %}
{% endif %} + {% block content %}{% endblock %} + + diff --git a/app/templates/mailmerge/_job_status.html b/app/templates/mailmerge/_job_status.html new file mode 100644 index 0000000..73ab620 --- /dev/null +++ b/app/templates/mailmerge/_job_status.html @@ -0,0 +1,25 @@ +
+

Vorlage: {{ job.template.name }}

+

Status: + {{ job.get_status_display }} +

+

Fortschritt: {{ job.processed_rows }} / {{ job.total_rows }}

+ + {% if job.status == "done" %} +

PDF herunterladen

+ {% endif %} + {% if job.error_message %} +

Fehler: {{ job.error_message }}

+ {% endif %} + +

Log

+
+ {% for entry in logs %} +
+ [{{ entry.timestamp|date:"H:i:s" }}] {{ entry.level|upper }} – {{ entry.message }} +
+ {% empty %} + Keine Einträge. + {% endfor %} +
+
diff --git a/app/templates/mailmerge/dashboard.html b/app/templates/mailmerge/dashboard.html new file mode 100644 index 0000000..de6c7d5 --- /dev/null +++ b/app/templates/mailmerge/dashboard.html @@ -0,0 +1,40 @@ +{% extends "base.html" %} +{% block content %} +

Übersicht

+ +

Vorlagen

+Neue Vorlage hochladen + + + + {% for t in templates %} + + + + + + {% empty %} + + {% endfor %} + +
NamePlatzhalterErstellt
{{ t.name }}{{ t.placeholders|join:", " }}{{ t.created_at|date:"d.m.Y H:i" }}
Noch keine Vorlagen vorhanden.
+ +

Aufträge

+Neuen Serienbrief erstellen + + + + {% for j in jobs %} + + + + + + + + {% empty %} + + {% endfor %} + +
IDVorlageStatusFortschrittErstellt
{{ j.id|stringformat:"s"|slice:":8" }}…{{ j.template.name }}{{ j.get_status_display }}{{ j.processed_rows }} / {{ j.total_rows }}{{ j.created_at|date:"d.m.Y H:i" }}
Noch keine Aufträge.
+{% endblock %} diff --git a/app/templates/mailmerge/job_detail.html b/app/templates/mailmerge/job_detail.html new file mode 100644 index 0000000..4248044 --- /dev/null +++ b/app/templates/mailmerge/job_detail.html @@ -0,0 +1,12 @@ +{% extends "base.html" %} +{% block content %} +

Auftrag {{ job.id|stringformat:"s"|slice:":8" }}…

+ +
+ {% include "mailmerge/_job_status.html" %} +
+ + +{% endblock %} diff --git a/app/templates/mailmerge/job_form.html b/app/templates/mailmerge/job_form.html new file mode 100644 index 0000000..ab7fe9b --- /dev/null +++ b/app/templates/mailmerge/job_form.html @@ -0,0 +1,10 @@ +{% extends "base.html" %} +{% block content %} +

Neuer Serienbrief

+

Vorlage und Empfänger-CSV auswählen. Die Spaltennamen der CSV müssen mit den Platzhaltern der Vorlage übereinstimmen (erste Zeile = Spaltennamen).

+
+ {% csrf_token %} + {{ form.as_p }} + +
+{% endblock %} diff --git a/app/templates/mailmerge/template_detail.html b/app/templates/mailmerge/template_detail.html new file mode 100644 index 0000000..b960965 --- /dev/null +++ b/app/templates/mailmerge/template_detail.html @@ -0,0 +1,11 @@ +{% extends "base.html" %} +{% block content %} +

{{ template.name }}

+

{{ template.description }}

+

Datei: {{ template.file.name }}

+

Erkannte Platzhalter:

+
    + {% for p in template.placeholders %}
  • {{ p }}
    • {% empty %}
  • Keine gefunden.
    • {% endfor %} +
+Serienbrief mit dieser Vorlage erstellen +{% endblock %} diff --git a/app/templates/mailmerge/template_form.html b/app/templates/mailmerge/template_form.html new file mode 100644 index 0000000..8f13700 --- /dev/null +++ b/app/templates/mailmerge/template_form.html @@ -0,0 +1,10 @@ +{% extends "base.html" %} +{% block content %} +

Neue Vorlage hochladen

+

DOCX-Datei mit Platzhaltern wie {{ vorname }}, {{ nachname }}, … Die Spaltennamen der späteren CSV müssen mit den Platzhaltern übereinstimmen.

+
+ {% csrf_token %} + {{ form.as_p }} + +
+{% endblock %} diff --git a/app/templates/registration/login.html b/app/templates/registration/login.html new file mode 100644 index 0000000..31a85e8 --- /dev/null +++ b/app/templates/registration/login.html @@ -0,0 +1,10 @@ +{% extends "base.html" %} +{% block title %}Anmelden{% endblock %} +{% block content %} +

Anmelden

+
+ {% csrf_token %} + {{ form.as_p }} + +
+{% endblock %} diff --git a/docker-compose.override.yml b/docker-compose.override.yml new file mode 100644 index 0000000..60668ef --- /dev/null +++ b/docker-compose.override.yml @@ -0,0 +1,65 @@ +# ============================================================================= +# Dev-Overrides – wird von "docker compose" automatisch mit gemerged. +# Für reine Produktion: `docker compose -f docker-compose.yml up -d` +# ============================================================================= +# Wichtig: Nur "web" hat einen build-Block. worker/beat teilen das gleiche +# Image und dürfen daher KEINEN build-Override haben – sonst versucht Bake, +# sie eigenständig zu bauen und scheitert mangels Dockerfile-Kontext. +# ============================================================================= + +services: + web: + build: + target: dev + args: + APP_UID: "${APP_UID:-1000}" + APP_GID: "${APP_GID:-1000}" + command: ["python", "manage.py", "runserver", "0.0.0.0:8000"] + environment: + DJANGO_SETTINGS_MODULE: config.settings.dev + ROLE: web + ports: + - "127.0.0.1:8000:8000" + - "127.0.0.1:5678:5678" + volumes: + - ./app:/app:cached + - static_files:/app/staticfiles + - media_files:/app/media + read_only: false + healthcheck: + disable: true + + worker: + command: ["celery", "-A", "config", "worker", "--loglevel=debug", "--concurrency=1"] + environment: + DJANGO_SETTINGS_MODULE: config.settings.dev + ROLE: worker + volumes: + - ./app:/app:cached + - media_files:/app/media + read_only: false + healthcheck: + disable: true + + beat: + environment: + DJANGO_SETTINGS_MODULE: config.settings.dev + volumes: + - ./app:/app:cached + read_only: false + + nginx: + depends_on: + web: + condition: service_started + + db: + ports: + - "127.0.0.1:5432:5432" + + redis: + ports: + - "127.0.0.1:6379:6379" + + backup: + profiles: ["never"] diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..781ac80 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,309 @@ +# ============================================================================= +# Serienbrief – Compose (Production) +# Ubuntu Server 22.04/24.04 LTS · Docker Engine 27.x · Compose v2 +# ============================================================================= +# Die App spricht intern nur HTTP. TLS terminiert ein vorgelagerter Nginx- +# Reverse-Proxy (außerhalb dieses Compose-Stacks, z.B. zentraler LAN-Proxy). +# Der hier enthaltene "nginx"-Service ist NUR ein App-interner Proxy für: +# - Ausspielen von static/media über X-Accel-Redirect +# - Connection-Pooling und einfache Rate-Limits +# ============================================================================= + +name: serienbrief + +x-logging: &default-logging + driver: "json-file" + options: + max-size: "10m" + max-file: "5" + tag: "{{.Name}}" + +x-restart: &default-restart + restart: unless-stopped + +x-security-opts: &default-security-opts + security_opt: + - no-new-privileges:true + +services: + # --------------------------------------------------------------------------- + # App-interner Nginx – nimmt Traffic vom äußeren Proxy entgegen (HTTP). + # Hört nur auf Loopback bzw. dem konfigurierten LAN-Bind. + # --------------------------------------------------------------------------- + nginx: + image: nginx:1.27-alpine + <<: [*default-restart, *default-security-opts] + depends_on: + web: + condition: service_healthy + ports: + # Standardmäßig nur lokal – der externe Proxy spricht über das Docker- + # Host-Interface. Für direkten LAN-Zugriff LAN_BIND_IP in .env setzen. + - "${APP_BIND_IP:-127.0.0.1}:${APP_BIND_PORT:-8080}:80" + volumes: + - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro + - ./nginx/conf.d:/etc/nginx/conf.d:ro + - static_files:/var/www/static:ro + - media_files:/var/www/media:ro + - nginx_cache:/var/cache/nginx + - nginx_run:/var/run + networks: + - frontend + logging: *default-logging + healthcheck: + test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://127.0.0.1/healthz"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 10s + deploy: + resources: + limits: + memory: 128M + cpus: "0.5" + + # --------------------------------------------------------------------------- + # Django – Gunicorn WSGI + # --------------------------------------------------------------------------- + web: + build: + context: ./app + dockerfile: Dockerfile + target: runtime + args: + APP_UID: ${APP_UID:-10001} + APP_GID: ${APP_GID:-10001} + image: serienbrief/web:${APP_VERSION:-latest} + <<: [*default-restart, *default-security-opts] + depends_on: + db: + condition: service_healthy + redis: + condition: service_healthy + env_file: .env + environment: + DJANGO_SETTINGS_MODULE: config.settings.production + ROLE: web + volumes: + - static_files:/app/staticfiles + - media_files:/app/media + networks: + - frontend + - backend + logging: *default-logging + healthcheck: + test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=3).status==200 else 1)"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 30s + read_only: true + tmpfs: + - /tmp:size=256M,mode=1777 + deploy: + resources: + limits: + memory: 1G + cpus: "1.5" + + # --------------------------------------------------------------------------- + # Celery Worker – DOCX→PDF-Generierung + # --------------------------------------------------------------------------- + worker: + image: serienbrief/web:${APP_VERSION:-latest} + <<: [*default-restart, *default-security-opts] + depends_on: + db: + condition: service_healthy + redis: + condition: service_healthy + web: + condition: service_started + env_file: .env + environment: + DJANGO_SETTINGS_MODULE: config.settings.production + ROLE: worker + command: ["celery", "-A", "config", "worker", "--loglevel=info", "--concurrency=2", "--max-tasks-per-child=50"] + volumes: + - media_files:/app/media + networks: + - backend + logging: *default-logging + healthcheck: + test: ["CMD-SHELL", "celery -A config inspect ping -d celery@$$HOSTNAME || exit 1"] + interval: 60s + timeout: 10s + retries: 3 + start_period: 30s + read_only: true + tmpfs: + - /tmp:size=512M,mode=1777 + deploy: + resources: + limits: + memory: 2G + cpus: "2.0" + + # --------------------------------------------------------------------------- + # Celery Beat – Scheduler (Retention, Cleanup) + # --------------------------------------------------------------------------- + beat: + image: serienbrief/web:${APP_VERSION:-latest} + <<: [*default-restart, *default-security-opts] + depends_on: + db: + condition: service_healthy + redis: + condition: service_healthy + env_file: .env + environment: + DJANGO_SETTINGS_MODULE: config.settings.production + ROLE: beat + command: ["celery", "-A", "config", "beat", "--loglevel=info", "--scheduler", "django_celery_beat.schedulers:DatabaseScheduler"] + networks: + - backend + logging: *default-logging + read_only: true + tmpfs: + - /tmp:size=64M,mode=1777 + deploy: + resources: + limits: + memory: 256M + cpus: "0.3" + + # --------------------------------------------------------------------------- + # PostgreSQL + # --------------------------------------------------------------------------- + db: + image: postgres:16-alpine + <<: [*default-restart, *default-security-opts] + environment: + POSTGRES_DB: ${POSTGRES_DB} + POSTGRES_USER: ${POSTGRES_USER} + POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password + PGDATA: /var/lib/postgresql/data/pgdata + secrets: + - postgres_password + volumes: + - postgres_data:/var/lib/postgresql/data + - ./postgres/init:/docker-entrypoint-initdb.d:ro + networks: + - backend + logging: *default-logging + healthcheck: + test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"] + interval: 10s + timeout: 5s + retries: 5 + start_period: 20s + shm_size: 256mb + deploy: + resources: + limits: + memory: 1G + cpus: "1.0" + + # --------------------------------------------------------------------------- + # Redis – Celery Broker + # --------------------------------------------------------------------------- + redis: + image: redis:7-alpine + <<: [*default-restart, *default-security-opts] + command: + - "redis-server" + - "--requirepass" + - "${REDIS_PASSWORD}" + - "--maxmemory" + - "256mb" + - "--maxmemory-policy" + - "allkeys-lru" + - "--save" + - "900 1" + - "--appendonly" + - "yes" + volumes: + - redis_data:/data + networks: + - backend + logging: *default-logging + healthcheck: + test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "--no-auth-warning", "ping"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 10s + deploy: + resources: + limits: + memory: 384M + cpus: "0.5" + + # --------------------------------------------------------------------------- + # Backup – pg_dump + Media-Tar, nightly, 14 Tage Retention + # --------------------------------------------------------------------------- + backup: + image: postgres:16-alpine + <<: [*default-restart, *default-security-opts] + depends_on: + db: + condition: service_healthy + environment: + PGHOST: db + PGUSER: ${POSTGRES_USER} + PGDATABASE: ${POSTGRES_DB} + PGPASSWORD_FILE: /run/secrets/postgres_password + BACKUP_RETENTION_DAYS: "14" + secrets: + - postgres_password + volumes: + - ./backups:/backups + - media_files:/media:ro + networks: + - backend + entrypoint: ["/bin/sh", "-c"] + command: + - | + apk add --no-cache tar gzip findutils >/dev/null && \ + while true; do + TS=$$(date +%Y%m%d_%H%M%S) + export PGPASSWORD=$$(cat $$PGPASSWORD_FILE) + echo "[$$(date -Iseconds)] starting backup $$TS" + pg_dump -Fc -f /backups/db_$$TS.dump && \ + tar czf /backups/media_$$TS.tar.gz -C /media . && \ + find /backups -name "db_*.dump" -mtime +$$BACKUP_RETENTION_DAYS -delete && \ + find /backups -name "media_*.tar.gz" -mtime +$$BACKUP_RETENTION_DAYS -delete && \ + echo "[$$(date -Iseconds)] backup done" + sleep 86400 + done + logging: *default-logging + deploy: + resources: + limits: + memory: 256M + cpus: "0.5" + +# ============================================================================= +networks: + frontend: + driver: bridge + driver_opts: + com.docker.network.bridge.name: br-sb-front + backend: + driver: bridge + driver_opts: + com.docker.network.bridge.name: br-sb-back + +# ============================================================================= +volumes: + postgres_data: + redis_data: + static_files: + media_files: + nginx_cache: + nginx_run: + +# ============================================================================= +secrets: + postgres_password: + file: ./secrets/postgres_password.txt diff --git a/nginx/conf.d/proxy_params.inc b/nginx/conf.d/proxy_params.inc new file mode 100644 index 0000000..906d055 --- /dev/null +++ b/nginx/conf.d/proxy_params.inc @@ -0,0 +1,9 @@ +proxy_http_version 1.1; +proxy_set_header Host $host; +proxy_set_header X-Real-IP $remote_addr; +proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; +proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto; +proxy_set_header X-Forwarded-Host $host; +proxy_set_header Connection ""; +proxy_redirect off; +proxy_buffering on; diff --git a/nginx/conf.d/serienbrief.conf b/nginx/conf.d/serienbrief.conf new file mode 100644 index 0000000..647e5a1 --- /dev/null +++ b/nginx/conf.d/serienbrief.conf @@ -0,0 +1,55 @@ +# ============================================================================= +# vHost – HTTP only. Security-Header & TLS sind Aufgabe des äußeren Proxys. +# ============================================================================= + +# Vom äußeren Proxy weitergereichte Header vertrauen – aber NUR aus dem +# Docker-Netz oder von der bekannten Proxy-IP. Bei Bedarf set_real_ip_from +# auf das CIDR des Proxys einschränken. +set_real_ip_from 0.0.0.0/0; +real_ip_header X-Forwarded-For; +real_ip_recursive on; + +server { + listen 80 default_server; + listen [::]:80 default_server; + server_name _; + + # Healthcheck (für äußeren Proxy & Compose-Healthcheck) + location = /healthz { + access_log off; + return 200 "ok\n"; + add_header Content-Type text/plain; + } + + limit_conn conn_per_ip 20; + + # Login strikter limitieren + location ~ ^/(accounts/login|admin/login) { + limit_req zone=login burst=3 nodelay; + proxy_pass http://django_app; + include /etc/nginx/conf.d/proxy_params.inc; + } + + # Statische Dateien + location /static/ { + alias /var/www/static/; + access_log off; + expires 7d; + add_header Cache-Control "public, immutable"; + } + + # Geschützte Media (PDFs) – nur per X-Accel-Redirect aus Django ausspielen + location /protected-media/ { + internal; + alias /var/www/media/; + } + + # App + location / { + limit_req zone=app burst=50 nodelay; + proxy_pass http://django_app; + include /etc/nginx/conf.d/proxy_params.inc; + proxy_read_timeout 120s; + proxy_send_timeout 120s; + } +} diff --git a/nginx/nginx.conf b/nginx/nginx.conf new file mode 100644 index 0000000..bdf3d22 --- /dev/null +++ b/nginx/nginx.conf @@ -0,0 +1,58 @@ +# ============================================================================= +# App-interner Nginx – HTTP only. +# TLS macht der äußere Proxy. +# ============================================================================= + +user nginx; +worker_processes auto; +worker_rlimit_nofile 8192; +pid /var/run/nginx.pid; +error_log /var/log/nginx/error.log warn; + +events { + worker_connections 2048; + multi_accept on; +} + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + log_format main_ext '$remote_addr - $remote_user [$time_iso8601] ' + '"$request" $status $body_bytes_sent ' + '"$http_referer" "$http_user_agent" ' + 'xff="$http_x_forwarded_for" ' + 'rt=$request_time urt="$upstream_response_time"'; + access_log /var/log/nginx/access.log main_ext; + + sendfile on; + tcp_nopush on; + tcp_nodelay on; + keepalive_timeout 65; + server_tokens off; + + # CSVs etc. – am äußeren Proxy spiegeln! + client_max_body_size 25M; + client_body_buffer_size 128k; + client_body_timeout 60s; + client_header_timeout 30s; + + gzip on; + gzip_vary on; + gzip_proxied any; + gzip_comp_level 6; + gzip_types text/plain text/css application/json application/javascript + text/xml application/xml application/xml+rss text/javascript + image/svg+xml; + + limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m; + limit_req_zone $binary_remote_addr zone=app:10m rate=30r/s; + limit_conn_zone $binary_remote_addr zone=conn_per_ip:10m; + + upstream django_app { + server web:8000 max_fails=3 fail_timeout=30s; + keepalive 32; + } + + include /etc/nginx/conf.d/*.conf; +} diff --git a/serienbrief/.devcontainer/devcontainer.json b/serienbrief/.devcontainer/devcontainer.json new file mode 100644 index 0000000..1d03a51 --- /dev/null +++ b/serienbrief/.devcontainer/devcontainer.json @@ -0,0 +1,57 @@ +{ + "name": "Serienbrief Dev", + "dockerComposeFile": [ + "../docker-compose.yml", + "../docker-compose.override.yml" + ], + "service": "web", + "workspaceFolder": "/app", + "remoteUser": "app", + "shutdownAction": "stopCompose", + + "customizations": { + "vscode": { + "extensions": [ + "ms-python.python", + "ms-python.vscode-pylance", + "ms-python.debugpy", + "charliermarsh.ruff", + "batisteo.vscode-django", + "ms-azuretools.vscode-docker", + "redhat.vscode-yaml", + "tamasfe.even-better-toml", + "eamodio.gitlens", + "njpwerner.autodocstring" + ], + "settings": { + "python.defaultInterpreterPath": "/usr/local/bin/python", + "python.terminal.activateEnvironment": false, + "python.testing.pytestEnabled": true, + "python.testing.pytestArgs": ["app"], + "[python]": { + "editor.formatOnSave": true, + "editor.defaultFormatter": "charliermarsh.ruff", + "editor.codeActionsOnSave": { + "source.organizeImports": "explicit", + "source.fixAll": "explicit" + } + }, + "files.exclude": { + "**/__pycache__": true, + "**/*.pyc": true + } + } + } + }, + + "forwardPorts": [8000, 5678, 5432, 6379, 8080], + "portsAttributes": { + "8000": { "label": "Django runserver" }, + "5678": { "label": "debugpy" }, + "8080": { "label": "App-Nginx" }, + "5432": { "label": "PostgreSQL" }, + "6379": { "label": "Redis" } + }, + + "postCreateCommand": "pip install --user -e . 2>/dev/null || true && python manage.py migrate --noinput || true" +} diff --git a/serienbrief/.env.example b/serienbrief/.env.example new file mode 100644 index 0000000..c6e34ed --- /dev/null +++ b/serienbrief/.env.example @@ -0,0 +1,46 @@ +# ============================================================================= +# Serienbrief – Environment +# Kopieren als .env. NICHT ins Git committen. +# ============================================================================= + +# --- Deployment ------------------------------------------------------------- +APP_VERSION=1.0.0 +APP_UID=10001 +APP_GID=10001 + +# Bind des internen Nginx-Sockets. Wird vom äußeren Reverse-Proxy angesprochen. +# - 127.0.0.1: nur lokal (z.B. wenn der äußere Proxy auf demselben Host läuft) +# - LAN-IP: wenn der äußere Proxy auf einem anderen Host steht +APP_BIND_IP=127.0.0.1 +APP_BIND_PORT=8080 + +# --- Django ----------------------------------------------------------------- +# openssl rand -base64 64 +DJANGO_SECRET_KEY=CHANGE_ME_LONG_RANDOM_STRING +DJANGO_DEBUG=False + +# Hosts, unter denen die App erreichbar ist (Domain des äußeren Proxys). +DJANGO_ALLOWED_HOSTS=serienbrief.lan,127.0.0.1,localhost + +# Trusted Origins für CSRF – HTTPS-URL des äußeren Proxys. +CSRF_TRUSTED_ORIGINS=https://serienbrief.lan + +# Wichtig hinter HTTPS-terminierendem Proxy: erlaubt Django, das Schema aus +# dem X-Forwarded-Proto-Header zu übernehmen. +USE_X_FORWARDED_HOST=True +SECURE_PROXY_SSL_HEADER=HTTP_X_FORWARDED_PROTO,https + +# --- PostgreSQL ------------------------------------------------------------- +POSTGRES_DB=serienbrief +POSTGRES_USER=serienbrief +# Passwort liegt in ./secrets/postgres_password.txt +DATABASE_URL=postgres://serienbrief:__FROM_SECRET__@db:5432/serienbrief + +# --- Redis ------------------------------------------------------------------ +# openssl rand -base64 32 +REDIS_PASSWORD=CHANGE_ME_REDIS_PASSWORD +CELERY_BROKER_URL=redis://:${REDIS_PASSWORD}@redis:6379/0 +CELERY_RESULT_BACKEND=redis://:${REDIS_PASSWORD}@redis:6379/1 + +# --- Retention -------------------------------------------------------------- +JOB_RETENTION_DAYS=30 diff --git a/serienbrief/.gitignore b/serienbrief/.gitignore new file mode 100644 index 0000000..4fd059b --- /dev/null +++ b/serienbrief/.gitignore @@ -0,0 +1,39 @@ +# --- Secrets / Env --- +.env +.env.* +!.env.example +secrets/ +*.key +*.pem + +# --- Python --- +__pycache__/ +*.py[cod] +*$py.class +*.so +.Python +.pytest_cache/ +.ruff_cache/ +.mypy_cache/ +.coverage +htmlcov/ +.tox/ + +# --- Django --- +*.sqlite3 +staticfiles/ +media/ + +# --- Backups & Data --- +backups/ +postgres_data/ +redis_data/ + +# --- IDE --- +.idea/ +*.swp +*.swo + +# --- OS --- +.DS_Store +Thumbs.db diff --git a/serienbrief/.vscode/extensions.json b/serienbrief/.vscode/extensions.json new file mode 100644 index 0000000..8fb2c50 --- /dev/null +++ b/serienbrief/.vscode/extensions.json @@ -0,0 +1,14 @@ +{ + "recommendations": [ + "ms-python.python", + "ms-python.vscode-pylance", + "ms-python.debugpy", + "charliermarsh.ruff", + "batisteo.vscode-django", + "ms-azuretools.vscode-docker", + "ms-vscode-remote.remote-containers", + "redhat.vscode-yaml", + "tamasfe.even-better-toml", + "eamodio.gitlens" + ] +} diff --git a/serienbrief/.vscode/launch.json b/serienbrief/.vscode/launch.json new file mode 100644 index 0000000..de50b70 --- /dev/null +++ b/serienbrief/.vscode/launch.json @@ -0,0 +1,53 @@ +{ + "version": "0.2.0", + "configurations": [ + { + "name": "Django: attach (debugpy in Container)", + "type": "debugpy", + "request": "attach", + "connect": { "host": "127.0.0.1", "port": 5678 }, + "pathMappings": [ + { + "localRoot": "${workspaceFolder}/app", + "remoteRoot": "/app" + } + ], + "justMyCode": false, + "django": true + }, + { + "name": "Django: runserver (lokal, ohne Docker)", + "type": "debugpy", + "request": "launch", + "program": "${workspaceFolder}/app/manage.py", + "args": ["runserver", "127.0.0.1:8000"], + "django": true, + "justMyCode": false, + "env": { + "DJANGO_SETTINGS_MODULE": "config.settings.dev" + } + }, + { + "name": "Celery Worker (lokal, ohne Docker)", + "type": "debugpy", + "request": "launch", + "module": "celery", + "args": ["-A", "config", "worker", "--loglevel=debug", "--concurrency=1"], + "cwd": "${workspaceFolder}/app", + "justMyCode": false, + "env": { + "DJANGO_SETTINGS_MODULE": "config.settings.dev" + } + }, + { + "name": "Pytest: aktuelle Datei", + "type": "debugpy", + "request": "launch", + "module": "pytest", + "args": ["${file}", "-v"], + "cwd": "${workspaceFolder}/app", + "console": "integratedTerminal", + "justMyCode": false + } + ] +} diff --git a/serienbrief/.vscode/settings.json b/serienbrief/.vscode/settings.json new file mode 100644 index 0000000..ef08bb5 --- /dev/null +++ b/serienbrief/.vscode/settings.json @@ -0,0 +1,38 @@ +{ + "python.analysis.extraPaths": ["app"], + "python.analysis.typeCheckingMode": "basic", + "python.testing.pytestEnabled": true, + "python.testing.pytestArgs": ["app"], + "python.testing.cwd": "${workspaceFolder}/app", + + "[python]": { + "editor.formatOnSave": true, + "editor.defaultFormatter": "charliermarsh.ruff", + "editor.codeActionsOnSave": { + "source.organizeImports": "explicit", + "source.fixAll": "explicit" + } + }, + + "ruff.lineLength": 100, + + "files.associations": { + "**/templates/**/*.html": "django-html", + "**/requirements*.txt": "pip-requirements" + }, + "emmet.includeLanguages": { + "django-html": "html" + }, + + "files.exclude": { + "**/__pycache__": true, + "**/*.pyc": true, + "**/.pytest_cache": true, + "**/.ruff_cache": true + }, + "search.exclude": { + "**/staticfiles": true, + "**/media": true, + "**/backups": true + } +} diff --git a/serienbrief/.vscode/tasks.json b/serienbrief/.vscode/tasks.json new file mode 100644 index 0000000..74b4cda --- /dev/null +++ b/serienbrief/.vscode/tasks.json @@ -0,0 +1,72 @@ +{ + "version": "2.0.0", + "tasks": [ + { + "label": "compose: up", + "type": "shell", + "command": "docker compose up -d", + "problemMatcher": [] + }, + { + "label": "compose: down", + "type": "shell", + "command": "docker compose down", + "problemMatcher": [] + }, + { + "label": "compose: logs web", + "type": "shell", + "command": "docker compose logs -f web", + "problemMatcher": [] + }, + { + "label": "compose: rebuild web", + "type": "shell", + "command": "docker compose build web && docker compose up -d web", + "problemMatcher": [] + }, + { + "label": "django: makemigrations", + "type": "shell", + "command": "docker compose exec web python manage.py makemigrations", + "problemMatcher": [] + }, + { + "label": "django: migrate", + "type": "shell", + "command": "docker compose exec web python manage.py migrate", + "problemMatcher": [] + }, + { + "label": "django: shell", + "type": "shell", + "command": "docker compose exec web python manage.py shell", + "problemMatcher": [] + }, + { + "label": "django: createsuperuser", + "type": "shell", + "command": "docker compose exec web python manage.py createsuperuser", + "problemMatcher": [] + }, + { + "label": "django: start debugpy", + "type": "shell", + "command": "docker compose exec web python -m debugpy --listen 0.0.0.0:5678 manage.py runserver 0.0.0.0:8000 --noreload", + "problemMatcher": [] + }, + { + "label": "tests: pytest", + "type": "shell", + "command": "docker compose exec web pytest -v", + "group": "test", + "problemMatcher": [] + }, + { + "label": "lint: ruff", + "type": "shell", + "command": "docker compose exec web ruff check .", + "problemMatcher": [] + } + ] +} diff --git a/serienbrief/README.md b/serienbrief/README.md new file mode 100644 index 0000000..db26d1a --- /dev/null +++ b/serienbrief/README.md @@ -0,0 +1,125 @@ +# Serienbrief + +Django-Webanwendung zur Erzeugung von Serienbriefen aus DOCX-Vorlagen und CSV-Empfängerlisten. +Läuft als Compose-Stack, HTTP-only — TLS terminiert ein **vorgelagerter Nginx-Reverse-Proxy** außerhalb dieses Stacks. + +## Architektur + +``` +LAN-Client ──HTTPS──▶ Externer Nginx-Proxy ──HTTP──▶ App-Stack (dieses Repo) + (TLS-Terminierung) │ + ├─ nginx (intern, :8080) + ├─ web (Gunicorn / Django) + ├─ worker (Celery) + ├─ beat (Celery Scheduler) + ├─ db (PostgreSQL 16) + ├─ redis (Celery Broker) + └─ backup (nightly pg_dump) +``` + +Der externe Proxy muss diese Header setzen: +- `X-Forwarded-Proto: https` +- `X-Forwarded-For: ` +- `Host: ` + +## Verzeichnisstruktur + +``` +serienbrief/ +├── docker-compose.yml # Prod-Stack +├── docker-compose.override.yml # Dev-Overrides (Auto-Merge) +├── .env.example # Konfig-Template +├── .devcontainer/ # VS Code Dev-Container +├── .vscode/ # Launch, Tasks, Settings +├── nginx/ # App-interner Nginx (HTTP) +├── app/ # Django-Projekt +│ ├── Dockerfile # Multi-Stage: builder/dev/runtime +│ ├── requirements.txt +│ ├── requirements-dev.txt +│ ├── pyproject.toml # Ruff & Pytest +│ ├── manage.py +│ ├── config/ # Settings, URLs, WSGI +│ └── mailmerge/ # Django-App: Models, Views, Tasks +├── postgres/init/ # SQL-Init-Scripts (optional) +├── secrets/ # Passwort-Files (chmod 600) +└── backups/ # DB- & Media-Dumps +``` + +## Erstinbetriebnahme (Prod) + +```bash +cp .env.example .env +nano .env # Werte setzen + +mkdir -p secrets +openssl rand -base64 32 > secrets/postgres_password.txt +chmod 600 secrets/postgres_password.txt + +docker compose -f docker-compose.yml build +docker compose -f docker-compose.yml up -d +docker compose exec web python manage.py createsuperuser +``` + +Der externe Proxy zeigt dann z.B. so auf den Stack: +```nginx +location / { + proxy_pass http://app-host:8080; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto https; +} +``` + +## Entwicklung in VS Code + +### Variante A — Dev-Container (empfohlen) + +1. VS Code öffnet das Projekt +2. Extension **Dev Containers** installiert +3. Command Palette → `Dev Containers: Reopen in Container` +4. VS Code startet Compose mit Override (Build-Target `dev`, Code-Mount, Hot-Reload) +5. Terminal im Container: `python manage.py migrate` +6. Browser: `http://localhost:8000` + +### Variante B — Lokal mit Compose im Hintergrund + +```bash +docker compose up -d # Override wird automatisch geladen +docker compose logs -f web +``` + +Code-Änderungen sind sofort wirksam (Bind-Mount + `runserver` mit Auto-Reload). + +### Debugging + +Im Container läuft `debugpy` auf Port **5678**. In VS Code: +- Run & Debug → **Django: attach (debugpy in Container)** +- Breakpoints überall im Code setzen + +Alternativ Task **django: start debugpy** ausführen und attachen. + +### Häufige VS-Code-Tasks + +`Strg+Shift+P` → `Tasks: Run Task`: + +| Task | Wirkung | +|---|---| +| compose: up | Stack starten | +| compose: logs web | Live-Logs | +| django: makemigrations | Migrations erzeugen | +| django: migrate | Migrations anwenden | +| django: shell | Django-Shell | +| django: createsuperuser | Admin-User anlegen | +| tests: pytest | Tests laufen lassen | +| lint: ruff | Linting | + +## Härtungs-Hinweise + +- Externe Proxy-Konfiguration (TLS, HSTS, CSP, Rate-Limit) **muss** vorhanden sein +- `APP_BIND_IP=127.0.0.1` in `.env` außer der externe Proxy läuft auf anderem Host +- `DJANGO_ALLOWED_HOSTS` und `CSRF_TRUSTED_ORIGINS` strikt setzen +- `SECURE_PROXY_SSL_HEADER` ist gesetzt — damit erkennt Django korrekt, dass der Client über HTTPS kam +- Postgres und Redis sind **nicht** nach außen exponiert (nur im backend-Netz) +- Container laufen non-root, `read_only`, mit `no-new-privileges` +- Backups regelmäßig auf zweites System spiegeln (Borg/Restic) diff --git a/serienbrief/app/Dockerfile b/serienbrief/app/Dockerfile new file mode 100644 index 0000000..78f3d64 --- /dev/null +++ b/serienbrief/app/Dockerfile @@ -0,0 +1,86 @@ +# ============================================================================= +# Multi-Stage: builder → dev → runtime +# Dev-Image enthält zusätzlich debugpy, ipython, django-debug-toolbar. +# ============================================================================= + +# ---------- Stage 1: Build ---------------------------------------------------- +FROM python:3.12-slim-bookworm AS builder + +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 \ + PIP_NO_CACHE_DIR=1 \ + PIP_DISABLE_PIP_VERSION_CHECK=1 + +RUN apt-get update && apt-get install -y --no-install-recommends \ + build-essential libpq-dev gcc \ + && rm -rf /var/lib/apt/lists/* + +WORKDIR /build +COPY requirements.txt requirements-dev.txt ./ +RUN pip wheel --wheel-dir=/wheels -r requirements.txt -r requirements-dev.txt + + +# ---------- Common Runtime Base ---------------------------------------------- +FROM python:3.12-slim-bookworm AS runtime-base + +ARG APP_UID=10001 +ARG APP_GID=10001 + +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 \ + PIP_NO_CACHE_DIR=1 \ + PYTHONPATH=/app + +RUN apt-get update && apt-get install -y --no-install-recommends \ + libpq5 \ + libreoffice libreoffice-writer \ + fonts-liberation fonts-dejavu \ + tini curl \ + && apt-get clean \ + && rm -rf /var/lib/apt/lists/* /var/cache/apt/* /tmp/* + +RUN groupadd -g ${APP_GID} app && \ + useradd -u ${APP_UID} -g ${APP_GID} -m -s /bin/bash app + +WORKDIR /app +COPY --from=builder /wheels /wheels + + +# ---------- Stage 2a: Runtime (Production) ------------------------------------ +FROM runtime-base AS runtime + +COPY requirements.txt . +RUN pip install --no-index --find-links=/wheels -r requirements.txt && \ + rm -rf /wheels + +COPY --chown=app:app . /app/ +RUN mkdir -p /app/staticfiles /app/media && chown -R app:app /app +USER app +EXPOSE 8000 +ENTRYPOINT ["/usr/bin/tini", "--", "/app/entrypoint.sh"] +CMD ["gunicorn", "config.wsgi:application", \ + "--bind", "0.0.0.0:8000", \ + "--workers", "3", "--threads", "2", \ + "--worker-class", "gthread", "--worker-tmp-dir", "/tmp", \ + "--access-logfile", "-", "--error-logfile", "-", \ + "--timeout", "120"] + + +# ---------- Stage 2b: Dev ----------------------------------------------------- +FROM runtime-base AS dev + +# Dev-Tools für Container & VS Code +RUN apt-get update && apt-get install -y --no-install-recommends \ + git bash-completion vim less procps iputils-ping \ + && apt-get clean && rm -rf /var/lib/apt/lists/* + +COPY requirements.txt requirements-dev.txt ./ +RUN pip install --no-index --find-links=/wheels -r requirements.txt -r requirements-dev.txt && \ + rm -rf /wheels + +# Code wird im Dev via Volume gemountet; nichts kopieren. +RUN mkdir -p /app/staticfiles /app/media && chown -R app:app /app +USER app +EXPOSE 8000 5678 +ENTRYPOINT ["/usr/bin/tini", "--", "/app/entrypoint.sh"] +CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"] diff --git a/serienbrief/app/config/__init__.py b/serienbrief/app/config/__init__.py new file mode 100644 index 0000000..370372a --- /dev/null +++ b/serienbrief/app/config/__init__.py @@ -0,0 +1,3 @@ +from .celery import app as celery_app + +__all__ = ["celery_app"] diff --git a/serienbrief/app/config/asgi.py b/serienbrief/app/config/asgi.py new file mode 100644 index 0000000..5096b23 --- /dev/null +++ b/serienbrief/app/config/asgi.py @@ -0,0 +1,6 @@ +import os + +from django.core.asgi import get_asgi_application + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production") +application = get_asgi_application() diff --git a/serienbrief/app/config/celery.py b/serienbrief/app/config/celery.py new file mode 100644 index 0000000..020f2a5 --- /dev/null +++ b/serienbrief/app/config/celery.py @@ -0,0 +1,9 @@ +import os + +from celery import Celery + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production") + +app = Celery("serienbrief") +app.config_from_object("django.conf:settings", namespace="CELERY") +app.autodiscover_tasks() diff --git a/serienbrief/app/config/settings/__init__.py b/serienbrief/app/config/settings/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/serienbrief/app/config/settings/base.py b/serienbrief/app/config/settings/base.py new file mode 100644 index 0000000..f10dce3 --- /dev/null +++ b/serienbrief/app/config/settings/base.py @@ -0,0 +1,144 @@ +""" +Basis-Settings. Werden von dev.py und production.py erweitert. +""" +from pathlib import Path + +import environ + +BASE_DIR = Path(__file__).resolve().parent.parent.parent + +env = environ.Env( + DJANGO_DEBUG=(bool, False), + USE_X_FORWARDED_HOST=(bool, True), + JOB_RETENTION_DAYS=(int, 30), +) + +# --- Core -------------------------------------------------------------------- +SECRET_KEY = env("DJANGO_SECRET_KEY", default="dev-insecure-change-me") +DEBUG = env("DJANGO_DEBUG") +ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=["*"]) +CSRF_TRUSTED_ORIGINS = env.list("CSRF_TRUSTED_ORIGINS", default=[]) + +# Hinter dem äußeren Reverse-Proxy +USE_X_FORWARDED_HOST = env("USE_X_FORWARDED_HOST") +_proxy_header = env("SECURE_PROXY_SSL_HEADER", default="") +if _proxy_header: + name, value = _proxy_header.split(",", 1) + SECURE_PROXY_SSL_HEADER = (name.strip(), value.strip()) + +# --- Apps -------------------------------------------------------------------- +INSTALLED_APPS = [ + "django.contrib.admin", + "django.contrib.auth", + "django.contrib.contenttypes", + "django.contrib.sessions", + "django.contrib.messages", + "django.contrib.staticfiles", + # 3rd party + "django_celery_beat", + "django_celery_results", + "django_htmx", + "axes", + # local + "mailmerge", +] + +MIDDLEWARE = [ + "django.middleware.security.SecurityMiddleware", + "django.contrib.sessions.middleware.SessionMiddleware", + "django.middleware.common.CommonMiddleware", + "django.middleware.csrf.CsrfViewMiddleware", + "django.contrib.auth.middleware.AuthenticationMiddleware", + "django.contrib.messages.middleware.MessageMiddleware", + "django.middleware.clickjacking.XFrameOptionsMiddleware", + "django_htmx.middleware.HtmxMiddleware", + "axes.middleware.AxesMiddleware", +] + +ROOT_URLCONF = "config.urls" + +TEMPLATES = [ + { + "BACKEND": "django.template.backends.django.DjangoTemplates", + "DIRS": [BASE_DIR / "templates"], + "APP_DIRS": True, + "OPTIONS": { + "context_processors": [ + "django.template.context_processors.debug", + "django.template.context_processors.request", + "django.contrib.auth.context_processors.auth", + "django.contrib.messages.context_processors.messages", + ], + }, + }, +] + +WSGI_APPLICATION = "config.wsgi.application" + +# --- Database ---------------------------------------------------------------- +DATABASES = { + "default": env.db_url( + "DATABASE_URL", default="sqlite:///" + str(BASE_DIR / "db.sqlite3") + ), +} + +# --- Auth -------------------------------------------------------------------- +AUTH_PASSWORD_VALIDATORS = [ + {"NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator"}, + {"NAME": "django.contrib.auth.password_validation.MinimumLengthValidator", + "OPTIONS": {"min_length": 12}}, + {"NAME": "django.contrib.auth.password_validation.CommonPasswordValidator"}, + {"NAME": "django.contrib.auth.password_validation.NumericPasswordValidator"}, +] +PASSWORD_HASHERS = [ + "django.contrib.auth.hashers.Argon2PasswordHasher", + "django.contrib.auth.hashers.PBKDF2PasswordHasher", +] + +AUTHENTICATION_BACKENDS = [ + "axes.backends.AxesStandaloneBackend", + "django.contrib.auth.backends.ModelBackend", +] + +LOGIN_URL = "/accounts/login/" +LOGIN_REDIRECT_URL = "/" +LOGOUT_REDIRECT_URL = "/accounts/login/" + +# django-axes – Brute-Force-Schutz +AXES_FAILURE_LIMIT = 5 +AXES_COOLOFF_TIME = 1 # Stunde +AXES_LOCKOUT_PARAMETERS = ["username", "ip_address"] + +# --- I18N / TZ --------------------------------------------------------------- +LANGUAGE_CODE = "de-at" +TIME_ZONE = "Europe/Vienna" +USE_I18N = True +USE_TZ = True + +# --- Static / Media ---------------------------------------------------------- +STATIC_URL = "/static/" +STATIC_ROOT = BASE_DIR / "staticfiles" +MEDIA_URL = "/media/" +MEDIA_ROOT = BASE_DIR / "media" + +# --- Celery ------------------------------------------------------------------ +CELERY_BROKER_URL = env("CELERY_BROKER_URL", default="redis://redis:6379/0") +CELERY_RESULT_BACKEND = env("CELERY_RESULT_BACKEND", default="django-db") +CELERY_TASK_TRACK_STARTED = True +CELERY_TASK_TIME_LIMIT = 600 # 10 Minuten Hard-Timeout +CELERY_TASK_SOFT_TIME_LIMIT = 540 +CELERY_WORKER_PREFETCH_MULTIPLIER = 1 +CELERY_TIMEZONE = TIME_ZONE + +# --- App --------------------------------------------------------------------- +JOB_RETENTION_DAYS = env("JOB_RETENTION_DAYS") +DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField" + +# --- Security Defaults ------------------------------------------------------- +SESSION_COOKIE_HTTPONLY = True +SESSION_COOKIE_SAMESITE = "Lax" +CSRF_COOKIE_HTTPONLY = False # Bleibt false, damit JS/HTMX-Forms funktionieren +CSRF_COOKIE_SAMESITE = "Lax" +X_FRAME_OPTIONS = "DENY" +SECURE_CONTENT_TYPE_NOSNIFF = True +SECURE_REFERRER_POLICY = "strict-origin-when-cross-origin" diff --git a/serienbrief/app/config/settings/dev.py b/serienbrief/app/config/settings/dev.py new file mode 100644 index 0000000..b42785a --- /dev/null +++ b/serienbrief/app/config/settings/dev.py @@ -0,0 +1,16 @@ +from .base import * # noqa: F401,F403 +from .base import INSTALLED_APPS, MIDDLEWARE + +DEBUG = True +ALLOWED_HOSTS = ["*"] + +# Debug-Toolbar nur lokal +INSTALLED_APPS = INSTALLED_APPS + ["debug_toolbar"] +MIDDLEWARE = ["debug_toolbar.middleware.DebugToolbarMiddleware", *MIDDLEWARE] +INTERNAL_IPS = ["127.0.0.1"] + +# Im Dev keine Auto-Lockouts beim Testen +AXES_ENABLED = False + +# E-Mails nur in die Konsole +EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend" diff --git a/serienbrief/app/config/settings/production.py b/serienbrief/app/config/settings/production.py new file mode 100644 index 0000000..6fa51d1 --- /dev/null +++ b/serienbrief/app/config/settings/production.py @@ -0,0 +1,32 @@ +from .base import * # noqa: F401,F403 + +DEBUG = False + +# Strikte Security-Defaults – TLS macht der äußere Proxy. +SESSION_COOKIE_SECURE = True +CSRF_COOKIE_SECURE = True +SECURE_HSTS_SECONDS = 0 # HSTS setzt der äußere Proxy +SECURE_HSTS_INCLUDE_SUBDOMAINS = False +SECURE_HSTS_PRELOAD = False + +LOGGING = { + "version": 1, + "disable_existing_loggers": False, + "formatters": { + "verbose": { + "format": "{asctime} {levelname} {name} {message}", + "style": "{", + }, + }, + "handlers": { + "console": { + "class": "logging.StreamHandler", + "formatter": "verbose", + }, + }, + "root": {"handlers": ["console"], "level": "INFO"}, + "loggers": { + "django.security": {"handlers": ["console"], "level": "WARNING", "propagate": False}, + "mailmerge": {"handlers": ["console"], "level": "INFO", "propagate": False}, + }, +} diff --git a/serienbrief/app/config/urls.py b/serienbrief/app/config/urls.py new file mode 100644 index 0000000..f44ac97 --- /dev/null +++ b/serienbrief/app/config/urls.py @@ -0,0 +1,27 @@ +from django.conf import settings +from django.conf.urls.static import static +from django.contrib import admin +from django.contrib.auth import views as auth_views +from django.http import HttpResponse +from django.urls import include, path + + +def healthz(_request): + return HttpResponse("ok", content_type="text/plain") + + +urlpatterns = [ + path("healthz", healthz), + path("admin/", admin.site.urls), + path("accounts/login/", auth_views.LoginView.as_view(), name="login"), + path("accounts/logout/", auth_views.LogoutView.as_view(), name="logout"), + path("", include("mailmerge.urls")), +] + +if settings.DEBUG: + urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) + try: + import debug_toolbar + urlpatterns = [path("__debug__/", include(debug_toolbar.urls)), *urlpatterns] + except ImportError: + pass diff --git a/serienbrief/app/config/wsgi.py b/serienbrief/app/config/wsgi.py new file mode 100644 index 0000000..949d08d --- /dev/null +++ b/serienbrief/app/config/wsgi.py @@ -0,0 +1,6 @@ +import os + +from django.core.wsgi import get_wsgi_application + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production") +application = get_wsgi_application() diff --git a/serienbrief/app/entrypoint.sh b/serienbrief/app/entrypoint.sh new file mode 100755 index 0000000..2c30784 --- /dev/null +++ b/serienbrief/app/entrypoint.sh @@ -0,0 +1,19 @@ +#!/bin/bash +set -euo pipefail +ROLE="${ROLE:-web}" + +if [[ "$ROLE" == "web" ]]; then + echo "[entrypoint] waiting for database..." + python manage.py wait_for_db --timeout 60 || true + + echo "[entrypoint] running migrations..." + python manage.py migrate --noinput + + if [[ "${DJANGO_DEBUG:-False}" != "True" ]]; then + echo "[entrypoint] collecting static files..." + python manage.py collectstatic --noinput --clear + fi +fi + +echo "[entrypoint] starting role=$ROLE: $*" +exec "$@" diff --git a/serienbrief/app/mailmerge/__init__.py b/serienbrief/app/mailmerge/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/serienbrief/app/mailmerge/admin.py b/serienbrief/app/mailmerge/admin.py new file mode 100644 index 0000000..5587a21 --- /dev/null +++ b/serienbrief/app/mailmerge/admin.py @@ -0,0 +1,25 @@ +from django.contrib import admin + +from .models import JobLogEntry, LetterTemplate, MailMergeJob + + +@admin.register(LetterTemplate) +class LetterTemplateAdmin(admin.ModelAdmin): + list_display = ("name", "created_by", "created_at") + search_fields = ("name",) + readonly_fields = ("id", "placeholders", "created_at", "updated_at") + + +@admin.register(MailMergeJob) +class MailMergeJobAdmin(admin.ModelAdmin): + list_display = ("id", "template", "status", "processed_rows", "total_rows", + "created_by", "created_at") + list_filter = ("status",) + readonly_fields = ("id", "created_at", "started_at", "finished_at", + "processed_rows", "total_rows", "error_message") + + +@admin.register(JobLogEntry) +class JobLogEntryAdmin(admin.ModelAdmin): + list_display = ("job", "level", "timestamp") + list_filter = ("level",) diff --git a/serienbrief/app/mailmerge/apps.py b/serienbrief/app/mailmerge/apps.py new file mode 100644 index 0000000..5e5874e --- /dev/null +++ b/serienbrief/app/mailmerge/apps.py @@ -0,0 +1,7 @@ +from django.apps import AppConfig + + +class MailmergeConfig(AppConfig): + default_auto_field = "django.db.models.BigAutoField" + name = "mailmerge" + verbose_name = "Serienbrief" diff --git a/serienbrief/app/mailmerge/forms.py b/serienbrief/app/mailmerge/forms.py new file mode 100644 index 0000000..c966b51 --- /dev/null +++ b/serienbrief/app/mailmerge/forms.py @@ -0,0 +1,34 @@ +from django import forms + +from .models import LetterTemplate, MailMergeJob + + +class LetterTemplateForm(forms.ModelForm): + class Meta: + model = LetterTemplate + fields = ["name", "description", "file"] + widgets = { + "description": forms.Textarea(attrs={"rows": 3}), + } + + def clean_file(self): + f = self.cleaned_data["file"] + if not f.name.lower().endswith(".docx"): + raise forms.ValidationError("Nur .docx-Dateien sind erlaubt.") + if f.size > 10 * 1024 * 1024: + raise forms.ValidationError("Datei zu groß (max. 10 MB).") + return f + + +class MailMergeJobForm(forms.ModelForm): + class Meta: + model = MailMergeJob + fields = ["template", "recipients_csv"] + + def clean_recipients_csv(self): + f = self.cleaned_data["recipients_csv"] + if not f.name.lower().endswith(".csv"): + raise forms.ValidationError("Nur .csv-Dateien sind erlaubt.") + if f.size > 20 * 1024 * 1024: + raise forms.ValidationError("Datei zu groß (max. 20 MB).") + return f diff --git a/serienbrief/app/mailmerge/management/__init__.py b/serienbrief/app/mailmerge/management/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/serienbrief/app/mailmerge/management/commands/__init__.py b/serienbrief/app/mailmerge/management/commands/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/serienbrief/app/mailmerge/management/commands/wait_for_db.py b/serienbrief/app/mailmerge/management/commands/wait_for_db.py new file mode 100644 index 0000000..e6bddd1 --- /dev/null +++ b/serienbrief/app/mailmerge/management/commands/wait_for_db.py @@ -0,0 +1,25 @@ +"""Wartet, bis die DB Connections annimmt.""" +import time + +from django.core.management.base import BaseCommand +from django.db import OperationalError, connections + + +class Command(BaseCommand): + help = "Wartet, bis die Standard-DB verfügbar ist." + + def add_arguments(self, parser): + parser.add_argument("--timeout", type=int, default=60) + + def handle(self, *args, **opts): + deadline = time.time() + opts["timeout"] + while time.time() < deadline: + try: + connections["default"].ensure_connection() + self.stdout.write(self.style.SUCCESS("DB ist bereit.")) + return + except OperationalError: + self.stdout.write("warte auf DB...") + time.sleep(2) + self.stderr.write(self.style.ERROR("DB nach Timeout nicht erreichbar.")) + raise SystemExit(1) diff --git a/serienbrief/app/mailmerge/migrations/__init__.py b/serienbrief/app/mailmerge/migrations/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/serienbrief/app/mailmerge/models.py b/serienbrief/app/mailmerge/models.py new file mode 100644 index 0000000..33d4160 --- /dev/null +++ b/serienbrief/app/mailmerge/models.py @@ -0,0 +1,86 @@ +""" +Datenmodell – Templates, Jobs, Log-Einträge. +""" +import uuid +from pathlib import Path + +from django.conf import settings +from django.db import models + + +def template_upload_path(instance, filename: str) -> str: + return f"templates/{instance.id}/{Path(filename).name}" + + +def csv_upload_path(instance, filename: str) -> str: + return f"jobs/{instance.id}/recipients/{Path(filename).name}" + + +def result_upload_path(instance, filename: str) -> str: + return f"jobs/{instance.id}/result/{Path(filename).name}" + + +class LetterTemplate(models.Model): + """DOCX-Vorlage mit Jinja-Platzhaltern.""" + + id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) + name = models.CharField(max_length=200) + description = models.TextField(blank=True) + file = models.FileField(upload_to=template_upload_path) + placeholders = models.JSONField(default=list, blank=True, + help_text="Aus dem DOCX extrahierte Variablen.") + created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT, + related_name="templates") + created_at = models.DateTimeField(auto_now_add=True) + updated_at = models.DateTimeField(auto_now=True) + + class Meta: + ordering = ["-created_at"] + + def __str__(self) -> str: + return self.name + + +class MailMergeJob(models.Model): + class Status(models.TextChoices): + PENDING = "pending", "Wartet" + RUNNING = "running", "Läuft" + DONE = "done", "Fertig" + FAILED = "failed", "Fehlgeschlagen" + + id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) + template = models.ForeignKey(LetterTemplate, on_delete=models.PROTECT, + related_name="jobs") + recipients_csv = models.FileField(upload_to=csv_upload_path) + status = models.CharField(max_length=20, choices=Status.choices, + default=Status.PENDING) + result_pdf = models.FileField(upload_to=result_upload_path, null=True, blank=True) + total_rows = models.PositiveIntegerField(default=0) + processed_rows = models.PositiveIntegerField(default=0) + error_message = models.TextField(blank=True) + created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT, + related_name="jobs") + created_at = models.DateTimeField(auto_now_add=True) + started_at = models.DateTimeField(null=True, blank=True) + finished_at = models.DateTimeField(null=True, blank=True) + + class Meta: + ordering = ["-created_at"] + + def __str__(self) -> str: + return f"Job {self.id} ({self.template.name})" + + +class JobLogEntry(models.Model): + class Level(models.TextChoices): + INFO = "info", "Info" + WARNING = "warning", "Warnung" + ERROR = "error", "Fehler" + + job = models.ForeignKey(MailMergeJob, on_delete=models.CASCADE, related_name="logs") + level = models.CharField(max_length=10, choices=Level.choices, default=Level.INFO) + message = models.TextField() + timestamp = models.DateTimeField(auto_now_add=True) + + class Meta: + ordering = ["timestamp"] diff --git a/serienbrief/app/mailmerge/services/__init__.py b/serienbrief/app/mailmerge/services/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/serienbrief/app/mailmerge/services/docx_renderer.py b/serienbrief/app/mailmerge/services/docx_renderer.py new file mode 100644 index 0000000..8e5a034 --- /dev/null +++ b/serienbrief/app/mailmerge/services/docx_renderer.py @@ -0,0 +1,74 @@ +""" +DOCX-Rendering und PDF-Konvertierung. +""" +from __future__ import annotations + +import logging +import re +import subprocess +import tempfile +from pathlib import Path + +from docx import Document +from docxtpl import DocxTemplate + +logger = logging.getLogger(__name__) + +PLACEHOLDER_RE = re.compile(r"\{\{\s*([A-Za-z_][A-Za-z0-9_]*)") + + +def extract_placeholders(docx_path: Path) -> list[str]: + """Liest die Jinja-Platzhalter aus einem DOCX und gibt sie sortiert zurück.""" + doc = Document(str(docx_path)) + found: set[str] = set() + for para in doc.paragraphs: + for m in PLACEHOLDER_RE.finditer(para.text): + found.add(m.group(1)) + for table in doc.tables: + for row in table.rows: + for cell in row.cells: + for para in cell.paragraphs: + for m in PLACEHOLDER_RE.finditer(para.text): + found.add(m.group(1)) + return sorted(found) + + +def render_docx(template_path: Path, context: dict, out_path: Path) -> Path: + """Füllt das DOCX-Template mit Kontext und schreibt das Ergebnis.""" + tpl = DocxTemplate(str(template_path)) + tpl.render(context) + tpl.save(str(out_path)) + return out_path + + +def docx_to_pdf(docx_path: Path, out_dir: Path) -> Path: + """Konvertiert DOCX nach PDF mit LibreOffice headless. + + LibreOffice braucht ein eigenes Profilverzeichnis, sonst kollidieren + parallele Worker. + """ + out_dir.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix="lo-profile-") as profile_dir: + cmd = [ + "soffice", + "--headless", + "--nologo", + "--norestore", + "--nolockcheck", + f"-env:UserInstallation=file://{profile_dir}", + "--convert-to", "pdf", + "--outdir", str(out_dir), + str(docx_path), + ] + logger.info("LibreOffice convert: %s", " ".join(cmd)) + result = subprocess.run( # noqa: S603 + cmd, capture_output=True, text=True, timeout=120, check=False + ) + if result.returncode != 0: + raise RuntimeError( + f"LibreOffice-Konvertierung fehlgeschlagen: {result.stderr}" + ) + pdf_path = out_dir / (docx_path.stem + ".pdf") + if not pdf_path.exists(): + raise FileNotFoundError(f"PDF nicht gefunden: {pdf_path}") + return pdf_path diff --git a/serienbrief/app/mailmerge/services/pdf_merge.py b/serienbrief/app/mailmerge/services/pdf_merge.py new file mode 100644 index 0000000..89d9294 --- /dev/null +++ b/serienbrief/app/mailmerge/services/pdf_merge.py @@ -0,0 +1,14 @@ +from pathlib import Path + +from pypdf import PdfWriter + + +def merge_pdfs(pdfs: list[Path], out_path: Path) -> Path: + writer = PdfWriter() + for pdf in pdfs: + writer.append(str(pdf)) + out_path.parent.mkdir(parents=True, exist_ok=True) + with out_path.open("wb") as f: + writer.write(f) + writer.close() + return out_path diff --git a/serienbrief/app/mailmerge/tasks.py b/serienbrief/app/mailmerge/tasks.py new file mode 100644 index 0000000..32ce487 --- /dev/null +++ b/serienbrief/app/mailmerge/tasks.py @@ -0,0 +1,87 @@ +""" +Celery-Tasks für die PDF-Erzeugung. +""" +from __future__ import annotations + +import csv +import io +import logging +import tempfile +from pathlib import Path + +from celery import shared_task +from django.core.files import File +from django.utils import timezone + +from .models import JobLogEntry, MailMergeJob +from .services.docx_renderer import docx_to_pdf, render_docx +from .services.pdf_merge import merge_pdfs + +logger = logging.getLogger(__name__) + + +def _log(job: MailMergeJob, level: str, msg: str) -> None: + JobLogEntry.objects.create(job=job, level=level, message=msg) + logger.log(getattr(logging, level.upper(), logging.INFO), "[job %s] %s", job.id, msg) + + +@shared_task(bind=True) +def run_mailmerge(self, job_id: str) -> str: + job = MailMergeJob.objects.select_related("template").get(pk=job_id) + job.status = MailMergeJob.Status.RUNNING + job.started_at = timezone.now() + job.save(update_fields=["status", "started_at"]) + _log(job, "info", f"Job gestartet (task={self.request.id}).") + + try: + # CSV einlesen + raw = job.recipients_csv.read().decode("utf-8-sig") + reader = csv.DictReader(io.StringIO(raw)) + rows = list(reader) + job.total_rows = len(rows) + job.save(update_fields=["total_rows"]) + _log(job, "info", f"{len(rows)} Empfänger gefunden.") + + if not rows: + raise ValueError("CSV enthält keine Datenzeilen.") + + # CSV-Felder vs. Template-Platzhalter prüfen + csv_fields = set(reader.fieldnames or []) + placeholders = set(job.template.placeholders or []) + missing = placeholders - csv_fields + if missing: + _log(job, "warning", + f"CSV fehlen Spalten: {', '.join(sorted(missing))}") + + # Jeden Brief rendern, alle zu einem PDF zusammenführen + with tempfile.TemporaryDirectory(prefix="mailmerge-") as tmpdir: + tmp = Path(tmpdir) + pdfs: list[Path] = [] + template_path = Path(job.template.file.path) + + for idx, row in enumerate(rows, start=1): + docx_out = tmp / f"letter_{idx:05d}.docx" + render_docx(template_path, row, docx_out) + pdf = docx_to_pdf(docx_out, tmp / "pdf") + pdfs.append(pdf) + job.processed_rows = idx + job.save(update_fields=["processed_rows"]) + + merged = tmp / f"serienbrief_{job.id}.pdf" + merge_pdfs(pdfs, merged) + with merged.open("rb") as f: + job.result_pdf.save(merged.name, File(f), save=False) + + job.status = MailMergeJob.Status.DONE + job.finished_at = timezone.now() + job.save() + _log(job, "info", "Job erfolgreich abgeschlossen.") + return str(job.id) + + except Exception as exc: # noqa: BLE001 + job.status = MailMergeJob.Status.FAILED + job.error_message = str(exc) + job.finished_at = timezone.now() + job.save() + _log(job, "error", f"Job fehlgeschlagen: {exc}") + raise diff --git a/serienbrief/app/mailmerge/urls.py b/serienbrief/app/mailmerge/urls.py new file mode 100644 index 0000000..72a9acf --- /dev/null +++ b/serienbrief/app/mailmerge/urls.py @@ -0,0 +1,12 @@ +from django.urls import path + +from . import views + +urlpatterns = [ + path("", views.dashboard, name="dashboard"), + path("templates/new/", views.template_upload, name="template-upload"), + path("templates//", views.template_detail, name="template-detail"), + path("jobs/new/", views.job_create, name="job-create"), + path("jobs//", views.job_detail, name="job-detail"), + path("jobs//download/", views.job_download, name="job-download"), +] diff --git a/serienbrief/app/mailmerge/views.py b/serienbrief/app/mailmerge/views.py new file mode 100644 index 0000000..e6c548c --- /dev/null +++ b/serienbrief/app/mailmerge/views.py @@ -0,0 +1,94 @@ +from pathlib import Path + +from django.contrib.auth.decorators import login_required +from django.http import FileResponse, Http404, HttpResponse +from django.shortcuts import get_object_or_404, redirect, render +from django.urls import reverse +from django.views.decorators.http import require_http_methods + +from .forms import LetterTemplateForm, MailMergeJobForm +from .models import LetterTemplate, MailMergeJob +from .services.docx_renderer import extract_placeholders +from .tasks import run_mailmerge + + +@login_required +def dashboard(request): + templates = LetterTemplate.objects.all()[:20] + jobs = MailMergeJob.objects.select_related("template")[:20] + return render(request, "mailmerge/dashboard.html", + {"templates": templates, "jobs": jobs}) + + +@login_required +@require_http_methods(["GET", "POST"]) +def template_upload(request): + form = LetterTemplateForm(request.POST or None, request.FILES or None) + if request.method == "POST" and form.is_valid(): + tpl = form.save(commit=False) + tpl.created_by = request.user + tpl.save() + # Platzhalter extrahieren – Datei liegt jetzt auf Disk + tpl.placeholders = extract_placeholders(Path(tpl.file.path)) + tpl.save(update_fields=["placeholders"]) + return redirect(reverse("template-detail", args=[tpl.id])) + return render(request, "mailmerge/template_form.html", {"form": form}) + + +@login_required +def template_detail(request, pk): + tpl = get_object_or_404(LetterTemplate, pk=pk) + return render(request, "mailmerge/template_detail.html", {"template": tpl}) + + +@login_required +@require_http_methods(["GET", "POST"]) +def job_create(request): + form = MailMergeJobForm(request.POST or None, request.FILES or None) + if request.method == "POST" and form.is_valid(): + job = form.save(commit=False) + job.created_by = request.user + job.save() + run_mailmerge.delay(str(job.id)) + return redirect(reverse("job-detail", args=[job.id])) + return render(request, "mailmerge/job_form.html", {"form": form}) + + +@login_required +def job_detail(request, pk): + job = get_object_or_404( + MailMergeJob.objects.select_related("template"), pk=pk + ) + logs = job.logs.all() + # HTMX partials: nur das Status-Fragment ausliefern + if request.headers.get("HX-Request"): + return render(request, "mailmerge/_job_status.html", + {"job": job, "logs": logs}) + return render(request, "mailmerge/job_detail.html", + {"job": job, "logs": logs}) + + +@login_required +def job_download(request, pk): + """PDF-Download – via X-Accel-Redirect in Production, direkter Stream im Dev.""" + job = get_object_or_404(MailMergeJob, pk=pk) + if not job.result_pdf: + raise Http404("Kein Ergebnis-PDF vorhanden.") + + # In Dev (settings.DEBUG) direkt streamen + from django.conf import settings + if settings.DEBUG: + return FileResponse(job.result_pdf.open("rb"), + as_attachment=True, + filename=Path(job.result_pdf.name).name) + + # In Production: Nginx serviert die Datei via internem Mount. + response = HttpResponse() + response["Content-Type"] = "application/pdf" + response["Content-Disposition"] = ( + f'attachment; filename="{Path(job.result_pdf.name).name}"' + ) + # Pfad relativ zu MEDIA_ROOT, gemappt auf /protected-media/ + relative = job.result_pdf.name + response["X-Accel-Redirect"] = f"/protected-media/{relative}" + return response diff --git a/serienbrief/app/manage.py b/serienbrief/app/manage.py new file mode 100644 index 0000000..fef1f23 --- /dev/null +++ b/serienbrief/app/manage.py @@ -0,0 +1,19 @@ +#!/usr/bin/env python +"""Django management entry point.""" +import os +import sys + + +def main() -> None: + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.dev") + try: + from django.core.management import execute_from_command_line + except ImportError as exc: + raise ImportError( + "Django konnte nicht importiert werden. Ist die venv aktiv?" + ) from exc + execute_from_command_line(sys.argv) + + +if __name__ == "__main__": + main() diff --git a/serienbrief/app/pyproject.toml b/serienbrief/app/pyproject.toml new file mode 100644 index 0000000..87a87b3 --- /dev/null +++ b/serienbrief/app/pyproject.toml @@ -0,0 +1,17 @@ +[tool.ruff] +line-length = 100 +target-version = "py312" +extend-exclude = ["migrations", "staticfiles", "media"] + +[tool.ruff.lint] +select = ["E", "F", "W", "I", "UP", "B", "DJ", "S"] +ignore = ["E501", "S101"] + +[tool.ruff.lint.per-file-ignores] +"**/tests/**" = ["S"] +"**/settings/**" = ["S105", "S106"] + +[tool.pytest.ini_options] +DJANGO_SETTINGS_MODULE = "config.settings.dev" +python_files = ["test_*.py", "*_test.py", "tests.py"] +addopts = "--reuse-db -ra" diff --git a/serienbrief/app/requirements-dev.txt b/serienbrief/app/requirements-dev.txt new file mode 100644 index 0000000..014fada --- /dev/null +++ b/serienbrief/app/requirements-dev.txt @@ -0,0 +1,8 @@ +# Nur im dev-Build installieren +debugpy==1.8.7 +django-debug-toolbar==4.4.6 +ipython==8.29.0 +ruff==0.7.4 +pytest==8.3.3 +pytest-django==4.9.0 +factory-boy==3.3.1 diff --git a/serienbrief/app/requirements.txt b/serienbrief/app/requirements.txt new file mode 100644 index 0000000..25827b4 --- /dev/null +++ b/serienbrief/app/requirements.txt @@ -0,0 +1,15 @@ +Django==5.1.4 +django-environ==0.11.2 +django-celery-beat==2.7.0 +django-celery-results==2.5.1 +psycopg[binary]==3.2.3 +gunicorn==23.0.0 +celery[redis]==5.4.0 +redis==5.2.0 +docxtpl==0.19.0 +python-docx==1.1.2 +pypdf==5.1.0 +argon2-cffi==23.1.0 +django-axes==7.0.1 +django-htmx==1.21.0 +python-dateutil==2.9.0.post0 diff --git a/serienbrief/app/templates/base.html b/serienbrief/app/templates/base.html new file mode 100644 index 0000000..55a413c --- /dev/null +++ b/serienbrief/app/templates/base.html @@ -0,0 +1,41 @@ + + + + + + {% block title %}Serienbrief{% endblock %} + + + + + {% if messages %}
    {% for m in messages %}
  • {{ m }}
    • {% endfor %}
{% endif %} + {% block content %}{% endblock %} + + diff --git a/serienbrief/app/templates/mailmerge/_job_status.html b/serienbrief/app/templates/mailmerge/_job_status.html new file mode 100644 index 0000000..73ab620 --- /dev/null +++ b/serienbrief/app/templates/mailmerge/_job_status.html @@ -0,0 +1,25 @@ +
+

Vorlage: {{ job.template.name }}

+

Status: + {{ job.get_status_display }} +

+

Fortschritt: {{ job.processed_rows }} / {{ job.total_rows }}

+ + {% if job.status == "done" %} +

PDF herunterladen

+ {% endif %} + {% if job.error_message %} +

Fehler: {{ job.error_message }}

+ {% endif %} + +

Log

+
+ {% for entry in logs %} +
+ [{{ entry.timestamp|date:"H:i:s" }}] {{ entry.level|upper }} – {{ entry.message }} +
+ {% empty %} + Keine Einträge. + {% endfor %} +
+
diff --git a/serienbrief/app/templates/mailmerge/dashboard.html b/serienbrief/app/templates/mailmerge/dashboard.html new file mode 100644 index 0000000..de6c7d5 --- /dev/null +++ b/serienbrief/app/templates/mailmerge/dashboard.html @@ -0,0 +1,40 @@ +{% extends "base.html" %} +{% block content %} +

Übersicht

+ +

Vorlagen

+Neue Vorlage hochladen + + + + {% for t in templates %} + + + + + + {% empty %} + + {% endfor %} + +
NamePlatzhalterErstellt
{{ t.name }}{{ t.placeholders|join:", " }}{{ t.created_at|date:"d.m.Y H:i" }}
Noch keine Vorlagen vorhanden.
+ +

Aufträge

+Neuen Serienbrief erstellen + + + + {% for j in jobs %} + + + + + + + + {% empty %} + + {% endfor %} + +
IDVorlageStatusFortschrittErstellt
{{ j.id|stringformat:"s"|slice:":8" }}…{{ j.template.name }}{{ j.get_status_display }}{{ j.processed_rows }} / {{ j.total_rows }}{{ j.created_at|date:"d.m.Y H:i" }}
Noch keine Aufträge.
+{% endblock %} diff --git a/serienbrief/app/templates/mailmerge/job_detail.html b/serienbrief/app/templates/mailmerge/job_detail.html new file mode 100644 index 0000000..4248044 --- /dev/null +++ b/serienbrief/app/templates/mailmerge/job_detail.html @@ -0,0 +1,12 @@ +{% extends "base.html" %} +{% block content %} +

Auftrag {{ job.id|stringformat:"s"|slice:":8" }}…

+ +
+ {% include "mailmerge/_job_status.html" %} +
+ + +{% endblock %} diff --git a/serienbrief/app/templates/mailmerge/job_form.html b/serienbrief/app/templates/mailmerge/job_form.html new file mode 100644 index 0000000..ab7fe9b --- /dev/null +++ b/serienbrief/app/templates/mailmerge/job_form.html @@ -0,0 +1,10 @@ +{% extends "base.html" %} +{% block content %} +

Neuer Serienbrief

+

Vorlage und Empfänger-CSV auswählen. Die Spaltennamen der CSV müssen mit den Platzhaltern der Vorlage übereinstimmen (erste Zeile = Spaltennamen).

+
+ {% csrf_token %} + {{ form.as_p }} + +
+{% endblock %} diff --git a/serienbrief/app/templates/mailmerge/template_detail.html b/serienbrief/app/templates/mailmerge/template_detail.html new file mode 100644 index 0000000..b960965 --- /dev/null +++ b/serienbrief/app/templates/mailmerge/template_detail.html @@ -0,0 +1,11 @@ +{% extends "base.html" %} +{% block content %} +

{{ template.name }}

+

{{ template.description }}

+

Datei: {{ template.file.name }}

+

Erkannte Platzhalter:

+
    + {% for p in template.placeholders %}
  • {{ p }}
    • {% empty %}
  • Keine gefunden.
    • {% endfor %} +
+Serienbrief mit dieser Vorlage erstellen +{% endblock %} diff --git a/serienbrief/app/templates/mailmerge/template_form.html b/serienbrief/app/templates/mailmerge/template_form.html new file mode 100644 index 0000000..8f13700 --- /dev/null +++ b/serienbrief/app/templates/mailmerge/template_form.html @@ -0,0 +1,10 @@ +{% extends "base.html" %} +{% block content %} +

Neue Vorlage hochladen

+

DOCX-Datei mit Platzhaltern wie {{ vorname }}, {{ nachname }}, … Die Spaltennamen der späteren CSV müssen mit den Platzhaltern übereinstimmen.

+
+ {% csrf_token %} + {{ form.as_p }} + +
+{% endblock %} diff --git a/serienbrief/app/templates/registration/login.html b/serienbrief/app/templates/registration/login.html new file mode 100644 index 0000000..31a85e8 --- /dev/null +++ b/serienbrief/app/templates/registration/login.html @@ -0,0 +1,10 @@ +{% extends "base.html" %} +{% block title %}Anmelden{% endblock %} +{% block content %} +

Anmelden

+
+ {% csrf_token %} + {{ form.as_p }} + +
+{% endblock %} diff --git a/serienbrief/docker-compose.override.yml b/serienbrief/docker-compose.override.yml new file mode 100644 index 0000000..f0b7a76 --- /dev/null +++ b/serienbrief/docker-compose.override.yml @@ -0,0 +1,62 @@ +# ============================================================================= +# Dev-Overrides – wird von "docker compose" automatisch mit gemerged. +# Für reine Produktion: `docker compose -f docker-compose.yml up -d` +# ============================================================================= + +services: + web: + build: + target: dev + command: ["python", "manage.py", "runserver", "0.0.0.0:8000"] + environment: + DJANGO_SETTINGS_MODULE: config.settings.dev + ROLE: web + ports: + # Direkter Zugriff von Host für VS Code & Browser + - "127.0.0.1:8000:8000" + # debugpy für VS Code Remote-Debug + - "127.0.0.1:5678:5678" + volumes: + # Live-Reload: lokaler Code wird in den Container gemountet + - ./app:/app:cached + - static_files:/app/staticfiles + - media_files:/app/media + read_only: false + healthcheck: + disable: true + + worker: + build: + target: dev + command: ["celery", "-A", "config", "worker", "--loglevel=debug", "--concurrency=1"] + environment: + DJANGO_SETTINGS_MODULE: config.settings.dev + ROLE: worker + volumes: + - ./app:/app:cached + - media_files:/app/media + read_only: false + healthcheck: + disable: true + + beat: + build: + target: dev + environment: + DJANGO_SETTINGS_MODULE: config.settings.dev + volumes: + - ./app:/app:cached + read_only: false + + db: + ports: + # Lokaler DB-Zugriff via DBeaver/psql – nur Loopback + - "127.0.0.1:5432:5432" + + redis: + ports: + - "127.0.0.1:6379:6379" + + # Backup im Dev nicht nötig + backup: + profiles: ["never"] diff --git a/serienbrief/docker-compose.yml b/serienbrief/docker-compose.yml new file mode 100644 index 0000000..781ac80 --- /dev/null +++ b/serienbrief/docker-compose.yml @@ -0,0 +1,309 @@ +# ============================================================================= +# Serienbrief – Compose (Production) +# Ubuntu Server 22.04/24.04 LTS · Docker Engine 27.x · Compose v2 +# ============================================================================= +# Die App spricht intern nur HTTP. TLS terminiert ein vorgelagerter Nginx- +# Reverse-Proxy (außerhalb dieses Compose-Stacks, z.B. zentraler LAN-Proxy). +# Der hier enthaltene "nginx"-Service ist NUR ein App-interner Proxy für: +# - Ausspielen von static/media über X-Accel-Redirect +# - Connection-Pooling und einfache Rate-Limits +# ============================================================================= + +name: serienbrief + +x-logging: &default-logging + driver: "json-file" + options: + max-size: "10m" + max-file: "5" + tag: "{{.Name}}" + +x-restart: &default-restart + restart: unless-stopped + +x-security-opts: &default-security-opts + security_opt: + - no-new-privileges:true + +services: + # --------------------------------------------------------------------------- + # App-interner Nginx – nimmt Traffic vom äußeren Proxy entgegen (HTTP). + # Hört nur auf Loopback bzw. dem konfigurierten LAN-Bind. + # --------------------------------------------------------------------------- + nginx: + image: nginx:1.27-alpine + <<: [*default-restart, *default-security-opts] + depends_on: + web: + condition: service_healthy + ports: + # Standardmäßig nur lokal – der externe Proxy spricht über das Docker- + # Host-Interface. Für direkten LAN-Zugriff LAN_BIND_IP in .env setzen. + - "${APP_BIND_IP:-127.0.0.1}:${APP_BIND_PORT:-8080}:80" + volumes: + - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro + - ./nginx/conf.d:/etc/nginx/conf.d:ro + - static_files:/var/www/static:ro + - media_files:/var/www/media:ro + - nginx_cache:/var/cache/nginx + - nginx_run:/var/run + networks: + - frontend + logging: *default-logging + healthcheck: + test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://127.0.0.1/healthz"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 10s + deploy: + resources: + limits: + memory: 128M + cpus: "0.5" + + # --------------------------------------------------------------------------- + # Django – Gunicorn WSGI + # --------------------------------------------------------------------------- + web: + build: + context: ./app + dockerfile: Dockerfile + target: runtime + args: + APP_UID: ${APP_UID:-10001} + APP_GID: ${APP_GID:-10001} + image: serienbrief/web:${APP_VERSION:-latest} + <<: [*default-restart, *default-security-opts] + depends_on: + db: + condition: service_healthy + redis: + condition: service_healthy + env_file: .env + environment: + DJANGO_SETTINGS_MODULE: config.settings.production + ROLE: web + volumes: + - static_files:/app/staticfiles + - media_files:/app/media + networks: + - frontend + - backend + logging: *default-logging + healthcheck: + test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=3).status==200 else 1)"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 30s + read_only: true + tmpfs: + - /tmp:size=256M,mode=1777 + deploy: + resources: + limits: + memory: 1G + cpus: "1.5" + + # --------------------------------------------------------------------------- + # Celery Worker – DOCX→PDF-Generierung + # --------------------------------------------------------------------------- + worker: + image: serienbrief/web:${APP_VERSION:-latest} + <<: [*default-restart, *default-security-opts] + depends_on: + db: + condition: service_healthy + redis: + condition: service_healthy + web: + condition: service_started + env_file: .env + environment: + DJANGO_SETTINGS_MODULE: config.settings.production + ROLE: worker + command: ["celery", "-A", "config", "worker", "--loglevel=info", "--concurrency=2", "--max-tasks-per-child=50"] + volumes: + - media_files:/app/media + networks: + - backend + logging: *default-logging + healthcheck: + test: ["CMD-SHELL", "celery -A config inspect ping -d celery@$$HOSTNAME || exit 1"] + interval: 60s + timeout: 10s + retries: 3 + start_period: 30s + read_only: true + tmpfs: + - /tmp:size=512M,mode=1777 + deploy: + resources: + limits: + memory: 2G + cpus: "2.0" + + # --------------------------------------------------------------------------- + # Celery Beat – Scheduler (Retention, Cleanup) + # --------------------------------------------------------------------------- + beat: + image: serienbrief/web:${APP_VERSION:-latest} + <<: [*default-restart, *default-security-opts] + depends_on: + db: + condition: service_healthy + redis: + condition: service_healthy + env_file: .env + environment: + DJANGO_SETTINGS_MODULE: config.settings.production + ROLE: beat + command: ["celery", "-A", "config", "beat", "--loglevel=info", "--scheduler", "django_celery_beat.schedulers:DatabaseScheduler"] + networks: + - backend + logging: *default-logging + read_only: true + tmpfs: + - /tmp:size=64M,mode=1777 + deploy: + resources: + limits: + memory: 256M + cpus: "0.3" + + # --------------------------------------------------------------------------- + # PostgreSQL + # --------------------------------------------------------------------------- + db: + image: postgres:16-alpine + <<: [*default-restart, *default-security-opts] + environment: + POSTGRES_DB: ${POSTGRES_DB} + POSTGRES_USER: ${POSTGRES_USER} + POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password + PGDATA: /var/lib/postgresql/data/pgdata + secrets: + - postgres_password + volumes: + - postgres_data:/var/lib/postgresql/data + - ./postgres/init:/docker-entrypoint-initdb.d:ro + networks: + - backend + logging: *default-logging + healthcheck: + test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"] + interval: 10s + timeout: 5s + retries: 5 + start_period: 20s + shm_size: 256mb + deploy: + resources: + limits: + memory: 1G + cpus: "1.0" + + # --------------------------------------------------------------------------- + # Redis – Celery Broker + # --------------------------------------------------------------------------- + redis: + image: redis:7-alpine + <<: [*default-restart, *default-security-opts] + command: + - "redis-server" + - "--requirepass" + - "${REDIS_PASSWORD}" + - "--maxmemory" + - "256mb" + - "--maxmemory-policy" + - "allkeys-lru" + - "--save" + - "900 1" + - "--appendonly" + - "yes" + volumes: + - redis_data:/data + networks: + - backend + logging: *default-logging + healthcheck: + test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "--no-auth-warning", "ping"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 10s + deploy: + resources: + limits: + memory: 384M + cpus: "0.5" + + # --------------------------------------------------------------------------- + # Backup – pg_dump + Media-Tar, nightly, 14 Tage Retention + # --------------------------------------------------------------------------- + backup: + image: postgres:16-alpine + <<: [*default-restart, *default-security-opts] + depends_on: + db: + condition: service_healthy + environment: + PGHOST: db + PGUSER: ${POSTGRES_USER} + PGDATABASE: ${POSTGRES_DB} + PGPASSWORD_FILE: /run/secrets/postgres_password + BACKUP_RETENTION_DAYS: "14" + secrets: + - postgres_password + volumes: + - ./backups:/backups + - media_files:/media:ro + networks: + - backend + entrypoint: ["/bin/sh", "-c"] + command: + - | + apk add --no-cache tar gzip findutils >/dev/null && \ + while true; do + TS=$$(date +%Y%m%d_%H%M%S) + export PGPASSWORD=$$(cat $$PGPASSWORD_FILE) + echo "[$$(date -Iseconds)] starting backup $$TS" + pg_dump -Fc -f /backups/db_$$TS.dump && \ + tar czf /backups/media_$$TS.tar.gz -C /media . && \ + find /backups -name "db_*.dump" -mtime +$$BACKUP_RETENTION_DAYS -delete && \ + find /backups -name "media_*.tar.gz" -mtime +$$BACKUP_RETENTION_DAYS -delete && \ + echo "[$$(date -Iseconds)] backup done" + sleep 86400 + done + logging: *default-logging + deploy: + resources: + limits: + memory: 256M + cpus: "0.5" + +# ============================================================================= +networks: + frontend: + driver: bridge + driver_opts: + com.docker.network.bridge.name: br-sb-front + backend: + driver: bridge + driver_opts: + com.docker.network.bridge.name: br-sb-back + +# ============================================================================= +volumes: + postgres_data: + redis_data: + static_files: + media_files: + nginx_cache: + nginx_run: + +# ============================================================================= +secrets: + postgres_password: + file: ./secrets/postgres_password.txt diff --git a/serienbrief/nginx/conf.d/proxy_params.inc b/serienbrief/nginx/conf.d/proxy_params.inc new file mode 100644 index 0000000..906d055 --- /dev/null +++ b/serienbrief/nginx/conf.d/proxy_params.inc @@ -0,0 +1,9 @@ +proxy_http_version 1.1; +proxy_set_header Host $host; +proxy_set_header X-Real-IP $remote_addr; +proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; +proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto; +proxy_set_header X-Forwarded-Host $host; +proxy_set_header Connection ""; +proxy_redirect off; +proxy_buffering on; diff --git a/serienbrief/nginx/conf.d/serienbrief.conf b/serienbrief/nginx/conf.d/serienbrief.conf new file mode 100644 index 0000000..647e5a1 --- /dev/null +++ b/serienbrief/nginx/conf.d/serienbrief.conf @@ -0,0 +1,55 @@ +# ============================================================================= +# vHost – HTTP only. Security-Header & TLS sind Aufgabe des äußeren Proxys. +# ============================================================================= + +# Vom äußeren Proxy weitergereichte Header vertrauen – aber NUR aus dem +# Docker-Netz oder von der bekannten Proxy-IP. Bei Bedarf set_real_ip_from +# auf das CIDR des Proxys einschränken. +set_real_ip_from 0.0.0.0/0; +real_ip_header X-Forwarded-For; +real_ip_recursive on; + +server { + listen 80 default_server; + listen [::]:80 default_server; + server_name _; + + # Healthcheck (für äußeren Proxy & Compose-Healthcheck) + location = /healthz { + access_log off; + return 200 "ok\n"; + add_header Content-Type text/plain; + } + + limit_conn conn_per_ip 20; + + # Login strikter limitieren + location ~ ^/(accounts/login|admin/login) { + limit_req zone=login burst=3 nodelay; + proxy_pass http://django_app; + include /etc/nginx/conf.d/proxy_params.inc; + } + + # Statische Dateien + location /static/ { + alias /var/www/static/; + access_log off; + expires 7d; + add_header Cache-Control "public, immutable"; + } + + # Geschützte Media (PDFs) – nur per X-Accel-Redirect aus Django ausspielen + location /protected-media/ { + internal; + alias /var/www/media/; + } + + # App + location / { + limit_req zone=app burst=50 nodelay; + proxy_pass http://django_app; + include /etc/nginx/conf.d/proxy_params.inc; + proxy_read_timeout 120s; + proxy_send_timeout 120s; + } +} diff --git a/serienbrief/nginx/nginx.conf b/serienbrief/nginx/nginx.conf new file mode 100644 index 0000000..bdf3d22 --- /dev/null +++ b/serienbrief/nginx/nginx.conf @@ -0,0 +1,58 @@ +# ============================================================================= +# App-interner Nginx – HTTP only. +# TLS macht der äußere Proxy. +# ============================================================================= + +user nginx; +worker_processes auto; +worker_rlimit_nofile 8192; +pid /var/run/nginx.pid; +error_log /var/log/nginx/error.log warn; + +events { + worker_connections 2048; + multi_accept on; +} + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + log_format main_ext '$remote_addr - $remote_user [$time_iso8601] ' + '"$request" $status $body_bytes_sent ' + '"$http_referer" "$http_user_agent" ' + 'xff="$http_x_forwarded_for" ' + 'rt=$request_time urt="$upstream_response_time"'; + access_log /var/log/nginx/access.log main_ext; + + sendfile on; + tcp_nopush on; + tcp_nodelay on; + keepalive_timeout 65; + server_tokens off; + + # CSVs etc. – am äußeren Proxy spiegeln! + client_max_body_size 25M; + client_body_buffer_size 128k; + client_body_timeout 60s; + client_header_timeout 30s; + + gzip on; + gzip_vary on; + gzip_proxied any; + gzip_comp_level 6; + gzip_types text/plain text/css application/json application/javascript + text/xml application/xml application/xml+rss text/javascript + image/svg+xml; + + limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m; + limit_req_zone $binary_remote_addr zone=app:10m rate=30r/s; + limit_conn_zone $binary_remote_addr zone=conn_per_ip:10m; + + upstream django_app { + server web:8000 max_fails=3 fail_timeout=30s; + keepalive 32; + } + + include /etc/nginx/conf.d/*.conf; +} diff --git a/serienbrief_testdata/empfaenger.csv b/serienbrief_testdata/empfaenger.csv new file mode 100644 index 0000000..b27863d --- /dev/null +++ b/serienbrief_testdata/empfaenger.csv @@ -0,0 +1,11 @@ +anrede,anrede_brief,vorname,nachname,strasse,plz,ort,datum,gz,personalnr,funktion,abteilung,durchwahl +Frau,geehrte Frau,Andrea,Huber,Hauptstraße 12,7000,Eisenstadt,21.05.2026,GB-DS-2026-0142,10342,Diplomierte Gesundheits- und Krankenpflegerin,Innere Medizin,1234 +Herr,geehrter Herr,Bernhard,Müller,Bahngasse 8,7100,Neusiedl am See,21.05.2026,GB-DS-2026-0143,10891,Stationsarzt,Chirurgie,1235 +Frau,geehrte Frau,Claudia,Berger,Schulgasse 3,7350,Oberpullendorf,21.05.2026,GB-DS-2026-0144,11204,Medizinisch-technische Assistentin,Labor,1236 +Herr,geehrter Herr,Daniel,Wagner,Kirchenplatz 5,7400,Oberwart,21.05.2026,GB-DS-2026-0145,11458,Hausarbeiter,Haustechnik,1237 +Frau,geehrte Frau,Eva,Schneider,Lindenweg 17,7540,Güssing,21.05.2026,GB-DS-2026-0146,11876,Verwaltungsangestellte,Patientenadministration,1238 +Herr,geehrter Herr,Florian,Maier,Ringstraße 22,7000,Eisenstadt,21.05.2026,GB-DS-2026-0147,12001,IT-Administrator,IT-Abteilung,1239 +Frau,geehrte Frau,Gabriele,Fischer,Marktplatz 4,7350,Oberpullendorf,21.05.2026,GB-DS-2026-0148,12245,Diätologin,Ernährungsmedizin,1240 +Herr,geehrter Herr,Hannes,Weber,Feldweg 9,7100,Neusiedl am See,21.05.2026,GB-DS-2026-0149,12567,Oberarzt,Anästhesiologie,1241 +Frau,geehrte Frau,Ingrid,Steiner,Bergstraße 14,7400,Oberwart,21.05.2026,GB-DS-2026-0150,12789,Stationsleitung,Geriatrie,1242 +Herr,geehrter Herr,Johann,Bauer,Wienerstraße 31,7540,Güssing,21.05.2026,GB-DS-2026-0151,13012,Reinigungskraft,Hauswirtschaft,1243 diff --git a/serienbrief_testdata/vorlage_dsgvo_mitarbeiterinfo.docx b/serienbrief_testdata/vorlage_dsgvo_mitarbeiterinfo.docx new file mode 100644 index 0000000..9bbe037 Binary files /dev/null and b/serienbrief_testdata/vorlage_dsgvo_mitarbeiterinfo.docx differ