307 lines
11 KiB
Markdown
307 lines
11 KiB
Markdown
# Ulti Backend
|
||
|
||
Backend monolithe Go orchestrant la Ulti Suite — alternative souveraine à Google Suite.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────┐
|
||
│ nginx (reverse proxy unique, port 80) │
|
||
├─────────────────────────────────────────────────────────┤
|
||
│ ultid (Go monolithe) │
|
||
│ ├─ /api/v1/mail — Ultimail (IMAP/SMTP custom) │
|
||
│ ├─ /api/v1/drive — Ultidrive (→ Nextcloud WebDAV) │
|
||
│ ├─ /api/v1/calendar — Agenda (→ Nextcloud CalDAV) │
|
||
│ ├─ /api/v1/contacts — Contacts (→ Nextcloud CardDAV) │
|
||
│ ├─ /api/v1/meet — Ultimeet (→ Jitsi JWT) │
|
||
│ ├─ /api/v1/photos — Ultiphotos (→ Immich API) │
|
||
│ ├─ /api/v1/admin — Administration │
|
||
│ ├─ /api/v1/search — Recherche transversale │
|
||
│ └─ /ws — WebSocket realtime │
|
||
├─────────────────────────────────────────────────────────┤
|
||
│ Services tiers (Docker) │
|
||
│ ├─ PostgreSQL 16 — DB partagée │
|
||
│ ├─ KeyDB — Cache/sessions │
|
||
│ ├─ RustFS — Object storage S3 │
|
||
│ ├─ Authentik — Auth OIDC/SAML │
|
||
│ ├─ Nextcloud (nginx+FPM) — Drive/Cal/Contacts (/cloud) │
|
||
│ ├─ Jitsi — Visioconférence │
|
||
│ └─ Immich — Photos/ML │
|
||
└─────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
## Quick Start
|
||
|
||
```bash
|
||
# 1. Copy environment file
|
||
cp .env.example .env
|
||
# Edit secrets once at the top of .env (POSTGRES_PASSWORD, RUSTFS_SECRET_KEY, etc.)
|
||
# Defaults use changeme — must match Authentik blueprints (deploy/authentik/blueprints/).
|
||
|
||
# 2. Start stack (core + modules enabled by flags)
|
||
./deploy/compose-up.sh up -d --build
|
||
```
|
||
|
||
**Auto-configured on first start:**
|
||
|
||
- SQL migrations (`ULTID_AUTO_MIGRATE=true`, embedded in `ultid`)
|
||
- Authentik OAuth apps **Ultimail** (`ulti-backend`) and **Nextcloud** via blueprints in `deploy/authentik/blueprints/`
|
||
- OIDC issuer for `ultid` via internal nginx: `ULTID_OIDC_ISSUER=http://nginx/auth/application/o/ulti/`
|
||
|
||
**Frontend** (`gmail-interface-clone`): copy `.env.example` → `.env.local`, then `pnpm dev` → http://localhost:3000 → login redirects to Authentik.
|
||
|
||
| Service | URL |
|
||
|---------|-----|
|
||
| API / Auth | http://localhost |
|
||
| Grafana | http://localhost:3002 |
|
||
| Frontend | http://localhost:3000 (Next dev) |
|
||
|
||
## Development
|
||
|
||
```bash
|
||
# Run locally (needs PG, KeyDB, RustFS, Authentik running; loads .env with {{VAR}} expansion)
|
||
go run ./cmd/ultid
|
||
|
||
# Build
|
||
go build -o ultid ./cmd/ultid
|
||
|
||
# Migrations run automatically on ultid start. To disable: ULTID_AUTO_MIGRATE=false
|
||
|
||
# Manual migrate (optional)
|
||
go run ./cmd/envexpand -in .env -out .env.resolved
|
||
source <(grep -v '^#' .env.resolved | sed 's/^/export /')
|
||
migrate -path migrations -database "$ULTID_DB_URL" up
|
||
```
|
||
|
||
### Reverse proxy (nginx)
|
||
|
||
Un seul **nginx** expose l’entrée HTTP (`:80`) et route :
|
||
|
||
| Chemin | Service |
|
||
|--------|---------|
|
||
| `/api/*` | ultid |
|
||
| `/ws` | ultid (WebSocket) |
|
||
| `/auth/*` | Authentik |
|
||
| `/meet/*` | Jitsi (si `JITSI_ENABLED=true`) |
|
||
| `/cloud/*` | Nextcloud nginx+FPM (si `NEXTCLOUD_ENABLED=true`) |
|
||
|
||
Nextcloud : FPM + nginx dédié ; ultid appelle `NEXTCLOUD_URL` en interne (`http://nextcloud:80`).
|
||
Caddy retiré : un seul proxy évite la double couche ; TLS plus tard (certbot, Traefik, ou `listen 443` nginx).
|
||
|
||
### Centralized secrets
|
||
|
||
Set passwords and keys **once** in the `Secrets` section at the top of `.env`. Derived values reference them with `{{POSTGRES_PASSWORD}}`, `{{RUSTFS_SECRET_KEY}}`, etc. Expansion runs via:
|
||
|
||
- `./deploy/compose-up.sh` — writes `.env.resolved` for Docker Compose
|
||
- `go run ./cmd/envexpand -in .env -out .env.resolved` — manual export for migrate/scripts
|
||
- `go run ./cmd/ultid` — expands `.env` in-process before reading config
|
||
|
||
Runtime secret files are also supported with `*_FILE` variables (example: `ULTID_OIDC_CLIENT_SECRET_FILE=/run/secrets/oidc_client_secret`).
|
||
|
||
Mail credentials are encrypted at rest with AES-GCM using `MAIL_CREDENTIAL_KEYS` (`key_id:base64key,...`) and `MAIL_ACTIVE_CREDENTIAL_KEY_ID`.
|
||
Secret rotation policy is enforced through:
|
||
|
||
- `SECRET_ROTATION_MAX_AGE`
|
||
- `ULTID_OIDC_CLIENT_SECRET_ROTATED_AT`
|
||
- `MAIL_CREDENTIAL_KEY_ROTATED_AT`
|
||
- `MAIL_WEBHOOK_SHARED_SECRET_ROTATED_AT`
|
||
|
||
### Observability (Prometheus / Grafana)
|
||
|
||
ultid exposes Prometheus metrics at `/metrics` (see `internal/observability/metrics.go`). The core Docker Compose stack includes Prometheus and Grafana with configs under `deploy/observability/`:
|
||
|
||
| File | Purpose |
|
||
|------|---------|
|
||
| `deploy/observability/prometheus/prometheus.yml` | Scrape ultid + self; loads alert rules |
|
||
| `deploy/observability/prometheus/alerts.yml` | Alert rules: IMAP sync stalled, outbox backlog, HTTP 5xx rate |
|
||
| `deploy/observability/grafana/ultid-baseline.json` | Baseline dashboard (HTTP latency/errors, IMAP sync, outbox, webhooks) |
|
||
| `deploy/observability/grafana/provisioning/` | Grafana datasource + dashboard auto-load |
|
||
|
||
Start with the rest of the stack (`./deploy/compose-up.sh up -d`), then open:
|
||
|
||
| Service | URL | Notes |
|
||
|---------|-----|-------|
|
||
| Prometheus | http://localhost:9090 | Targets: `ultid`, `prometheus` |
|
||
| Grafana | http://localhost:3002 | Login from `.env` (`GRAFANA_ADMIN_USER` / `GRAFANA_ADMIN_PASSWORD`, default `admin` / `admin`); dashboard **Ultid Baseline** under folder **Ultid** |
|
||
|
||
**Alertmanager** — not included in compose; route labels `service=ultid` and `severity` (`critical`, `warning`) to your on-call channels when you add it.
|
||
|
||
## Stack
|
||
|
||
| Component | Technology |
|
||
|-----------|-----------|
|
||
| Backend | Go 1.25+ (chi, pgx, go-imap, go-smtp) |
|
||
| Database | PostgreSQL 16 |
|
||
| Cache | KeyDB (Redis-compatible, multi-threaded) |
|
||
| Object Storage | RustFS (S3-compatible, Apache 2.0) |
|
||
| Auth | Authentik (OIDC/SAML) |
|
||
| Files/Cal/Contacts | Nextcloud headless (WebDAV/CalDAV/CardDAV) |
|
||
| Video | Jitsi Meet (JWT auth) |
|
||
| Photos | Immich (ML classification) |
|
||
| Reverse Proxy | nginx (TLS à ajouter via certbot ou autre) |
|
||
| Search | PostgreSQL tsvector + GIN |
|
||
|
||
## Calendar API (mini doc)
|
||
|
||
Base path: `/api/v1/calendar` (module actif seulement si `NEXTCLOUD_ENABLED=true`).
|
||
|
||
### Endpoints principaux
|
||
|
||
| Endpoint | Méthode | Usage |
|
||
|----------|---------|-------|
|
||
| `/` | `GET` | Lister calendriers |
|
||
| `/{calID}/events` | `GET` | Lister événements d'un calendrier |
|
||
| `/{calID}/events` | `POST` | Créer événement |
|
||
| `/events/*` | `PUT` | Modifier événement avec `If-Match` |
|
||
| `/events/*` | `DELETE` | Supprimer événement |
|
||
| `/events/response/*` | `POST` | Répondre invitation participant |
|
||
| `/freebusy` | `POST` | Obtenir disponibilités (free/busy) |
|
||
| `/events/meet-link/*` | `POST` | Générer lien Meet et l'injecter dans l'événement |
|
||
|
||
`*` représente un chemin CalDAV absolu, ex:
|
||
`/remote.php/dav/calendars/{user}/{calendar}/{uid}.ics`
|
||
|
||
### Event payload (create / update)
|
||
|
||
```json
|
||
{
|
||
"uid": "team-sync-2026-05-23",
|
||
"summary": "Team sync",
|
||
"description": "Weekly project update",
|
||
"location": "Paris office",
|
||
"start": "20260523T090000Z",
|
||
"end": "20260523T100000Z",
|
||
"organizer": "alice@example.com",
|
||
"attendees": [
|
||
{ "email": "bob@example.com", "name": "Bob", "status": "NEEDS-ACTION" },
|
||
{ "email": "carol@example.com", "name": "Carol", "status": "ACCEPTED" }
|
||
]
|
||
}
|
||
```
|
||
|
||
### Update avec ETag / If-Match
|
||
|
||
Requête:
|
||
|
||
```http
|
||
PUT /api/v1/calendar/events/remote.php/dav/calendars/alice/work/team-sync-2026-05-23.ics
|
||
If-Match: "68031b8f4d18f"
|
||
Content-Type: application/json
|
||
```
|
||
|
||
Réponse succès:
|
||
|
||
```json
|
||
{ "etag": "\"68031b8f5f901\"" }
|
||
```
|
||
|
||
Si ETag obsolète, API renvoie `412` avec code `etag_mismatch`.
|
||
|
||
### Réponse invitation participant
|
||
|
||
```json
|
||
POST /api/v1/calendar/events/response/remote.php/dav/calendars/alice/work/team-sync-2026-05-23.ics
|
||
{
|
||
"email": "bob@example.com",
|
||
"response": "accepted",
|
||
"if_match": "\"68031b8f5f901\""
|
||
}
|
||
```
|
||
|
||
`response` autorise: `accepted`, `declined`, `tentative`.
|
||
|
||
### Free/Busy
|
||
|
||
```json
|
||
POST /api/v1/calendar/freebusy
|
||
{
|
||
"start": "2026-05-23T08:00:00Z",
|
||
"end": "2026-05-23T18:00:00Z",
|
||
"attendees": ["bob@example.com", "carol@example.com"]
|
||
}
|
||
```
|
||
|
||
Réponse (exemple):
|
||
|
||
```json
|
||
{
|
||
"attendees": [
|
||
{
|
||
"email": "bob@example.com",
|
||
"periods": [
|
||
{ "start": "20260523T090000Z", "end": "20260523T100000Z", "type": "BUSY" }
|
||
]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
### Création lien Meet depuis event
|
||
|
||
```json
|
||
POST /api/v1/calendar/events/meet-link/remote.php/dav/calendars/alice/work/team-sync-2026-05-23.ics
|
||
{
|
||
"if_match": "\"68031b8f5f901\""
|
||
}
|
||
```
|
||
|
||
Réponse:
|
||
|
||
```json
|
||
{
|
||
"meet_url": "https://your-domain/meet/team-sync-2026-05-23?jwt=...",
|
||
"etag": "\"68031b8f8b102\""
|
||
}
|
||
```
|
||
|
||
Si module Meet désactivé (`JITSI_ENABLED=false`), API renvoie `409` avec code `meet_disabled`.
|
||
|
||
### Erreurs API Calendar
|
||
|
||
| HTTP | `code` | Cas |
|
||
|------|--------|-----|
|
||
| `400` | `invalid_request_body` | JSON invalide, champ manquant, email participant invalide, format date invalide |
|
||
| `400` | `request_body_too_large` | Payload dépasse limite handler (`256KB`) |
|
||
| `401` | `auth.*` | Token manquant/invalide |
|
||
| `403` | `auth.forbidden` | Permission calendrier insuffisante (`read`/`write`) |
|
||
| `404` | `attendee_not_found` | Email donné absent des `ATTENDEE` de l'événement |
|
||
| `409` | `meet_disabled` | Endpoint meet-link appelé alors que Jitsi désactivé |
|
||
| `412` | `etag_mismatch` | `If-Match` obsolète sur update event/response/meet-link |
|
||
| `500` | `internal_error` | Erreur Nextcloud/CalDAV ou erreur interne backend |
|
||
|
||
Format standard:
|
||
|
||
```json
|
||
{
|
||
"code": "etag_mismatch",
|
||
"message": "etag does not match current resource version",
|
||
"trace_id": "req_01HZ..."
|
||
}
|
||
```
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
├── cmd/ultid/ — Entry point
|
||
├── internal/
|
||
│ ├── api/ — HTTP handlers (mail, admin, drive, calendar, contacts, meet, photos)
|
||
│ ├── mail/ — IMAP sync, SMTP send, rules engine, webhooks
|
||
│ ├── nextcloud/ — WebDAV/CalDAV/CardDAV client
|
||
│ ├── meet/ — Jitsi JWT generation
|
||
│ ├── photos/ — Immich API client
|
||
│ ├── auth/ — OIDC verification
|
||
│ ├── search/ — Full-text search
|
||
│ ├── realtime/ — WebSocket hub
|
||
│ └── config/ — Environment config
|
||
├── migrations/ — SQL migrations
|
||
├── deploy/ — Docker Compose configs
|
||
│ ├── docker-compose.yml — Core stack
|
||
│ ├── observability/ — Prometheus alerts + Grafana dashboard
|
||
│ ├── nginx/
|
||
│ ├── nextcloud/
|
||
│ ├── jitsi/
|
||
│ └── immich/
|
||
├── Dockerfile — Multi-stage build
|
||
└── .env.example
|
||
```
|