Complete backend for the Ulti-Suite
Go to file
R3D347HR4Y 747e0d4bb4 Add CI workflow and unit tests for mail API
- Created a CI workflow in `.github/workflows/ci.yml` to run Go tests and verify database migrations.
- Added unit tests for the mail API in `internal/api/mail/handlers_test.go`, covering message listing, retrieval, sending, and label updating.
- Introduced a service interface for the mail handler in `internal/api/mail/service_iface.go`.
- Updated mail handler initialization to accept a service API in `internal/api/mail/handlers.go`.
- Implemented test authentication middleware for testing purposes in `internal/api/middleware/testauth.go`.
- Added various test cases for IMAP and SMTP functionalities, ensuring robust error handling and validation.
- Enhanced project documentation with checklist updates for testing and CI integration.
2026-05-22 17:02:37 +02:00
.github/workflows Add CI workflow and unit tests for mail API 2026-05-22 17:02:37 +02:00
cmd Add observability features with Prometheus and Grafana integration 2026-05-22 16:17:10 +02:00
deploy Add observability features with Prometheus and Grafana integration 2026-05-22 16:17:10 +02:00
internal Add CI workflow and unit tests for mail API 2026-05-22 17:02:37 +02:00
migrations Initialize Ulti Backend project with Docker setup, environment configuration, and core services. Added .dockerignore, .env.example, Dockerfile, and docker-compose files for PostgreSQL, KeyDB, RustFS, Authentik, Nextcloud, Jitsi, and Immich. Implemented main application structure in Go with API handlers and environment variable expansion. Included README for project overview and setup instructions. 2026-05-22 16:02:53 +02:00
project-plan Add CI workflow and unit tests for mail API 2026-05-22 17:02:37 +02:00
.dockerignore Initialize Ulti Backend project with Docker setup, environment configuration, and core services. Added .dockerignore, .env.example, Dockerfile, and docker-compose files for PostgreSQL, KeyDB, RustFS, Authentik, Nextcloud, Jitsi, and Immich. Implemented main application structure in Go with API handlers and environment variable expansion. Included README for project overview and setup instructions. 2026-05-22 16:02:53 +02:00
.env.example Add observability features with Prometheus and Grafana integration 2026-05-22 16:17:10 +02:00
.gitignore Initialize Ulti Backend project with Docker setup, environment configuration, and core services. Added .dockerignore, .env.example, Dockerfile, and docker-compose files for PostgreSQL, KeyDB, RustFS, Authentik, Nextcloud, Jitsi, and Immich. Implemented main application structure in Go with API handlers and environment variable expansion. Included README for project overview and setup instructions. 2026-05-22 16:02:53 +02:00
Dockerfile Initialize Ulti Backend project with Docker setup, environment configuration, and core services. Added .dockerignore, .env.example, Dockerfile, and docker-compose files for PostgreSQL, KeyDB, RustFS, Authentik, Nextcloud, Jitsi, and Immich. Implemented main application structure in Go with API handlers and environment variable expansion. Included README for project overview and setup instructions. 2026-05-22 16:02:53 +02:00
go.mod Add observability features with Prometheus and Grafana integration 2026-05-22 16:17:10 +02:00
go.sum Add observability features with Prometheus and Grafana integration 2026-05-22 16:17:10 +02:00
README.md Add observability features with Prometheus and Grafana integration 2026-05-22 16:17:10 +02:00

Ulti Backend

Backend monolithe Go orchestrant la Ulti Suite — alternative souveraine à Google Suite.

Architecture

┌─────────────────────────────────────────────────────────┐
│  nginx (reverse proxy unique, port 80)                  │
├─────────────────────────────────────────────────────────┤
│  ultid (Go monolithe)                                   │
│  ├─ /api/v1/mail      — Ultimail (IMAP/SMTP custom)    │
│  ├─ /api/v1/drive     — Ultidrive (→ Nextcloud WebDAV) │
│  ├─ /api/v1/calendar  — Agenda (→ Nextcloud CalDAV)    │
│  ├─ /api/v1/contacts  — Contacts (→ Nextcloud CardDAV) │
│  ├─ /api/v1/meet      — Ultimeet (→ Jitsi JWT)        │
│  ├─ /api/v1/photos    — Ultiphotos (→ Immich API)     │
│  ├─ /api/v1/admin     — Administration                 │
│  ├─ /api/v1/search    — Recherche transversale         │
│  └─ /ws               — WebSocket realtime             │
├─────────────────────────────────────────────────────────┤
│  Services tiers (Docker)                                │
│  ├─ PostgreSQL 16     — DB partagée                    │
│  ├─ KeyDB             — Cache/sessions                  │
│  ├─ RustFS            — Object storage S3              │
│  ├─ Authentik         — Auth OIDC/SAML                 │
│  ├─ Nextcloud (nginx+FPM) — Drive/Cal/Contacts (/cloud) │
│  ├─ Jitsi             — Visioconférence                │
│  └─ Immich            — Photos/ML                      │
└─────────────────────────────────────────────────────────┘

Quick Start

# 1. Copy environment file
cp .env.example .env
# Edit secrets once at the top of .env (POSTGRES_PASSWORD, RUSTFS_SECRET_KEY, etc.)
# Other variables use {{VAR}} placeholders expanded at launch.
# Toggle modules with flags:
# NEXTCLOUD_ENABLED=true|false
# JITSI_ENABLED=true|false
# IMMICH_ENABLED=true|false

# 2. Start stack (core + modules enabled by flags)
./deploy/compose-up.sh up -d

Development

# Run locally (needs PG, KeyDB, RustFS running; loads .env with {{VAR}} expansion)
go run ./cmd/ultid

# Build
go build -o ultid ./cmd/ultid

# Expand .env for external tools (docker, migrate)
go run ./cmd/envexpand -in .env -out .env.resolved
source <(grep -v '^#' .env.resolved | sed 's/^/export /')

# Run migrations (use expanded ULTID_DB_URL; host may need localhost instead of postgres)
migrate -path migrations -database "$ULTID_DB_URL" up

Reverse proxy (nginx)

Un seul nginx expose lentrée HTTP (:80) et route :

Chemin Service
/api/* ultid
/ws ultid (WebSocket)
/auth/* Authentik
/meet/* Jitsi (si JITSI_ENABLED=true)
/cloud/* Nextcloud nginx+FPM (si NEXTCLOUD_ENABLED=true)

Nextcloud : FPM + nginx dédié ; ultid appelle NEXTCLOUD_URL en interne (http://nextcloud:80).
Caddy retiré : un seul proxy évite la double couche ; TLS plus tard (certbot, Traefik, ou listen 443 nginx).

Centralized secrets

Set passwords and keys once in the Secrets section at the top of .env. Derived values reference them with {{POSTGRES_PASSWORD}}, {{RUSTFS_SECRET_KEY}}, etc. Expansion runs via:

  • ./deploy/compose-up.sh — writes .env.resolved for Docker Compose
  • go run ./cmd/envexpand -in .env -out .env.resolved — manual export for migrate/scripts
  • go run ./cmd/ultid — expands .env in-process before reading config

Runtime secret files are also supported with *_FILE variables (example: ULTID_OIDC_CLIENT_SECRET_FILE=/run/secrets/oidc_client_secret).

Mail credentials are encrypted at rest with AES-GCM using MAIL_CREDENTIAL_KEYS (key_id:base64key,...) and MAIL_ACTIVE_CREDENTIAL_KEY_ID. Secret rotation policy is enforced through:

  • SECRET_ROTATION_MAX_AGE
  • ULTID_OIDC_CLIENT_SECRET_ROTATED_AT
  • MAIL_CREDENTIAL_KEY_ROTATED_AT
  • MAIL_WEBHOOK_SHARED_SECRET_ROTATED_AT

Observability (Prometheus / Grafana)

ultid exposes Prometheus metrics at /metrics (see internal/observability/metrics.go). The core Docker Compose stack includes Prometheus and Grafana with configs under deploy/observability/:

File Purpose
deploy/observability/prometheus/prometheus.yml Scrape ultid + self; loads alert rules
deploy/observability/prometheus/alerts.yml Alert rules: IMAP sync stalled, outbox backlog, HTTP 5xx rate
deploy/observability/grafana/ultid-baseline.json Baseline dashboard (HTTP latency/errors, IMAP sync, outbox, webhooks)
deploy/observability/grafana/provisioning/ Grafana datasource + dashboard auto-load

Start with the rest of the stack (./deploy/compose-up.sh up -d), then open:

Service URL Notes
Prometheus http://localhost:9090 Targets: ultid, prometheus
Grafana http://localhost:3000 Login from .env (GRAFANA_ADMIN_USER / GRAFANA_ADMIN_PASSWORD, default admin / admin); dashboard Ultid Baseline under folder Ultid

Alertmanager — not included in compose; route labels service=ultid and severity (critical, warning) to your on-call channels when you add it.

Stack

Component Technology
Backend Go 1.23+ (chi, pgx, go-imap, go-smtp)
Database PostgreSQL 16
Cache KeyDB (Redis-compatible, multi-threaded)
Object Storage RustFS (S3-compatible, Apache 2.0)
Auth Authentik (OIDC/SAML)
Files/Cal/Contacts Nextcloud headless (WebDAV/CalDAV/CardDAV)
Video Jitsi Meet (JWT auth)
Photos Immich (ML classification)
Reverse Proxy nginx (TLS à ajouter via certbot ou autre)
Search PostgreSQL tsvector + GIN

Project Structure

├── cmd/ultid/          — Entry point
├── internal/
│   ├── api/            — HTTP handlers (mail, admin, drive, calendar, contacts, meet, photos)
│   ├── mail/           — IMAP sync, SMTP send, rules engine, webhooks
│   ├── nextcloud/      — WebDAV/CalDAV/CardDAV client
│   ├── meet/           — Jitsi JWT generation
│   ├── photos/         — Immich API client
│   ├── auth/           — OIDC verification
│   ├── search/         — Full-text search
│   ├── realtime/       — WebSocket hub
│   └── config/         — Environment config
├── migrations/         — SQL migrations
├── deploy/             — Docker Compose configs
│   ├── docker-compose.yml  — Core stack
│   ├── observability/      — Prometheus alerts + Grafana dashboard
│   ├── nginx/
│   ├── nextcloud/
│   ├── jitsi/
│   └── immich/
├── Dockerfile          — Multi-stage build
└── .env.example