UltiSuite/ultisuite-backend
UltiSuite/ultisuite-backend
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Lots of changes

Browse Source
This commit is contained in:
R3D347HR4Y 2026-06-04 00:12:11 +02:00
parent cd0a80f5e8
commit 25d3ac4cd9
63 changed files with 5833 additions and 153 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
.cursor/rules/local-env.mdc Normal file
View File

@ -0,0 +1,12 @@
---
description: Fichiers .env locaux — les agents peuvent les lire et les modifier
alwaysApply: true
---
# Environnement local (test)
Les fichiers `.env` et `.env.resolved` à la racine de ce dépôt contiennent **uniquement des valeurs de test locales**.
- Les agents **peuvent et doivent** les lire et les modifier quand la tâche l'exige (URLs, ports, secrets de dev, etc.).
- Ne pas les committer : ils restent dans `.gitignore`.
- Pour documenter les variables, mettre à jour `.env.example`.

4
.cursorignore Normal file
View File

@ -0,0 +1,4 @@
# Environnement local (test) — accessible aux agents Cursor.
# Contre la liste d'ignore par défaut de Cursor ; fichiers toujours dans .gitignore.
!.env
!.env.resolved

28
.env.example
View File

@ -21,6 +21,8 @@ MAIL_ACTIVE_CREDENTIAL_KEY_ID=v1
MAIL_WEBHOOK_SHARED_SECRET=changeme-webhook-signing-secret
NC_ADMIN_PASSWORD=changeme
NC_OIDC_CLIENT_SECRET=changeme
ONLYOFFICE_OIDC_CLIENT_SECRET=changeme
IMMICH_OIDC_CLIENT_SECRET=changeme
JITSI_APP_SECRET=changeme-jwt-secret
JITSI_INTERNAL_AUTH_PASSWORD=changeme
KEYDB_PASSWORD=
@ -104,6 +106,10 @@ AUTHENTIK_REDIS__HOST=keydb
AUTHENTIK_WEB__PATH=/auth/
# URL publique affichee dans les redirects OIDC (navigateur)
AUTHENTIK_HOST=http://{{DOMAIN}}
# API interne (ultid → authentik-server) pour provisionner les apps OIDC au demarrage
AUTHENTIK_API_URL=http://authentik-server:9000
# Token admin Authentik (Flows & Stages → Tokens) — active le provisioning API ultid
# AUTHENTIK_API_TOKEN — defini dans la section Secrets (optionnel ; sinon blueprints seuls)
# -----------------------------------------------------------------------------
# Nextcloud (Drive / Calendar / Contacts)
@ -136,6 +142,22 @@ NC_S3_KEY={{RUSTFS_ACCESS_KEY}}
NC_S3_SECRET={{RUSTFS_SECRET_KEY}}
NC_S3_SSL=false
# -----------------------------------------------------------------------------
# OnlyOffice (Docs / Sheets / Slides)
# -----------------------------------------------------------------------------
ONLYOFFICE_ENABLED=false
ONLYOFFICE_URL=http://onlyoffice
ONLYOFFICE_PUBLIC_URL=http://{{DOMAIN}}/office
# URL ultid joignable depuis le conteneur OnlyOffice (fetch doc + callback)
ONLYOFFICE_API_INTERNAL_URL=http://ultid:8080
# URL Nextcloud joignable depuis OnlyOffice (host nginx Docker). Nginx route /index.php/* → Nextcloud.
# NC_ONLYOFFICE_STORAGE_URL=http://nginx
# Docker compose OnlyOffice also sets ALLOW_PRIVATE_IP_ADDRESS=true (required for internal URLs)
ONLYOFFICE_JWT_SECRET=changeme-onlyoffice-jwt
ONLYOFFICE_OIDC_CLIENT_ID=ulti-onlyoffice
# ONLYOFFICE_OIDC_CLIENT_SECRET — defini dans la section Secrets
ULTID_PUBLIC_URL=http://{{DOMAIN}}
# -----------------------------------------------------------------------------
# Jitsi Meet (Visioconference)
# Mode local : Jitsi deploye dans la stack
@ -211,7 +233,11 @@ MAIL_MICROSOFT_OAUTH_CLIENT_ID=
MAIL_MICROSOFT_OAUTH_CLIENT_SECRET=
MAIL_MICROSOFT_OAUTH_TENANT=common
MAIL_OAUTH_REDIRECT_URL=
MAIL_APP_URL=http://localhost:3000
MAIL_APP_URL=http://localhost/mail
# Cible nginx → frontend mail (dev: Next sur l'hôte ; prod: ultimail:3000 si container)
MAIL_FRONTEND_UPSTREAM=host.docker.internal:3000
# Dev: pnpm dev drive-suite sur :3001 | Prod sans dev local: drive-suite:3000
DRIVE_FRONTEND_UPSTREAM=host.docker.internal:3001
MAIL_WEBHOOK_SHARED_SECRET_ROTATED_AT=2026-01-01T00:00:00Z
# -----------------------------------------------------------------------------

10
README.md
View File

@ -48,13 +48,17 @@ cp .env.example .env
- Authentik OAuth apps **Ultimail** (`ulti-backend`) and **Nextcloud** via blueprints in `deploy/authentik/blueprints/`
- OIDC issuer for `ultid` via internal nginx: `ULTID_OIDC_ISSUER=http://nginx/auth/application/o/ulti/`
**Frontend** (`gmail-interface-clone`): copy `.env.example` → `.env.local`, then `pnpm dev` → http://localhost:3000 → login redirects to Authentik.
**Frontends** (stack + `pnpm dev` sur l’hôte, nginx route tout sur le port 80) :
- **Ultimail** (`gmail-interface-clone`) : `.env.local` avec `NEXT_PUBLIC_APP_URL=http://localhost`, puis `pnpm dev` → http://localhost/mail/
- **UltiDrive** (`drive-suite`) : `.env.local` avec `NEXT_PUBLIC_APP_URL=http://localhost/drive` et `NEXT_PUBLIC_BASE_PATH=/drive`, puis `pnpm dev` → http://localhost/drive/
| Service | URL |
|---------|-----|
| API / Auth | http://localhost |
| Ultimail | http://localhost/mail/ |
| UltiDrive | http://localhost/drive/ |
| Grafana | http://localhost:3002 |
| Frontend | http://localhost:3000 (Next dev) |
## Development
@ -84,6 +88,8 @@ Un seul **nginx** expose l’entrée HTTP (`:80`) et route :
| `/auth/*` | Authentik |
| `/meet/*` | Jitsi (si `JITSI_ENABLED=true`) |
| `/cloud/*` | Nextcloud nginx+FPM (si `NEXTCLOUD_ENABLED=true`) |
| `/mail/*` | Ultimail (`MAIL_FRONTEND_UPSTREAM`, défaut `host.docker.internal:3000`) |
| `/drive/*` | UltiDrive (`drive-suite`) |
Nextcloud : FPM + nginx dédié ; ultid appelle `NEXTCLOUD_URL` en interne (`http://nextcloud:80`).
Caddy retiré : un seul proxy évite la double couche ; TLS plus tard (certbot, Traefik, ou `listen 443` nginx).

22
cmd/ultid/main.go
View File

@ -22,11 +22,13 @@ import (
"github.com/ultisuite/ulti-backend/internal/api/calendar"
"github.com/ultisuite/ulti-backend/internal/api/contacts"
"github.com/ultisuite/ulti-backend/internal/api/drive"
"github.com/ultisuite/ulti-backend/internal/api/office"
mailapi "github.com/ultisuite/ulti-backend/internal/api/mail"
"github.com/ultisuite/ulti-backend/internal/api/mail/sendguard"
meetapi "github.com/ultisuite/ulti-backend/internal/api/meet"
"github.com/ultisuite/ulti-backend/internal/api/middleware"
photosapi "github.com/ultisuite/ulti-backend/internal/api/photos"
"github.com/ultisuite/ulti-backend/internal/authentik"
"github.com/ultisuite/ulti-backend/internal/auth"
"github.com/ultisuite/ulti-backend/internal/dbmigrate"
"github.com/ultisuite/ulti-backend/internal/config"
@ -72,6 +74,8 @@ func main() {
}
defer pool.Close()
authentik.StartProvisioner(ctx, pool, cfg)
rdb := redis.NewClient(&redis.Options{Addr: cfg.KeyDBAddr})
defer rdb.Close()
@ -210,6 +214,22 @@ func main() {
r.Get("/ws", hub.HandleWS)
r.Get("/api/v1/mail/accounts/oauth/callback", mailHandler.OAuthCallback)
var driveSvc *drive.Service
if ncClient != nil {
driveSvc = drive.NewService(ncClient, hub)
}
if ncClient != nil && cfg.OnlyOfficeEnabled && driveSvc != nil {
officeSvc := office.NewService(ncClient, office.Config{
Enabled: true,
DocumentURL: cfg.OnlyOfficeURL,
PublicURL: cfg.OnlyOfficePublicURL,
APIInternalURL: cfg.OnlyOfficeAPIInternalURL,
JWTSecret: cfg.OnlyOfficeJWTSecret,
})
officeHandler := office.NewHandler(officeSvc, driveSvc)
r.Mount("/api/v1/office", officeHandler.Routes(middleware.Auth(verifier, pool, auditLogger)))
}
r.Group(func(r chi.Router) {
r.Use(middleware.Auth(verifier, pool, auditLogger))
@ -227,7 +247,7 @@ func main() {
}).Search)
if ncClient != nil {
r.Mount("/api/v1/drive", drive.NewHandler(ncClient).Routes())
r.Mount("/api/v1/drive", drive.NewHandler(ncClient, hub).Routes())
r.Mount("/api/v1/calendar", calendar.NewHandler(ncClient, meetCfg).Routes())
r.Mount("/api/v1/contacts", contacts.NewHandler(ncClient, pool).Routes())
}

13
deploy/authentik/README.md
View File

@ -9,6 +9,7 @@ Blueprints in `blueprints/` are mounted into Authentik at `/blueprints/custom` a
| `03-ulti-suite-groups.yaml` | Claim OIDC `groups` (RBAC contacts/calendar/drive/photos) |
| `ulti-oidc.yaml` | App OIDC Ultimail |
| `nextcloud-oidc.yaml` | App OIDC Nextcloud |
| `onlyoffice-oidc.yaml` | App OIDC OnlyOffice |
Assets branding : générés depuis le frontend (`pnpm run brand:authentik` dans `gmail-interface-clone`) :
@ -60,9 +61,21 @@ Modifier le logo : `pnpm run brand:authentik` dans le repo frontend, puis redém
|-----|------|-----------|------------------|
| Ultimail | `ulti` | `ulti-backend` | `ULTID_OIDC_CLIENT_SECRET` |
| Nextcloud | `nextcloud` | `ulti-nextcloud` | `NC_OIDC_CLIENT_SECRET` |
| OnlyOffice | `onlyoffice` | `ulti-onlyoffice` | `ONLYOFFICE_OIDC_CLIENT_SECRET` |
| Immich | `immich` | `ulti-immich` | `IMMICH_OIDC_CLIENT_SECRET` |
Defaults blueprints : `changeme` — sync avec `.env`.
## Provisioning automatique (ultid)
Si `AUTHENTIK_API_TOKEN` est défini, **ultid** provisionne au démarrage les applications OIDC des briques activées (`NEXTCLOUD_ENABLED`, `ONLYOFFICE_ENABLED`, `IMMICH_ENABLED`, etc.) via l’API Authentik, puis enregistre l’état dans Postgres (`suite_authentik_provisioned`) pour ne pas recréer à chaque restart.
1. Authentik Admin → **Directory** → **Tokens** → créer un token API (intent: `api`)
2. `.env` : `AUTHENTIK_API_TOKEN=<token>`
3. Redémarrer `ultid`
Sans token : les blueprints ci-dessus restent la source (worker Authentik au boot).
## Appliquer / vérifier
```bash

40
deploy/authentik/blueprints/onlyoffice-oidc.yaml Normal file
View File

@ -0,0 +1,40 @@
# Authentik blueprint — OnlyOffice OIDC (when ONLYOFFICE_ENABLED=true)
# Client secret must match ONLYOFFICE_OIDC_CLIENT_SECRET in .env
version: 1
metadata:
name: OnlyOffice OIDC
labels:
blueprints.goauthentik.io/instantiate: "true"
entries:
- model: authentik_providers_oauth2.oauth2provider
id: oo-oauth-provider
identifiers:
name: ulti-onlyoffice-provider
attrs:
name: ulti-onlyoffice-provider
authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
property_mappings:
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, openid]]
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, email]]
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, profile]]
client_type: confidential
client_id: ulti-onlyoffice
client_secret: changeme
redirect_uris:
- matching_mode: strict
url: http://localhost/office/
- matching_mode: strict
url: http://localhost/office/oauth2/callback
signing_key: !Find [authentik_crypto.certificatekeypair, [name, authentik Self-signed Certificate]]
- model: authentik_core.application
identifiers:
slug: onlyoffice
attrs:
name: OnlyOffice
slug: onlyoffice
group: Ulti Suite
provider: !KeyOf oo-oauth-provider
meta_launch_url: http://localhost/office
policy_engine_mode: any

10
deploy/authentik/blueprints/ulti-oidc.yaml
View File

@ -25,6 +25,14 @@ entries:
client_id: ulti-backend
client_secret: changeme
redirect_uris:
- matching_mode: strict
url: http://localhost/api/auth/callback
- matching_mode: strict
url: http://127.0.0.1/api/auth/callback
- matching_mode: strict
url: http://localhost/drive/api/auth/callback
- matching_mode: strict
url: http://127.0.0.1/drive/api/auth/callback
- matching_mode: strict
url: http://localhost:3000/api/auth/callback
- matching_mode: strict
@ -43,5 +51,5 @@ entries:
slug: ulti
group: Ulti Suite
provider: !KeyOf ulti-oauth-provider
meta_launch_url: http://localhost:3000/
meta_launch_url: http://localhost/mail/inbox
policy_engine_mode: any

4
deploy/compose-up.sh
View File

@ -50,4 +50,8 @@ if [[ "$(to_bool "${IMMICH_ENABLED:-false}")" == "true" ]]; then
compose_files+=("-f" "deploy/immich/docker-compose.immich.yml")
fi
if [[ "$(to_bool "${ONLYOFFICE_ENABLED:-false}")" == "true" ]]; then
compose_files+=("-f" "deploy/onlyoffice/docker-compose.onlyoffice.yml")
fi
exec docker compose --env-file .env.resolved "${compose_files[@]}" "$@"

22
deploy/docker-compose.yml
View File

@ -8,7 +8,11 @@ services:
- ./nginx/default.conf.template:/etc/nginx/templates/default.conf.template:ro
environment:
DOMAIN: ${DOMAIN:-localhost}
MAIL_FRONTEND_UPSTREAM: ${MAIL_FRONTEND_UPSTREAM:-host.docker.internal:3000}
DRIVE_FRONTEND_UPSTREAM: ${DRIVE_FRONTEND_UPSTREAM:-host.docker.internal:3001}
env_file: ../.env.resolved
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- ulti-net
depends_on:
@ -198,6 +202,24 @@ services:
prometheus:
condition: service_started
drive-suite:
build:
context: ../../drive-suite
dockerfile: Dockerfile
restart: unless-stopped
environment:
- ULTI_PROXY_ORIGIN=http://nginx
- NEXT_PUBLIC_API_URL=/api/v1
- NEXT_PUBLIC_APP_URL=http://${DOMAIN:-localhost}/drive
- OIDC_CLIENT_SECRET=${ULTID_OIDC_CLIENT_SECRET:-changeme}
- NEXT_PUBLIC_OIDC_ISSUER=http://${DOMAIN:-localhost}/auth/application/o/ulti/
- NEXT_PUBLIC_OIDC_CLIENT_ID=${ULTID_OIDC_CLIENT_ID:-ulti-backend}
- NEXT_PUBLIC_ONLYOFFICE_URL=http://${DOMAIN:-localhost}/office
networks:
- ulti-net
depends_on:
- ultid
networks:
ulti-net:
driver: bridge

65
deploy/nextcloud/configure-onlyoffice.sh Executable file
View File

@ -0,0 +1,65 @@
#!/bin/sh
# Configure the OnlyOffice Nextcloud connector for server-side document/PDF previews.
# Idempotent — safe to run on every container start (before-starting hook).
set -e
case "$(printf '%s' "${ONLYOFFICE_ENABLED:-false}" | tr '[:upper:]' '[:lower:]')" in
1|true|yes|on) ;;
*)
echo "OnlyOffice disabled — skipping Nextcloud connector configuration."
exit 0
;;
esac
OCC="php /var/www/html/occ"
if ! $OCC status 2>/dev/null | grep -q "installed: true"; then
echo "Nextcloud not installed yet — skipping OnlyOffice configuration."
exit 0
fi
if ! $OCC app:list 2>/dev/null | grep -qE 'onlyoffice:'; then
echo "Installing OnlyOffice Nextcloud app…"
$OCC app:install onlyoffice || true
fi
$OCC app:enable onlyoffice || true
if ! $OCC app:list --enabled 2>/dev/null | grep -qE 'onlyoffice:'; then
echo "OnlyOffice app unavailable — skipping connector configuration."
exit 0
fi
DS_INTERNAL="${ONLYOFFICE_URL:-http://onlyoffice}"
DS_PUBLIC="${ONLYOFFICE_PUBLIC_URL:-$DS_INTERNAL}"
# OnlyOffice fetches files via the edge nginx. StorageUrl replaces the host part of
# Nextcloud absolute URLs; nginx routes /index.php/* to Nextcloud (see default.conf.template).
STORAGE="${NC_ONLYOFFICE_STORAGE_URL:-http://nginx}"
JWT="${ONLYOFFICE_JWT_SECRET:-}"
# Trailing slash required by the connector.
case "$DS_PUBLIC" in */) ;; *) DS_PUBLIC="${DS_PUBLIC}/" ;; esac
case "$DS_INTERNAL" in */) ;; *) DS_INTERNAL="${DS_INTERNAL}/" ;; esac
case "$STORAGE" in */) ;; *) STORAGE="${STORAGE}/" ;; esac
$OCC config:app:set onlyoffice DocumentServerUrl --value="$DS_PUBLIC"
$OCC config:app:set onlyoffice DocumentServerInternalUrl --value="$DS_INTERNAL"
$OCC config:app:set onlyoffice StorageUrl --value="$STORAGE"
$OCC config:app:set onlyoffice advanced --value="true"
$OCC config:app:set onlyoffice preview --value="true"
$OCC config:app:set onlyoffice demo --value="false"
if [ -n "$JWT" ]; then
$OCC config:app:set onlyoffice jwt_secret --value="$JWT"
fi
$OCC config:system:set enable_previews --value=true --type=boolean
$OCC config:system:set preview_max_x --value=2048 --type=integer
$OCC config:system:set preview_max_y --value=2048 --type=integer
echo "OnlyOffice connector configured (document preview enabled)."
if $OCC onlyoffice:documentserver --check 2>/dev/null; then
echo "OnlyOffice Document Server connection OK."
else
echo "OnlyOffice Document Server not reachable yet — previews will work once it is healthy."
fi

9
deploy/nextcloud/docker-compose.nextcloud.yml
View File

@ -7,6 +7,9 @@ services:
condition: service_healthy
keydb:
condition: service_healthy
onlyoffice:
condition: service_healthy
required: false
environment:
- POSTGRES_HOST=postgres
- POSTGRES_DB=nextcloud
@ -32,9 +35,15 @@ services:
- NC_OIDC_CLIENT_ID=${NC_OIDC_CLIENT_ID:-ulti-nextcloud}
- NC_OIDC_CLIENT_SECRET=${NC_OIDC_CLIENT_SECRET:-changeme}
- NC_OIDC_DISCOVERY_URL=${NC_OIDC_DISCOVERY_URL:-http://nginx/auth/application/o/nextcloud/.well-known/openid-configuration}
- ONLYOFFICE_ENABLED=${ONLYOFFICE_ENABLED:-false}
- ONLYOFFICE_URL=${ONLYOFFICE_URL:-http://onlyoffice}
- ONLYOFFICE_PUBLIC_URL=${ONLYOFFICE_PUBLIC_URL:-http://localhost/office}
- ONLYOFFICE_JWT_SECRET=${ONLYOFFICE_JWT_SECRET:-}
- NC_ONLYOFFICE_STORAGE_URL=${NC_ONLYOFFICE_STORAGE_URL:-http://nginx}
volumes:
- nextcloud_data:/var/www/html
- ./nextcloud/init.sh:/docker-entrypoint-hooks.d/post-installation/init.sh:ro
- ./nextcloud/configure-onlyoffice.sh:/docker-entrypoint-hooks.d/before-starting/50-configure-onlyoffice.sh:ro
networks:
- ulti-net
healthcheck:

5
deploy/nextcloud/nginx.conf
View File

@ -10,6 +10,11 @@ server {
client_max_body_size 10G;
fastcgi_buffers 64 4K;
# Internal Docker requests may include /cloud (OVERWRITEWEBROOT) — strip it.
location ^~ /cloud/ {
rewrite ^/cloud/(.*)$ /$1 last;
}
location / {
rewrite ^ /index.php;
}

264
deploy/nginx/default.conf.template
View File

@ -18,6 +18,7 @@ server {
client_max_body_size 10G;
# ultid API (must stay after ^~ /api/auth/ — mail OIDC routes)
location /api/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $ultid_upstream ultid:8080;
@ -64,6 +65,20 @@ server {
proxy_set_header X-Forwarded-Proto $scheme;
}
# Ultimail OIDC post-login — before Authentik /auth/ (path collision)
location ^~ /auth/complete {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location /auth/ {
proxy_pass http://authentik-server:9000;
proxy_http_version 1.1;
@ -79,6 +94,7 @@ server {
location /meet/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $jitsi_upstream jitsi-web;
rewrite ^/meet/(.*)$ /$1 break;
proxy_pass http://$jitsi_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
@ -90,7 +106,8 @@ server {
location /cloud/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $nc_upstream nextcloud;
proxy_pass http://$nc_upstream/;
rewrite ^/cloud/(.*)$ /$1 break;
proxy_pass http://$nc_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
@ -102,8 +119,251 @@ server {
return 301 /cloud/;
}
# OnlyOffice fetches Nextcloud after StorageUrl host rewrite (URLs lack /cloud prefix).
location ^~ /index.php {
resolver 127.0.0.11 valid=10s ipv6=off;
set $nc_upstream nextcloud;
proxy_pass http://$nc_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
}
location /office/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $oo_upstream onlyoffice;
rewrite ^/office/(.*)$ /$1 break;
proxy_pass http://$oo_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host/office;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_read_timeout 3600s;
}
location = /office {
return 301 /office/;
}
# UltiDrive — variable proxy_pass must not include a URI path (passes client URI as-is).
location ^~ /drive/api/auth/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
rewrite ^/drive/api/auth/(.*)$ /api/auth/$1 break;
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location ^~ /drive/_next/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
# Next.js dev-only assets (no /drive prefix in generated URLs with assetPrefix-only setup)
location ^~ /__nextjs_font/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
location = /__nextjs_original-stack-frames {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
location ^~ /__nextjs_source-map {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
location ^~ /drive/login {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
rewrite ^/drive/login(.*)$ /login$1 break;
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location ^~ /drive/auth/complete {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
rewrite ^/drive/auth/complete(.*)$ /auth/complete$1 break;
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location /drive/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location = /drive {
resolver 127.0.0.11 valid=10s ipv6=off;
set $drive_upstream ${DRIVE_FRONTEND_UPSTREAM};
proxy_pass http://$drive_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
# Ultimail frontend — dev: pnpm dev on host (MAIL_FRONTEND_UPSTREAM=host.docker.internal:3000)
# Prod: set MAIL_FRONTEND_UPSTREAM=ultimail:3000 when the container exists.
location ^~ /api/auth/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location ^~ /mail/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
# Do not 301 /mail → /mail/ (Next 308 /mail/ → /mail causes a redirect loop).
location = /mail {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location = /mail/ {
return 302 /mail/inbox;
}
location ^~ /login {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location ^~ /_next/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
location ^~ /brand/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
location ^~ /mail-backgrounds/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Public assets at repo root (launcher icons, etc.)
location ~* ^/[^/]+\.(svg|png|ico|webp)$ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $mail_upstream ${MAIL_FRONTEND_UPSTREAM};
proxy_pass http://$mail_upstream$request_uri;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
location = / {
return 302 /mail/inbox;
}
location / {
default_type text/plain;
return 200 "Ulti Suite\n";
return 404 "Not found\n";
}
}

20
deploy/onlyoffice/docker-compose.onlyoffice.yml Normal file
View File

@ -0,0 +1,20 @@
services:
onlyoffice:
image: onlyoffice/documentserver:8.2
restart: unless-stopped
environment:
- JWT_ENABLED=true
- JWT_SECRET=${ONLYOFFICE_JWT_SECRET:-changeme-onlyoffice-jwt}
- JWT_HEADER=Authorization
- JWT_IN_BODY=true
# ultid/nginx URLs in editor config resolve to Docker private IPs
- ALLOW_PRIVATE_IP_ADDRESS=true
- ALLOW_META_IP_ADDRESS=true
networks:
- ulti-net
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://127.0.0.1/healthcheck >/dev/null || exit 1"]
interval: 30s
timeout: 10s
retries: 5
start_period: 90s

2
go.mod
View File

@ -16,6 +16,7 @@ require (
github.com/minio/minio-go/v7 v7.0.80
github.com/prometheus/client_golang v1.23.2
github.com/redis/go-redis/v9 v9.7.0
golang.org/x/text v0.28.0
golang.org/x/time v0.15.0
)
@ -53,6 +54,5 @@ require (
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/sync v0.16.0 // indirect
golang.org/x/sys v0.35.0 // indirect
golang.org/x/text v0.28.0 // indirect
google.golang.org/protobuf v1.36.8 // indirect
)

25
internal/api/drive/blank_office.go Normal file
View File

@ -0,0 +1,25 @@
package drive
import _ "embed"
//go:embed testdata/blank.docx
var blankDocx []byte
//go:embed testdata/blank.xlsx
var blankXlsx []byte
//go:embed testdata/blank.pptx
var blankPptx []byte
func blankOfficeFile(kind NewFileKind) ([]byte, string) {
switch kind {
case NewFileDocument:
return blankDocx, "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
case NewFileSpreadsheet:
return blankXlsx, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
case NewFilePresentation:
return blankPptx, "application/vnd.openxmlformats-officedocument.presentationml.presentation"
default:
return nil, ""
}
}

369
internal/api/drive/handlers.go
View File

@ -14,8 +14,10 @@ import (
"github.com/ultisuite/ulti-backend/internal/api/apivalidate"
"github.com/ultisuite/ulti-backend/internal/api/middleware"
"github.com/ultisuite/ulti-backend/internal/api/query"
"github.com/ultisuite/ulti-backend/internal/auth"
"github.com/ultisuite/ulti-backend/internal/nextcloud"
"github.com/ultisuite/ulti-backend/internal/permission"
"github.com/ultisuite/ulti-backend/internal/realtime"
)
type Handler struct {
@ -23,38 +25,64 @@ type Handler struct {
logger *slog.Logger
}
func NewHandler(nc *nextcloud.Client) *Handler {
func NewHandler(nc *nextcloud.Client, hub *realtime.Hub) *Handler {
return &Handler{
svc: NewService(nc),
svc: NewService(nc, hub),
logger: slog.Default().With("component", "drive-api"),
}
}
func (h *Handler) nextcloudUser(w http.ResponseWriter, r *http.Request, claims *auth.Claims) (string, bool) {
userID, err := h.svc.EnsureNextcloudUser(r.Context(), claims)
if err != nil {
h.logger.Error("ensure nextcloud user", "error", err, "sub", claims.Sub, "email", claims.Email)
apivalidate.WriteInternal(w, r)
return "", false
}
return userID, true
}
func (h *Handler) Routes() chi.Router {
r := chi.NewRouter()
read := middleware.RequirePermission(permission.ResourceDrive, permission.LevelRead)
write := middleware.RequirePermission(permission.ResourceDrive, permission.LevelWrite)
admin := middleware.RequirePermission(permission.ResourceDrive, permission.LevelAdmin)
r.With(read).Get("/files/*", h.ListFiles)
r.With(read).Get("/quota", h.GetQuota)
r.With(read).Get("/trash", h.ListTrash)
r.With(read).Get("/recent", h.ListRecent)
r.With(read).Get("/starred", h.ListStarred)
r.With(read).Get("/starred/*", h.ListStarred)
r.With(read).Get("/shared", h.ListSharedWithMe)
r.With(read).Get("/search", h.Search)
r.With(read).Get("/shares", h.ListShares)
r.With(read).Get("/shares/recipients/lookup", h.LookupShareRecipient)
r.With(read).Get("/download/*", h.Download)
r.With(read).Get("/preview/*", h.Preview)
r.With(read).Get("/files/*", h.ListFiles)
r.With(write).Post("/files/*", h.Upload)
r.With(write).Post("/files/new", h.CreateNewFile)
r.With(write).Delete("/files/*", h.DeleteFile)
r.With(write).Post("/folders/*", h.CreateFolder)
r.With(write).Post("/move", h.Move)
r.With(write).Post("/copy", h.Copy)
r.With(write).Post("/rename", h.Rename)
r.With(write).Post("/trash/restore", h.RestoreTrash)
r.With(write).Post("/favorite", h.SetFavorite)
r.With(admin).Post("/shares", h.CreateShare)
r.With(admin).Post("/shares/{shareID}/send-email", h.SendShareEmail)
r.With(admin).Put("/shares/{shareID}", h.UpdateShare)
r.With(admin).Delete("/shares/{shareID}", h.DeleteShare)
return r
}
func (h *Handler) ListFiles(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
params, err := query.ParseListRequest(r)
if err != nil {
apivalidate.WriteQueryError(w, r, err)
@ -62,7 +90,7 @@ func (h *Handler) ListFiles(w http.ResponseWriter, r *http.Request) {
}
path := chi.URLParam(r, "*")
result, err := h.svc.ListFiles(r.Context(), claims.Sub, path, params)
result, err := h.svc.ListFiles(r.Context(), ncUser, path, params)
if err != nil {
h.logger.Error("list files", "error", err)
apivalidate.WriteInternal(w, r)
@ -73,6 +101,10 @@ func (h *Handler) ListFiles(w http.ResponseWriter, r *http.Request) {
func (h *Handler) Upload(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
path := chi.URLParam(r, "*")
if verr := validatePath(path); verr != nil {
apivalidate.WriteValidationError(w, r, verr)
@ -83,7 +115,7 @@ func (h *Handler) Upload(w http.ResponseWriter, r *http.Request) {
apivalidate.WriteValidationError(w, r, verr)
return
} else if ok {
if err := h.svc.UploadChunk(r.Context(), claims.Sub, chunk.UploadID, path, chunk.ChunkUpload, r.Body, r.Header.Get("Content-Type")); err != nil {
if err := h.svc.UploadChunk(r.Context(), ncUser, chunk.UploadID, path, chunk.ChunkUpload, r.Body, r.Header.Get("Content-Type")); err != nil {
h.logger.Error("upload chunk", "error", err)
writeDriveError(w, r, err)
return
@ -105,23 +137,28 @@ func (h *Handler) Upload(w http.ResponseWriter, r *http.Request) {
return
}
if err := h.svc.Upload(r.Context(), claims.Sub, path, r.Body, r.Header.Get("Content-Type"), r.ContentLength); err != nil {
if err := h.svc.Upload(r.Context(), ncUser, path, r.Body, r.Header.Get("Content-Type"), r.ContentLength); err != nil {
h.logger.Error("upload", "error", err)
writeDriveError(w, r, err)
return
}
h.svc.notifyFileChanged(claims.Sub, path)
apiresponse.WriteJSON(w, http.StatusCreated, map[string]string{"status": "uploaded", "path": path})
}
func (h *Handler) Download(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
path := chi.URLParam(r, "*")
if verr := validatePath(path); verr != nil {
apivalidate.WriteValidationError(w, r, verr)
return
}
body, contentType, err := h.svc.Download(r.Context(), claims.Sub, path)
body, contentType, err := h.svc.Download(r.Context(), ncUser, path)
if err != nil {
writeDriveError(w, r, err)
return
@ -132,31 +169,67 @@ func (h *Handler) Download(w http.ResponseWriter, r *http.Request) {
io.Copy(w, body)
}
func (h *Handler) DeleteFile(w http.ResponseWriter, r *http.Request) {
func (h *Handler) Preview(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
path := chi.URLParam(r, "*")
if verr := validatePath(path); verr != nil {
apivalidate.WriteValidationError(w, r, verr)
return
}
if err := h.svc.Delete(r.Context(), claims.Sub, path); err != nil {
width, _ := strconv.Atoi(r.URL.Query().Get("w"))
height, _ := strconv.Atoi(r.URL.Query().Get("h"))
body, contentType, err := h.svc.Preview(r.Context(), ncUser, path, width, height)
if err != nil {
writeDriveError(w, r, err)
return
}
defer body.Close()
w.Header().Set("Content-Type", contentType)
w.Header().Set("Cache-Control", "private, max-age=300")
io.Copy(w, body)
}
func (h *Handler) DeleteFile(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
path := chi.URLParam(r, "*")
if verr := validatePath(path); verr != nil {
apivalidate.WriteValidationError(w, r, verr)
return
}
if err := h.svc.Delete(r.Context(), ncUser, path); err != nil {
h.logger.Error("delete file", "error", err)
writeDriveError(w, r, err)
return
}
h.svc.notifyFileChanged(claims.Sub, path)
w.WriteHeader(http.StatusNoContent)
}
func (h *Handler) CreateFolder(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
path := chi.URLParam(r, "*")
if verr := validatePath(path); verr != nil {
apivalidate.WriteValidationError(w, r, verr)
return
}
if err := h.svc.CreateFolder(r.Context(), claims.Sub, path); err != nil {
if err := h.svc.CreateFolder(r.Context(), ncUser, path); err != nil {
h.logger.Error("create folder", "error", err)
writeDriveError(w, r, err)
return
@ -166,6 +239,10 @@ func (h *Handler) CreateFolder(w http.ResponseWriter, r *http.Request) {
func (h *Handler) Move(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
var req moveRequest
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
@ -176,16 +253,21 @@ func (h *Handler) Move(w http.ResponseWriter, r *http.Request) {
return
}
if err := h.svc.Move(r.Context(), claims.Sub, req.Source, req.Destination); err != nil {
if err := h.svc.Move(r.Context(), ncUser, req.Source, req.Destination); err != nil {
h.logger.Error("move", "error", err)
writeDriveError(w, r, err)
return
}
h.svc.notifyFileChanged(claims.Sub, req.Destination)
w.WriteHeader(http.StatusNoContent)
}
func (h *Handler) Copy(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
var req copyRequest
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
@ -196,7 +278,7 @@ func (h *Handler) Copy(w http.ResponseWriter, r *http.Request) {
return
}
if err := h.svc.Copy(r.Context(), claims.Sub, req.Source, req.Destination); err != nil {
if err := h.svc.Copy(r.Context(), ncUser, req.Source, req.Destination); err != nil {
h.logger.Error("copy", "error", err)
writeDriveError(w, r, err)
return
@ -206,6 +288,10 @@ func (h *Handler) Copy(w http.ResponseWriter, r *http.Request) {
func (h *Handler) Rename(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
var req renameRequest
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
@ -216,7 +302,7 @@ func (h *Handler) Rename(w http.ResponseWriter, r *http.Request) {
return
}
if err := h.svc.Rename(r.Context(), claims.Sub, req.Path, req.NewName); err != nil {
if err := h.svc.Rename(r.Context(), ncUser, req.Path, req.NewName); err != nil {
h.logger.Error("rename", "error", err)
writeDriveError(w, r, err)
return
@ -226,12 +312,16 @@ func (h *Handler) Rename(w http.ResponseWriter, r *http.Request) {
func (h *Handler) ListTrash(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
params, err := query.ParseListRequest(r)
if err != nil {
apivalidate.WriteQueryError(w, r, err)
return
}
result, err := h.svc.ListTrash(r.Context(), claims.Sub, params)
result, err := h.svc.ListTrash(r.Context(), ncUser, params)
if err != nil {
h.logger.Error("list trash", "error", err)
writeDriveError(w, r, err)
@ -242,12 +332,16 @@ func (h *Handler) ListTrash(w http.ResponseWriter, r *http.Request) {
func (h *Handler) ListRecent(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
params, err := query.ParseListRequest(r)
if err != nil {
apivalidate.WriteQueryError(w, r, err)
return
}
result, err := h.svc.ListRecent(r.Context(), claims.Sub, params)
result, err := h.svc.ListRecent(r.Context(), ncUser, params)
if err != nil {
h.logger.Error("list recent", "error", err)
writeDriveError(w, r, err)
@ -258,13 +352,17 @@ func (h *Handler) ListRecent(w http.ResponseWriter, r *http.Request) {
func (h *Handler) ListStarred(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
params, err := query.ParseListRequest(r)
if err != nil {
apivalidate.WriteQueryError(w, r, err)
return
}
basePath := strings.TrimPrefix(chi.URLParam(r, "*"), "/")
result, err := h.svc.ListStarred(r.Context(), claims.Sub, basePath, params)
result, err := h.svc.ListStarred(r.Context(), ncUser, basePath, params)
if err != nil {
h.logger.Error("list starred", "error", err)
writeDriveError(w, r, err)
@ -273,8 +371,32 @@ func (h *Handler) ListStarred(w http.ResponseWriter, r *http.Request) {
apiresponse.WriteJSON(w, http.StatusOK, result)
}
func (h *Handler) ListSharedWithMe(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
params, err := query.ParseListRequest(r)
if err != nil {
apivalidate.WriteQueryError(w, r, err)
return
}
result, err := h.svc.ListSharedWithMe(r.Context(), ncUser, params)
if err != nil {
h.logger.Error("list shared with me", "error", err)
writeDriveError(w, r, err)
return
}
apiresponse.WriteJSON(w, http.StatusOK, result)
}
func (h *Handler) CreateShare(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
var req createShareRequest
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
@ -292,15 +414,228 @@ func (h *Handler) CreateShare(w http.ResponseWriter, r *http.Request) {
}
}
share, err := h.svc.CreateShare(r.Context(), claims.Sub, req.Path, req.ShareType, permissions)
share, err := h.svc.CreateShare(r.Context(), ncUser, req.Path, req, permissions)
if err != nil {
h.logger.Error("create share", "error", err)
writeDriveError(w, r, err)
return
}
h.svc.notifyShareUpdated(claims.Sub, req.Path)
apiresponse.WriteJSON(w, http.StatusCreated, share)
}
func (h *Handler) GetQuota(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
quota, err := h.svc.GetQuota(r.Context(), ncUser)
if err != nil {
h.logger.Error("get quota", "error", err, "nc_user", ncUser)
writeDriveError(w, r, err)
return
}
apiresponse.WriteJSON(w, http.StatusOK, quota)
}
func (h *Handler) Search(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
params, err := query.ParseListRequest(r)
if err != nil {
apivalidate.WriteQueryError(w, r, err)
return
}
basePath := r.URL.Query().Get("path")
result, err := h.svc.Search(r.Context(), ncUser, basePath, params)
if err != nil {
writeDriveError(w, r, err)
return
}
apiresponse.WriteJSON(w, http.StatusOK, result)
}
func (h *Handler) ListShares(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
filePath := r.URL.Query().Get("path")
if filePath == "" {
apivalidate.WriteValidationError(w, r, apivalidate.NewValidationError(
apivalidate.FieldDetail{Field: "path", Message: "required"},
))
return
}
shares, err := h.svc.ListShares(r.Context(), ncUser, filePath)
if err != nil {
writeDriveError(w, r, err)
return
}
apiresponse.WriteJSON(w, http.StatusOK, map[string]any{"shares": shares})
}
func (h *Handler) LookupShareRecipient(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
_ = ncUser
email := strings.TrimSpace(strings.ToLower(r.URL.Query().Get("email")))
if email == "" {
apivalidate.WriteValidationError(w, r, apivalidate.NewValidationError(
apivalidate.FieldDetail{Field: "email", Message: "required"},
))
return
}
registered, err := h.svc.UserExists(r.Context(), email)
if err != nil {
h.logger.Error("lookup share recipient", "error", err, "email", email)
writeDriveError(w, r, err)
return
}
apiresponse.WriteJSON(w, http.StatusOK, map[string]any{
"email": email,
"registered": registered,
})
}
func (h *Handler) SendShareEmail(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
shareID := chi.URLParam(r, "shareID")
var req struct {
Password string `json:"password"`
}
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
return
}
if err := h.svc.SendShareEmail(r.Context(), ncUser, shareID, req.Password); err != nil {
h.logger.Error("send share email", "error", err, "share_id", shareID)
writeDriveError(w, r, err)
return
}
w.WriteHeader(http.StatusNoContent)
}
func (h *Handler) UpdateShare(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
shareID := chi.URLParam(r, "shareID")
var req updateShareRequest
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
return
}
permissions := req.Permissions
if permissions == 0 && strings.TrimSpace(req.Role) != "" {
if mapped, ok := sharePermissionsForRole(req.Role); ok {
permissions = mapped
}
}
share, err := h.svc.UpdateShare(r.Context(), ncUser, shareID, permissions, req.ExpireDate, req.Password)
if err != nil {
writeDriveError(w, r, err)
return
}
h.svc.notifyShareUpdated(claims.Sub, share.Path)
apiresponse.WriteJSON(w, http.StatusOK, share)
}
func (h *Handler) DeleteShare(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
shareID := chi.URLParam(r, "shareID")
if err := h.svc.DeleteShare(r.Context(), ncUser, shareID); err != nil {
writeDriveError(w, r, err)
return
}
h.svc.notifyShareUpdated(claims.Sub, "")
w.WriteHeader(http.StatusNoContent)
}
func (h *Handler) RestoreTrash(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
var req restoreTrashRequest
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
return
}
if verr := validateRestoreTrashRequest(&req); verr != nil {
apivalidate.WriteValidationError(w, r, verr)
return
}
if err := h.svc.RestoreTrash(r.Context(), ncUser, req.Name); err != nil {
writeDriveError(w, r, err)
return
}
h.svc.notifyFileChanged(claims.Sub, req.Name)
w.WriteHeader(http.StatusNoContent)
}
func (h *Handler) SetFavorite(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
var req favoriteRequest
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
return
}
if verr := validateFavoriteRequest(&req); verr != nil {
apivalidate.WriteValidationError(w, r, verr)
return
}
if err := h.svc.SetFavorite(r.Context(), ncUser, req.Path, req.Favorite); err != nil {
writeDriveError(w, r, err)
return
}
h.svc.notifyFileChanged(claims.Sub, req.Path)
w.WriteHeader(http.StatusNoContent)
}
func (h *Handler) CreateNewFile(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, ok := h.nextcloudUser(w, r, claims)
if !ok {
return
}
var req newFileRequest
if err := apivalidate.DecodeJSON(w, r, maxJSONRequestBody, &req); err != nil {
return
}
if verr := validateNewFileRequest(&req); verr != nil {
apivalidate.WriteValidationError(w, r, verr)
return
}
kind := NewFileKind(strings.TrimSpace(strings.ToLower(req.Kind)))
target, err := h.svc.CreateNewFile(r.Context(), ncUser, req.ParentPath, req.Name, kind)
if err != nil {
writeDriveError(w, r, err)
return
}
h.svc.notifyFileChanged(claims.Sub, target)
apiresponse.WriteJSON(w, http.StatusCreated, map[string]string{"path": target})
}
func writeDriveError(w http.ResponseWriter, r *http.Request, err error) {
switch {
case errors.Is(err, ErrNotFound):

210
internal/api/drive/service.go
View File

@ -1,6 +1,7 @@
package drive
import (
"bytes"
"context"
"errors"
"io"
@ -13,7 +14,9 @@ import (
"github.com/ultisuite/ulti-backend/internal/api/paginate"
"github.com/ultisuite/ulti-backend/internal/api/query"
"github.com/ultisuite/ulti-backend/internal/auth"
"github.com/ultisuite/ulti-backend/internal/nextcloud"
"github.com/ultisuite/ulti-backend/internal/realtime"
)
var (
@ -26,18 +29,41 @@ var (
type Service struct {
nc *nextcloud.Client
hub *realtime.Hub
maxUploadBytes int64
quotaReserveByte int64
}
func NewService(nc *nextcloud.Client) *Service {
func NewService(nc *nextcloud.Client, hub *realtime.Hub) *Service {
return &Service{
nc: nc,
hub: hub,
maxUploadBytes: envInt64("ULTID_DRIVE_MAX_UPLOAD_BYTES", 0),
quotaReserveByte: envInt64("ULTID_DRIVE_QUOTA_RESERVED_BYTES", 0),
}
}
func (s *Service) EnsureNextcloudUser(ctx context.Context, claims *auth.Claims) (string, error) {
if s.nc == nil {
return "", errors.New("nextcloud unavailable")
}
return s.nc.EnsurePrincipal(ctx, claims.Email, claims.Sub, claims.Name)
}
func (s *Service) notifyFileChanged(platformUserID, path string) {
if s.hub == nil || platformUserID == "" {
return
}
s.hub.Broadcast(platformUserID, realtime.NewDriveFileChangedEvent(path))
}
func (s *Service) notifyShareUpdated(platformUserID, path string) {
if s.hub == nil || platformUserID == "" {
return
}
s.hub.Broadcast(platformUserID, realtime.NewDriveShareUpdatedEvent(path))
}
type FilesList struct {
Files []nextcloud.FileInfo `json:"files"`
Pagination query.PaginationMeta `json:"pagination,omitempty"`
@ -85,6 +111,19 @@ func (s *Service) ListRecent(ctx context.Context, userID string, params query.Li
}, nil
}
func (s *Service) ListSharedWithMe(ctx context.Context, userID string, params query.ListParams) (FilesList, error) {
files, err := s.nc.ListSharedWithMe(ctx, userID)
if err != nil {
return FilesList{}, mapDriveError(err)
}
filtered := filterFiles(files, params.Q)
page, total := paginate.Slice(filtered, params.Offset(), params.Limit())
return FilesList{
Files: page,
Pagination: params.Meta(&total),
}, nil
}
func (s *Service) ListStarred(ctx context.Context, userID, basePath string, params query.ListParams) (FilesList, error) {
if basePath == "" {
basePath = "/"
@ -140,6 +179,19 @@ func (s *Service) Download(ctx context.Context, userID, path string) (io.ReadClo
return body, contentType, nil
}
func (s *Service) Preview(ctx context.Context, userID, filePath string, width, height int) (io.ReadCloser, string, error) {
filePath = nextcloud.NormalizeClientFilePath(userID, filePath)
fileID, err := s.nc.FileID(ctx, userID, filePath)
if err != nil {
return nil, "", mapDriveError(err)
}
body, contentType, err := s.nc.Preview(ctx, userID, fileID, width, height)
if err != nil {
return nil, "", mapDriveError(err)
}
return body, contentType, nil
}
func (s *Service) Delete(ctx context.Context, userID, path string) error {
return mapDriveError(s.nc.Delete(ctx, userID, path))
}
@ -165,14 +217,163 @@ func (s *Service) Rename(ctx context.Context, userID, filePath, newName string)
return mapDriveError(s.nc.Move(ctx, userID, filePath, destination))
}
func (s *Service) CreateShare(ctx context.Context, userID, filePath string, shareType, permissions int) (*nextcloud.ShareInfo, error) {
share, err := s.nc.CreateShare(ctx, userID, filePath, shareType, permissions)
func (s *Service) CreateShare(ctx context.Context, userID, filePath string, req createShareRequest, permissions int) (*nextcloud.ShareInfo, error) {
opts, err := s.buildCreateShareOptions(ctx, req, permissions)
if err != nil {
return nil, err
}
share, err := s.nc.CreateShare(ctx, userID, filePath, opts)
if err != nil {
return nil, mapDriveError(err)
}
if shouldSendShareEmail(opts, req) {
if sendErr := s.nc.SendShareEmail(ctx, userID, share.ID, ""); sendErr != nil {
return share, nil
}
}
return share, nil
}
func (s *Service) UserExists(ctx context.Context, email string) (bool, error) {
userID := nextcloud.UserIDFromClaims(strings.TrimSpace(email), "")
if userID == "" {
return false, nil
}
return s.nc.UserExists(ctx, userID)
}
func (s *Service) SendShareEmail(ctx context.Context, userID, shareID, password string) error {
return mapDriveError(s.nc.SendShareEmail(ctx, userID, shareID, password))
}
func shouldSendShareEmail(opts nextcloud.CreateShareOptions, req createShareRequest) bool {
// Fallback when NC did not send the invitation email on create (common for mail shares).
if opts.ShareType != 4 {
return false
}
return req.SendMail == nil || *req.SendMail
}
func (s *Service) buildCreateShareOptions(ctx context.Context, req createShareRequest, permissions int) (nextcloud.CreateShareOptions, error) {
opts := nextcloud.CreateShareOptions{Permissions: permissions}
mode := strings.TrimSpace(strings.ToLower(req.Mode))
sendMail := req.SendMail == nil || *req.SendMail
switch mode {
case "internal":
opts.ShareType = 3
opts.Label = "internal"
opts.AccessMode = "internal"
case "contact":
email := strings.TrimSpace(strings.ToLower(req.ShareWith))
if email == "" {
return opts, ErrInvalid
}
registered, err := s.UserExists(ctx, email)
if err != nil {
return opts, err
}
opts.Note = strings.TrimSpace(req.Note)
opts.SendMail = sendMail
if registered {
opts.ShareType = 0
opts.ShareWith = nextcloud.UserIDFromClaims(email, "")
opts.AccessMode = "user"
} else {
opts.ShareType = 4
opts.ShareWith = email
opts.AccessMode = "email"
}
case "public":
opts.ShareType = 3
opts.AccessMode = "public"
default:
opts.ShareType = req.ShareType
if opts.ShareType == 0 {
opts.ShareType = 3
}
if opts.ShareType == 3 {
opts.AccessMode = "public"
}
}
return opts, nil
}
func (s *Service) GetQuota(ctx context.Context, userID string) (nextcloud.UserQuota, error) {
quota, err := s.nc.GetQuota(ctx, userID)
if err != nil {
return nextcloud.UserQuota{}, mapDriveError(err)
}
return quota, nil
}
func (s *Service) ListShares(ctx context.Context, userID, filePath string) ([]nextcloud.ShareInfo, error) {
shares, err := s.nc.ListShares(ctx, userID, filePath)
if err != nil {
return nil, mapDriveError(err)
}
return shares, nil
}
func (s *Service) UpdateShare(ctx context.Context, userID, shareID string, permissions int, expireDate, password string) (*nextcloud.ShareInfo, error) {
share, err := s.nc.UpdateShare(ctx, userID, shareID, permissions, expireDate, password)
if err != nil {
return nil, mapDriveError(err)
}
return share, nil
}
func (s *Service) DeleteShare(ctx context.Context, userID, shareID string) error {
return mapDriveError(s.nc.DeleteShare(ctx, userID, shareID))
}
func (s *Service) RestoreTrash(ctx context.Context, userID, trashName string) error {
return mapDriveError(s.nc.RestoreFromTrash(ctx, userID, trashName))
}
func (s *Service) SetFavorite(ctx context.Context, userID, filePath string, favorite bool) error {
return mapDriveError(s.nc.SetFavorite(ctx, userID, filePath, favorite))
}
func (s *Service) Search(ctx context.Context, userID, basePath string, params query.ListParams) (FilesList, error) {
if basePath == "" {
basePath = "/"
}
files, err := s.nc.ListFiles(ctx, userID, basePath)
if err != nil {
return FilesList{}, mapDriveError(err)
}
filtered := filterFiles(files, params.Q)
page, total := paginate.Slice(filtered, params.Offset(), params.Limit())
return FilesList{
Files: page,
Pagination: params.Meta(&total),
}, nil
}
type NewFileKind string
const (
NewFileDocument NewFileKind = "document"
NewFileSpreadsheet NewFileKind = "spreadsheet"
NewFilePresentation NewFileKind = "presentation"
)
func (s *Service) CreateNewFile(ctx context.Context, userID, parentPath, name string, kind NewFileKind) (string, error) {
if strings.TrimSpace(name) == "" {
return "", ErrInvalid
}
content, contentType := blankOfficeFile(kind)
if content == nil {
return "", ErrInvalid
}
target := path.Join(strings.TrimSuffix(parentPath, "/"), name)
if err := mapDriveError(s.nc.Upload(ctx, userID, target, bytes.NewReader(content), contentType)); err != nil {
return "", err
}
return target, nil
}
func filterFiles(files []nextcloud.FileInfo, q string) []nextcloud.FileInfo {
q = strings.ToLower(strings.TrimSpace(q))
if q == "" {
@ -223,6 +424,9 @@ func mapDriveError(err error) error {
if err == nil {
return nil
}
if errors.Is(err, nextcloud.ErrDAVCredentialsMissing) {
return ErrForbidden
}
var statusErr *nextcloud.HTTPStatusError
if !errors.As(err, &statusErr) {
return err

BIN
internal/api/drive/testdata/blank.docx vendored Normal file
View File

Binary file not shown.

BIN
internal/api/drive/testdata/blank.pptx vendored Normal file
View File

Binary file not shown.

BIN
internal/api/drive/testdata/blank.xlsx vendored Normal file
View File

Binary file not shown.

69
internal/api/drive/validate.go
View File

@ -73,6 +73,10 @@ type createShareRequest struct {
ShareType int `json:"share_type"`
Permissions int `json:"permissions"`
Role string `json:"role"`
Mode string `json:"mode"`
ShareWith string `json:"share_with"`
Note string `json:"note"`
SendMail *bool `json:"send_mail"`
}
func sharePermissionsForRole(role string) (int, bool) {
@ -98,12 +102,77 @@ func validateCreateShareRequest(req *createShareRequest) *apivalidate.Validation
details = append(details, apivalidate.FieldDetail{Field: "role", Message: "invalid"})
}
}
mode := strings.TrimSpace(strings.ToLower(req.Mode))
switch mode {
case "", "public", "internal":
case "contact":
if strings.TrimSpace(req.ShareWith) == "" {
details = append(details, apivalidate.FieldDetail{Field: "share_with", Message: "required"})
}
default:
details = append(details, apivalidate.FieldDetail{Field: "mode", Message: "invalid"})
}
if len(details) == 0 {
return nil
}
return apivalidate.NewValidationError(details...)
}
type restoreTrashRequest struct {
Name string `json:"name"`
}
func validateRestoreTrashRequest(req *restoreTrashRequest) *apivalidate.ValidationError {
if strings.TrimSpace(req.Name) == "" {
return apivalidate.NewValidationError(apivalidate.FieldDetail{
Field: "name", Message: "required",
})
}
return nil
}
type favoriteRequest struct {
Path string `json:"path"`
Favorite bool `json:"favorite"`
}
func validateFavoriteRequest(req *favoriteRequest) *apivalidate.ValidationError {
if strings.TrimSpace(req.Path) == "" {
return apivalidate.NewValidationError(apivalidate.FieldDetail{
Field: "path", Message: "required",
})
}
return nil
}
type updateShareRequest struct {
Permissions int `json:"permissions"`
Role string `json:"role"`
ExpireDate string `json:"expire_date"`
Password string `json:"password"`
}
type newFileRequest struct {
ParentPath string `json:"parent_path"`
Name string `json:"name"`
Kind string `json:"kind"`
}
func validateNewFileRequest(req *newFileRequest) *apivalidate.ValidationError {
var details []apivalidate.FieldDetail
if strings.TrimSpace(req.Name) == "" {
details = append(details, apivalidate.FieldDetail{Field: "name", Message: "required"})
}
k := strings.TrimSpace(strings.ToLower(req.Kind))
if k != "document" && k != "spreadsheet" && k != "presentation" {
details = append(details, apivalidate.FieldDetail{Field: "kind", Message: "invalid"})
}
if len(details) > 0 {
return apivalidate.NewValidationError(details...)
}
return nil
}
func validatePath(path string) *apivalidate.ValidationError {
if strings.TrimSpace(path) == "" {
return apivalidate.NewValidationError(apivalidate.FieldDetail{

6
internal/api/drive/validate_test.go
View File

@ -81,6 +81,12 @@ func TestValidateCreateShareRequest(t *testing.T) {
if err := validateCreateShareRequest(&createShareRequest{Path: "/docs/file.pdf", Role: "admin"}); !hasFieldDetail(err, "role", "invalid") {
t.Fatal("expected invalid role error")
}
if err := validateCreateShareRequest(&createShareRequest{Path: "/docs/file.pdf", Mode: "contact"}); !hasFieldDetail(err, "share_with", "required") {
t.Fatal("expected missing share_with for contact mode")
}
if validateCreateShareRequest(&createShareRequest{Path: "/docs/file.pdf", Mode: "internal"}) != nil {
t.Fatal("expected valid internal share request")
}
}
func TestSharePermissionsForRole(t *testing.T) {

29
internal/api/mail/attachments.go
View File

@ -5,6 +5,8 @@ import (
"encoding/json"
"errors"
"io"
"path/filepath"
"strings"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
@ -72,8 +74,8 @@ func (s *Service) MessageAttachmentCIDMap(ctx context.Context, externalID, messa
}
rows, err := s.db.Query(ctx, `
SELECT id, content_id FROM attachments
WHERE message_id = $1 AND content_id <> ''
SELECT id, content_id, filename, is_inline FROM attachments
WHERE message_id = $1 AND (content_id <> '' OR is_inline)
`, messageID)
if err != nil {
return nil, err
@ -82,15 +84,32 @@ func (s *Service) MessageAttachmentCIDMap(ctx context.Context, externalID, messa
mapping := make(map[string]string)
for rows.Next() {
var id, contentID string
if err := rows.Scan(&id, &contentID); err != nil {
var id, contentID, filename string
var isInline bool
if err := rows.Scan(&id, &contentID, &filename, &isInline); err != nil {
return nil, err
}
mapping[contentID] = id
if contentID != "" {
registerCIDMapKeys(mapping, contentID, id)
}
if isInline && filename != "" {
registerCIDMapKeys(mapping, filepath.Base(filename), id)
}
}
return mapping, rows.Err()
}
func registerCIDMapKeys(mapping map[string]string, contentID, attachmentID string) {
key := strings.Trim(contentID, "<> \t")
if key == "" {
return
}
mapping[key] = attachmentID
mapping[strings.ToLower(key)] = attachmentID
mapping["cid:"+key] = attachmentID
mapping["cid:"+strings.ToLower(key)] = attachmentID
}
func (s *Service) UploadMessageAttachment(
ctx context.Context, externalID, messageID, filename, contentType, contentID string,
isInline bool, reader io.Reader, size int64,

1
internal/api/mail/handlers.go
View File

@ -114,6 +114,7 @@ func (h *Handler) Routes() chi.Router {
r.Get("/messages", h.ListMessages)
r.Get("/messages/{messageID}/attachments", h.ListMessageAttachments)
r.Get("/messages/{messageID}/attachments/cid-map", h.MessageAttachmentCIDMap)
r.Post("/messages/{messageID}/attachments/reindex", h.ReindexMessageAttachments)
r.Post("/messages/{messageID}/attachments", h.UploadMessageAttachment)
r.Post("/messages/{messageID}/list-unsubscribe-mailto", h.SendListUnsubscribeMailto)
r.Get("/messages/{messageID}", h.GetMessage)

36
internal/api/mail/handlers_account_maintenance.go
View File

@ -4,17 +4,22 @@ import (
"context"
"errors"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
"github.com/ultisuite/ulti-backend/internal/api/apiresponse"
"github.com/ultisuite/ulti-backend/internal/api/apivalidate"
"github.com/ultisuite/ulti-backend/internal/api/middleware"
mailimap "github.com/ultisuite/ulti-backend/internal/mail/imap"
)
// AccountSyncTrigger runs an immediate IMAP sync for one mail account.
// AccountSyncTrigger runs on-demand IMAP maintenance for owned mail accounts.
type AccountSyncTrigger interface {
SyncAccountForUser(ctx context.Context, externalID, accountID string) error
ForceSyncAccountForUser(ctx context.Context, externalID, accountID string) error
RefetchAccountBodiesForUser(ctx context.Context, externalID, accountID string) (mailimap.RefetchBodiesResult, error)
ReindexMessageAttachments(ctx context.Context, externalID, messageID string) error
}
func (h *Handler) ResanitizeAccountBodies(w http.ResponseWriter, r *http.Request) {
@ -25,6 +30,24 @@ func (h *Handler) ResanitizeAccountBodies(w http.ResponseWriter, r *http.Request
return
}
if h.accountSync != nil {
result, err := h.accountSync.RefetchAccountBodiesForUser(r.Context(), claims.Sub, accountID)
if err != nil {
if errors.Is(err, ErrAccountNotFound) || strings.Contains(err.Error(), "account not found") {
apivalidate.WriteNotFound(w, r, "not found")
return
}
h.logger.Error("refetch account bodies", "account_id", accountID, "error", err)
apivalidate.WriteInternal(w, r)
return
}
apiresponse.WriteJSON(w, http.StatusOK, map[string]int{
"scanned": result.Scanned,
"updated": result.Updated,
})
return
}
result, err := h.svc.ResanitizeAccountBodies(r.Context(), claims.Sub, accountID)
if err != nil {
if errors.Is(err, ErrAccountNotFound) {
@ -61,8 +84,15 @@ func (h *Handler) SyncAccountNow(w http.ResponseWriter, r *http.Request) {
return
}
if err := h.accountSync.SyncAccountForUser(r.Context(), claims.Sub, accountID); err != nil {
h.logger.Error("sync account", "account_id", accountID, "error", err)
force := r.URL.Query().Get("force") == "true"
var syncErr error
if force {
syncErr = h.accountSync.ForceSyncAccountForUser(r.Context(), claims.Sub, accountID)
} else {
syncErr = h.accountSync.SyncAccountForUser(r.Context(), claims.Sub, accountID)
}
if syncErr != nil {
h.logger.Error("sync account", "account_id", accountID, "force", force, "error", syncErr)
apiresponse.WriteError(w, r, http.StatusBadGateway, "sync_failed", "imap sync failed", nil)
return
}

28
internal/api/mail/handlers_attachments.go
View File

@ -51,6 +51,34 @@ func (h *Handler) MessageAttachmentCIDMap(w http.ResponseWriter, r *http.Request
apiresponse.WriteJSON(w, http.StatusOK, map[string]any{"cid_map": mapping})
}
func (h *Handler) ReindexMessageAttachments(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
messageID := chi.URLParam(r, "messageID")
if h.accountSync == nil {
apiresponse.WriteError(w, r, http.StatusServiceUnavailable, "sync_unavailable", "mail sync is not configured", nil)
return
}
if _, err := h.svc.GetMessage(r.Context(), claims.Sub, messageID); err != nil {
if errors.Is(err, ErrNotFound) {
apivalidate.WriteNotFound(w, r, "not found")
return
}
h.logger.Error("load message for attachment reindex", "message_id", messageID, "error", err)
apivalidate.WriteInternal(w, r)
return
}
if err := h.accountSync.ReindexMessageAttachments(r.Context(), claims.Sub, messageID); err != nil {
if strings.Contains(err.Error(), "not found") {
apivalidate.WriteNotFound(w, r, "not found")
return
}
h.logger.Error("reindex message attachments", "message_id", messageID, "error", err)
apiresponse.WriteError(w, r, http.StatusBadGateway, "reindex_failed", "attachment reindex failed", nil)
return
}
apiresponse.WriteJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
func (h *Handler) UploadMessageAttachment(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
messageID := chi.URLParam(r, "messageID")

6
internal/api/mail/service.go
View File

@ -227,9 +227,9 @@ func (s *Service) ListMessages(ctx context.Context, externalID string, filter Me
return MessagesList{}, err
}
bodyTextSample, bodyHTMLSample = imap.RepairStoredBodies(bodyTextSample, bodyHTMLSample)
preview := imap.RepairSnippet(imap.SnippetFromBodies(bodyTextSample, bodyHTMLSample, 200))
preview := imap.RepairSnippetWithBodies(snippet, bodyTextSample, bodyHTMLSample)
if preview == "" {
preview = imap.RepairSnippet(snippet)
preview = imap.SnippetFromBodies(bodyTextSample, bodyHTMLSample, 200)
}
entry := map[string]any{
"id": id, "message_id": messageID,
@ -294,7 +294,7 @@ func (s *Service) GetMessage(ctx context.Context, externalID, messageID string)
}
bodyText, bodyHTML := imap.RepairStoredBodies(msg.Text, msg.HTML)
subject := imap.RepairSubject(msg.Subject, bodyText, bodyHTML, nil)
repairedSnippet := imap.RepairSnippet(imap.SnippetFromBodies(bodyText, bodyHTML, 200))
repairedSnippet := imap.RepairSnippetWithBodies(imap.SnippetFromBodies(bodyText, bodyHTML, 200), bodyText, bodyHTML)
if bodyText != msg.Text || bodyHTML != msg.HTML || subject != msg.Subject {
_, _ = s.db.Exec(ctx, `
UPDATE messages SET body_text = $1, body_html = $2, snippet = $3, subject = $4, updated_at = NOW()

177
internal/api/office/handlers.go Normal file
View File

@ -0,0 +1,177 @@
package office
import (
"encoding/json"
"io"
"log/slog"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
"github.com/ultisuite/ulti-backend/internal/api/apiresponse"
"github.com/ultisuite/ulti-backend/internal/api/apivalidate"
"github.com/ultisuite/ulti-backend/internal/api/drive"
"github.com/ultisuite/ulti-backend/internal/api/middleware"
"github.com/ultisuite/ulti-backend/internal/permission"
)
type Handler struct {
svc *Service
drive *drive.Service
logger *slog.Logger
}
func NewHandler(svc *Service, driveSvc *drive.Service) *Handler {
return &Handler{
svc: svc,
drive: driveSvc,
logger: slog.Default().With("component", "office-api"),
}
}
func (h *Handler) PublicRoutes() chi.Router {
r := chi.NewRouter()
r.Get("/document", h.ServeDocument)
r.Post("/callback", h.Callback)
return r
}
func (h *Handler) ProtectedRoutes() chi.Router {
r := chi.NewRouter()
read := middleware.RequirePermission(permission.ResourceDrive, permission.LevelRead)
write := middleware.RequirePermission(permission.ResourceDrive, permission.LevelWrite)
r.With(read).Post("/session", h.CreateSession)
r.With(write).Post("/create", h.CreateDocument)
return r
}
// Routes registers public OnlyOffice callbacks and authenticated session endpoints on one router.
func (h *Handler) Routes(authMiddleware func(http.Handler) http.Handler) chi.Router {
r := chi.NewRouter()
r.Get("/document", h.ServeDocument)
r.Post("/callback", h.Callback)
r.Group(func(pr chi.Router) {
pr.Use(authMiddleware)
read := middleware.RequirePermission(permission.ResourceDrive, permission.LevelRead)
write := middleware.RequirePermission(permission.ResourceDrive, permission.LevelWrite)
pr.With(read).Post("/session", h.CreateSession)
pr.With(write).Post("/create", h.CreateDocument)
})
return r
}
type sessionRequest struct {
Path string `json:"path"`
Mode string `json:"mode"`
}
func (h *Handler) CreateSession(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, err := h.svc.EnsureNextcloudUser(r.Context(), claims)
if err != nil {
h.logger.Error("ensure nextcloud user", "error", err)
apivalidate.WriteInternal(w, r)
return
}
var req sessionRequest
if err := apivalidate.DecodeJSON(w, r, 32<<10, &req); err != nil {
return
}
if strings.TrimSpace(req.Path) == "" {
apivalidate.WriteValidationError(w, r, apivalidate.NewValidationError(
apivalidate.FieldDetail{Field: "path", Message: "required"},
))
return
}
mode := strings.TrimSpace(req.Mode)
if mode == "" {
mode = "edit"
}
cfg, err := h.svc.EditorConfig(r.Context(), ncUser, req.Path, mode, claims.Name)
if err != nil {
h.logger.Error("editor config", "error", err)
apivalidate.WriteInternal(w, r)
return
}
apiresponse.WriteJSON(w, http.StatusOK, map[string]any{
"config": cfg,
"serverUrl": h.svc.PublicURL(),
})
}
type createRequest struct {
ParentPath string `json:"parent_path"`
Name string `json:"name"`
Kind string `json:"kind"`
}
func (h *Handler) CreateDocument(w http.ResponseWriter, r *http.Request) {
claims := middleware.ClaimsFromContext(r.Context())
ncUser, err := h.svc.EnsureNextcloudUser(r.Context(), claims)
if err != nil {
apivalidate.WriteInternal(w, r)
return
}
var req createRequest
if err := apivalidate.DecodeJSON(w, r, 32<<10, &req); err != nil {
return
}
kind := drive.NewFileKind(strings.TrimSpace(strings.ToLower(req.Kind)))
target, err := h.drive.CreateNewFile(r.Context(), ncUser, req.ParentPath, req.Name, kind)
if err != nil {
apiresponse.WriteError(w, r, http.StatusBadRequest, apiresponse.CodeInvalidRequest, err.Error(), nil)
return
}
apiresponse.WriteJSON(w, http.StatusCreated, map[string]string{"path": target})
}
func (h *Handler) ServeDocument(w http.ResponseWriter, r *http.Request) {
ncUser := r.URL.Query().Get("user")
filePath := r.URL.Query().Get("path")
sig := r.URL.Query().Get("sig")
if h.svc.Cfg.JWTSecret != "" && !VerifyDocAccess(ncUser, filePath, sig, h.svc.Cfg.JWTSecret) {
http.Error(w, "forbidden", http.StatusForbidden)
return
}
body, contentType, err := h.svc.OpenDocument(r.Context(), ncUser, filePath)
if err != nil {
http.Error(w, "not found", http.StatusNotFound)
return
}
defer body.Close()
if contentType != "" {
w.Header().Set("Content-Type", contentType)
}
io.Copy(w, body)
}
func (h *Handler) Callback(w http.ResponseWriter, r *http.Request) {
ncUser := r.URL.Query().Get("user")
filePath := r.URL.Query().Get("path")
var payload struct {
Status int `json:"status"`
URL string `json:"url"`
Key string `json:"key"`
}
if err := json.NewDecoder(r.Body).Decode(&payload); err != nil {
apiresponse.WriteJSON(w, http.StatusOK, map[string]int{"error": 1})
return
}
// status 2 = ready for saving, 6 = must force save
if payload.Status == 2 || payload.Status == 6 {
if payload.URL != "" {
resp, err := http.Get(payload.URL)
if err == nil && resp.StatusCode == http.StatusOK {
defer resp.Body.Close()
ct := resp.Header.Get("Content-Type")
_ = h.svc.SaveDocument(r.Context(), ncUser, filePath, resp.Body, ct)
}
}
}
apiresponse.WriteJSON(w, http.StatusOK, map[string]int{"error": 0})
}

75
internal/api/office/jwt.go Normal file
View File

@ -0,0 +1,75 @@
package office
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
)
func signJWT(payload any, secret string) (string, error) {
if secret == "" {
return "", nil
}
header := base64.RawURLEncoding.EncodeToString([]byte(`{"alg":"HS256","typ":"JWT"}`))
bodyBytes, err := json.Marshal(payload)
if err != nil {
return "", err
}
body := base64.RawURLEncoding.EncodeToString(bodyBytes)
mac := hmac.New(sha256.New, []byte(secret))
_, _ = mac.Write([]byte(header + "." + body))
sig := base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
return header + "." + body + "." + sig, nil
}
func wrapConfig(config map[string]any, secret string) (map[string]any, error) {
if secret == "" {
return config, nil
}
token, err := signJWT(config, secret)
if err != nil {
return nil, err
}
config["token"] = token
return config, nil
}
func verifyJWT(token, secret string) (map[string]any, error) {
if secret == "" || token == "" {
return nil, fmt.Errorf("missing token or secret")
}
parts := splitJWT(token)
if len(parts) != 3 {
return nil, fmt.Errorf("invalid token")
}
mac := hmac.New(sha256.New, []byte(secret))
_, _ = mac.Write([]byte(parts[0] + "." + parts[1]))
expected := base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
if !hmac.Equal([]byte(expected), []byte(parts[2])) {
return nil, fmt.Errorf("invalid signature")
}
raw, err := base64.RawURLEncoding.DecodeString(parts[1])
if err != nil {
return nil, err
}
var payload map[string]any
if err := json.Unmarshal(raw, &payload); err != nil {
return nil, err
}
return payload, nil
}
func splitJWT(token string) []string {
var parts []string
start := 0
for i := 0; i < len(token); i++ {
if token[i] == '.' {
parts = append(parts, token[start:i])
start = i + 1
}
}
parts = append(parts, token[start:])
return parts
}

174
internal/api/office/service.go Normal file
View File

@ -0,0 +1,174 @@
package office
import (
"context"
"crypto/sha256"
"encoding/hex"
"io"
"net/url"
"path"
"strings"
"time"
"github.com/ultisuite/ulti-backend/internal/auth"
"github.com/ultisuite/ulti-backend/internal/nextcloud"
)
type Config struct {
Enabled bool
DocumentURL string // OnlyOffice Document Server API base (e.g. http://onlyoffice)
PublicURL string // Browser-facing Document Server URL
APIInternalURL string // ultid base reachable from OnlyOffice container (doc fetch + callback)
JWTSecret string
}
type Service struct {
nc *nextcloud.Client
Cfg Config
}
func NewService(nc *nextcloud.Client, cfg Config) *Service {
return &Service{nc: nc, Cfg: cfg}
}
func (s *Service) PublicURL() string {
return strings.TrimRight(s.Cfg.PublicURL, "/")
}
func (s *Service) EnsureNextcloudUser(ctx context.Context, claims *auth.Claims) (string, error) {
return s.nc.EnsurePrincipal(ctx, claims.Email, claims.Sub, claims.Name)
}
func (s *Service) EditorConfig(ctx context.Context, ncUser, filePath, mode, userName string) (map[string]any, error) {
filePath = normalizePath(filePath)
docType := documentType(filePath)
key := documentKey(ncUser, filePath)
apiBase := strings.TrimRight(s.Cfg.APIInternalURL, "/")
sig := ""
if s.Cfg.JWTSecret != "" {
var err error
sig, err = signDocAccess(ncUser, filePath, s.Cfg.JWTSecret)
if err != nil {
return nil, err
}
}
downloadURL := buildOfficeEndpointURL(apiBase, "/api/v1/office/document", ncUser, filePath, sig)
callbackURL := buildOfficeEndpointURL(apiBase, "/api/v1/office/callback", ncUser, filePath, "")
edit := mode == "edit"
document := map[string]any{
"fileType": fileExt(filePath),
"key": key,
"title": path.Base(filePath),
"url": downloadURL,
"permissions": map[string]any{
"comment": true,
"copy": true,
"deleteCommentAuthorOnly": false,
"download": true,
"edit": edit,
"editCommentAuthorOnly": false,
"fillForms": edit,
"modifyContentControl": edit,
"modifyFilter": edit,
"print": true,
"review": true,
},
}
editorCfg := map[string]any{
"mode": mode,
"user": map[string]any{
"id": ncUser,
"name": userName,
},
"callbackUrl": callbackURL,
"customization": map[string]any{
"forcesave": true,
},
}
config := map[string]any{
"documentType": docType,
"document": document,
"editorConfig": editorCfg,
}
return wrapConfig(config, s.Cfg.JWTSecret)
}
func (s *Service) OpenDocument(ctx context.Context, ncUser, filePath string) (io.ReadCloser, string, error) {
return s.nc.Download(ctx, ncUser, normalizePath(filePath))
}
func (s *Service) SaveDocument(ctx context.Context, ncUser, filePath string, body io.Reader, contentType string) error {
return s.nc.Upload(ctx, ncUser, normalizePath(filePath), body, contentType)
}
func documentKey(ncUser, filePath string) string {
h := sha256.Sum256([]byte(ncUser + "|" + filePath + "|" + time.Now().Format("2006-01-02")))
return hex.EncodeToString(h[:16])
}
func documentType(filePath string) string {
ext := strings.ToLower(path.Ext(filePath))
switch ext {
case ".xlsx", ".xls", ".xlsb", ".xlsm", ".xlt", ".xltm", ".xltx",
".ods", ".ots", ".csv", ".tsv", ".fods", ".et", ".ett", ".sxc":
return "cell"
case ".pptx", ".ppt", ".pptm", ".pot", ".potm", ".potx",
".pps", ".ppsm", ".ppsx", ".odp", ".otp", ".odg", ".fodp",
".dps", ".dpt", ".sxi":
return "slide"
case ".vsdm", ".vsdx", ".vssm", ".vssx", ".vstm", ".vstx":
return "diagram"
default:
return "word"
}
}
func fileExt(filePath string) string {
return strings.TrimPrefix(strings.ToLower(path.Ext(filePath)), ".")
}
func normalizePath(p string) string {
p = strings.TrimSpace(p)
if p == "" {
return "/"
}
if !strings.HasPrefix(p, "/") {
p = "/" + p
}
return p
}
func buildOfficeEndpointURL(base, endpoint, ncUser, filePath, sig string) string {
q := url.Values{}
q.Set("path", normalizePath(filePath))
q.Set("user", ncUser)
if sig != "" {
q.Set("sig", sig)
}
return strings.TrimRight(base, "/") + endpoint + "?" + q.Encode()
}
func signDocAccess(ncUser, filePath, secret string) (string, error) {
payload := map[string]any{
"user": ncUser,
"path": normalizePath(filePath),
"exp": time.Now().Add(2 * time.Hour).Unix(),
}
return signJWT(payload, secret)
}
func VerifyDocAccess(ncUser, filePath, sig, secret string) bool {
payload, err := verifyJWT(sig, secret)
if err != nil {
return false
}
if payload["user"] != ncUser || payload["path"] != normalizePath(filePath) {
return false
}
if exp, ok := payload["exp"].(float64); ok && int64(exp) < time.Now().Unix() {
return false
}
return true
}

171
internal/authentik/catalog.go Normal file
View File

@ -0,0 +1,171 @@
package authentik
import (
"fmt"
"strings"
"github.com/ultisuite/ulti-backend/internal/config"
)
const suiteGroup = "Ulti Suite"
// AppSpec describes one Authentik OAuth application to provision when enabled.
type AppSpec struct {
Key string
Slug string
ProviderName string
DisplayName string
ClientID func(*config.Config) string
ClientSecret func(*config.Config) string
LaunchURL func(*config.Config) string
RedirectURIs func(*config.Config) []string
Enabled func(*config.Config) bool
OfflineAccess bool
}
func baseURL(cfg *config.Config) string {
host := strings.TrimSpace(cfg.Domain)
if host == "" {
host = "localhost"
}
scheme := "http"
if cfg.AuthentikPublicHTTPS {
scheme = "https"
}
return fmt.Sprintf("%s://%s", scheme, host)
}
func Catalog(cfg *config.Config) []AppSpec {
return []AppSpec{
{
Key: "ultimail",
Slug: "ulti",
ProviderName: "ulti-backend-provider",
DisplayName: "Ultimail",
ClientID: func(c *config.Config) string { return c.OIDCClientID },
ClientSecret: func(c *config.Config) string { return c.OIDCClientSecret },
LaunchURL: func(c *config.Config) string { return c.MailAppURL + "/" },
RedirectURIs: ultimailRedirectURIs,
Enabled: func(c *config.Config) bool { return c.OIDCClientID != "" && c.OIDCClientSecret != "" },
OfflineAccess: true,
},
{
Key: "nextcloud",
Slug: "nextcloud",
ProviderName: "ulti-nextcloud-provider",
DisplayName: "Nextcloud",
ClientID: func(c *config.Config) string { return c.NCOIDCClientID },
ClientSecret: func(c *config.Config) string { return c.NCOIDCClientSecret },
LaunchURL: func(c *config.Config) string { return baseURL(c) + "/cloud/" },
RedirectURIs: nextcloudRedirectURIs,
Enabled: func(c *config.Config) bool { return c.NextcloudEnabled },
},
{
Key: "onlyoffice",
Slug: "onlyoffice",
ProviderName: "ulti-onlyoffice-provider",
DisplayName: "OnlyOffice",
ClientID: func(c *config.Config) string { return c.OnlyOfficeOIDCClientID },
ClientSecret: func(c *config.Config) string { return c.OnlyOfficeOIDCClientSecret },
LaunchURL: func(c *config.Config) string { return c.OnlyOfficePublicURL },
RedirectURIs: onlyofficeRedirectURIs,
Enabled: func(c *config.Config) bool { return c.OnlyOfficeEnabled },
},
{
Key: "immich",
Slug: "immich",
ProviderName: "ulti-immich-provider",
DisplayName: "Ultiphotos",
ClientID: func(c *config.Config) string { return c.ImmichOIDCClientID },
ClientSecret: func(c *config.Config) string { return c.ImmichOIDCClientSecret },
LaunchURL: func(c *config.Config) string { return baseURL(c) + "/photos/" },
RedirectURIs: immichRedirectURIs,
Enabled: func(c *config.Config) bool { return c.ImmichEnabled },
},
{
Key: "ultidrive",
Slug: "ultidrive",
ProviderName: "ulti-drive-provider",
DisplayName: "UltiDrive",
ClientID: func(c *config.Config) string { return c.DriveOIDCClientID },
ClientSecret: func(c *config.Config) string { return c.DriveOIDCClientSecret },
LaunchURL: func(c *config.Config) string { return baseURL(c) + "/drive/" },
RedirectURIs: driveRedirectURIs,
Enabled: func(c *config.Config) bool {
return c.NextcloudEnabled && c.DriveOIDCClientID != ""
},
},
}
}
func ultimailRedirectURIs(cfg *config.Config) []string {
base := baseURL(cfg)
mail := strings.TrimRight(cfg.MailAppURL, "/")
drive := strings.TrimRight(base+"/drive", "/")
return uniqueURIs(
mail+"/api/auth/callback",
"http://localhost:3000/api/auth/callback",
"http://127.0.0.1:3000/api/auth/callback",
drive+"/api/auth/callback",
"http://localhost:3001/api/auth/callback",
"http://127.0.0.1:3001/api/auth/callback",
base+"/api/auth/callback",
)
}
func nextcloudRedirectURIs(cfg *config.Config) []string {
base := baseURL(cfg)
return uniqueURIs(
base+"/cloud/apps/user_oidc/code",
"http://localhost/cloud/apps/user_oidc/code",
"http://127.0.0.1/cloud/apps/user_oidc/code",
)
}
func onlyofficeRedirectURIs(cfg *config.Config) []string {
base := baseURL(cfg)
office := strings.TrimRight(cfg.OnlyOfficePublicURL, "/")
return uniqueURIs(
office+"/",
office+"/oauth2/callback",
base+"/office/",
base+"/office/oauth2/callback",
"http://localhost/office/",
)
}
func immichRedirectURIs(cfg *config.Config) []string {
base := baseURL(cfg)
return uniqueURIs(
base+"/photos/auth/login",
base+"/photos/user-settings",
"http://localhost/photos/auth/login",
)
}
func driveRedirectURIs(cfg *config.Config) []string {
base := baseURL(cfg)
drive := strings.TrimRight(base+"/drive", "/")
return uniqueURIs(
drive+"/api/auth/callback",
"http://localhost:3001/api/auth/callback",
"http://127.0.0.1:3001/api/auth/callback",
)
}
func uniqueURIs(uris ...string) []string {
seen := map[string]struct{}{}
var out []string
for _, u := range uris {
u = strings.TrimSpace(u)
if u == "" {
continue
}
if _, ok := seen[u]; ok {
continue
}
seen[u] = struct{}{}
out = append(out, u)
}
return out
}

297
internal/authentik/client.go Normal file
View File

@ -0,0 +1,297 @@
package authentik
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
)
type Client struct {
baseURL string
token string
httpClient *http.Client
}
func NewClient(baseURL, token string) *Client {
return &Client{
baseURL: strings.TrimRight(baseURL, "/"),
token: token,
httpClient: &http.Client{
Timeout: 30 * time.Second,
},
}
}
type listResponse[T any] struct {
Results []T `json:"results"`
}
type flowRef struct {
PK string `json:"pk"`
Slug string `json:"slug"`
}
type oauth2Provider struct {
PK int `json:"pk"`
Name string `json:"name"`
ClientID string `json:"client_id"`
RedirectURIs string `json:"redirect_uris"`
}
type application struct {
PK int `json:"pk"`
Slug string `json:"slug"`
Name string `json:"name"`
Provider int `json:"provider"`
}
type scopeMapping struct {
PK string `json:"pk"`
ScopeName string `json:"scope_name"`
}
type certKeyPair struct {
PK string `json:"pk"`
Name string `json:"name"`
}
func (c *Client) Ping(ctx context.Context) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+"/api/v3/root/config/", nil)
if err != nil {
return err
}
c.authorize(req)
resp, err := c.httpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("authentik ping: %d", resp.StatusCode)
}
return nil
}
func (c *Client) authorize(req *http.Request) {
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
req.Header.Set("Accept", "application/json")
}
func (c *Client) FindApplicationBySlug(ctx context.Context, slug string) (*application, bool, error) {
q := url.Values{}
q.Set("slug", slug)
var out listResponse[application]
if err := c.getJSON(ctx, "/api/v3/core/applications/?"+q.Encode(), &out); err != nil {
return nil, false, err
}
if len(out.Results) == 0 {
return nil, false, nil
}
return &out.Results[0], true, nil
}
func (c *Client) FindOAuth2ProviderByName(ctx context.Context, name string) (*oauth2Provider, bool, error) {
q := url.Values{}
q.Set("name", name)
var out listResponse[oauth2Provider]
if err := c.getJSON(ctx, "/api/v3/providers/oauth2/?"+q.Encode(), &out); err != nil {
return nil, false, err
}
if len(out.Results) == 0 {
return nil, false, nil
}
return &out.Results[0], true, nil
}
func (c *Client) FindFlowBySlug(ctx context.Context, slug string) (string, error) {
q := url.Values{}
q.Set("slug", slug)
var out listResponse[flowRef]
if err := c.getJSON(ctx, "/api/v3/flows/instances/?"+q.Encode(), &out); err != nil {
return "", err
}
if len(out.Results) == 0 {
return "", fmt.Errorf("flow not found: %s", slug)
}
return out.Results[0].PK, nil
}
func (c *Client) FindScopeMappingPK(ctx context.Context, scopeName string) (string, error) {
q := url.Values{}
q.Set("scope_name", scopeName)
var out listResponse[scopeMapping]
if err := c.getJSON(ctx, "/api/v3/propertymappings/provider/oauth2/?"+q.Encode(), &out); err != nil {
return "", err
}
if len(out.Results) == 0 {
return "", fmt.Errorf("scope mapping not found: %s", scopeName)
}
return out.Results[0].PK, nil
}
func (c *Client) FindSigningKeyPK(ctx context.Context) (string, error) {
var out listResponse[certKeyPair]
if err := c.getJSON(ctx, "/api/v3/crypto/certificatekeypairs/", &out); err != nil {
return "", err
}
for _, k := range out.Results {
if strings.Contains(k.Name, "authentik") && strings.Contains(k.Name, "Certificate") {
return k.PK, nil
}
}
if len(out.Results) > 0 {
return out.Results[0].PK, nil
}
return "", fmt.Errorf("no signing key found")
}
type CreateOAuth2ProviderRequest struct {
Name string
ClientID string
ClientSecret string
RedirectURIs []string
AuthorizationFlowPK string
InvalidationFlowPK string
SigningKeyPK string
PropertyMappingPKs []string
OfflineAccess bool
}
func (c *Client) CreateOAuth2Provider(ctx context.Context, req CreateOAuth2ProviderRequest) (int, error) {
mappings := req.PropertyMappingPKs
if req.OfflineAccess {
if pk, err := c.FindScopeMappingPK(ctx, "offline_access"); err == nil {
mappings = append(mappings, pk)
}
}
body := map[string]any{
"name": req.Name,
"authorization_flow": req.AuthorizationFlowPK,
"invalidation_flow": req.InvalidationFlowPK,
"property_mappings": mappings,
"client_type": "confidential",
"client_id": req.ClientID,
"client_secret": req.ClientSecret,
"redirect_uris": joinRedirectURIs(req.RedirectURIs),
"signing_key": req.SigningKeyPK,
"access_token_validity": "hours=1",
"refresh_token_validity": "days=365",
}
var created oauth2Provider
if err := c.postJSON(ctx, "/api/v3/providers/oauth2/", body, &created); err != nil {
return 0, err
}
return created.PK, nil
}
func (c *Client) UpdateOAuth2ProviderRedirects(ctx context.Context, providerPK int, redirectURIs []string) error {
body := map[string]any{
"redirect_uris": joinRedirectURIs(redirectURIs),
}
return c.patchJSON(ctx, fmt.Sprintf("/api/v3/providers/oauth2/%d/", providerPK), body)
}
type CreateApplicationRequest struct {
Name string
Slug string
Group string
LaunchURL string
Provider int
}
func (c *Client) CreateApplication(ctx context.Context, req CreateApplicationRequest) (int, error) {
body := map[string]any{
"name": req.Name,
"slug": req.Slug,
"group": req.Group,
"provider": req.Provider,
"meta_launch_url": req.LaunchURL,
"policy_engine_mode": "any",
}
var created application
if err := c.postJSON(ctx, "/api/v3/core/applications/", body, &created); err != nil {
return 0, err
}
return created.PK, nil
}
func joinRedirectURIs(uris []string) string {
seen := make(map[string]struct{}, len(uris))
var lines []string
for _, u := range uris {
u = strings.TrimSpace(u)
if u == "" {
continue
}
if _, ok := seen[u]; ok {
continue
}
seen[u] = struct{}{}
lines = append(lines, u)
}
return strings.Join(lines, "\n")
}
func (c *Client) getJSON(ctx context.Context, path string, dest any) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+path, nil)
if err != nil {
return err
}
c.authorize(req)
resp, err := c.httpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
return decodeResponse(resp, dest)
}
func (c *Client) postJSON(ctx context.Context, path string, body any, dest any) error {
return c.doJSON(ctx, http.MethodPost, path, body, dest)
}
func (c *Client) patchJSON(ctx context.Context, path string, body any) error {
return c.doJSON(ctx, http.MethodPatch, path, body, nil)
}
func (c *Client) doJSON(ctx context.Context, method, path string, body any, dest any) error {
var reader io.Reader
if body != nil {
raw, err := json.Marshal(body)
if err != nil {
return err
}
reader = bytes.NewReader(raw)
}
req, err := http.NewRequestWithContext(ctx, method, c.baseURL+path, reader)
if err != nil {
return err
}
c.authorize(req)
req.Header.Set("Content-Type", "application/json")
resp, err := c.httpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
return decodeResponse(resp, dest)
}
func decodeResponse(resp *http.Response, dest any) error {
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
b, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
return fmt.Errorf("authentik api %s: %d %s", resp.Request.URL.Path, resp.StatusCode, strings.TrimSpace(string(b)))
}
if dest == nil {
return nil
}
return json.NewDecoder(resp.Body).Decode(dest)
}

268
internal/authentik/provision.go Normal file
View File

@ -0,0 +1,268 @@
package authentik
import (
"context"
"log/slog"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/ultisuite/ulti-backend/internal/config"
)
const (
provisionAttempts = 30
provisionDelay = 5 * time.Second
)
// StartProvisioner runs Authentik suite app provisioning in the background until success or ctx cancel.
func StartProvisioner(ctx context.Context, pool *pgxpool.Pool, cfg *config.Config) {
if cfg == nil || cfg.AuthentikAPIToken == "" {
slog.Info("authentik api provisioning skipped (AUTHENTIK_API_TOKEN not set; using blueprints)")
return
}
go func() {
for attempt := 1; attempt <= provisionAttempts; attempt++ {
if ctx.Err() != nil {
return
}
if err := EnsureSuiteApplications(ctx, pool, cfg); err != nil {
slog.Warn("authentik suite provisioning failed, retrying",
"attempt", attempt,
"max", provisionAttempts,
"error", err,
)
select {
case <-ctx.Done():
return
case <-time.After(provisionDelay):
}
continue
}
if attempt > 1 {
slog.Info("authentik suite provisioning ready", "attempt", attempt)
}
return
}
slog.Error("authentik suite provisioning gave up after retries", "attempts", provisionAttempts)
}()
}
// EnsureSuiteApplications creates or adopts Authentik OAuth apps for enabled suite integrations.
func EnsureSuiteApplications(ctx context.Context, pool *pgxpool.Pool, cfg *config.Config) error {
if cfg.AuthentikAPIToken == "" {
return nil
}
client := NewClient(cfg.AuthentikAPIURL, cfg.AuthentikAPIToken)
if err := client.Ping(ctx); err != nil {
return err
}
authFlow, err := client.FindFlowBySlug(ctx, "default-provider-authorization-implicit-consent")
if err != nil {
return err
}
invalidFlow, err := client.FindFlowBySlug(ctx, "default-provider-invalidation-flow")
if err != nil {
return err
}
signingKey, err := client.FindSigningKeyPK(ctx)
if err != nil {
return err
}
scopePKs, err := loadStandardScopeMappings(ctx, client)
if err != nil {
return err
}
for _, spec := range Catalog(cfg) {
if spec.Enabled != nil && !spec.Enabled(cfg) {
continue
}
clientID, clientSecret, ok := resolveCredentials(spec, cfg)
if !ok {
slog.Warn("authentik app skipped: missing client credentials", "app", spec.Key)
continue
}
redirects := spec.RedirectURIs(cfg)
launchURL := ""
if spec.LaunchURL != nil {
launchURL = spec.LaunchURL(cfg)
}
if provisioned, err := IsProvisioned(ctx, pool, spec.Key); err != nil {
return err
} else if provisioned {
if err := syncRedirects(ctx, pool, client, spec.Key, redirects); err != nil {
slog.Warn("authentik redirect sync failed", "app", spec.Key, "error", err)
}
continue
}
providerPK, appPK, err := ensureApp(ctx, client, spec, clientID, clientSecret, launchURL, redirects, authFlow, invalidFlow, signingKey, scopePKs)
if err != nil {
slog.Warn("authentik app provision failed", "app", spec.Key, "error", err)
continue
}
now := time.Now()
if err := SaveProvisioned(ctx, pool, ProvisionRecord{
AppKey: spec.Key,
AuthentikSlug: spec.Slug,
ClientID: clientID,
ProviderID: &providerPK,
ApplicationID: &appPK,
ProvisionedAt: now,
}); err != nil {
return err
}
slog.Info("authentik app provisioned", "app", spec.Key, "slug", spec.Slug, "client_id", clientID)
}
return nil
}
func syncRedirects(ctx context.Context, pool *pgxpool.Pool, client *Client, appKey string, redirects []string) error {
rec, err := GetProvisioned(ctx, pool, appKey)
if err != nil {
return err
}
if rec.ProviderID == nil || *rec.ProviderID == 0 {
return nil
}
return client.UpdateOAuth2ProviderRedirects(ctx, *rec.ProviderID, redirects)
}
func ensureApp(
ctx context.Context,
client *Client,
spec AppSpec,
clientID, clientSecret, launchURL string,
redirects []string,
authFlow, invalidFlow, signingKey string,
scopePKs []string,
) (providerPK, appPK int, err error) {
if app, found, err := client.FindApplicationBySlug(ctx, spec.Slug); err != nil {
return 0, 0, err
} else if found {
providerPK = app.Provider
if providerPK == 0 {
if prov, ok, err := client.FindOAuth2ProviderByName(ctx, spec.ProviderName); err != nil {
return 0, 0, err
} else if ok {
providerPK = prov.PK
}
}
if providerPK != 0 {
_ = client.UpdateOAuth2ProviderRedirects(ctx, providerPK, redirects)
}
return providerPK, app.PK, nil
}
if prov, found, err := client.FindOAuth2ProviderByName(ctx, spec.ProviderName); err != nil {
return 0, 0, err
} else if found {
providerPK = prov.PK
_ = client.UpdateOAuth2ProviderRedirects(ctx, providerPK, redirects)
} else {
mappings := scopePKs
providerPK, err = client.CreateOAuth2Provider(ctx, CreateOAuth2ProviderRequest{
Name: spec.ProviderName,
ClientID: clientID,
ClientSecret: clientSecret,
RedirectURIs: redirects,
AuthorizationFlowPK: authFlow,
InvalidationFlowPK: invalidFlow,
SigningKeyPK: signingKey,
PropertyMappingPKs: mappings,
OfflineAccess: spec.OfflineAccess,
})
if err != nil {
return 0, 0, err
}
}
appPK, err = client.CreateApplication(ctx, CreateApplicationRequest{
Name: spec.DisplayName,
Slug: spec.Slug,
Group: suiteGroup,
LaunchURL: launchURL,
Provider: providerPK,
})
if err != nil {
return providerPK, 0, err
}
return providerPK, appPK, nil
}
func loadStandardScopeMappings(ctx context.Context, client *Client) ([]string, error) {
var pks []string
for _, name := range []string{"openid", "email", "profile"} {
pk, err := client.FindScopeMappingPK(ctx, name)
if err != nil {
return nil, err
}
pks = append(pks, pk)
}
return pks, nil
}
func resolveCredentials(spec AppSpec, cfg *config.Config) (clientID, clientSecret string, ok bool) {
if spec.ClientID != nil {
clientID = spec.ClientID(cfg)
}
if spec.ClientSecret != nil {
clientSecret = spec.ClientSecret(cfg)
}
if clientID == "" {
clientID = defaultClientID(spec.Key)
}
if clientSecret == "" {
clientSecret = defaultClientSecret(spec.Key, cfg)
}
if clientID == "" || clientSecret == "" {
return "", "", false
}
return clientID, clientSecret, true
}
func defaultClientID(key string) string {
switch key {
case "ultimail":
return "ulti-backend"
case "nextcloud":
return "ulti-nextcloud"
case "onlyoffice":
return "ulti-onlyoffice"
case "immich":
return "ulti-immich"
case "ultidrive":
return "ulti-drive"
default:
return ""
}
}
func defaultClientSecret(key string, cfg *config.Config) string {
switch key {
case "ultimail":
return cfg.OIDCClientSecret
case "nextcloud":
return secretOrChangeme(cfg.NCOIDCClientSecret)
case "onlyoffice":
return secretOrChangeme(cfg.OnlyOfficeOIDCClientSecret)
case "immich":
return secretOrChangeme(cfg.ImmichOIDCClientSecret)
case "ultidrive":
return secretOrChangeme(cfg.DriveOIDCClientSecret)
default:
return "changeme"
}
}
func secretOrChangeme(s string) string {
if s != "" {
return s
}
return "changeme"
}

22
internal/authentik/provision_test.go Normal file
View File

@ -0,0 +1,22 @@
package authentik
import "testing"
func TestJoinRedirectURIsDedupes(t *testing.T) {
got := joinRedirectURIs([]string{
"http://localhost/a",
"http://localhost/a",
"http://localhost/b",
})
want := "http://localhost/a\nhttp://localhost/b"
if got != want {
t.Fatalf("got %q want %q", got, want)
}
}
func TestUniqueURIs(t *testing.T) {
uris := uniqueURIs("", "http://a", "http://a", "http://b")
if len(uris) != 2 {
t.Fatalf("expected 2 uris, got %d", len(uris))
}
}

54
internal/authentik/store.go Normal file
View File

@ -0,0 +1,54 @@
package authentik
import (
"context"
"errors"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
type ProvisionRecord struct {
AppKey string
AuthentikSlug string
ClientID string
ProviderID *int
ApplicationID *int
ProvisionedAt time.Time
}
func IsProvisioned(ctx context.Context, pool *pgxpool.Pool, appKey string) (bool, error) {
var exists bool
err := pool.QueryRow(ctx,
`SELECT EXISTS(SELECT 1 FROM suite_authentik_provisioned WHERE app_key = $1)`,
appKey,
).Scan(&exists)
return exists, err
}
func SaveProvisioned(ctx context.Context, pool *pgxpool.Pool, rec ProvisionRecord) error {
_, err := pool.Exec(ctx, `
INSERT INTO suite_authentik_provisioned (app_key, authentik_slug, client_id, provider_id, application_id, provisioned_at)
VALUES ($1, $2, $3, $4, $5, COALESCE($6, NOW()))
ON CONFLICT (app_key) DO UPDATE SET
authentik_slug = EXCLUDED.authentik_slug,
client_id = EXCLUDED.client_id,
provider_id = EXCLUDED.provider_id,
application_id = EXCLUDED.application_id,
provisioned_at = EXCLUDED.provisioned_at
`, rec.AppKey, rec.AuthentikSlug, rec.ClientID, rec.ProviderID, rec.ApplicationID, rec.ProvisionedAt)
return err
}
func GetProvisioned(ctx context.Context, pool *pgxpool.Pool, appKey string) (ProvisionRecord, error) {
var rec ProvisionRecord
err := pool.QueryRow(ctx, `
SELECT app_key, authentik_slug, client_id, provider_id, application_id, provisioned_at
FROM suite_authentik_provisioned WHERE app_key = $1
`, appKey).Scan(&rec.AppKey, &rec.AuthentikSlug, &rec.ClientID, &rec.ProviderID, &rec.ApplicationID, &rec.ProvisionedAt)
if errors.Is(err, pgx.ErrNoRows) {
return rec, err
}
return rec, err
}

51
internal/config/config.go
View File

@ -38,12 +38,35 @@ type Config struct {
OIDCClientID string
OIDCClientSecret string
// Authentik API (suite app auto-provisioning)
AuthentikAPIURL string
AuthentikAPIToken string
AuthentikPublicHTTPS bool
// Suite OIDC clients (Authentik applications)
NCOIDCClientID string
NCOIDCClientSecret string
OnlyOfficeOIDCClientID string
OnlyOfficeOIDCClientSecret string
ImmichOIDCClientID string
ImmichOIDCClientSecret string
DriveOIDCClientID string
DriveOIDCClientSecret string
// Nextcloud
NextcloudEnabled bool
NextcloudURL string
NCAdminUser string
NCAdminPass string
// OnlyOffice
OnlyOfficeEnabled bool
OnlyOfficeURL string
OnlyOfficePublicURL string
OnlyOfficeAPIInternalURL string
OnlyOfficeJWTSecret string
UltidPublicURL string
// Jitsi
JitsiEnabled bool
JitsiDomain string
@ -127,11 +150,31 @@ func Load() (*Config, error) {
OIDCClientID: os.Getenv("ULTID_OIDC_CLIENT_ID"),
OIDCClientSecret: secrets.Env("ULTID_OIDC_CLIENT_SECRET"),
AuthentikAPIURL: envOrDefault("AUTHENTIK_API_URL", "http://authentik-server:9000"),
AuthentikAPIToken: secrets.Env("AUTHENTIK_API_TOKEN"),
AuthentikPublicHTTPS: authentikPublicHTTPS(),
NCOIDCClientID: envOrDefault("NC_OIDC_CLIENT_ID", "ulti-nextcloud"),
NCOIDCClientSecret: secrets.Env("NC_OIDC_CLIENT_SECRET"),
OnlyOfficeOIDCClientID: envOrDefault("ONLYOFFICE_OIDC_CLIENT_ID", "ulti-onlyoffice"),
OnlyOfficeOIDCClientSecret: secrets.Env("ONLYOFFICE_OIDC_CLIENT_SECRET"),
ImmichOIDCClientID: envOrDefault("IMMICH_OIDC_CLIENT_ID", "ulti-immich"),
ImmichOIDCClientSecret: secrets.Env("IMMICH_OIDC_CLIENT_SECRET"),
DriveOIDCClientID: os.Getenv("DRIVE_OIDC_CLIENT_ID"),
DriveOIDCClientSecret: secrets.Env("DRIVE_OIDC_CLIENT_SECRET"),
NextcloudEnabled: envBool("NEXTCLOUD_ENABLED", true),
NextcloudURL: envOrDefault("NEXTCLOUD_URL", "http://nextcloud:80"),
NCAdminUser: envOrDefault("NC_ADMIN_USER", "admin"),
NCAdminPass: envOrDefaultSecret("NC_ADMIN_PASSWORD", "changeme"),
OnlyOfficeEnabled: envBool("ONLYOFFICE_ENABLED", false),
OnlyOfficeURL: envOrDefault("ONLYOFFICE_URL", "http://onlyoffice"),
OnlyOfficePublicURL: envOrDefault("ONLYOFFICE_PUBLIC_URL", "http://localhost/office"),
OnlyOfficeAPIInternalURL: envOrDefault("ONLYOFFICE_API_INTERNAL_URL", "http://ultid:8080"),
OnlyOfficeJWTSecret: secrets.Env("ONLYOFFICE_JWT_SECRET"),
UltidPublicURL: envOrDefault("ULTID_PUBLIC_URL", "http://localhost"),
JitsiEnabled: envBool("JITSI_ENABLED", true),
JitsiDomain: envOrDefault("JITSI_DOMAIN", "meet.jitsi"),
JitsiAppID: envOrDefault("JITSI_APP_ID", "ulti"),
@ -305,3 +348,11 @@ func defaultHealthJitsiURL(publicURL string) string {
trimmed = strings.TrimSuffix(trimmed, "/meet")
return trimmed + "/about/health"
}
func authentikPublicHTTPS() bool {
if envBool("AUTHENTIK_PUBLIC_HTTPS", false) {
return true
}
host := strings.TrimSpace(os.Getenv("AUTHENTIK_HOST"))
return strings.HasPrefix(strings.ToLower(host), "https://")
}

56
internal/mail/imap/attachments.go
View File

@ -24,7 +24,8 @@ type AttachmentPart struct {
// ExtractAttachments parses raw RFC 822 message bytes and returns attachment parts.
// Body text/plain and text/html parts are skipped. Non-text parts are collected when
// Content-Disposition is attachment, or inline with a filename. Parts exceeding
// Content-Disposition is attachment, inline (with filename or Content-ID), or when a
// related inline part has Content-ID (typical cid: image references). Parts exceeding
// limits.MaxAttachmentBytes are skipped; collection stops at limits.MaxAttachmentsPerMessage.
func ExtractAttachments(raw []byte) ([]AttachmentPart, error) {
if len(raw) == 0 {
@ -100,13 +101,26 @@ func partToAttachment(part *multipart.Part, mediaType string, typeParams map[str
filename = typeParams["name"]
}
contentID := normalizeContentID(part.Header.Get("Content-ID"))
isInline := strings.EqualFold(disposition, "inline")
isAttachment := strings.EqualFold(disposition, "attachment")
if !isAttachment && !(isInline && filename != "") {
switch {
case isAttachment:
// always store
case isInline && (filename != "" || contentID != ""):
// inline image / resource referenced via cid:
case contentID != "":
// multipart/related parts often omit Content-Disposition
isInline = true
default:
return AttachmentPart{}, false
}
if filename == "" {
filename = inlineAttachmentFilename(contentID, mediaType)
}
data, err := io.ReadAll(part)
if err != nil {
return AttachmentPart{}, false
@ -120,7 +134,7 @@ func partToAttachment(part *multipart.Part, mediaType string, typeParams map[str
return AttachmentPart{
Filename: filename,
ContentType: mediaType,
ContentID: normalizeContentID(part.Header.Get("Content-ID")),
ContentID: contentID,
IsInline: isInline,
Data: data,
}, true
@ -130,6 +144,42 @@ func normalizeContentID(raw string) string {
return strings.Trim(raw, "<> \t")
}
func inlineAttachmentFilename(contentID, mediaType string) string {
base := "inline"
if contentID != "" {
base = strings.Map(func(r rune) rune {
switch r {
case '<', '>', '/', '\\', ':', '"', '\'', '?', '*':
return '_'
default:
return r
}
}, contentID)
}
ext := extensionFromMediaType(mediaType)
if ext != "" && !strings.HasSuffix(strings.ToLower(base), ext) {
return base + ext
}
return base
}
func extensionFromMediaType(mediaType string) string {
switch strings.ToLower(strings.TrimSpace(strings.Split(mediaType, ";")[0])) {
case "image/jpeg", "image/jpg":
return ".jpg"
case "image/png":
return ".png"
case "image/gif":
return ".gif"
case "image/webp":
return ".webp"
case "image/svg+xml":
return ".svg"
default:
return ""
}
}
func decodePartBody(transferEncoding string, data []byte) ([]byte, error) {
switch strings.ToLower(strings.TrimSpace(transferEncoding)) {
case "base64":

106
internal/mail/imap/attachments_test.go
View File

@ -89,6 +89,112 @@ func TestExtractAttachments_inlineWithCID(t *testing.T) {
}
}
func TestExtractAttachments_inlineCIDWithoutFilename(t *testing.T) {
pngData := []byte{0x89, 'P', 'N', 'G', '\r', '\n', 0x1a, '\n'}
raw := buildMultipartMessage(t, "related", []mimePart{
{
contentType: "text/html",
body: []byte(`<html><body><img src="cid:part1"></body></html>`),
},
{
contentType: "image/png",
disposition: "inline",
contentID: "<part1>",
body: pngData,
transferEnc: "base64",
},
})
attachments, err := ExtractAttachments(raw)
if err != nil {
t.Fatalf("ExtractAttachments() error = %v", err)
}
if len(attachments) != 1 {
t.Fatalf("len(attachments) = %d, want 1", len(attachments))
}
att := attachments[0]
if att.ContentID != "part1" {
t.Fatalf("ContentID = %q, want part1", att.ContentID)
}
if !att.IsInline {
t.Fatal("IsInline = false, want true")
}
if att.Filename == "" {
t.Fatal("Filename empty, want generated name")
}
if string(att.Data) != string(pngData) {
t.Fatalf("Data mismatch")
}
}
func TestExtractAttachments_nestedMixedRelatedInline(t *testing.T) {
pngData := []byte{0x89, 'P', 'N', 'G', '\r', '\n', 0x1a, '\n'}
pdfData := []byte("%PDF-1.4\n")
raw := []byte(
"From: ikea@example.com\r\n" +
"To: user@example.com\r\n" +
"Subject: ticket\r\n" +
"MIME-Version: 1.0\r\n" +
"Content-Type: multipart/mixed; boundary=\"mix\"\r\n" +
"\r\n" +
"--mix\r\n" +
"Content-Type: multipart/related; boundary=\"rel\"\r\n" +
"\r\n" +
"--rel\r\n" +
"Content-Type: multipart/alternative; boundary=\"alt\"\r\n" +
"\r\n" +
"--alt\r\n" +
"Content-Type: text/plain\r\n" +
"\r\n" +
"Merci pour votre achat\r\n" +
"--alt\r\n" +
"Content-Type: text/html\r\n" +
"\r\n" +
"<img src=\"cid:Color.png\" alt=\"Logo\">\r\n" +
"--alt--\r\n" +
"--rel\r\n" +
"Content-Type: image/png; name=\"Color.png\"\r\n" +
"Content-Transfer-Encoding: base64\r\n" +
"Content-ID: <Color.png>\r\n" +
"Content-Disposition: inline; filename=\"Color.png\"\r\n" +
"\r\n" +
base64.StdEncoding.EncodeToString(pngData) + "\r\n" +
"--rel--\r\n" +
"--mix\r\n" +
"Content-Type: application/pdf; name=\"ticket.pdf\"\r\n" +
"Content-Transfer-Encoding: base64\r\n" +
"Content-Disposition: attachment; filename=\"ticket.pdf\"\r\n" +
"\r\n" +
base64.StdEncoding.EncodeToString(pdfData) + "\r\n" +
"--mix--\r\n",
)
attachments, err := ExtractAttachments(raw)
if err != nil {
t.Fatalf("ExtractAttachments() error = %v", err)
}
if len(attachments) != 2 {
t.Fatalf("len(attachments) = %d, want 2 (inline png + pdf)", len(attachments))
}
var inlineFound, pdfFound bool
for _, att := range attachments {
switch {
case att.IsInline && att.ContentID == "Color.png":
inlineFound = true
case !att.IsInline && att.Filename == "ticket.pdf":
pdfFound = true
}
}
if !inlineFound {
t.Fatal("inline Color.png attachment missing")
}
if !pdfFound {
t.Fatal("pdf attachment missing")
}
}
func TestExtractAttachments_skipsBodyParts(t *testing.T) {
raw := buildMultipartMessage(t, "alternative", []mimePart{
{

46
internal/mail/imap/body_repair.go
View File

@ -13,6 +13,8 @@ const minBareBase64Len = 24
// RepairStoredBodies fixes bodies stored as raw MIME, quoted-printable, or base64.
func RepairStoredBodies(text, html string) (string, string) {
text = repairLegacyCharsetString(text)
html = repairLegacyCharsetString(html)
text, html = repairRawMIME(text, html)
text = decodeBareQuotedPrintableIfNeeded(text)
html = decodeBareQuotedPrintableIfNeeded(html)
@ -42,9 +44,12 @@ func repairRawMIME(text, html string) (string, string) {
// RepairSnippet fixes list/search previews stored as undecoded base64 or raw MIME.
func RepairSnippet(snippet string) string {
if snippet == "" {
return snippet
}
return RepairSnippetWithBodies(snippet, "", "")
}
// RepairSnippetWithBodies decodes a stored snippet and optionally rebuilds from bodies.
func RepairSnippetWithBodies(snippet, bodyText, bodyHTML string) string {
snippet = stripSnippetMarkup(snippet)
if decoded := decodeBareQuotedPrintableIfNeeded(snippet); decoded != snippet {
snippet = decoded
}
@ -58,20 +63,27 @@ func RepairSnippet(snippet string) string {
return SnippetFromBodies(t, h, 200)
}
}
return snippet
}
// SnippetFromBodies builds a short preview from repaired plain/html bodies.
func SnippetFromBodies(text, html string, maxLen int) string {
text = strings.TrimSpace(text)
if text != "" {
return truncate(text, maxLen)
bodyText, bodyHTML = RepairStoredBodies(bodyText, bodyHTML)
if bodyText != "" || bodyHTML != "" {
if rebuilt := SnippetFromBodies(bodyText, bodyHTML, 200); rebuilt != "" {
if SnippetLooksLowQuality(snippet) || snippet == "" {
return rebuilt
}
if SnippetLooksLowQuality(rebuilt) {
return snippet
}
if snippetLineScore(rebuilt) > snippetLineScore(snippet) {
return rebuilt
}
}
}
html = strings.TrimSpace(stripHTMLForSnippet(html))
if html != "" {
return truncate(html, maxLen)
if snippet == "" {
return ""
}
return ""
if SnippetLooksLowQuality(snippet) {
return ""
}
return stripSnippetMarkup(snippet)
}
func stripPlainTextPreheaderPadding(text string) string {
@ -112,7 +124,7 @@ func decodeBareQuotedPrintableIfNeeded(s string) string {
if err != nil || len(decoded) == 0 || !isMostlyReadableText(decoded) {
return s
}
return string(decoded)
return decodeBodyBytesToUTF8(decoded, "")
}
func looksLikeQuotedPrintable(s string) bool {
@ -141,7 +153,7 @@ func decodeBareBase64IfNeeded(s string) string {
if err != nil || len(decoded) == 0 || !isMostlyReadableText(decoded) {
return s
}
return string(decoded)
return decodeBodyBytesToUTF8(decoded, "")
}
func stripBase64Whitespace(s string) string {

86
internal/mail/imap/charset.go Normal file
View File

@ -0,0 +1,86 @@
package imap
import (
"mime"
"strings"
"unicode/utf8"
"golang.org/x/text/encoding"
"golang.org/x/text/encoding/charmap"
"golang.org/x/text/encoding/htmlindex"
"golang.org/x/text/transform"
)
func charsetFromContentType(contentType string) string {
if contentType == "" {
return ""
}
_, params, err := mime.ParseMediaType(contentType)
if err != nil {
return ""
}
return strings.Trim(strings.TrimSpace(params["charset"]), `"`)
}
func isUTF8Charset(charset string) bool {
switch strings.ToLower(strings.TrimSpace(charset)) {
case "", "utf-8", "utf8", "unicode-1-1-utf-8":
return true
default:
return false
}
}
// decodeBodyBytesToUTF8 converts a MIME part payload to UTF-8 using Content-Type charset.
func decodeBodyBytesToUTF8(data []byte, contentType string) string {
if len(data) == 0 {
return ""
}
charset := charsetFromContentType(contentType)
if charset != "" && !isUTF8Charset(charset) {
if decoded := decodeBytesWithCharset(data, charset); decoded != "" {
return decoded
}
}
if utf8.Valid(data) {
return string(data)
}
return repairRawBytesToUTF8(data)
}
func decodeBytesWithCharset(data []byte, charset string) string {
enc, err := htmlindex.Get(charset)
if err != nil || enc == nil {
return ""
}
decoded, err := enc.NewDecoder().Bytes(data)
if err != nil || !utf8.Valid(decoded) {
return ""
}
return string(decoded)
}
// repairRawBytesToUTF8 fixes bodies stored without charset conversion (Latin-1 / Windows-1252).
func repairRawBytesToUTF8(data []byte) string {
if len(data) == 0 {
return ""
}
if utf8.Valid(data) {
return string(data)
}
for _, enc := range []encoding.Encoding{charmap.Windows1252, charmap.ISO8859_1} {
decoded, _, err := transform.Bytes(enc.NewDecoder(), data)
if err == nil && utf8.Valid(decoded) && isMostlyReadableText(decoded) {
return string(decoded)
}
}
return strings.ToValidUTF8(string(data), "")
}
// repairLegacyCharsetString fixes text already loaded as a Go string with invalid UTF-8 bytes.
func repairLegacyCharsetString(s string) string {
if s == "" || utf8.ValidString(s) {
return s
}
return repairRawBytesToUTF8([]byte(s))
}

46
internal/mail/imap/charset_test.go Normal file
View File

@ -0,0 +1,46 @@
package imap
import (
"strings"
"testing"
)
func TestParseBody_iso88591Charset(t *testing.T) {
body := []byte("Vous avez un rendez-vous programm\xe9.\r\nLien de la r\xe9union.")
var b strings.Builder
b.WriteString("From: calendar@google.com\r\n")
b.WriteString("To: user@example.com\r\n")
b.WriteString("Subject: Invitation\r\n")
b.WriteString("Content-Type: text/plain; charset=iso-8859-1\r\n")
b.WriteString("Content-Transfer-Encoding: 8bit\r\n")
b.WriteString("\r\n")
b.Write(body)
text, html := parseBody([]byte(b.String()))
if html != "" {
t.Fatalf("html = %q, want empty", html)
}
if !strings.Contains(text, "programmé") {
t.Fatalf("text = %q, want iso-8859-1 accents", text)
}
if !strings.Contains(text, "réunion") {
t.Fatalf("text = %q, want réunion", text)
}
}
func TestRepairLegacyCharsetString_latin1BytesInString(t *testing.T) {
// Simulates DB row stored before charset decode (raw Latin-1 bytes in text column).
raw := string([]byte{0x56, 0x6f, 0x75, 0x73, 0x20, 0x72, 0xe9, 0x75, 0x6e, 0x69, 0x6f, 0x6e})
repaired := repairLegacyCharsetString(raw)
if repaired != "Vous réunion" {
t.Fatalf("repaired = %q", repaired)
}
}
func TestRepairStoredBodies_legacyLatin1(t *testing.T) {
raw := string([]byte("programm\xe9"))
text, _ := RepairStoredBodies(raw, "")
if text != "programmé" {
t.Fatalf("text = %q", text)
}
}

35
internal/mail/imap/parse.go
View File

@ -53,7 +53,7 @@ func parseBody(raw []byte) (text string, html string) {
if text != "" || html != "" {
return finalizeDecodedBody(text), finalizeDecodedBody(html)
}
fallback := string(raw)
fallback := repairRawBytesToUTF8(raw)
return finalizeDecodedBody(fallback), ""
}
@ -75,19 +75,19 @@ func parseBodyFromRFC822(raw []byte) (text string, html string) {
mediaType, params, err := mime.ParseMediaType(contentType)
if err != nil {
body, _ := readDecodedBody(msg.Body, msg.Header.Get("Content-Transfer-Encoding"))
return string(body), ""
body, _ := readDecodedBody(msg.Body, contentType, msg.Header.Get("Content-Transfer-Encoding"))
return body, ""
}
if strings.HasPrefix(mediaType, "multipart/") {
return parseMultipart(msg.Body, params["boundary"])
}
body, _ := readDecodedBody(msg.Body, msg.Header.Get("Content-Transfer-Encoding"))
body, _ := readDecodedBody(msg.Body, contentType, msg.Header.Get("Content-Transfer-Encoding"))
if mediaType == "text/html" {
return "", string(body)
return "", body
}
outText := string(body)
outText := body
if looksLikeEmbeddedMIME(raw) {
if t, h, ok := parseEmbeddedMIME(raw); ok {
return t, h
@ -112,14 +112,14 @@ func parseMultipart(r io.Reader, boundary string) (text string, html string) {
switch {
case mediaType == "text/plain":
body, _ := readDecodedBody(part, part.Header.Get("Content-Transfer-Encoding"))
body, _ := readDecodedBody(part, partType, part.Header.Get("Content-Transfer-Encoding"))
if text == "" {
text = string(body)
text = body
}
case mediaType == "text/html":
body, _ := readDecodedBody(part, part.Header.Get("Content-Transfer-Encoding"))
if len(body) > 0 {
html = string(body)
case mediaType == "text/html", mediaType == "text/x-html", mediaType == "application/xhtml+xml":
body, _ := readDecodedBody(part, partType, part.Header.Get("Content-Transfer-Encoding"))
if len(body) > len(html) {
html = body
}
case strings.HasPrefix(mediaType, "multipart/"):
t, h := parseMultipart(part, params["boundary"])
@ -134,12 +134,16 @@ func parseMultipart(r io.Reader, boundary string) (text string, html string) {
return text, html
}
func readDecodedBody(r io.Reader, transferEncoding string) ([]byte, error) {
func readDecodedBody(r io.Reader, contentType, transferEncoding string) (string, error) {
data, err := io.ReadAll(r)
if err != nil {
return nil, err
return "", err
}
return decodePartBody(transferEncoding, data)
data, err = decodePartBody(transferEncoding, data)
if err != nil {
return "", err
}
return decodeBodyBytesToUTF8(data, contentType), nil
}
func parseEmbeddedMIME(raw []byte) (text string, html string, ok bool) {
@ -247,5 +251,6 @@ func parseThreadHeaders(raw []byte) (references []string, inReplyTo string) {
}
func toValidUTF8(s string) string {
s = repairLegacyCharsetString(s)
return strings.ToValidUTF8(s, "")
}

288
internal/mail/imap/snippet.go Normal file
View File

@ -0,0 +1,288 @@
package imap
import (
stdhtml "html"
"regexp"
"strings"
"unicode"
"golang.org/x/net/html"
"github.com/ultisuite/ulti-backend/internal/mail/sanitize"
)
var snippetHTMLTagRE = regexp.MustCompile(`(?is)<[^>]*>`)
var snippetSkipTags = map[string]bool{
"script": true, "style": true, "head": true, "noscript": true,
"meta": true, "link": true, "title": true, "svg": true,
}
var snippetBlockTags = map[string]bool{
"p": true, "li": true, "td": true, "th": true,
"h1": true, "h2": true, "h3": true, "h4": true, "h5": true, "h6": true,
"div": true, "span": true, "a": true,
}
// SnippetFromBodies builds a short list-preview from plain and HTML bodies.
func SnippetFromBodies(text, html string, maxLen int) string {
candidates := snippetCandidates(text, html)
best := pickBestSnippetLine(candidates)
if best == "" {
return ""
}
return truncate(stripSnippetMarkup(best), maxLen)
}
// stripSnippetMarkup removes HTML tags and entities from preview text.
func stripSnippetMarkup(s string) string {
s = strings.TrimSpace(s)
if s == "" {
return ""
}
if snippetHTMLTagRE.MatchString(s) {
s = snippetHTMLTagRE.ReplaceAllString(s, " ")
}
s = stdhtml.UnescapeString(s)
return strings.Join(strings.Fields(s), " ")
}
func snippetCandidates(text, html string) []string {
var out []string
text = strings.TrimSpace(stripPlainTextPreheaderPadding(text))
if text != "" {
out = append(out, splitSnippetSegments(text)...)
}
out = append(out, htmlSnippetCandidates(html)...)
return out
}
func splitSnippetSegments(s string) []string {
s = strings.ReplaceAll(s, "\r\n", "\n")
raw := strings.FieldsFunc(s, func(r rune) bool {
return r == '\n'
})
var segments []string
for _, line := range raw {
line = stripSnippetMarkup(line)
if line == "" {
continue
}
segments = append(segments, line)
}
return segments
}
func htmlSnippetCandidates(raw string) []string {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil
}
raw = sanitize.StripHiddenEmailHTML(raw)
doc, err := html.Parse(strings.NewReader(raw))
if err != nil {
if flat := strings.TrimSpace(stripHTMLForSnippet(raw)); flat != "" {
return splitSnippetSegments(flat)
}
return nil
}
seen := make(map[string]struct{})
var candidates []string
add := func(s string) {
s = stripSnippetMarkup(s)
if s == "" {
return
}
if _, ok := seen[s]; ok {
return
}
seen[s] = struct{}{}
candidates = append(candidates, s)
}
var walk func(*html.Node)
walk = func(n *html.Node) {
if n.Type == html.ElementNode {
tag := strings.ToLower(n.Data)
if snippetSkipTags[tag] {
return
}
if snippetBlockTags[tag] {
add(textFromHTMLSubtree(n))
return
}
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
walk(c)
}
}
walk(doc)
if len(candidates) == 0 {
add(textFromHTMLSubtree(doc))
}
return candidates
}
func textFromHTMLSubtree(n *html.Node) string {
var buf strings.Builder
var walk func(*html.Node)
walk = func(node *html.Node) {
if node.Type == html.ElementNode && snippetSkipTags[strings.ToLower(node.Data)] {
return
}
if node.Type == html.TextNode {
t := strings.TrimSpace(node.Data)
if t != "" {
if buf.Len() > 0 {
buf.WriteRune(' ')
}
buf.WriteString(t)
}
}
for c := node.FirstChild; c != nil; c = c.NextSibling {
walk(c)
}
}
walk(n)
return strings.TrimSpace(buf.String())
}
func pickBestSnippetLine(candidates []string) string {
var best string
bestScore := -1
for _, c := range candidates {
c = stripSnippetMarkup(c)
if c == "" || isSnippetBoilerplate(c) {
continue
}
score := snippetLineScore(c)
if score > bestScore {
bestScore = score
best = c
}
}
return best
}
func snippetLineScore(s string) int {
letters := 0
for _, r := range s {
if unicode.IsLetter(r) {
letters++
}
}
if letters < 8 {
return 0
}
score := letters * 4
if len(s) > 40 && len(s) < 280 {
score += 40
}
if len(s) >= 280 {
score += 10
}
return score
}
func isSnippetBoilerplate(s string) bool {
s = strings.TrimSpace(s)
if s == "" || len(s) < 4 {
return true
}
lower := strings.ToLower(s)
if looksLikeCSSSnippet(s) {
return true
}
if isMostlySeparatorLine(s) {
return true
}
boilerplate := []string{
"afficher dans le navigateur",
"view in browser",
"view this email in your browser",
"voir ce message en ligne",
"si vous ne visualisez pas",
"si vous n'arrivez pas à lire",
"si vous n'arrivez pas a lire",
"problems viewing this email",
"having trouble viewing",
"cliquer ici",
"click here",
"unsubscribe",
"se désabonner",
"se desabonner",
"manage your preferences",
"gérer vos préférences",
}
for _, phrase := range boilerplate {
if strings.Contains(lower, phrase) && len(s) < 160 {
return true
}
}
if strings.HasPrefix(lower, "http://") || strings.HasPrefix(lower, "https://") {
return len(s) < 100
}
if strings.HasPrefix(s, "[") && strings.HasSuffix(s, "]") && len(s) < 80 {
return true
}
letterRatio := snippetLetterRatio(s)
return letterRatio < 0.35
}
func looksLikeCSSSnippet(s string) bool {
lower := strings.ToLower(s)
if strings.Contains(lower, ":root") ||
strings.Contains(lower, "color-scheme:") ||
strings.Contains(lower, "@media") ||
strings.Contains(lower, "<!--") {
return true
}
if strings.HasPrefix(strings.TrimSpace(s), "/*") {
return true
}
semis := strings.Count(s, ";")
braces := strings.Count(s, "{") + strings.Count(s, "}")
if braces >= 2 && semis >= 2 {
return true
}
if strings.Contains(s, "{") && strings.Contains(s, "}") &&
(strings.Contains(lower, "font-") || strings.Contains(lower, "margin:") || strings.Contains(lower, "padding:")) {
return true
}
return false
}
func isMostlySeparatorLine(s string) bool {
if len(s) < 8 {
return false
}
sep := 0
for _, r := range s {
switch r {
case '-', '_', '*', '=', '·', '—':
sep++
}
}
return float64(sep)/float64(len(s)) >= 0.6
}
func snippetLetterRatio(s string) float64 {
if len(s) == 0 {
return 0
}
letters := 0
for _, r := range s {
if unicode.IsLetter(r) || unicode.IsNumber(r) {
letters++
}
}
return float64(letters) / float64(len([]rune(s)))
}
// SnippetLooksLowQuality reports whether a stored snippet should be recomputed from bodies.
func SnippetLooksLowQuality(snippet string) bool {
snippet = strings.TrimSpace(snippet)
if snippet == "" {
return true
}
return isSnippetBoilerplate(snippet) || looksLikeCSSSnippet(snippet) || snippetHTMLTagRE.MatchString(snippet)
}

78
internal/mail/imap/snippet_test.go Normal file
View File

@ -0,0 +1,78 @@
package imap
import (
"strings"
"testing"
)
func TestSnippetFromBodies_skipsStyleBlock(t *testing.T) {
html := `<html><head><style>:root { color-scheme: light dark; }</style></head>
<body><p>Meta for Business vous informe des nouveautés publicitaires.</p></body></html>`
got := SnippetFromBodies("", html, 200)
if strings.Contains(got, ":root") || strings.Contains(got, "color-scheme") {
t.Fatalf("snippet = %q, want body text not CSS", got)
}
if !strings.Contains(got, "nouveautés") {
t.Fatalf("snippet = %q, want meaningful body", got)
}
}
func TestSnippetFromBodies_skipsViewInBrowser(t *testing.T) {
html := `<html><body>
<p><a href="#">Afficher dans le navigateur</a></p>
<p>Webinar J-14 : boostez la recherche de vos données matériaux.</p>
</body></html>`
got := SnippetFromBodies("", html, 200)
lower := strings.ToLower(got)
if strings.Contains(lower, "afficher dans le navigateur") {
t.Fatalf("snippet = %q, want to skip boilerplate", got)
}
if !strings.Contains(got, "Webinar") {
t.Fatalf("snippet = %q, want real content", got)
}
}
func TestSnippetFromBodies_skipsSeparatorLine(t *testing.T) {
text := "----------------------------------------------------------------\nUn festival rétro au Château de Tilloloy arrive cet été."
got := SnippetFromBodies(text, "", 200)
if strings.HasPrefix(got, "---") {
t.Fatalf("snippet = %q, want content after separator", got)
}
if !strings.Contains(got, "festival") {
t.Fatalf("snippet = %q", got)
}
}
func TestSnippetFromBodies_stripsHTMLTags(t *testing.T) {
text := "<b>Bonjour</b> Eliott, votre <strong>commande</strong> est prête."
got := SnippetFromBodies(text, "", 200)
if strings.Contains(got, "<") || strings.Contains(got, ">") {
t.Fatalf("snippet = %q, want plain text without tags", got)
}
if !strings.Contains(got, "Bonjour") || !strings.Contains(got, "commande") {
t.Fatalf("snippet = %q, want readable text", got)
}
}
func TestRepairSnippetWithBodies_stripsStoredHTMLTags(t *testing.T) {
stored := "<span style=\"color:red\">Offre</span> limitée &amp; exclusive"
got := RepairSnippetWithBodies(stored, "", "")
if strings.Contains(got, "<") {
t.Fatalf("snippet = %q, want tags stripped", got)
}
if !strings.Contains(got, "Offre") || !strings.Contains(got, "&") {
t.Fatalf("snippet = %q, want unescaped text", got)
}
}
func TestRepairSnippetWithBodies_replacesCSSPreview(t *testing.T) {
stored := "FacebookMeta for Business :root { color-scheme: light dark;"
html := `<html><body><p>Inclure automatiquement des informations plus détaillées sur le compte.</p></body></html>`
got := RepairSnippetWithBodies(stored, "", html)
if strings.Contains(got, ":root") {
t.Fatalf("snippet = %q", got)
}
if !strings.Contains(got, "automatiquement") {
t.Fatalf("snippet = %q, want rebuilt from html", got)
}
}

313
internal/mail/imap/sync.go
View File

@ -19,6 +19,7 @@ import (
mailoauth "github.com/ultisuite/ulti-backend/internal/mail/oauth"
"github.com/ultisuite/ulti-backend/internal/mail/limits"
"github.com/ultisuite/ulti-backend/internal/mail/rules"
"github.com/ultisuite/ulti-backend/internal/mail/sanitize"
"github.com/ultisuite/ulti-backend/internal/mail/storage"
"github.com/ultisuite/ulti-backend/internal/mail/threading"
"github.com/ultisuite/ulti-backend/internal/observability"
@ -126,6 +127,16 @@ func (w *SyncWorker) syncAllAccounts(ctx context.Context) error {
// SyncAccountForUser triggers an immediate IMAP sync for a single owned account.
func (w *SyncWorker) SyncAccountForUser(ctx context.Context, externalID, accountID string) error {
return w.syncAccountForUser(ctx, externalID, accountID, false)
}
// ForceSyncAccountForUser resets sync cursors then re-fetches every message
// from IMAP, re-applying sanitization. Existing rows are updated in place.
func (w *SyncWorker) ForceSyncAccountForUser(ctx context.Context, externalID, accountID string) error {
return w.syncAccountForUser(ctx, externalID, accountID, true)
}
func (w *SyncWorker) syncAccountForUser(ctx context.Context, externalID, accountID string, force bool) error {
var (
host string
port int
@ -144,6 +155,12 @@ func (w *SyncWorker) SyncAccountForUser(ctx context.Context, externalID, account
}
return err
}
if force {
if err := resetAccountSyncCursors(ctx, w.db, accountID); err != nil {
return fmt.Errorf("reset sync cursors: %w", err)
}
w.logger.Info("force sync: reset cursors", "account_id", accountID)
}
return w.syncAccount(ctx, accountID, host, port, useTLS, creds)
}
@ -428,10 +445,7 @@ func (w *SyncWorker) processMessage(ctx context.Context, msg *imapclient.FetchMe
toAddrs := addressesToJSON(envelope.To)
ccAddrs := addressesToJSON(envelope.Cc)
bodyText, bodyHTML := parseBody(bodyContent)
snippet := truncate(bodyText, 200)
if snippet == "" && bodyHTML != "" {
snippet = SnippetFromBodies(bodyText, bodyHTML, 200)
}
snippet := SnippetFromBodies(bodyText, bodyHTML, 200)
headerRefs, headerInReplyTo := parseThreadHeaders(bodyContent)
inReplyTo := headerInReplyTo
@ -449,7 +463,7 @@ func (w *SyncWorker) processMessage(ctx context.Context, msg *imapclient.FetchMe
subject := RepairSubject(envelope.Subject, bodyText, bodyHTML, bodyContent)
snippet = toValidUTF8(snippet)
bodyText = toValidUTF8(bodyText)
bodyHTML = toValidUTF8(bodyHTML)
bodyHTML = toValidUTF8(sanitize.SanitizeHTML(bodyHTML))
var existed bool
_ = w.db.QueryRow(ctx, `
@ -537,13 +551,6 @@ func (w *SyncWorker) storeAttachments(ctx context.Context, userID, messageID str
if w.storage == nil || len(raw) == 0 {
return nil
}
if messageExisted {
var attCount int
_ = w.db.QueryRow(ctx, `SELECT COUNT(*) FROM attachments WHERE message_id = $1`, messageID).Scan(&attCount)
if attCount > 0 {
return nil
}
}
parts, err := ExtractAttachments(raw)
if err != nil || len(parts) == 0 {
@ -559,6 +566,9 @@ func (w *SyncWorker) storeAttachments(ctx context.Context, userID, messageID str
if err := limits.ValidateAttachmentSize(int64(len(part.Data))); err != nil {
continue
}
if messageExisted && attachmentPartExists(ctx, w.db, messageID, part) {
continue
}
objectKey := storage.MessageObjectKey(userID, messageID, part.Filename)
if err := w.storage.Put(ctx, objectKey, bytes.NewReader(part.Data), int64(len(part.Data)), part.ContentType); err != nil {
return err
@ -577,6 +587,285 @@ func (w *SyncWorker) storeAttachments(ctx context.Context, userID, messageID str
return err
}
func attachmentPartExists(ctx context.Context, db *pgxpool.Pool, messageID string, part AttachmentPart) bool {
var count int
if part.ContentID != "" {
_ = db.QueryRow(ctx, `
SELECT COUNT(*) FROM attachments
WHERE message_id = $1 AND (content_id = $2 OR filename = $3)
`, messageID, part.ContentID, part.Filename).Scan(&count)
return count > 0
}
_ = db.QueryRow(ctx, `
SELECT COUNT(*) FROM attachments WHERE message_id = $1 AND filename = $2
`, messageID, part.Filename).Scan(&count)
return count > 0
}
// ReindexMessageAttachments fetches one message from IMAP and stores missing inline parts.
func (w *SyncWorker) ReindexMessageAttachments(ctx context.Context, externalID, messageID string) error {
if w.storage == nil {
return errors.New("object storage unavailable")
}
var accountID, userID, folderName string
var uid uint32
err := w.db.QueryRow(ctx, `
SELECT m.account_id, ma.user_id, mf.remote_name, m.uid
FROM messages m
JOIN mail_accounts ma ON m.account_id = ma.id
JOIN users u ON ma.user_id = u.id
JOIN mail_folders mf ON m.folder_id = mf.id
WHERE m.id = $1 AND u.external_id = $2 AND ma.is_active = true
`, messageID, externalID).Scan(&accountID, &userID, &folderName, &uid)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return fmt.Errorf("message not found")
}
return err
}
if uid == 0 {
return fmt.Errorf("message has no IMAP uid")
}
var host string
var port int
var useTLS bool
var creds []byte
err = w.db.QueryRow(ctx, `SELECT imap_host, imap_port, imap_tls, credentials FROM mail_accounts WHERE id = $1`, accountID).
Scan(&host, &port, &useTLS, &creds)
if err != nil {
return err
}
addr := fmt.Sprintf("%s:%d", host, port)
var client *imapclient.Client
if useTLS {
client, err = imapclient.DialTLS(addr, &imapclient.Options{})
} else {
client, err = imapclient.DialStartTLS(addr, &imapclient.Options{})
}
if err != nil {
return fmt.Errorf("dial: %w", err)
}
defer client.Close()
cred, err := w.resolveCredential(ctx, accountID, creds)
if err != nil {
return err
}
if err := connect.AuthenticateIMAP(client, cred); err != nil {
return fmt.Errorf("login: %w", err)
}
if _, err := client.Select(folderName, nil).Wait(); err != nil {
return fmt.Errorf("select: %w", err)
}
bodyContent, err := fetchIMAPMessageRawBody(client, uid)
if err != nil {
return err
}
return w.storeAttachments(ctx, userID, messageID, bodyContent, true)
}
type RefetchBodiesResult struct {
Scanned int `json:"scanned"`
Updated int `json:"updated"`
}
const refetchBodiesBatchSize = 100
// RefetchAccountBodiesForUser re-downloads raw MIME from IMAP and re-parses bodies
// (text/html + inline attachments). Fixes messages imported with outdated sanitization.
func (w *SyncWorker) RefetchAccountBodiesForUser(ctx context.Context, externalID, accountID string) (RefetchBodiesResult, error) {
var result RefetchBodiesResult
if w.storage == nil {
return result, errors.New("object storage unavailable")
}
var host string
var port int
var useTLS bool
var creds []byte
err := w.db.QueryRow(ctx, `
SELECT ma.imap_host, ma.imap_port, ma.imap_tls, ma.credentials
FROM mail_accounts ma
JOIN users u ON ma.user_id = u.id
WHERE ma.id = $1 AND u.external_id = $2 AND ma.is_active = true
`, accountID, externalID).Scan(&host, &port, &useTLS, &creds)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return result, fmt.Errorf("account not found")
}
return result, err
}
type messageRef struct {
id string
userID string
folderName string
uid uint32
}
var lastID string
for {
rows, err := w.db.Query(ctx, `
SELECT m.id, ma.user_id, mf.remote_name, m.uid
FROM messages m
JOIN mail_accounts ma ON m.account_id = ma.id
JOIN mail_folders mf ON m.folder_id = mf.id
WHERE m.account_id = $1
AND m.uid > 0
AND ($2 = '' OR m.id > $2::uuid)
ORDER BY m.id
LIMIT $3
`, accountID, lastID, refetchBodiesBatchSize)
if err != nil {
return result, err
}
batch := make([]messageRef, 0, refetchBodiesBatchSize)
for rows.Next() {
var ref messageRef
if err := rows.Scan(&ref.id, &ref.userID, &ref.folderName, &ref.uid); err != nil {
rows.Close()
return result, err
}
batch = append(batch, ref)
lastID = ref.id
}
if err := rows.Err(); err != nil {
rows.Close()
return result, err
}
rows.Close()
if len(batch) == 0 {
break
}
byFolder := make(map[string][]messageRef)
for _, ref := range batch {
byFolder[ref.folderName] = append(byFolder[ref.folderName], ref)
}
addr := fmt.Sprintf("%s:%d", host, port)
var client *imapclient.Client
if useTLS {
client, err = imapclient.DialTLS(addr, &imapclient.Options{})
} else {
client, err = imapclient.DialStartTLS(addr, &imapclient.Options{})
}
if err != nil {
return result, fmt.Errorf("dial: %w", err)
}
cred, err := w.resolveCredential(ctx, accountID, creds)
if err != nil {
client.Close()
return result, err
}
if err := connect.AuthenticateIMAP(client, cred); err != nil {
client.Close()
return result, fmt.Errorf("login: %w", err)
}
for folderName, refs := range byFolder {
if _, err := client.Select(folderName, nil).Wait(); err != nil {
w.logger.Warn("refetch bodies: select folder failed", "folder", folderName, "error", err)
result.Scanned += len(refs)
continue
}
for _, ref := range refs {
result.Scanned++
updated, err := w.refetchStoredMessageBody(ctx, client, ref.userID, ref.id, ref.uid)
if err != nil {
w.logger.Warn("refetch message body failed", "message_id", ref.id, "error", err)
continue
}
if updated {
result.Updated++
}
}
}
client.Close()
if len(batch) < refetchBodiesBatchSize {
break
}
}
return result, nil
}
func (w *SyncWorker) refetchStoredMessageBody(ctx context.Context, client *imapclient.Client, userID, messageID string, uid uint32) (bool, error) {
bodyContent, err := fetchIMAPMessageRawBody(client, uid)
if err != nil {
return false, err
}
if len(bodyContent) == 0 {
return false, nil
}
bodyText, bodyHTML := parseBody(bodyContent)
bodyText = toValidUTF8(bodyText)
bodyHTML = toValidUTF8(sanitize.SanitizeHTML(bodyHTML))
snippet := SnippetFromBodies(bodyText, bodyHTML, 200)
tag, err := w.db.Exec(ctx, `
UPDATE messages
SET body_text = $2,
body_html = $3,
snippet = CASE WHEN $4 <> '' THEN $4 ELSE snippet END,
updated_at = NOW()
WHERE id = $1
AND (body_text IS DISTINCT FROM $2 OR body_html IS DISTINCT FROM $3)
`, messageID, bodyText, bodyHTML, snippet)
if err != nil {
return false, err
}
if err := w.storeAttachments(ctx, userID, messageID, bodyContent, true); err != nil {
return false, err
}
return tag.RowsAffected() > 0, nil
}
func fetchIMAPMessageRawBody(client *imapclient.Client, uid uint32) ([]byte, error) {
seqSet := imap.UIDSet{}
seqSet.AddNum(imap.UID(uid))
fetchCmd := client.Fetch(seqSet, &imap.FetchOptions{
UID: true,
BodySection: []*imap.FetchItemBodySection{{}},
})
msg := fetchCmd.Next()
if msg == nil {
return nil, fmt.Errorf("message not found on server")
}
var bodyContent []byte
for {
item := msg.Next()
if item == nil {
break
}
if data, ok := item.(imapclient.FetchItemDataBodySection); ok && data.Literal != nil {
buf := make([]byte, 0, 4096)
b := make([]byte, 4096)
for {
n, readErr := data.Literal.Read(b)
buf = append(buf, b[:n]...)
if readErr != nil {
break
}
}
bodyContent = buf
}
}
return bodyContent, nil
}
func isEmptyFromJSON(fromAddr []byte) bool {
if len(fromAddr) == 0 || string(fromAddr) == "[]" || string(fromAddr) == "null" {
return true

12
internal/mail/imap/sync_state.go
View File

@ -43,3 +43,15 @@ func resetFolderMessages(ctx context.Context, db *pgxpool.Pool, folderID string)
_, err := db.Exec(ctx, `DELETE FROM messages WHERE folder_id = $1`, folderID)
return err
}
// resetAccountSyncCursors zeroes last_uid and highest_modseq for all folders
// of an account so the next sync re-fetches every message from the IMAP server.
// Existing messages are kept; ON CONFLICT DO UPDATE overwrites body_html.
func resetAccountSyncCursors(ctx context.Context, db *pgxpool.Pool, accountID string) error {
_, err := db.Exec(ctx, `
UPDATE mail_folders
SET last_uid = 0, highest_modseq = 0, updated_at = NOW()
WHERE account_id = $1
`, accountID)
return err
}

59
internal/mail/sanitize/debug_test.go Normal file
View File

@ -0,0 +1,59 @@
package sanitize
import (
"strings"
"testing"
)
func TestStripHiddenEmailHTML_preservesFontSize0WithChildElements(t *testing.T) {
in := `<div style="font-size:0;text-align:center">` +
`<div style="display:inline-block;font-size:14px;width:200px">Column 1</div>` +
`<div style="display:inline-block;font-size:14px;width:200px">Column 2</div></div>`
out := StripHiddenEmailHTML(in)
if !strings.Contains(out, "Column 1") || !strings.Contains(out, "Column 2") {
t.Fatalf("inline-block columns removed by font-size:0 stripping: %q", out)
}
}
func TestStripHiddenEmailHTML_preservesMultiColumnTable(t *testing.T) {
in := `<table width="600"><tr><td style="font-size:0">` +
`<table width="300" style="display:inline-block"><tr><td style="font-size:14px;padding:10px">Left</td></tr></table>` +
`<table width="300" style="display:inline-block"><tr><td style="font-size:14px;padding:10px">Right</td></tr></table>` +
`</td></tr></table>`
out := StripHiddenEmailHTML(in)
if !strings.Contains(out, "Left") || !strings.Contains(out, "Right") {
t.Fatalf("multi-column table content removed: %q", out)
}
}
func TestStripHiddenEmailHTML_stillStripsFontSize0Preheader(t *testing.T) {
in := `<span style="font-size:0">Hidden preview text</span><p>Visible content</p>`
out := StripHiddenEmailHTML(in)
if strings.Contains(out, "Hidden preview text") {
t.Fatalf("font-size:0 preheader text should be stripped: %q", out)
}
if !strings.Contains(out, "Visible content") {
t.Fatalf("visible content missing: %q", out)
}
}
func TestSanitizeHTML_preservesFontSize0InlineBlockLayout(t *testing.T) {
in := `<div style="font-size:0;text-align:center">` +
`<div style="display:inline-block;font-size:14px;width:200px">Column 1</div>` +
`<div style="display:inline-block;font-size:14px;width:200px">Column 2</div></div>`
out := SanitizeHTML(in)
if !strings.Contains(out, "Column 1") || !strings.Contains(out, "Column 2") {
t.Fatalf("full sanitize pipeline removed inline-block content: %q", out)
}
}
func TestSanitizeHTML_preservesTableFontSize0Wrapper(t *testing.T) {
in := `<table width="600"><tr><td style="font-size:0">` +
`<table width="300" style="display:inline-block"><tr><td style="font-size:14px">Left content</td></tr></table>` +
`<table width="300" style="display:inline-block"><tr><td style="font-size:14px">Right content</td></tr></table>` +
`</td></tr></table>`
out := SanitizeHTML(in)
if !strings.Contains(out, "Left content") || !strings.Contains(out, "Right content") {
t.Fatalf("table multi-column content lost after sanitize: %q", out)
}
}

58
internal/mail/sanitize/mjml_test.go Normal file
View File

@ -0,0 +1,58 @@
package sanitize
import (
"strings"
"testing"
)
func TestSanitizeHTML_MJMLNewsletter(t *testing.T) {
// Real MJML pattern from Pennylane newsletter
in := `<!doctype html><html><head><style type="text/css">
#outlook a { padding: 0; }
body { margin: 0; padding: 0; }
table, td { border-collapse: collapse; }
.mj-column-per-100 { width:100% !important; max-width:100%; }
</style></head><body style="word-spacing:normal;background-color:#f8f9fa;">
<div style="background-color:#f8f9fa;">
<div style="background:white;background-color:white;margin:0px auto;max-width:600px;">
<table border="0" cellpadding="0" cellspacing="0" role="presentation" align="center" style="background:white;width:100%;">
<tbody><tr><td style="direction:ltr;font-size:0px;padding:10px 0;text-align:center;">
<div class="mj-outlook-group-fix mj-column-per-100" style="font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;">
<table border="0" cellpadding="0" cellspacing="0" role="presentation" width="100%" style="vertical-align:top;">
<tbody><tr><td align="center" style="font-size:0px;padding:10px 25px;word-break:break-word;">
<div style="font-family:sans-serif;font-size:16px;font-weight:400;line-height:24px;text-align:center;color:#000000;">
<img src="https://assets.pennylane.com/logo.png" alt="Pennylane logo" style="max-height:34px;max-width:100%" />
</div></td></tr></tbody></table></div>
</td></tr></tbody></table></div>
<div style="background:white;margin:0px auto;max-width:600px;">
<table border="0" cellpadding="0" cellspacing="0" align="center" style="background:white;width:100%;">
<tbody><tr><td style="direction:ltr;font-size:0px;padding:10px 0;text-align:center;">
<div class="mj-column-per-100" style="font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;">
<table border="0" cellpadding="0" cellspacing="0" width="100%" style="vertical-align:top;">
<tbody><tr><td align="left" style="font-size:0px;padding:10px 25px;word-break:break-word;">
<div style="font-size:22px;font-weight:600;line-height:32px;color:#006666;">5 notifications non lues</div>
</td></tr><tr><td align="left" style="font-size:0px;padding:10px 25px;word-break:break-word;">
<div style="font-size:14px;line-height:24px;color:#000000;">Bonjour Eliott,</div>
</td></tr><tr><td align="left" style="font-size:0px;padding:10px 25px;word-break:break-word;">
<div style="font-size:14px;line-height:24px;color:#000000;">Vous avez 5 notifications non lues</div>
</td></tr></tbody></table></div>
</td></tr></tbody></table></div>
</div></body></html>`
out := SanitizeHTML(in)
checks := []string{
"Pennylane logo",
"5 notifications non lues",
"Bonjour Eliott",
"Vous avez 5 notifications",
}
for _, want := range checks {
if !strings.Contains(out, want) {
t.Errorf("MJML content %q missing from output.\nOutput: %s", want, out)
}
}
if strings.TrimSpace(out) == "" {
t.Fatal("sanitizer produced empty output for MJML newsletter")
}
}

38
internal/mail/sanitize/preheader.go
View File

@ -63,13 +63,36 @@ func shouldStripHiddenElement(n *html.Node) bool {
return false
}
styleCompact := strings.ReplaceAll(style, " ", "")
return strings.Contains(styleCompact, "display:none") ||
if strings.Contains(styleCompact, "display:none") ||
strings.Contains(styleCompact, "mso-hide:all") ||
strings.Contains(styleCompact, "max-height:0") ||
strings.Contains(styleCompact, "opacity:0") ||
strings.Contains(styleCompact, "font-size:0") ||
strings.Contains(styleCompact, "visibility:hidden") ||
strings.Contains(styleCompact, "overflow:hidden") && strings.Contains(styleCompact, "max-height:0")
strings.Contains(styleCompact, "overflow:hidden") && strings.Contains(styleCompact, "max-height:0") {
return true
}
if strings.Contains(styleCompact, "font-size:0") && !hasSignificantChildElements(n) {
return true
}
return false
}
// hasSignificantChildElements returns true when n contains child elements
// beyond trivial void elements (br, wbr, hr). Parents with font-size:0 that
// contain real child elements are layout wrappers (inline-block whitespace
// collapse), not hidden preheader text.
func hasSignificantChildElements(n *html.Node) bool {
for c := n.FirstChild; c != nil; c = c.NextSibling {
if c.Type != html.ElementNode {
continue
}
tag := strings.ToLower(c.Data)
if tag == "br" || tag == "wbr" || tag == "hr" {
continue
}
return true
}
return false
}
func attrVal(n *html.Node, key string) string {
@ -102,7 +125,8 @@ func isInvisiblePaddingRune(r rune) bool {
}
}
// StripInvisibleTextRuns removes repeated invisible Unicode padding from plain text previews.
// StripInvisibleTextRuns removes repeated invisible Unicode padding from plain text.
// Line breaks are preserved so reply quotes stay splittable in the UI.
func StripInvisibleTextRuns(s string) string {
if s == "" {
return s
@ -115,5 +139,9 @@ func StripInvisibleTextRuns(s string) string {
}
b.WriteRune(r)
}
return strings.Join(strings.Fields(b.String()), " ")
lines := strings.Split(b.String(), "\n")
for i, line := range lines {
lines[i] = strings.Join(strings.Fields(line), " ")
}
return strings.TrimSpace(strings.Join(lines, "\n"))
}

11
internal/mail/sanitize/preheader_test.go
View File

@ -39,3 +39,14 @@ func TestStripInvisibleTextRuns(t *testing.T) {
t.Fatalf("got %q", got)
}
}
func TestStripInvisibleTextRuns_preservesNewlines(t *testing.T) {
in := "Reply line one\nOn 22/04/2026 wrote:\n> quoted"
got := StripInvisibleTextRuns(in)
if !strings.Contains(got, "\n") {
t.Fatalf("newlines collapsed: %q", got)
}
if !strings.Contains(got, "> quoted") {
t.Fatalf("quote line missing: %q", got)
}
}

14
internal/nextcloud/client.go
View File

@ -52,6 +52,10 @@ func (c *Client) doRequest(ctx context.Context, method, path string, body io.Rea
}
func (c *Client) DoAsUser(ctx context.Context, method, path string, body io.Reader, userID string, headers map[string]string) (*http.Response, error) {
return c.doAsUser(ctx, method, path, body, userID, headers, false)
}
func (c *Client) doAsUser(ctx context.Context, method, path string, body io.Reader, userID string, headers map[string]string, retried bool) (*http.Response, error) {
token, err := c.userDAVToken(ctx, userID)
if err != nil {
return nil, err
@ -73,8 +77,13 @@ func (c *Client) DoAsUser(ctx context.Context, method, path string, body io.Read
return nil, err
}
if resp.StatusCode == http.StatusUnauthorized && c.credStore != nil {
_ = c.credStore.DeleteToken(ctx, userID)
resp.Body.Close()
_ = c.credStore.DeleteToken(ctx, userID)
if !retried {
if refreshErr := c.RefreshPrincipalCredentials(ctx, userID); refreshErr == nil {
return c.doAsUser(ctx, method, path, body, userID, headers, true)
}
}
return nil, ErrDAVCredentialsMissing
}
return resp, nil
@ -91,6 +100,3 @@ func (c *Client) userDAVToken(ctx context.Context, userID string) (string, error
return token, nil
}
func (c *Client) WebDAVPath(userID, path string) string {
return fmt.Sprintf("/remote.php/dav/files/%s/%s", userID, path)
}

171
internal/nextcloud/dav_path.go Normal file
View File

@ -0,0 +1,171 @@
package nextcloud
import (
"net/url"
"strings"
)
// WebDAVPath builds an encoded WebDAV URL for a logical client path (may contain spaces).
func (c *Client) WebDAVPath(userID, path string) string {
userSeg := url.PathEscape(strings.TrimSpace(userID))
logical := strings.Trim(path, "/")
var encoded string
if logical == "" {
encoded = ""
} else {
parts := strings.Split(logical, "/")
for i, p := range parts {
parts[i] = url.PathEscape(p)
}
encoded = strings.Join(parts, "/")
}
if encoded == "" {
return "/remote.php/dav/files/" + userSeg
}
return "/remote.php/dav/files/" + userSeg + "/" + encoded
}
func decodeDAVSegment(seg string) string {
if seg == "" {
return seg
}
dec, err := url.PathUnescape(seg)
if err != nil {
return seg
}
return dec
}
// decodeDAVPath turns a slash-separated (possibly encoded) path into a logical path.
func decodeDAVPath(p string) string {
p = strings.TrimSpace(p)
p = strings.Trim(p, "/")
if p == "" {
return "/"
}
parts := strings.Split(p, "/")
for i, seg := range parts {
parts[i] = decodeDAVSegment(seg)
}
return "/" + strings.Join(parts, "/")
}
// clientPathFromDAVHref maps a WebDAV href to a logical path (/Folder/My File.docx).
func clientPathFromDAVHref(href string) string {
href = strings.TrimSpace(href)
if href == "" {
return "/"
}
if marker := "/dav/files/"; strings.Contains(href, marker) {
idx := strings.Index(href, marker)
rest := strings.TrimSuffix(href[idx+len(marker):], "/")
slash := strings.Index(rest, "/")
if slash < 0 {
return "/"
}
return decodeDAVPath("/" + rest[slash+1:])
}
if i := strings.LastIndex(href, "/trash/"); i >= 0 {
return decodeDAVPath("/" + href[i+len("/trash/"):])
}
return decodeDAVPath(href)
}
func fileNameFromDAVProp(displayName, href string) string {
if dn := strings.TrimSpace(displayName); dn != "" {
return dn
}
href = strings.TrimSuffix(strings.TrimSpace(href), "/")
if href == "" {
return ""
}
if i := strings.LastIndex(href, "/"); i >= 0 {
href = href[i+1:]
}
return decodeDAVSegment(href)
}
// NormalizeClientPath decodes segments and ensures a leading slash.
func NormalizeClientPath(path string) string {
return decodeDAVPath(path)
}
// JoinClientPath joins a directory path with a file or folder name.
func JoinClientPath(dir, name string) string {
name = strings.TrimSpace(name)
dir = NormalizeClientPath(dir)
if name == "" {
return dir
}
if dir == "/" {
return "/" + name
}
return dir + "/" + name
}
// NormalizeClientFilePath maps OCS/WebDAV paths to logical paths under the user files root.
func NormalizeClientFilePath(userID, path string) string {
path = strings.TrimSpace(path)
if path == "" {
return "/"
}
path = NormalizeClientPath(path)
uid := strings.TrimSpace(userID)
if uid == "" {
return path
}
trimmed := strings.Trim(path, "/")
if trimmed == "" {
return "/"
}
parts := strings.Split(trimmed, "/")
if len(parts) >= 2 {
head := decodeDAVSegment(parts[0])
if head == uid && parts[1] == "files" {
return NormalizeClientPath("/" + strings.Join(parts[2:], "/"))
}
if parts[0] == "files" && decodeDAVSegment(parts[1]) == uid {
return NormalizeClientPath("/" + strings.Join(parts[2:], "/"))
}
}
return path
}
// EnsureClientFilePath joins name when path is a parent directory (Nextcloud recent API).
func EnsureClientFilePath(path, name string) string {
path = NormalizeClientPath(path)
name = strings.TrimSpace(name)
if name == "" {
return path
}
if path == "/" {
return "/" + name
}
if strings.HasSuffix(path, "/"+name) {
return path
}
base := path[strings.LastIndex(path, "/")+1:]
if base == name {
return path
}
return JoinClientPath(path, name)
}
// ResolvePropfindClientPath resolves a PROPFIND child href against the listed directory.
func ResolvePropfindClientPath(listDir, href, fileName string) string {
if strings.Contains(href, "/dav/files/") {
return clientPathFromDAVHref(href)
}
base := NormalizeClientPath(listDir)
rel := strings.TrimPrefix(clientPathFromDAVHref(href), "/")
if rel == "" {
rel = strings.TrimSpace(fileName)
}
if rel == "" {
return base
}
if base == "/" {
return "/" + rel
}
return base + "/" + rel
}

67
internal/nextcloud/dav_path_test.go Normal file
View File

@ -0,0 +1,67 @@
package nextcloud
import "testing"
func TestDecodeDAVPath(t *testing.T) {
got := decodeDAVPath("/Documents/My%20File.docx")
want := "/Documents/My File.docx"
if got != want {
t.Fatalf("decodeDAVPath() = %q, want %q", got, want)
}
}
func TestClientPathFromDAVHref(t *testing.T) {
href := "/remote.php/dav/files/user%40example.com/Documents/Hello%20World/"
got := clientPathFromDAVHref(href)
want := "/Documents/Hello World"
if got != want {
t.Fatalf("clientPathFromDAVHref() = %q, want %q", got, want)
}
}
func TestFileNameFromDAVPropPrefersDisplayName(t *testing.T) {
got := fileNameFromDAVProp("My File", "/remote.php/dav/files/u/x%20y")
if got != "My File" {
t.Fatalf("got %q", got)
}
}
func TestFileNameFromDAVPropDecodesHref(t *testing.T) {
got := fileNameFromDAVProp("", "/remote.php/dav/files/u/Report%20Q1.pdf")
if got != "Report Q1.pdf" {
t.Fatalf("got %q", got)
}
}
func TestWebDAVPathEncodesSpaces(t *testing.T) {
c := &Client{}
got := c.WebDAVPath("user@example.com", "/Documents/My File")
want := "/remote.php/dav/files/user@example.com/Documents/My%20File"
if got != want {
t.Fatalf("WebDAVPath() = %q, want %q", got, want)
}
}
func TestResolvePropfindClientPathRelativeHref(t *testing.T) {
got := ResolvePropfindClientPath("/Documents", "photo.jpg", "photo.jpg")
want := "/Documents/photo.jpg"
if got != want {
t.Fatalf("ResolvePropfindClientPath() = %q, want %q", got, want)
}
}
func TestNormalizeClientFilePathStripsOCSPrefix(t *testing.T) {
got := NormalizeClientFilePath("alice", "/alice/files/Photos/vacation.jpg")
want := "/Photos/vacation.jpg"
if got != want {
t.Fatalf("NormalizeClientFilePath() = %q, want %q", got, want)
}
}
func TestEnsureClientFilePathJoinsName(t *testing.T) {
got := EnsureClientFilePath("/Documents", "report.pdf")
want := "/Documents/report.pdf"
if got != want {
t.Fatalf("EnsureClientFilePath() = %q, want %q", got, want)
}
}

623
internal/nextcloud/drive.go
View File

@ -4,9 +4,12 @@ import (
"context"
"encoding/json"
"encoding/xml"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"sort"
"strconv"
"strings"
"time"
@ -20,16 +23,38 @@ type FileInfo struct {
MimeType string `json:"mime_type"`
LastModified string `json:"last_modified"`
ETag string `json:"etag"`
FileID int64 `json:"file_id,omitempty"`
IsFavorite bool `json:"is_favorite"`
IsShared bool `json:"is_shared"`
}
type ShareInfo struct {
ID string `json:"id"`
Path string `json:"path"`
ShareType int `json:"share_type"`
Permissions int `json:"permissions"`
URL string `json:"url"`
ExpiresAt string `json:"expires_at,omitempty"`
ID string `json:"id"`
Path string `json:"path"`
ShareType int `json:"share_type"`
Permissions int `json:"permissions"`
URL string `json:"url"`
InternalURL string `json:"internal_url,omitempty"`
AccessMode string `json:"access_mode,omitempty"`
ExpiresAt string `json:"expires_at,omitempty"`
ShareWith string `json:"share_with,omitempty"`
ShareWithDisplayName string `json:"share_with_displayname,omitempty"`
Label string `json:"label,omitempty"`
Token string `json:"token,omitempty"`
}
// CreateShareOptions holds optional OCS share creation parameters.
type CreateShareOptions struct {
ShareType int
Permissions int
ShareWith string
Password string
ExpireDate string
Note string
Label string
Attributes string
SendMail bool
AccessMode string
}
type HTTPStatusError struct {
@ -61,6 +86,8 @@ func (c *Client) ListFiles(ctx context.Context, userID, path string) ([]FileInfo
<oc:fileid/>
<oc:size/>
<oc:favorite/>
<oc:share-types/>
<d:displayname/>
</d:prop>
</d:propfind>`
@ -77,7 +104,7 @@ func (c *Client) ListFiles(ctx context.Context, userID, path string) ([]FileInfo
return nil, &HTTPStatusError{Operation: "propfind", StatusCode: resp.StatusCode}
}
return parsePropfindResponse(resp.Body, davPath)
return parsePropfindResponse(resp.Body, path)
}
func (c *Client) Upload(ctx context.Context, userID, path string, content io.Reader, contentType string) error {
@ -100,6 +127,7 @@ func (c *Client) Upload(ctx context.Context, userID, path string, content io.Rea
}
func (c *Client) Download(ctx context.Context, userID, path string) (io.ReadCloser, string, error) {
path = NormalizeClientFilePath(userID, path)
davPath := c.WebDAVPath(userID, path)
resp, err := c.DoAsUser(ctx, "GET", davPath, nil, userID, nil)
if err != nil {
@ -228,7 +256,7 @@ func (c *Client) AbortChunkUpload(ctx context.Context, userID, uploadID string)
}
func (c *Client) ListTrash(ctx context.Context, userID string) ([]FileInfo, error) {
basePath := fmt.Sprintf("/remote.php/dav/trashbin/%s/trash", userID)
basePath := fmt.Sprintf("/remote.php/dav/trashbin/%s/trash", url.PathEscape(strings.TrimSpace(userID)))
body := `<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
<d:prop>
@ -239,6 +267,7 @@ func (c *Client) ListTrash(ctx context.Context, userID string) ([]FileInfo, erro
<d:resourcetype/>
<oc:size/>
<oc:favorite/>
<d:displayname/>
</d:prop>
</d:propfind>`
@ -253,17 +282,27 @@ func (c *Client) ListTrash(ctx context.Context, userID string) ([]FileInfo, erro
if resp.StatusCode != 207 {
return nil, &HTTPStatusError{Operation: "list trash", StatusCode: resp.StatusCode}
}
return parsePropfindResponse(resp.Body, basePath)
return parsePropfindResponse(resp.Body, "/")
}
func (c *Client) ListRecent(ctx context.Context, userID string, limit int) ([]FileInfo, error) {
path := "/ocs/v2.php/apps/files/api/v1/recent"
if limit > 0 {
path = fmt.Sprintf("%s?limit=%d", path, limit)
files, err := c.listRecentOCS(ctx, userID, limit)
if err == nil {
return files, nil
}
resp, err := c.DoAsUser(ctx, "GET", path, nil, userID, map[string]string{
"Accept": "application/json",
})
var statusErr *HTTPStatusError
if errors.As(err, &statusErr) && (statusErr.StatusCode == http.StatusNotFound || statusErr.StatusCode == http.StatusMethodNotAllowed) {
return c.listRecentFromRoot(ctx, userID, limit)
}
return nil, err
}
func (c *Client) listRecentOCS(ctx context.Context, userID string, limit int) ([]FileInfo, error) {
path := "/ocs/v2.php/apps/files/api/v1/recent?format=json"
if limit > 0 {
path = fmt.Sprintf("%s&limit=%d", path, limit)
}
resp, err := c.DoAsUser(ctx, "GET", path, nil, userID, ocsJSONHeaders())
if err != nil {
return nil, err
}
@ -298,8 +337,12 @@ func (c *Client) ListRecent(ctx context.Context, userID string, limit int) ([]Fi
if ts := parseAnyInt64(item.MTime); ts > 0 {
lastModified = time.Unix(ts, 0).UTC().Format(time.RFC3339)
}
logicalPath := EnsureClientFilePath(
NormalizeClientFilePath(userID, item.Path),
item.Name,
)
files = append(files, FileInfo{
Path: item.Path,
Path: logicalPath,
Name: item.Name,
Type: fileType,
Size: item.Size,
@ -311,10 +354,320 @@ func (c *Client) ListRecent(ctx context.Context, userID string, limit int) ([]Fi
return files, nil
}
func (c *Client) CreateShare(ctx context.Context, userID, path string, shareType int, permissions int) (*ShareInfo, error) {
formData := fmt.Sprintf("path=%s&shareType=%d&permissions=%d", path, shareType, permissions)
// listRecentFromRoot approximates recents when the Files app recent API is unavailable.
func (c *Client) listRecentFromRoot(ctx context.Context, userID string, limit int) ([]FileInfo, error) {
if limit <= 0 {
limit = 50
}
all, err := c.ListFiles(ctx, userID, "/")
if err != nil {
return nil, err
}
files := make([]FileInfo, 0, len(all))
for _, f := range all {
if f.Type == "file" {
files = append(files, f)
}
}
sort.Slice(files, func(i, j int) bool {
return fileModifiedTime(files[i].LastModified).After(fileModifiedTime(files[j].LastModified))
})
if len(files) > limit {
files = files[:limit]
}
return files, nil
}
func (c *Client) ListSharedWithMe(ctx context.Context, userID string) ([]FileInfo, error) {
path := "/ocs/v2.php/apps/files_sharing/api/v1/shares?shared_with_me=true&format=json"
resp, err := c.DoAsUser(ctx, "GET", path, nil, userID, ocsJSONHeaders())
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, &HTTPStatusError{Operation: "list shared with me", StatusCode: resp.StatusCode}
}
var payload struct {
OCS struct {
Data []struct {
Path string `json:"path"`
Name string `json:"name"`
ItemType string `json:"item_type"`
MimeType string `json:"mimetype"`
ETag string `json:"etag"`
Size int64 `json:"storage"`
MTime any `json:"mtime"`
} `json:"data"`
} `json:"ocs"`
}
if err := json.NewDecoder(resp.Body).Decode(&payload); err != nil {
return nil, err
}
files := make([]FileInfo, 0, len(payload.OCS.Data))
for _, item := range payload.OCS.Data {
name := strings.TrimSpace(item.Name)
if name == "" {
name = pathBaseName(item.Path)
}
fileType := "file"
if strings.EqualFold(item.ItemType, "folder") ||
strings.EqualFold(item.ItemType, "dir") ||
strings.EqualFold(item.ItemType, "directory") ||
strings.HasPrefix(item.MimeType, "httpd/unix-directory") {
fileType = "directory"
}
lastModified := ""
if ts := parseAnyInt64(item.MTime); ts > 0 {
lastModified = time.Unix(ts, 0).UTC().Format(time.RFC3339)
}
logicalPath := EnsureClientFilePath(
NormalizeClientFilePath(userID, item.Path),
name,
)
files = append(files, FileInfo{
Path: logicalPath,
Name: name,
Type: fileType,
Size: item.Size,
MimeType: item.MimeType,
LastModified: lastModified,
ETag: strings.Trim(item.ETag, "\""),
IsShared: true,
})
}
return files, nil
}
func pathBaseName(raw string) string {
raw = strings.TrimSpace(raw)
raw = strings.Trim(raw, "/")
if raw == "" {
return ""
}
if idx := strings.LastIndex(raw, "/"); idx >= 0 {
return raw[idx+1:]
}
return raw
}
func (c *Client) ListShares(ctx context.Context, userID, filePath string) ([]ShareInfo, error) {
path := "/ocs/v2.php/apps/files_sharing/api/v1/shares?path=" + url.QueryEscape(filePath)
resp, err := c.DoAsUser(ctx, "GET", path, nil, userID, map[string]string{
"Accept": "application/json",
})
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, &HTTPStatusError{Operation: "list shares", StatusCode: resp.StatusCode}
}
var ocsResp struct {
OCS struct {
Data []struct {
ID int `json:"id"`
Path string `json:"path"`
ShareType int `json:"share_type"`
Permissions int `json:"permissions"`
URL string `json:"url"`
Expiration string `json:"expiration"`
ShareWith string `json:"share_with"`
ShareWithDisplayName string `json:"share_with_displayname"`
Label string `json:"label"`
Token string `json:"token"`
} `json:"data"`
} `json:"ocs"`
}
if err := json.NewDecoder(resp.Body).Decode(&ocsResp); err != nil {
return nil, err
}
out := make([]ShareInfo, 0, len(ocsResp.OCS.Data))
for _, item := range ocsResp.OCS.Data {
out = append(out, mapOCSShareItem(item.ID, item.Path, item.ShareType, item.Permissions, item.URL, item.Expiration, item.ShareWith, item.ShareWithDisplayName, item.Label, item.Token))
}
return out, nil
}
func (c *Client) UpdateShare(ctx context.Context, userID, shareID string, permissions int, expireDate, password string) (*ShareInfo, error) {
form := fmt.Sprintf("permissions=%d", permissions)
if expireDate != "" {
form += "&expireDate=" + expireDate
}
if password != "" {
form += "&password=" + password
}
apiPath := fmt.Sprintf("/ocs/v2.php/apps/files_sharing/api/v1/shares/%s", shareID)
resp, err := c.DoAsUser(ctx, "PUT", apiPath, strings.NewReader(form), userID, map[string]string{
"Content-Type": "application/x-www-form-urlencoded",
"Accept": "application/json",
})
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, &HTTPStatusError{Operation: "update share", StatusCode: resp.StatusCode}
}
return decodeShareResponse(resp.Body, "")
}
func (c *Client) DeleteShare(ctx context.Context, userID, shareID string) error {
apiPath := fmt.Sprintf("/ocs/v2.php/apps/files_sharing/api/v1/shares/%s", shareID)
resp, err := c.DoAsUser(ctx, "DELETE", apiPath, nil, userID, map[string]string{
"Accept": "application/json",
})
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusNoContent {
return &HTTPStatusError{Operation: "delete share", StatusCode: resp.StatusCode}
}
return nil
}
func (c *Client) RestoreFromTrash(ctx context.Context, userID, trashName string) error {
userSeg := url.PathEscape(strings.TrimSpace(userID))
logical := strings.Trim(strings.TrimPrefix(trashName, "/"), "/")
var nameSeg string
if logical != "" {
parts := strings.Split(logical, "/")
for i, p := range parts {
parts[i] = url.PathEscape(p)
}
nameSeg = strings.Join(parts, "/")
}
src := fmt.Sprintf("/remote.php/dav/trashbin/%s/trash/%s", userSeg, nameSeg)
destURL := c.baseURL + c.WebDAVPath(userID, "/"+strings.TrimPrefix(trashName, "/"))
resp, err := c.DoAsUser(ctx, "MOVE", src, nil, userID, map[string]string{
"Destination": destURL,
"Overwrite": "T",
})
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusCreated && resp.StatusCode != http.StatusNoContent {
return &HTTPStatusError{Operation: "restore trash", StatusCode: resp.StatusCode}
}
return nil
}
func (c *Client) SetFavorite(ctx context.Context, userID, filePath string, favorite bool) error {
davPath := c.WebDAVPath(userID, filePath)
val := "0"
if favorite {
val = "1"
}
body := fmt.Sprintf(`<?xml version="1.0"?>
<d:propertyupdate xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns">
<d:set>
<d:prop><oc:favorite>%s</oc:favorite></d:prop>
</d:set>
</d:propertyupdate>`, val)
resp, err := c.DoAsUser(ctx, "PROPPATCH", davPath, strings.NewReader(body), userID, map[string]string{
"Content-Type": "application/xml",
})
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusMultiStatus && resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusNoContent {
return &HTTPStatusError{Operation: "set favorite", StatusCode: resp.StatusCode}
}
return nil
}
func decodeShareResponse(body io.Reader, path string) (*ShareInfo, error) {
var ocsResp struct {
OCS struct {
Data struct {
ID int `json:"id"`
URL string `json:"url"`
Permissions int `json:"permissions"`
Expiration string `json:"expiration"`
ShareType int `json:"share_type"`
ShareWith string `json:"share_with"`
ShareWithDisplayName string `json:"share_with_displayname"`
Label string `json:"label"`
Token string `json:"token"`
} `json:"data"`
} `json:"ocs"`
}
if err := json.NewDecoder(body).Decode(&ocsResp); err != nil {
return nil, err
}
item := mapOCSShareItem(
ocsResp.OCS.Data.ID,
path,
ocsResp.OCS.Data.ShareType,
ocsResp.OCS.Data.Permissions,
ocsResp.OCS.Data.URL,
ocsResp.OCS.Data.Expiration,
ocsResp.OCS.Data.ShareWith,
ocsResp.OCS.Data.ShareWithDisplayName,
ocsResp.OCS.Data.Label,
ocsResp.OCS.Data.Token,
)
return &item, nil
}
func mapOCSShareItem(id int, path string, shareType, permissions int, shareURL, expiration, shareWith, shareWithDisplayName, label, token string) ShareInfo {
info := ShareInfo{
ID: fmt.Sprintf("%d", id),
Path: path,
ShareType: shareType,
Permissions: permissions,
URL: shareURL,
ExpiresAt: expiration,
ShareWith: shareWith,
ShareWithDisplayName: shareWithDisplayName,
Label: label,
Token: token,
}
if shareType == 3 {
if strings.EqualFold(strings.TrimSpace(label), "internal") {
info.AccessMode = "internal"
} else {
info.AccessMode = "public"
}
}
return info
}
func (c *Client) CreateShare(ctx context.Context, userID, path string, opts CreateShareOptions) (*ShareInfo, error) {
form := url.Values{}
form.Set("path", path)
form.Set("shareType", strconv.Itoa(opts.ShareType))
form.Set("permissions", strconv.Itoa(opts.Permissions))
if shareWith := strings.TrimSpace(opts.ShareWith); shareWith != "" {
form.Set("shareWith", shareWith)
}
if password := strings.TrimSpace(opts.Password); password != "" {
form.Set("password", password)
}
if expireDate := strings.TrimSpace(opts.ExpireDate); expireDate != "" {
form.Set("expireDate", expireDate)
}
if note := strings.TrimSpace(opts.Note); note != "" {
form.Set("note", note)
}
if label := strings.TrimSpace(opts.Label); label != "" {
form.Set("label", label)
}
if attributes := strings.TrimSpace(opts.Attributes); attributes != "" {
form.Set("attributes", attributes)
}
if opts.SendMail {
form.Set("sendMail", "true")
}
resp, err := c.DoAsUser(ctx, "POST", "/ocs/v2.php/apps/files_sharing/api/v1/shares",
strings.NewReader(formData), userID, map[string]string{
strings.NewReader(form.Encode()), userID, map[string]string{
"Content-Type": "application/x-www-form-urlencoded",
"Accept": "application/json",
})
@ -326,65 +679,202 @@ func (c *Client) CreateShare(ctx context.Context, userID, path string, shareType
return nil, &HTTPStatusError{Operation: "create share", StatusCode: resp.StatusCode}
}
var ocsResp struct {
OCS struct {
Data struct {
ID int `json:"id"`
URL string `json:"url"`
Permissions int `json:"permissions"`
Expiration string `json:"expiration"`
} `json:"data"`
} `json:"ocs"`
}
if err := json.NewDecoder(resp.Body).Decode(&ocsResp); err != nil {
share, err := decodeShareResponse(resp.Body, path)
if err != nil {
return nil, err
}
if opts.AccessMode != "" {
share.AccessMode = opts.AccessMode
} else if share.AccessMode == "" && opts.ShareType == 3 {
share.AccessMode = "public"
}
if opts.ShareType == 4 {
share.AccessMode = "email"
}
if opts.ShareType == 0 {
share.AccessMode = "user"
}
return share, nil
}
return &ShareInfo{
ID: fmt.Sprintf("%d", ocsResp.OCS.Data.ID),
Path: path,
ShareType: shareType,
Permissions: ocsResp.OCS.Data.Permissions,
URL: ocsResp.OCS.Data.URL,
ExpiresAt: ocsResp.OCS.Data.Expiration,
}, nil
func (c *Client) SendShareEmail(ctx context.Context, userID, shareID, password string) error {
form := url.Values{}
if password := strings.TrimSpace(password); password != "" {
form.Set("password", password)
}
apiPath := fmt.Sprintf("/ocs/v2.php/apps/files_sharing/api/v1/shares/%s/send-email", url.PathEscape(shareID))
resp, err := c.DoAsUser(ctx, "POST", apiPath, strings.NewReader(form.Encode()), userID, map[string]string{
"Content-Type": "application/x-www-form-urlencoded",
"Accept": "application/json",
})
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return &HTTPStatusError{Operation: "send share email", StatusCode: resp.StatusCode}
}
return nil
}
func (c *Client) GetQuota(ctx context.Context, userID string) (UserQuota, error) {
resp, err := c.DoAsUser(ctx, "GET", fmt.Sprintf("/ocs/v1.php/cloud/users/%s?format=json", userID), nil, userID, map[string]string{
"Accept": "application/json",
var lastErr error
if q, err := c.getQuotaOCSCurrentUser(ctx, userID); err == nil {
return q, nil
} else {
lastErr = err
}
if q, err := c.getQuotaOCSUserRecord(ctx, userID); err == nil {
return q, nil
} else {
lastErr = err
}
if q, err := c.getQuotaWebDAV(ctx, userID); err == nil {
return q, nil
} else {
lastErr = err
}
return UserQuota{}, lastErr
}
func (c *Client) getQuotaOCSCurrentUser(ctx context.Context, userID string) (UserQuota, error) {
resp, err := c.DoAsUser(ctx, "GET", "/ocs/v2.php/cloud/user?format=json", nil, userID, ocsJSONHeaders())
if err != nil {
return UserQuota{}, err
}
defer resp.Body.Close()
return decodeOCSQuotaResponse(resp, "get quota (cloud/user)")
}
func (c *Client) getQuotaOCSUserRecord(ctx context.Context, userID string) (UserQuota, error) {
path := fmt.Sprintf("/ocs/v1.php/cloud/users/%s?format=json", url.PathEscape(userID))
resp, err := c.DoAsUser(ctx, "GET", path, nil, userID, ocsJSONHeaders())
if err != nil {
return UserQuota{}, err
}
defer resp.Body.Close()
return decodeOCSQuotaResponse(resp, "get quota (cloud/users)")
}
func (c *Client) getQuotaWebDAV(ctx context.Context, userID string) (UserQuota, error) {
davPath := c.WebDAVPath(userID, "")
body := `<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:">
<d:prop>
<d:quota-available-bytes/>
<d:quota-used-bytes/>
</d:prop>
</d:propfind>`
resp, err := c.DoAsUser(ctx, "PROPFIND", davPath, strings.NewReader(body), userID, map[string]string{
"Depth": "0",
"Content-Type": "application/xml",
})
if err != nil {
return UserQuota{}, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return UserQuota{}, &HTTPStatusError{Operation: "get quota", StatusCode: resp.StatusCode}
if resp.StatusCode != http.StatusMultiStatus && resp.StatusCode != http.StatusOK {
return UserQuota{}, &HTTPStatusError{Operation: "get quota (webdav)", StatusCode: resp.StatusCode}
}
return parseQuotaPropfind(resp.Body)
}
func decodeOCSQuotaResponse(resp *http.Response, op string) (UserQuota, error) {
if resp.StatusCode != http.StatusOK {
return UserQuota{}, &HTTPStatusError{Operation: op, StatusCode: resp.StatusCode}
}
raw, err := io.ReadAll(resp.Body)
if err != nil {
return UserQuota{}, err
}
var payload struct {
OCS struct {
Meta struct {
StatusCode int `json:"statuscode"`
} `json:"meta"`
Data struct {
Quota struct {
Free any `json:"free"`
Used any `json:"used"`
Total any `json:"total"`
Relative int64 `json:"relative"`
Free any `json:"free"`
Used any `json:"used"`
Total any `json:"total"`
Relative any `json:"relative"`
} `json:"quota"`
} `json:"data"`
} `json:"ocs"`
}
if err := json.NewDecoder(resp.Body).Decode(&payload); err != nil {
if err := json.Unmarshal(raw, &payload); err != nil {
return UserQuota{}, err
}
if code := payload.OCS.Meta.StatusCode; code != 0 && code != 100 && code != 200 {
return UserQuota{}, fmt.Errorf("%s: ocs status %d", op, code)
}
q := payload.OCS.Data.Quota
used := parseAnyInt64(q.Used)
free := parseAnyInt64(q.Free)
total := parseAnyInt64(q.Total)
if total <= 0 && used >= 0 && free >= 0 {
total = used + free
}
return UserQuota{
Used: parseAnyInt64(payload.OCS.Data.Quota.Used),
Free: parseAnyInt64(payload.OCS.Data.Quota.Free),
Total: parseAnyInt64(payload.OCS.Data.Quota.Total),
Relative: payload.OCS.Data.Quota.Relative,
Used: used,
Free: free,
Total: total,
Relative: parseAnyInt64(q.Relative),
}, nil
}
func parseQuotaPropfind(body io.Reader) (UserQuota, error) {
var ms struct {
XMLName xml.Name `xml:"multistatus"`
Responses []struct {
Propstat struct {
Prop struct {
Available int64 `xml:"quota-available-bytes"`
Used int64 `xml:"quota-used-bytes"`
} `xml:"prop"`
} `xml:"propstat"`
} `xml:"response"`
}
if err := xml.NewDecoder(body).Decode(&ms); err != nil {
return UserQuota{}, err
}
if len(ms.Responses) == 0 {
return UserQuota{}, fmt.Errorf("quota propfind: empty response")
}
used := ms.Responses[0].Propstat.Prop.Used
free := ms.Responses[0].Propstat.Prop.Available
total := used + free
var relative int64
if total > 0 {
relative = (used * 100) / total
}
return UserQuota{Used: used, Free: free, Total: total, Relative: relative}, nil
}
func ocsJSONHeaders() map[string]string {
return map[string]string{
"Accept": "application/json",
"OCS-APIRequest": "true",
}
}
func fileModifiedTime(raw string) time.Time {
raw = strings.TrimSpace(raw)
if raw == "" {
return time.Time{}
}
if ts, err := time.Parse(time.RFC3339, raw); err == nil {
return ts
}
if ts, err := time.Parse(time.RFC1123, raw); err == nil {
return ts
}
if ts, err := http.ParseTime(raw); err == nil {
return ts
}
return time.Time{}
}
// PROPFIND XML response parsing
type multistatus struct {
XMLName xml.Name `xml:"multistatus"`
@ -409,6 +899,8 @@ type prop struct {
ResourceType resourceType `xml:"resourcetype"`
Size int64 `xml:"size"`
Favorite int `xml:"favorite"`
ShareTypes shareTypes `xml:"http://owncloud.org/ns share-types"`
FileID string `xml:"http://owncloud.org/ns fileid"`
DisplayName string `xml:"displayname"`
CalendarColor string `xml:"calendar-color"`
}
@ -417,7 +909,11 @@ type resourceType struct {
Collection *struct{} `xml:"collection"`
}
func parsePropfindResponse(body io.Reader, basePath string) ([]FileInfo, error) {
type shareTypes struct {
ShareType []string `xml:"http://owncloud.org/ns share-type"`
}
func parsePropfindResponse(body io.Reader, listDir string) ([]FileInfo, error) {
var ms multistatus
if err := xml.NewDecoder(body).Decode(&ms); err != nil {
return nil, err
@ -429,11 +925,8 @@ func parsePropfindResponse(body io.Reader, basePath string) ([]FileInfo, error)
continue // skip the folder itself
}
name := r.Href
if idx := strings.LastIndex(strings.TrimSuffix(name, "/"), "/"); idx >= 0 {
name = name[idx+1:]
}
name = strings.TrimSuffix(name, "/")
name := fileNameFromDAVProp(r.Propstat.Prop.DisplayName, r.Href)
clientPath := ResolvePropfindClientPath(listDir, r.Href, name)
fileType := "file"
if r.Propstat.Prop.ResourceType.Collection != nil {
@ -446,20 +939,34 @@ func parsePropfindResponse(body io.Reader, basePath string) ([]FileInfo, error)
}
files = append(files, FileInfo{
Path: r.Href,
Path: clientPath,
Name: name,
Type: fileType,
Size: size,
MimeType: r.Propstat.Prop.ContentType,
LastModified: r.Propstat.Prop.LastModified,
ETag: strings.Trim(r.Propstat.Prop.ETag, "\""),
FileID: parseFileID(r.Propstat.Prop.FileID),
IsFavorite: r.Propstat.Prop.Favorite == 1,
IsShared: len(r.Propstat.Prop.ShareTypes.ShareType) > 0,
})
}
return files, nil
}
func parseFileID(raw string) int64 {
raw = strings.TrimSpace(raw)
if raw == "" {
return 0
}
id, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return 0
}
return id
}
func parseInt64(raw string) int64 {
n, err := strconv.ParseInt(strings.TrimSpace(raw), 10, 64)
if err != nil {

69
internal/nextcloud/drive_quota_test.go Normal file
View File

@ -0,0 +1,69 @@
package nextcloud
import (
"io"
"net/http"
"strings"
"testing"
)
func TestFileModifiedTimeRFC3339(t *testing.T) {
raw := "2024-06-01T12:00:00Z"
got := fileModifiedTime(raw)
if got.IsZero() {
t.Fatalf("expected parsed time for %q", raw)
}
}
func TestFileModifiedTimeHTTPDate(t *testing.T) {
raw := "Wed, 01 Jun 2024 12:00:00 GMT"
got := fileModifiedTime(raw)
if got.IsZero() {
t.Fatalf("expected parsed time for %q", raw)
}
}
func TestDecodeOCSQuotaResponse(t *testing.T) {
body := strings.NewReader(`{
"ocs": {
"meta": { "statuscode": 100 },
"data": {
"quota": {
"free": 900,
"used": 100,
"total": 1000,
"relative": 10
}
}
}
}`)
resp := &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(body)}
q, err := decodeOCSQuotaResponse(resp, "test")
if err != nil {
t.Fatal(err)
}
if q.Total != 1000 || q.Used != 100 || q.Free != 900 {
t.Fatalf("unexpected quota: %+v", q)
}
}
func TestParseQuotaPropfind(t *testing.T) {
raw := `<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:">
<d:response>
<d:propstat>
<d:prop>
<d:quota-available-bytes>900</d:quota-available-bytes>
<d:quota-used-bytes>100</d:quota-used-bytes>
</d:prop>
</d:propstat>
</d:response>
</d:multistatus>`
q, err := parseQuotaPropfind(strings.NewReader(raw))
if err != nil {
t.Fatal(err)
}
if q.Total != 1000 || q.Used != 100 || q.Free != 900 {
t.Fatalf("unexpected quota: %+v", q)
}
}

96
internal/nextcloud/preview.go Normal file
View File

@ -0,0 +1,96 @@
package nextcloud
import (
"context"
"encoding/xml"
"fmt"
"io"
"net/http"
"strconv"
"strings"
)
func (c *Client) FileID(ctx context.Context, userID, filePath string) (int64, error) {
filePath = NormalizeClientFilePath(userID, filePath)
davPath := c.WebDAVPath(userID, filePath)
body := `<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns">
<d:prop>
<oc:fileid/>
</d:prop>
</d:propfind>`
resp, err := c.DoAsUser(ctx, "PROPFIND", davPath, strings.NewReader(body), userID, map[string]string{
"Depth": "0",
"Content-Type": "application/xml",
})
if err != nil {
return 0, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusMultiStatus && resp.StatusCode != http.StatusOK {
return 0, &HTTPStatusError{Operation: "propfind fileid", StatusCode: resp.StatusCode}
}
var ms struct {
Responses []struct {
Propstat struct {
Prop struct {
FileID string `xml:"http://owncloud.org/ns fileid"`
} `xml:"prop"`
} `xml:"propstat"`
} `xml:"response"`
}
if err := xml.NewDecoder(resp.Body).Decode(&ms); err != nil {
return 0, err
}
if len(ms.Responses) == 0 {
return 0, fmt.Errorf("fileid propfind: empty response")
}
raw := strings.TrimSpace(ms.Responses[0].Propstat.Prop.FileID)
if raw == "" {
return 0, fmt.Errorf("fileid propfind: missing fileid")
}
// Nextcloud may return "00001234" — keep numeric part.
id, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return 0, fmt.Errorf("fileid propfind: invalid fileid %q", raw)
}
return id, nil
}
func (c *Client) Preview(ctx context.Context, userID string, fileID int64, width, height int) (io.ReadCloser, string, error) {
if width <= 0 {
width = 400
}
if height <= 0 {
height = 300
}
if width > 2048 {
width = 2048
}
if height > 2048 {
height = 2048
}
path := fmt.Sprintf(
"/index.php/core/preview?fileId=%d&x=%d&y=%d&a=1&mode=cover&mimeFallback=true",
fileID, width, height,
)
resp, err := c.DoAsUser(ctx, "GET", path, nil, userID, nil)
if err != nil {
return nil, "", err
}
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
return nil, "", &HTTPStatusError{Operation: "preview", StatusCode: resp.StatusCode}
}
contentType := resp.Header.Get("Content-Type")
if contentType == "" {
contentType = "image/jpeg"
}
return resp.Body, contentType, nil
}

29
internal/nextcloud/users.go
View File

@ -77,6 +77,35 @@ func (c *Client) EnsurePrincipal(ctx context.Context, email, sub, displayName st
return userID, nil
}
// RefreshPrincipalCredentials rotates the Nextcloud login password and app password for an existing user.
func (c *Client) RefreshPrincipalCredentials(ctx context.Context, userID string) error {
if c.credStore == nil {
return fmt.Errorf("nextcloud dav credentials store not configured")
}
userID = strings.TrimSpace(userID)
if userID == "" {
return fmt.Errorf("nextcloud user id is empty")
}
loginPassword, err := generateNextcloudPassword()
if err != nil {
return err
}
if err := c.setUserPassword(ctx, userID, loginPassword); err != nil {
return err
}
appPassword, err := c.createAppPassword(ctx, userID, loginPassword)
if err != nil {
return err
}
return c.credStore.SaveToken(ctx, userID, appPassword)
}
// UserExists reports whether a Nextcloud account exists for the given user id (typically email).
func (c *Client) UserExists(ctx context.Context, userID string) (bool, error) {
return c.userExists(ctx, userID)
}
func (c *Client) userExists(ctx context.Context, userID string) (bool, error) {
path := fmt.Sprintf("/ocs/v1.php/cloud/users/%s?format=json", url.PathEscape(userID))
resp, err := c.doRequest(ctx, http.MethodGet, path, nil, map[string]string{

21
internal/realtime/events.go
View File

@ -7,6 +7,8 @@ const (
TypeMailDeleted = "mail.deleted"
TypeOutboxUpdated = "outbox.updated"
TypeContactUpdated = "contact.updated"
TypeDriveFileChanged = "drive.file_changed"
TypeDriveShareUpdated = "drive.share_updated"
TypeWSPing = "ws.ping"
TypeWSPong = "ws.pong"
@ -77,3 +79,22 @@ func NewContactUpdatedEvent(contactID, accountID string) Event {
},
}
}
// DriveEventPayload is the payload for drive.file_changed and drive.share_updated.
type DriveEventPayload struct {
Path string `json:"path"`
}
func NewDriveFileChangedEvent(path string) Event {
return Event{
Type: TypeDriveFileChanged,
Payload: DriveEventPayload{Path: path},
}
}
func NewDriveShareUpdatedEvent(path string) Event {
return Event{
Type: TypeDriveShareUpdated,
Payload: DriveEventPayload{Path: path},
}
}

1053
mailpennylane.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1
migrations/000021_suite_authentik_provisioned.down.sql Normal file
View File

@ -0,0 +1 @@
DROP TABLE IF EXISTS suite_authentik_provisioned;

12
migrations/000021_suite_authentik_provisioned.up.sql Normal file
View File

@ -0,0 +1,12 @@
-- Tracks Authentik OAuth applications provisioned by ultid (idempotent startup).
CREATE TABLE IF NOT EXISTS suite_authentik_provisioned (
app_key TEXT PRIMARY KEY,
authentik_slug TEXT NOT NULL,
client_id TEXT NOT NULL DEFAULT '',
provider_id INTEGER,
application_id INTEGER,
provisioned_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_suite_authentik_provisioned_slug
ON suite_authentik_provisioned (authentik_slug);