customer-installer/wiki/Architecture.md

# Architektur

Diese Seite beschreibt die technische Architektur des Customer Installer Systems.

## 📐 System-Übersicht

```
┌─────────────────────────────────────────────────────────────────┐
│                         Proxmox VE Host                         │
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐ │
│  │              LXC Container (Debian 12)                    │ │
│  │                                                           │ │
│  │  ┌─────────────────────────────────────────────────────┐ │ │
│  │  │           Docker Compose Stack                      │ │ │
│  │  │                                                     │ │ │
│  │  │  ┌──────────────┐  ┌──────────────┐  ┌─────────┐  │ │ │
│  │  │  │  PostgreSQL  │  │  PostgREST   │  │   n8n   │  │ │ │
│  │  │  │  + pgvector  │◄─┤  (REST API)  │◄─┤ Workflow│  │ │ │
│  │  │  │              │  │              │  │ Engine  │  │ │ │
│  │  │  └──────────────┘  └──────────────┘  └─────────┘  │ │ │
│  │  │         │                  │              │        │ │ │
│  │  │         └──────────────────┴──────────────┘        │ │ │
│  │  │                    Docker Network                  │ │ │
│  │  │                  (customer-net)                    │ │ │
│  │  └─────────────────────────────────────────────────────┘ │ │
│  │                                                           │ │
│  │  ┌─────────────────────────────────────────────────────┐ │ │
│  │  │              Systemd Services                       │ │ │
│  │  │  - docker.service                                   │ │ │
│  │  │  - n8n-workflow-reload.service                      │ │ │
│  │  └─────────────────────────────────────────────────────┘ │ │
│  └───────────────────────────────────────────────────────────┘ │
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐ │
│  │              NGINX Reverse Proxy (OPNsense)             │ │
│  │  https://sb-<timestamp>.userman.de → http://<ip>:5678  │ │
│  └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
                   ┌──────────────────┐
                   │  Ollama Server   │
                   │  (External Host) │
                   │  Port: 11434     │
                   └──────────────────┘
```

## 🏗️ Komponenten-Architektur

### 1. Proxmox LXC Container

**Technologie:** Linux Container (LXC)  
**OS:** Debian 12 (Bookworm)  
**Typ:** Unprivileged (Standard) oder Privileged (optional)

**Ressourcen:**
- CPU: Unlimited (konfigurierbar)
- RAM: 4096 MB (Standard)
- Swap: 512 MB
- Disk: 50 GB (Standard)
- Netzwerk: Bridge mit VLAN-Support

**Features:**
- Automatische CTID-Generierung (customer-safe)
- DHCP oder statische IP
- VLAN-Tagging
- APT-Proxy-Support

### 2. Docker Stack

**Technologie:** Docker Compose v2  
**Netzwerk:** Bridge Network (customer-net)  
**Volumes:** Named Volumes für Persistenz

#### 2.1 PostgreSQL Container

**Image:** `postgres:16-alpine`  
**Name:** `customer-postgres`  
**Port:** 5432 (intern)

**Features:**
- pgvector Extension (v0.5.1)
- Automatische Datenbank-Initialisierung
- Persistente Daten via Volume
- Health Checks

**Datenbank-Schema:**
```sql
-- documents Tabelle für RAG
CREATE TABLE documents (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  content TEXT NOT NULL,
  metadata JSONB,
  embedding vector(384),  -- nomic-embed-text Dimension
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Index für Vektor-Suche
CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);

-- RPC-Funktion für Similarity Search
CREATE FUNCTION match_documents(
  query_embedding vector(384),
  match_count int DEFAULT 5
) RETURNS TABLE (
  id UUID,
  content TEXT,
  metadata JSONB,
  similarity FLOAT
) AS $$
  SELECT
    id,
    content,
    metadata,
    1 - (embedding <=> query_embedding) AS similarity
  FROM documents
  ORDER BY embedding <=> query_embedding
  LIMIT match_count;
$$ LANGUAGE sql STABLE;
```

#### 2.2 PostgREST Container

**Image:** `postgrest/postgrest:v12.0.2`  
**Name:** `customer-postgrest`  
**Port:** 3000 (extern + intern)

**Features:**
- Supabase-kompatible REST API
- JWT-basierte Authentikation
- Automatische OpenAPI-Dokumentation
- RPC-Funktionen-Support

**Endpoints:**
- `GET /documents` - Dokumente abrufen
- `POST /documents` - Dokument erstellen
- `POST /rpc/match_documents` - Vektor-Suche

**Authentication:**
- `anon` Role: Lesezugriff
- `service_role`: Voller Zugriff

#### 2.3 n8n Container

**Image:** `n8nio/n8n:latest`  
**Name:** `n8n`  
**Port:** 5678 (extern + intern)

**Features:**
- PostgreSQL als Backend
- Workflow-Automation
- Webhook-Support
- Credentials-Management
- Execution-History

**Workflows:**
- RAG KI-Bot (Chat-Interface)
- Document Upload (Form)
- Vector Embedding (Ollama)
- Similarity Search (PostgreSQL)

**Environment:**
```bash
DB_TYPE=postgresdb
DB_POSTGRESDB_HOST=postgres
DB_POSTGRESDB_PORT=5432
DB_POSTGRESDB_DATABASE=customer
DB_POSTGRESDB_USER=customer
DB_POSTGRESDB_PASSWORD=<generated>
N8N_ENCRYPTION_KEY=<generated>
WEBHOOK_URL=https://sb-<timestamp>.userman.de
N8N_DIAGNOSTICS_ENABLED=false
N8N_PERSONALIZATION_ENABLED=false
```

### 3. Systemd Services

#### 3.1 docker.service

Standard Docker Service für Container-Management.

#### 3.2 n8n-workflow-reload.service

**Typ:** oneshot  
**Trigger:** Container-Start  
**Funktion:** Automatisches Workflow-Reload

```ini
[Unit]
Description=Reload n8n workflow on container start
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/opt/customer-stack/reload-workflow.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
```

### 4. Netzwerk-Architektur

#### 4.1 Docker Network

**Name:** `customer-stack_customer-net`  
**Typ:** Bridge  
**Subnet:** Automatisch (Docker)

**DNS-Resolution:**
- `postgres` → PostgreSQL Container
- `postgrest` → PostgREST Container
- `n8n` → n8n Container

#### 4.2 LXC Network

**Interface:** eth0  
**Bridge:** vmbr0 (Standard)  
**VLAN:** 90 (Standard)  
**IP:** DHCP oder statisch

#### 4.3 External Access

**NGINX Reverse Proxy:**
```
https://sb-<timestamp>.userman.de
  ↓
http://<container-ip>:5678
```

**Direct Access:**
- n8n: `http://<ip>:5678`
- PostgREST: `http://<ip>:3000`

### 5. Storage-Architektur

#### 5.1 Container Storage

**Location:** `/var/lib/lxc/<ctid>/rootfs`  
**Type:** ZFS (Standard) oder Directory  
**Size:** 50 GB (Standard)

#### 5.2 Docker Volumes

```
/opt/customer-stack/volumes/
├── postgres-data/     # PostgreSQL Daten
├── n8n-data/         # n8n Workflows & Credentials
└── postgrest-data/   # PostgREST Cache (optional)
```

**Permissions:**
- postgres-data: 999:999 (postgres user)
- n8n-data: 1000:1000 (node user)

#### 5.3 Configuration Files

```
/opt/customer-stack/
├── docker-compose.yml        # Stack-Definition
├── .env                      # Environment-Variablen
├── workflow-template.json    # n8n Workflow-Template
├── reload-workflow.sh        # Reload-Script
└── volumes/                  # Persistente Daten
```

## 🔄 Datenfluss

### RAG Chat-Flow

```
1. User → Chat-Webhook
   POST https://sb-<timestamp>.userman.de/webhook/rag-chat-webhook/chat
   Body: {"query": "Was ist...?"}

2. n8n → Ollama (Embedding)
   POST http://ollama:11434/api/embeddings
   Body: {"model": "nomic-embed-text", "prompt": "Was ist...?"}

3. n8n → PostgreSQL (Vector Search)
   POST http://postgrest:3000/rpc/match_documents
   Body: {"query_embedding": [...], "match_count": 5}

4. PostgreSQL → n8n (Relevant Documents)
   Response: [{"content": "...", "similarity": 0.85}, ...]

5. n8n → Ollama (Chat Completion)
   POST http://ollama:11434/api/generate
   Body: {"model": "ministral-3:3b", "prompt": "Context: ... Question: ..."}

6. n8n → User (Response)
   Response: {"answer": "...", "sources": [...]}
```

### Document Upload-Flow

```
1. User → Upload-Form
   POST https://sb-<timestamp>.userman.de/form/rag-upload-form
   Body: FormData with file

2. n8n → Text Extraction
   Extract text from PDF/DOCX/TXT

3. n8n → Text Chunking
   Split text into chunks (max 1000 chars)

4. n8n → Ollama (Embeddings)
   For each chunk:
   POST http://ollama:11434/api/embeddings
   Body: {"model": "nomic-embed-text", "prompt": "<chunk>"}

5. n8n → PostgreSQL (Store)
   For each chunk:
   POST http://postgrest:3000/documents
   Body: {"content": "<chunk>", "embedding": [...], "metadata": {...}}

6. n8n → User (Confirmation)
   Response: {"status": "success", "chunks": 42}
```

## 🔐 Security-Architektur

### 1. Container-Isolation

- **Unprivileged LXC:** Prozesse laufen als unprivilegierte User
- **AppArmor:** Kernel-Level Security
- **Seccomp:** Syscall-Filtering

### 2. Network-Isolation

- **Docker Network:** Isoliertes Bridge-Network
- **Firewall:** Nur notwendige Ports exponiert
- **VLAN:** Netzwerk-Segmentierung

### 3. Authentication

- **JWT-Tokens:** Für PostgREST API
- **n8n Credentials:** Verschlüsselt mit N8N_ENCRYPTION_KEY
- **PostgreSQL:** Passwort-basiert, nur intern erreichbar

### 4. Data Protection

- **Encryption at Rest:** Optional via ZFS
- **Encryption in Transit:** HTTPS via NGINX
- **Credentials:** Gespeichert in .gitignore-geschütztem Verzeichnis

## 📊 Performance-Architektur

### 1. Database Optimization

- **pgvector Index:** IVFFlat für schnelle Vektor-Suche
- **Connection Pooling:** Via PostgREST
- **Query Optimization:** Prepared Statements

### 2. Caching

- **PostgREST:** Schema-Cache
- **n8n:** Workflow-Cache
- **Docker:** Layer-Cache

### 3. Resource Management

- **CPU:** Unlimited (kann limitiert werden)
- **Memory:** 4 GB (kann angepasst werden)
- **Disk I/O:** ZFS mit Compression

## 🔧 Deployment-Architektur

### 1. Installation-Flow

```
1. install.sh
   ↓
2. Parameter-Validierung
   ↓
3. CTID-Generierung
   ↓
4. Template-Download (Debian 12)
   ↓
5. LXC-Container-Erstellung
   ↓
6. Container-Start
   ↓
7. System-Update (APT)
   ↓
8. Docker-Installation
   ↓
9. Stack-Deployment (docker-compose.yml)
   ↓
10. Database-Initialization (pgvector, schema)
   ↓
11. n8n-Setup (owner, credentials, workflow)
   ↓
12. Workflow-Reload-Service
   ↓
13. NGINX-Proxy-Setup (optional)
   ↓
14. Credentials-Save
   ↓
15. JSON-Output
```

### 2. Update-Flow

```
1. update_credentials.sh
   ↓
2. Load Credentials
   ↓
3. n8n API Login
   ↓
4. Update Credentials (Ollama, etc.)
   ↓
5. Reload Workflow (optional)
   ↓
6. Verify Changes
```

### 3. Backup-Flow

```
1. Stop Container
   ↓
2. Backup Volumes
   - /opt/customer-stack/volumes/postgres-data
   - /opt/customer-stack/volumes/n8n-data
   ↓
3. Backup Configuration
   - /opt/customer-stack/.env
   - /opt/customer-stack/docker-compose.yml
   ↓
4. Start Container
```

## 📚 Technologie-Stack

### Core Technologies

- **Proxmox VE:** Virtualisierung
- **LXC:** Container-Technologie
- **Docker:** Container-Runtime
- **Docker Compose:** Orchestrierung

### Database Stack

- **PostgreSQL 16:** Relationale Datenbank
- **pgvector:** Vektor-Extension
- **PostgREST:** REST API

### Application Stack

- **n8n:** Workflow-Automation
- **Node.js:** Runtime für n8n
- **Ollama:** LLM-Integration

### Infrastructure

- **Debian 12:** Base OS
- **Systemd:** Service-Management
- **NGINX:** Reverse Proxy

## 🔗 Integration-Points

### 1. Ollama Integration

**Connection:** HTTP REST API  
**Endpoint:** `http://192.168.45.3:11434`  
**Models:**
- Chat: `ministral-3:3b`
- Embeddings: `nomic-embed-text:latest`

### 2. NGINX Integration

**Connection:** HTTP Reverse Proxy  
**Configuration:** OPNsense NGINX Plugin  
**SSL:** Let's Encrypt (optional)

### 3. Monitoring Integration

**Potential Integrations:**
- Prometheus (Metrics)
- Grafana (Visualization)
- Loki (Logs)
- Alertmanager (Alerts)

## 📚 Weiterführende Dokumentation

- [Installation](Installation.md) - Installations-Anleitung
- [Configuration](Configuration.md) - Konfiguration
- [Deployment](Deployment.md) - Deployment-Strategien
- [API-Referenz](API-Reference.md) - API-Dokumentation

---

**Design-Prinzipien:**
1. **Modularität:** Komponenten sind austauschbar
2. **Skalierbarkeit:** Horizontal und vertikal skalierbar
3. **Wartbarkeit:** Klare Struktur und Dokumentation
4. **Sicherheit:** Defense in Depth
5. **Performance:** Optimiert für RAG-Workloads