admin/automat26

Fork 0

Files

root c67c43d08d Altes Projekt hinzugefuegt

2026-02-25 22:47:22 +01:00

14 KiB

Raw Blame History

Architektur

Diese Seite beschreibt die technische Architektur des Customer Installer Systems.

📐 System-Übersicht

┌─────────────────────────────────────────────────────────────────┐
│                         Proxmox VE Host                         │
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐ │
│  │              LXC Container (Debian 12)                    │ │
│  │                                                           │ │
│  │  ┌─────────────────────────────────────────────────────┐ │ │
│  │  │           Docker Compose Stack                      │ │ │
│  │  │                                                     │ │ │
│  │  │  ┌──────────────┐  ┌──────────────┐  ┌─────────┐  │ │ │
│  │  │  │  PostgreSQL  │  │  PostgREST   │  │   n8n   │  │ │ │
│  │  │  │  + pgvector  │◄─┤  (REST API)  │◄─┤ Workflow│  │ │ │
│  │  │  │              │  │              │  │ Engine  │  │ │ │
│  │  │  └──────────────┘  └──────────────┘  └─────────┘  │ │ │
│  │  │         │                  │              │        │ │ │
│  │  │         └──────────────────┴──────────────┘        │ │ │
│  │  │                    Docker Network                  │ │ │
│  │  │                  (customer-net)                    │ │ │
│  │  └─────────────────────────────────────────────────────┘ │ │
│  │                                                           │ │
│  │  ┌─────────────────────────────────────────────────────┐ │ │
│  │  │              Systemd Services                       │ │ │
│  │  │  - docker.service                                   │ │ │
│  │  │  - n8n-workflow-reload.service                      │ │ │
│  │  └─────────────────────────────────────────────────────┘ │ │
│  └───────────────────────────────────────────────────────────┘ │
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐ │
│  │              NGINX Reverse Proxy (OPNsense)             │ │
│  │  https://sb-<timestamp>.userman.de → http://<ip>:5678  │ │
│  └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
                   ┌──────────────────┐
                   │  Ollama Server   │
                   │  (External Host) │
                   │  Port: 11434     │
                   └──────────────────┘

🏗️ Komponenten-Architektur

1. Proxmox LXC Container

Technologie: Linux Container (LXC)
OS: Debian 12 (Bookworm)
Typ: Unprivileged (Standard) oder Privileged (optional)

Ressourcen:

CPU: Unlimited (konfigurierbar)
RAM: 4096 MB (Standard)
Swap: 512 MB
Disk: 50 GB (Standard)
Netzwerk: Bridge mit VLAN-Support

Features:

Automatische CTID-Generierung (customer-safe)
DHCP oder statische IP
VLAN-Tagging
APT-Proxy-Support

2. Docker Stack

Technologie: Docker Compose v2
Netzwerk: Bridge Network (customer-net)
Volumes: Named Volumes für Persistenz

2.1 PostgreSQL Container

Image: postgres:16-alpine
Name: customer-postgres
Port: 5432 (intern)

Features:

pgvector Extension (v0.5.1)
Automatische Datenbank-Initialisierung
Persistente Daten via Volume
Health Checks

Datenbank-Schema:

-- documents Tabelle für RAG
CREATE TABLE documents (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  content TEXT NOT NULL,
  metadata JSONB,
  embedding vector(384),  -- nomic-embed-text Dimension
  created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Index für Vektor-Suche
CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);

-- RPC-Funktion für Similarity Search
CREATE FUNCTION match_documents(
  query_embedding vector(384),
  match_count int DEFAULT 5
) RETURNS TABLE (
  id UUID,
  content TEXT,
  metadata JSONB,
  similarity FLOAT
) AS $$
  SELECT
    id,
    content,
    metadata,
    1 - (embedding <=> query_embedding) AS similarity
  FROM documents
  ORDER BY embedding <=> query_embedding
  LIMIT match_count;
$$ LANGUAGE sql STABLE;

2.2 PostgREST Container

Image: postgrest/postgrest:v12.0.2
Name: customer-postgrest
Port: 3000 (extern + intern)

Features:

Supabase-kompatible REST API
JWT-basierte Authentikation
Automatische OpenAPI-Dokumentation
RPC-Funktionen-Support

Endpoints:

GET /documents - Dokumente abrufen
POST /documents - Dokument erstellen
POST /rpc/match_documents - Vektor-Suche

Authentication:

anon Role: Lesezugriff
service_role: Voller Zugriff

2.3 n8n Container

Image: n8nio/n8n:latest
Name: n8n
Port: 5678 (extern + intern)

Features:

PostgreSQL als Backend
Workflow-Automation
Webhook-Support
Credentials-Management
Execution-History

Workflows:

RAG KI-Bot (Chat-Interface)
Document Upload (Form)
Vector Embedding (Ollama)
Similarity Search (PostgreSQL)

Environment:

DB_TYPE=postgresdb
DB_POSTGRESDB_HOST=postgres
DB_POSTGRESDB_PORT=5432
DB_POSTGRESDB_DATABASE=customer
DB_POSTGRESDB_USER=customer
DB_POSTGRESDB_PASSWORD=<generated>
N8N_ENCRYPTION_KEY=<generated>
WEBHOOK_URL=https://sb-<timestamp>.userman.de
N8N_DIAGNOSTICS_ENABLED=false
N8N_PERSONALIZATION_ENABLED=false

3. Systemd Services

3.1 docker.service

Standard Docker Service für Container-Management.

3.2 n8n-workflow-reload.service

Typ: oneshot
Trigger: Container-Start
Funktion: Automatisches Workflow-Reload

[Unit]
Description=Reload n8n workflow on container start
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/opt/customer-stack/reload-workflow.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

4. Netzwerk-Architektur

4.1 Docker Network

Name: customer-stack_customer-net
Typ: Bridge
Subnet: Automatisch (Docker)

DNS-Resolution:

postgres → PostgreSQL Container
postgrest → PostgREST Container
n8n → n8n Container

4.2 LXC Network

Interface: eth0
Bridge: vmbr0 (Standard)
VLAN: 90 (Standard)
IP: DHCP oder statisch

4.3 External Access

NGINX Reverse Proxy:

https://sb-<timestamp>.userman.de
  ↓
http://<container-ip>:5678

Direct Access:

n8n: http://<ip>:5678
PostgREST: http://<ip>:3000

5. Storage-Architektur

5.1 Container Storage

Location: /var/lib/lxc/<ctid>/rootfs
Type: ZFS (Standard) oder Directory
Size: 50 GB (Standard)

5.2 Docker Volumes

/opt/customer-stack/volumes/
├── postgres-data/     # PostgreSQL Daten
├── n8n-data/         # n8n Workflows & Credentials
└── postgrest-data/   # PostgREST Cache (optional)

Permissions:

postgres-data: 999:999 (postgres user)
n8n-data: 1000:1000 (node user)

5.3 Configuration Files

/opt/customer-stack/
├── docker-compose.yml        # Stack-Definition
├── .env                      # Environment-Variablen
├── workflow-template.json    # n8n Workflow-Template
├── reload-workflow.sh        # Reload-Script
└── volumes/                  # Persistente Daten

🔄 Datenfluss

RAG Chat-Flow

1. User → Chat-Webhook
   POST https://sb-<timestamp>.userman.de/webhook/rag-chat-webhook/chat
   Body: {"query": "Was ist...?"}

2. n8n → Ollama (Embedding)
   POST http://ollama:11434/api/embeddings
   Body: {"model": "nomic-embed-text", "prompt": "Was ist...?"}

3. n8n → PostgreSQL (Vector Search)
   POST http://postgrest:3000/rpc/match_documents
   Body: {"query_embedding": [...], "match_count": 5}

4. PostgreSQL → n8n (Relevant Documents)
   Response: [{"content": "...", "similarity": 0.85}, ...]

5. n8n → Ollama (Chat Completion)
   POST http://ollama:11434/api/generate
   Body: {"model": "ministral-3:3b", "prompt": "Context: ... Question: ..."}

6. n8n → User (Response)
   Response: {"answer": "...", "sources": [...]}

Document Upload-Flow

1. User → Upload-Form
   POST https://sb-<timestamp>.userman.de/form/rag-upload-form
   Body: FormData with file

2. n8n → Text Extraction
   Extract text from PDF/DOCX/TXT

3. n8n → Text Chunking
   Split text into chunks (max 1000 chars)

4. n8n → Ollama (Embeddings)
   For each chunk:
   POST http://ollama:11434/api/embeddings
   Body: {"model": "nomic-embed-text", "prompt": "<chunk>"}

5. n8n → PostgreSQL (Store)
   For each chunk:
   POST http://postgrest:3000/documents
   Body: {"content": "<chunk>", "embedding": [...], "metadata": {...}}

6. n8n → User (Confirmation)
   Response: {"status": "success", "chunks": 42}

🔐 Security-Architektur

1. Container-Isolation

Unprivileged LXC: Prozesse laufen als unprivilegierte User
AppArmor: Kernel-Level Security
Seccomp: Syscall-Filtering

2. Network-Isolation

Docker Network: Isoliertes Bridge-Network
Firewall: Nur notwendige Ports exponiert
VLAN: Netzwerk-Segmentierung

3. Authentication

JWT-Tokens: Für PostgREST API
n8n Credentials: Verschlüsselt mit N8N_ENCRYPTION_KEY
PostgreSQL: Passwort-basiert, nur intern erreichbar

4. Data Protection

Encryption at Rest: Optional via ZFS
Encryption in Transit: HTTPS via NGINX
Credentials: Gespeichert in .gitignore-geschütztem Verzeichnis

📊 Performance-Architektur

1. Database Optimization

pgvector Index: IVFFlat für schnelle Vektor-Suche
Connection Pooling: Via PostgREST
Query Optimization: Prepared Statements

2. Caching

PostgREST: Schema-Cache
n8n: Workflow-Cache
Docker: Layer-Cache

3. Resource Management

CPU: Unlimited (kann limitiert werden)
Memory: 4 GB (kann angepasst werden)
Disk I/O: ZFS mit Compression

🔧 Deployment-Architektur

1. Installation-Flow

1. install.sh
   ↓
2. Parameter-Validierung
   ↓
3. CTID-Generierung
   ↓
4. Template-Download (Debian 12)
   ↓
5. LXC-Container-Erstellung
   ↓
6. Container-Start
   ↓
7. System-Update (APT)
   ↓
8. Docker-Installation
   ↓
9. Stack-Deployment (docker-compose.yml)
   ↓
10. Database-Initialization (pgvector, schema)
   ↓
11. n8n-Setup (owner, credentials, workflow)
   ↓
12. Workflow-Reload-Service
   ↓
13. NGINX-Proxy-Setup (optional)
   ↓
14. Credentials-Save
   ↓
15. JSON-Output

2. Update-Flow

1. update_credentials.sh
   ↓
2. Load Credentials
   ↓
3. n8n API Login
   ↓
4. Update Credentials (Ollama, etc.)
   ↓
5. Reload Workflow (optional)
   ↓
6. Verify Changes

3. Backup-Flow

1. Stop Container
   ↓
2. Backup Volumes
   - /opt/customer-stack/volumes/postgres-data
   - /opt/customer-stack/volumes/n8n-data
   ↓
3. Backup Configuration
   - /opt/customer-stack/.env
   - /opt/customer-stack/docker-compose.yml
   ↓
4. Start Container

📚 Technologie-Stack

Core Technologies

Proxmox VE: Virtualisierung
LXC: Container-Technologie
Docker: Container-Runtime
Docker Compose: Orchestrierung

Database Stack

PostgreSQL 16: Relationale Datenbank
pgvector: Vektor-Extension
PostgREST: REST API

Application Stack

n8n: Workflow-Automation
Node.js: Runtime für n8n
Ollama: LLM-Integration

Infrastructure

Debian 12: Base OS
Systemd: Service-Management
NGINX: Reverse Proxy

🔗 Integration-Points

1. Ollama Integration

Connection: HTTP REST API
Endpoint: http://192.168.45.3:11434
Models:

Chat: ministral-3:3b
Embeddings: nomic-embed-text:latest

2. NGINX Integration

Connection: HTTP Reverse Proxy
Configuration: OPNsense NGINX Plugin
SSL: Let's Encrypt (optional)

3. Monitoring Integration

Potential Integrations:

Prometheus (Metrics)
Grafana (Visualization)
Loki (Logs)
Alertmanager (Alerts)

📚 Weiterführende Dokumentation

Installation - Installations-Anleitung
Configuration - Konfiguration
Deployment - Deployment-Strategien
API-Referenz - API-Dokumentation

Design-Prinzipien:

Modularität: Komponenten sind austauschbar
Skalierbarkeit: Horizontal und vertikal skalierbar
Wartbarkeit: Klare Struktur und Dokumentation
Sicherheit: Defense in Depth
Performance: Optimiert für RAG-Workloads

14 KiB Raw Blame History

Architektur

📐 System-Übersicht

🏗️ Komponenten-Architektur

1. Proxmox LXC Container

2. Docker Stack

2.1 PostgreSQL Container

2.2 PostgREST Container

2.3 n8n Container

3. Systemd Services

3.1 docker.service

3.2 n8n-workflow-reload.service

4. Netzwerk-Architektur

4.1 Docker Network

4.2 LXC Network

4.3 External Access

5. Storage-Architektur

5.1 Container Storage

5.2 Docker Volumes

5.3 Configuration Files

🔄 Datenfluss

RAG Chat-Flow

Document Upload-Flow

🔐 Security-Architektur

1. Container-Isolation

2. Network-Isolation

3. Authentication

4. Data Protection

📊 Performance-Architektur

1. Database Optimization

2. Caching

3. Resource Management

🔧 Deployment-Architektur

1. Installation-Flow

2. Update-Flow

3. Backup-Flow

📚 Technologie-Stack

Core Technologies

Database Stack

Application Stack

Infrastructure

🔗 Integration-Points

1. Ollama Integration

2. NGINX Integration

3. Monitoring Integration

📚 Weiterführende Dokumentation

14 KiB

Raw Blame History