MediaMetz/customer-installer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Add comprehensive Wiki documentation

Browse Source 
- Add Wiki home page with navigation
- Add Installation guide with all parameters
- Add Credentials-Management documentation
- Add Testing guide with all test suites
- Add Architecture documentation with diagrams
- Add Troubleshooting guide with solutions
- Add FAQ with common questions

Wiki includes:
- Complete installation instructions
- Credentials management workflows
- Testing procedures (40+ tests)
- System architecture diagrams
- Troubleshooting for common issues
- FAQ covering all aspects
- Cross-referenced documentation
This commit is contained in:
Wolfgang
2026-01-24 22:48:04 +01:00
parent aa00fb9d29
commit 1a91f23044
7 changed files with 2809 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
wiki/Architecture.md
+503
View File
@@ -0,0 +1,503 @@
# Architektur
Diese Seite beschreibt die technische Architektur des Customer Installer Systems.
## 📐 System-Übersicht
```
┌─────────────────────────────────────────────────────────────────┐
│ Proxmox VE Host │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ LXC Container (Debian 12) │ │
│ │ │ │
│ │ ┌─────────────────────────────────────────────────────┐ │ │
│ │ │ Docker Compose Stack │ │ │
│ │ │ │ │ │
│ │ │ ┌──────────────┐ ┌──────────────┐ ┌─────────┐ │ │ │
│ │ │ │ PostgreSQL │ │ PostgREST │ │ n8n │ │ │ │
│ │ │ │ + pgvector │◄─┤ (REST API) │◄─┤ Workflow│ │ │ │
│ │ │ │ │ │ │ │ Engine │ │ │ │
│ │ │ └──────────────┘ └──────────────┘ └─────────┘ │ │ │
│ │ │ │ │ │ │ │ │
│ │ │ └──────────────────┴──────────────┘ │ │ │
│ │ │ Docker Network │ │ │
│ │ │ (customer-net) │ │ │
│ │ └─────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌─────────────────────────────────────────────────────┐ │ │
│ │ │ Systemd Services │ │ │
│ │ │ - docker.service │ │ │
│ │ │ - n8n-workflow-reload.service │ │ │
│ │ └─────────────────────────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ NGINX Reverse Proxy (OPNsense) │ │
│ │ https://sb-<timestamp>.userman.de → http://<ip>:5678 │ │
│ └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────┐
│ Ollama Server │
│ (External Host) │
│ Port: 11434 │
└──────────────────┘
```
## 🏗️ Komponenten-Architektur
### 1. Proxmox LXC Container
**Technologie:** Linux Container (LXC)
**OS:** Debian 12 (Bookworm)
**Typ:** Unprivileged (Standard) oder Privileged (optional)
**Ressourcen:**
- CPU: Unlimited (konfigurierbar)
- RAM: 4096 MB (Standard)
- Swap: 512 MB
- Disk: 50 GB (Standard)
- Netzwerk: Bridge mit VLAN-Support
**Features:**
- Automatische CTID-Generierung (customer-safe)
- DHCP oder statische IP
- VLAN-Tagging
- APT-Proxy-Support
### 2. Docker Stack
**Technologie:** Docker Compose v2
**Netzwerk:** Bridge Network (customer-net)
**Volumes:** Named Volumes für Persistenz
#### 2.1 PostgreSQL Container
**Image:** `postgres:16-alpine`
**Name:** `customer-postgres`
**Port:** 5432 (intern)
**Features:**
- pgvector Extension (v0.5.1)
- Automatische Datenbank-Initialisierung
- Persistente Daten via Volume
- Health Checks
**Datenbank-Schema:**
```sql
-- documents Tabelle für RAG
CREATE TABLE documents (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
content TEXT NOT NULL,
metadata JSONB,
embedding vector(384), -- nomic-embed-text Dimension
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Index für Vektor-Suche
CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);
-- RPC-Funktion für Similarity Search
CREATE FUNCTION match_documents(
query_embedding vector(384),
match_count int DEFAULT 5
) RETURNS TABLE (
id UUID,
content TEXT,
metadata JSONB,
similarity FLOAT
) AS $$
SELECT
id,
content,
metadata,
1 - (embedding <=> query_embedding) AS similarity
FROM documents
ORDER BY embedding <=> query_embedding
LIMIT match_count;
$$ LANGUAGE sql STABLE;
```
#### 2.2 PostgREST Container
**Image:** `postgrest/postgrest:v12.0.2`
**Name:** `customer-postgrest`
**Port:** 3000 (extern + intern)
**Features:**
- Supabase-kompatible REST API
- JWT-basierte Authentikation
- Automatische OpenAPI-Dokumentation
- RPC-Funktionen-Support
**Endpoints:**
- `GET /documents` - Dokumente abrufen
- `POST /documents` - Dokument erstellen
- `POST /rpc/match_documents` - Vektor-Suche
**Authentication:**
- `anon` Role: Lesezugriff
- `service_role`: Voller Zugriff
#### 2.3 n8n Container
**Image:** `n8nio/n8n:latest`
**Name:** `n8n`
**Port:** 5678 (extern + intern)
**Features:**
- PostgreSQL als Backend
- Workflow-Automation
- Webhook-Support
- Credentials-Management
- Execution-History
**Workflows:**
- RAG KI-Bot (Chat-Interface)
- Document Upload (Form)
- Vector Embedding (Ollama)
- Similarity Search (PostgreSQL)
**Environment:**
```bash
DB_TYPE=postgresdb
DB_POSTGRESDB_HOST=postgres
DB_POSTGRESDB_PORT=5432
DB_POSTGRESDB_DATABASE=customer
DB_POSTGRESDB_USER=customer
DB_POSTGRESDB_PASSWORD=<generated>
N8N_ENCRYPTION_KEY=<generated>
WEBHOOK_URL=https://sb-<timestamp>.userman.de
N8N_DIAGNOSTICS_ENABLED=false
N8N_PERSONALIZATION_ENABLED=false
```
### 3. Systemd Services
#### 3.1 docker.service
Standard Docker Service für Container-Management.
#### 3.2 n8n-workflow-reload.service
**Typ:** oneshot
**Trigger:** Container-Start
**Funktion:** Automatisches Workflow-Reload
```ini
[Unit]
Description=Reload n8n workflow on container start
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
ExecStart=/opt/customer-stack/reload-workflow.sh
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
```
### 4. Netzwerk-Architektur
#### 4.1 Docker Network
**Name:** `customer-stack_customer-net`
**Typ:** Bridge
**Subnet:** Automatisch (Docker)
**DNS-Resolution:**
- `postgres` → PostgreSQL Container
- `postgrest` → PostgREST Container
- `n8n` → n8n Container
#### 4.2 LXC Network
**Interface:** eth0
**Bridge:** vmbr0 (Standard)
**VLAN:** 90 (Standard)
**IP:** DHCP oder statisch
#### 4.3 External Access
**NGINX Reverse Proxy:**
```
https://sb-<timestamp>.userman.de
↓
http://<container-ip>:5678
```
**Direct Access:**
- n8n: `http://<ip>:5678`
- PostgREST: `http://<ip>:3000`
### 5. Storage-Architektur
#### 5.1 Container Storage
**Location:** `/var/lib/lxc/<ctid>/rootfs`
**Type:** ZFS (Standard) oder Directory
**Size:** 50 GB (Standard)
#### 5.2 Docker Volumes
```
/opt/customer-stack/volumes/
├── postgres-data/ # PostgreSQL Daten
├── n8n-data/ # n8n Workflows & Credentials
└── postgrest-data/ # PostgREST Cache (optional)
```
**Permissions:**
- postgres-data: 999:999 (postgres user)
- n8n-data: 1000:1000 (node user)
#### 5.3 Configuration Files
```
/opt/customer-stack/
├── docker-compose.yml # Stack-Definition
├── .env # Environment-Variablen
├── workflow-template.json # n8n Workflow-Template
├── reload-workflow.sh # Reload-Script
└── volumes/ # Persistente Daten
```
## 🔄 Datenfluss
### RAG Chat-Flow
```
1. User → Chat-Webhook
POST https://sb-<timestamp>.userman.de/webhook/rag-chat-webhook/chat
Body: {"query": "Was ist...?"}
2. n8n → Ollama (Embedding)
POST http://ollama:11434/api/embeddings
Body: {"model": "nomic-embed-text", "prompt": "Was ist...?"}
3. n8n → PostgreSQL (Vector Search)
POST http://postgrest:3000/rpc/match_documents
Body: {"query_embedding": [...], "match_count": 5}
4. PostgreSQL → n8n (Relevant Documents)
Response: [{"content": "...", "similarity": 0.85}, ...]
5. n8n → Ollama (Chat Completion)
POST http://ollama:11434/api/generate
Body: {"model": "ministral-3:3b", "prompt": "Context: ... Question: ..."}
6. n8n → User (Response)
Response: {"answer": "...", "sources": [...]}
```
### Document Upload-Flow
```
1. User → Upload-Form
POST https://sb-<timestamp>.userman.de/form/rag-upload-form
Body: FormData with file
2. n8n → Text Extraction
Extract text from PDF/DOCX/TXT
3. n8n → Text Chunking
Split text into chunks (max 1000 chars)
4. n8n → Ollama (Embeddings)
For each chunk:
POST http://ollama:11434/api/embeddings
Body: {"model": "nomic-embed-text", "prompt": "<chunk>"}
5. n8n → PostgreSQL (Store)
For each chunk:
POST http://postgrest:3000/documents
Body: {"content": "<chunk>", "embedding": [...], "metadata": {...}}
6. n8n → User (Confirmation)
Response: {"status": "success", "chunks": 42}
```
## 🔐 Security-Architektur
### 1. Container-Isolation
- **Unprivileged LXC:** Prozesse laufen als unprivilegierte User
- **AppArmor:** Kernel-Level Security
- **Seccomp:** Syscall-Filtering
### 2. Network-Isolation
- **Docker Network:** Isoliertes Bridge-Network
- **Firewall:** Nur notwendige Ports exponiert
- **VLAN:** Netzwerk-Segmentierung
### 3. Authentication
- **JWT-Tokens:** Für PostgREST API
- **n8n Credentials:** Verschlüsselt mit N8N_ENCRYPTION_KEY
- **PostgreSQL:** Passwort-basiert, nur intern erreichbar
### 4. Data Protection
- **Encryption at Rest:** Optional via ZFS
- **Encryption in Transit:** HTTPS via NGINX
- **Credentials:** Gespeichert in .gitignore-geschütztem Verzeichnis
## 📊 Performance-Architektur
### 1. Database Optimization
- **pgvector Index:** IVFFlat für schnelle Vektor-Suche
- **Connection Pooling:** Via PostgREST
- **Query Optimization:** Prepared Statements
### 2. Caching
- **PostgREST:** Schema-Cache
- **n8n:** Workflow-Cache
- **Docker:** Layer-Cache
### 3. Resource Management
- **CPU:** Unlimited (kann limitiert werden)
- **Memory:** 4 GB (kann angepasst werden)
- **Disk I/O:** ZFS mit Compression
## 🔧 Deployment-Architektur
### 1. Installation-Flow
```
1. install.sh
↓
2. Parameter-Validierung
↓
3. CTID-Generierung
↓
4. Template-Download (Debian 12)
↓
5. LXC-Container-Erstellung
↓
6. Container-Start
↓
7. System-Update (APT)
↓
8. Docker-Installation
↓
9. Stack-Deployment (docker-compose.yml)
↓
10. Database-Initialization (pgvector, schema)
↓
11. n8n-Setup (owner, credentials, workflow)
↓
12. Workflow-Reload-Service
↓
13. NGINX-Proxy-Setup (optional)
↓
14. Credentials-Save
↓
15. JSON-Output
```
### 2. Update-Flow
```
1. update_credentials.sh
↓
2. Load Credentials
↓
3. n8n API Login
↓
4. Update Credentials (Ollama, etc.)
↓
5. Reload Workflow (optional)
↓
6. Verify Changes
```
### 3. Backup-Flow
```
1. Stop Container
↓
2. Backup Volumes
- /opt/customer-stack/volumes/postgres-data
- /opt/customer-stack/volumes/n8n-data
↓
3. Backup Configuration
- /opt/customer-stack/.env
- /opt/customer-stack/docker-compose.yml
↓
4. Start Container
```
## 📚 Technologie-Stack
### Core Technologies
- **Proxmox VE:** Virtualisierung
- **LXC:** Container-Technologie
- **Docker:** Container-Runtime
- **Docker Compose:** Orchestrierung
### Database Stack
- **PostgreSQL 16:** Relationale Datenbank
- **pgvector:** Vektor-Extension
- **PostgREST:** REST API
### Application Stack
- **n8n:** Workflow-Automation
- **Node.js:** Runtime für n8n
- **Ollama:** LLM-Integration
### Infrastructure
- **Debian 12:** Base OS
- **Systemd:** Service-Management
- **NGINX:** Reverse Proxy
## 🔗 Integration-Points
### 1. Ollama Integration
**Connection:** HTTP REST API
**Endpoint:** `http://192.168.45.3:11434`
**Models:**
- Chat: `ministral-3:3b`
- Embeddings: `nomic-embed-text:latest`
### 2. NGINX Integration
**Connection:** HTTP Reverse Proxy
**Configuration:** OPNsense NGINX Plugin
**SSL:** Let's Encrypt (optional)
### 3. Monitoring Integration
**Potential Integrations:**
- Prometheus (Metrics)
- Grafana (Visualization)
- Loki (Logs)
- Alertmanager (Alerts)
## 📚 Weiterführende Dokumentation
- [Installation](Installation.md) - Installations-Anleitung
- [Configuration](Configuration.md) - Konfiguration
- [Deployment](Deployment.md) - Deployment-Strategien
- [API-Referenz](API-Reference.md) - API-Dokumentation
---
**Design-Prinzipien:**
1. **Modularität:** Komponenten sind austauschbar
2. **Skalierbarkeit:** Horizontal und vertikal skalierbar
3. **Wartbarkeit:** Klare Struktur und Dokumentation
4. **Sicherheit:** Defense in Depth
5. **Performance:** Optimiert für RAG-Workloads