Peter123/privacy-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial: Privacy Gateway Projekt mit Team-Implementierung

Browse Source
This commit is contained in:
root
2026-05-09 05:58:21 +00:00
commit a25b234405
49 changed files with 5107 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PROJECT.md
+63
View File
@@ -0,0 +1,63 @@
# Privacy Gateway - Projektübersicht
## Vision
Ein Docker-basierter Proxy für KI-Prompts, der automatisch persönliche Daten (PII) anonymisiert, an externe KI-Modelle weiterleitet und die Antworten re-identifiziert.
## Kernfunktionen
1. **Multi-Chat-Interface** - Mehrere parallele Konversationen (wie OpenWebUI)
2. **PII-Erkennung & Anonymisierung** - Lokales LLM für Datenschutz
3. **KI-Proxy** - Weiterleitung an Ollama/OpenWebUI
4. **Re-Identifizierung** - Zurückfüllen der Originaldaten in Antworten
5. **Session-Management** - Persistente Chats mit History
## Technologie-Stack
- **Frontend:** React + TypeScript (Chat-UI)
- **Backend:** Node.js/Express oder Python/FastAPI
- **Anonymisierung:** Ollama mit lokalem Modell (z.B. gemma oder mistral)
- **Datenbank:** PostgreSQL (Sessions, Mappings)
- **Container:** Docker + Docker Compose
- **API-Proxy:** Eigener Ollama-kompatibler Endpoint
## Architektur
```
┌─────────────┐ ┌──────────────┐ ┌──────────────┐ ┌─────────────┐
│ Client │────▶│ Gateway │────▶│ Anonymizer │────▶│ Ollama │
│ (Chat-UI) │◄────│ (API) │◄────│ (lokales LLM)│◄────│ (KI-Model) │
└─────────────┘ └──────────────┘ └──────────────┘ └─────────────┘
│
▼
┌──────────────┐
│ PostgreSQL │
│ (Mapping DB) │
└──────────────┘
```
## PII-Kategorien (Phase 1)
- Namen (Personen, Firmen)
- Adressen
- E-Mail-Adressen
- Telefonnummern
- Geburtsdaten
- Kontonummern / IBAN
- Personalnummern
- Kreditkartennummern
## Projektphasen
1. **Setup** - Grundstruktur, Docker, Datenbank
2. **Anonymisierung** - PII-Erkennung mit lokalem LLM
3. **Backend API** - Proxy-Endpoint, Session-Management
4. **Frontend** - Chat-UI mit Multi-Window
5. **Integration** - Mapping-Speicherung, Re-Identifizierung
6. **Testing & Polish** - E2E-Tests, Feinschliff
## Team
- **Peter (ich)** - Projektleitung, Architektur, Integration
- **Sub-Agent: Backend-Dev** - API, Datenbank, Ollama-Proxy
- **Sub-Agent: Frontend-Dev** - React-UI, Chat-Komponenten
- **Sub-Agent: Anonymisierung** - PII-Erkennung, Prompt-Engineering
## Start-Datum
2026-05-09
## Status
🟡 Planung - Team wird zusammengestellt
anonymizer/Dockerfile
+47
View File
@@ -0,0 +1,47 @@
# Dockerfile für den Anonymizer Service
# Läuft im Backend-Container oder standalone
FROM node:20-alpine AS base
# Installiere nötige Pakete
RUN apk add --no-cache dumb-init
WORKDIR /app
# Kopiere Package-Dateien
COPY package.json tsconfig.json ./
# Installiere Dependencies
RUN npm ci --only=production
# Dev-Stage für Entwicklung
FROM base AS dev
RUN npm ci
COPY . .
EXPOSE 3000
CMD ["dumb-init", "npm", "run", "dev"]
# Production-Stage
FROM base AS production
# Kopiere kompilierte Dateien (von Build-Stage)
COPY --from=builder /app/dist ./dist
USER node
EXPOSE 3000
CMD ["dumb-init", "node", "dist/index.js"]
# Builder-Stage (wird für Production benötigt)
FROM base AS builder
RUN npm ci
COPY . .
RUN npm run build
anonymizer/README.md
+161
View File
@@ -0,0 +1,161 @@
# 🔐 Privacy Gateway Anonymizer
PII-Erkennungs- und Anonymisierungs-Service für das Privacy Gateway.
Nutzt lokale LLMs (Ollama) zur Erkennung persönlicher Daten mit Pattern-basiertem Fallback.
## Features
- 🤖 **LLM-basierte PII-Erkennung** via Ollama
- 🛡️ **10 PII-Typen** erkannt: Namen, Adressen, E-Mails, Telefonnummern, IBANs, etc.
- 🔄 **Re-Identifizierung** für nachgelagerte Prozesse
- ⚡ **Pattern-Fallback** bei LLM-Ausfall
- 📊 **Sensitivitäts-Bewertung** (low/medium/high/critical)
- 💾 **Mapping-Serialisierung** für Persistenz
## Schnellstart
```typescript
import { anonymize, reIdentify } from './index.js';
// Anonymisieren
const result = await anonymize(
'Hallo Max Mustermann, kontaktiere mich unter max@beispiel.de'
);
console.log(result.anonymizedText);
// "Hallo [NAME_1], kontaktiere mich unter [EMAIL_1]"
// Re-Identifizieren
const restored = reIdentify(result.anonymizedText, result.mapping);
console.log(restored.reidentifiedText);
// "Hallo Max Mustermann, kontaktiere mich unter max@beispiel.de"
```
## Unterstützte PII-Typen
| Typ | Beschreibung | Beispiel | Sensitivität |
|-----|-------------|----------|--------------|
| `name_person` | Personenname | Max Mustermann | 🔴 high |
| `name_company` | Firmenname | Musterfirma GmbH | 🟡 medium |
| `address` | Vollständige Adresse | Musterstraße 1, 12345 Berlin | 🔴 high |
| `email` | E-Mail-Adresse | max@beispiel.de | 🔴 high |
| `phone` | Telefonnummer (DE) | +49 170 12345678 | 🟡 medium |
| `birthdate` | Geburtsdatum | 15.03.1985 | 🔴 high |
| `account_number` | Kontonummer | 1234567890 | 🔴 critical |
| `iban` | IBAN | DE89 3704... | 🔴 critical |
| `employee_id` | Personalnummer | EMP-12345 | 🟡 medium |
| `credit_card` | Kreditkartennummer | 4111 1111... | 🔴 critical |
## API-Referenz
### `anonymize(text, config?)`
Anonymisiert einen Text durch Erkennung und Ersetzung von PII.
**Parameter:**
- `text` (string): Zu anonymisierender Text
- `config` (optional): Konfigurationsobjekt
**Rückgabe:** `AnonymizeResult`
```typescript
{
success: boolean;
anonymizedText: string;
mapping: ReverseMapping;
sensitivityLevel: 'low' | 'medium' | 'high' | 'critical';
piiCount: number;
processingTimeMs: number;
error?: string;
usedFallback?: boolean;
}
```
### `reIdentify(anonymizedText, mapping, options?)`
Stellt einen anonymisierten Text wieder her.
**Parameter:**
- `anonymizedText` (string): Anonymisierter Text mit Platzhaltern
- `mapping` (ReverseMapping): Mapping von Platzhalter zu Originalwert
- `options` (optional): Optionen für die Re-Identifizierung
**Rückgabe:** `ReidentifyResult`
```typescript
{
success: boolean;
reidentifiedText: string;
replacementsMade: number;
errors: string[];
}
```
### `createAnonymizer(config?)`
Erstellt eine konfigurierte Anonymizer-Instanz.
```typescript
const anonymizer = createAnonymizer({
ollamaUrl: 'http://localhost:11434',
model: 'llama3.2',
timeoutMs: 30000,
maxRetries: 2,
fallbackEnabled: true,
});
```
## Konfiguration
Umgebungsvariablen:
```bash
OLLAMA_URL=http://192.168.2.122:11434
OLLAMA_MODEL=llama3.2
OLLAMA_TIMEOUT=30000
```
## Integration im Backend
```typescript
import { getAnonymizer, maskForLog } from '@privacy-gateway/anonymizer';
const anonymizer = getAnonymizer();
// Vor dem Senden an externe KI
const { anonymizedText, mapping, sensitivityLevel } = await anonymizer.anonymize(userInput);
// Log mit maskierten Werten
console.log(`Sensitivität: ${sensitivityLevel}, PII: ${piiCount}`);
// Nach Antwort der KI
const restored = anonymizer.reIdentify(aiResponse, mapping);
```
## Tests
```bash
npm test
```
## Projektstruktur
```
anonymizer/
├── src/
│ ├── index.ts # Haupt-Export
│ ├── anonymizer.ts # Kern-Logik
│ ├── reverser.ts # Re-Identifizierung
│ ├── pii-types.ts # Typ-Definitionen
│ ├── prompts/
│ │ └── pii-detection.ts # LLM-Prompts
│ ├── utils/
│ │ └── text.ts # Hilfsfunktionen
│ └── test/
│ └── test-cases.ts # Testfälle
├── package.json
├── tsconfig.json
├── Dockerfile
└── README.md
```
## Lizenz
MIT
anonymizer/package.json
+38
View File
@@ -0,0 +1,38 @@
{
"name": "@privacy-gateway/anonymizer",
"version": "1.0.0",
"description": "PII-Erkennungs- und Anonymisierungs-Service für das Privacy Gateway",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"scripts": {
"build": "tsc",
"dev": "tsx watch src/index.ts",
"start": "node dist/index.js",
"test": "node --test dist/test/*.js",
"lint": "eslint src/**/*.ts",
"typecheck": "tsc --noEmit"
},
"keywords": [
"pii",
"anonymization",
"privacy",
"gdpr",
"dsgvo",
"ollama",
"llm"
],
"author": "Privacy Gateway Team",
"license": "MIT",
"dependencies": {
"zod": "^3.23.8"
},
"devDependencies": {
"@types/node": "^20.0.0",
"tsx": "^4.0.0",
"typescript": "^5.0.0"
},
"engines": {
"node": ">=18.0.0"
}
}
anonymizer/src/anonymizer.ts
+271
View File
@@ -0,0 +1,271 @@
/**
* Haupt-Logik für PII-Erkennung und Anonymisierung
* Interagiert mit Ollama für LLM-basierte PII-Erkennung
*/
import {
PiiType,
PiiInstance,
PiiDetectionResult,
ReverseMapping,
PII_TYPE_METADATA,
} from './pii-types.js';
import {
generatePlaceholder,
replaceInText,
cleanJsonResponse,
validatePiiResult,
calculateSensitivityLevel,
mightContainPii,
} from './utils/text.js';
import { generatePrompt, PromptConfig } from './prompts/pii-detection.js';
export interface AnonymizerConfig {
ollamaUrl: string;
model: string;
timeoutMs: number;
maxRetries: number;
fallbackEnabled: boolean;
}
export const DEFAULT_CONFIG: AnonymizerConfig = {
ollamaUrl: process.env.OLLAMA_URL || 'http://192.168.2.122:11434',
model: process.env.OLLAMA_MODEL || 'llama3.2',
timeoutMs: parseInt(process.env.OLLAMA_TIMEOUT || '30000', 10),
maxRetries: 2,
fallbackEnabled: true,
};
export interface AnonymizeResult {
success: boolean;
anonymizedText: string;
mapping: ReverseMapping;
sensitivityLevel: 'low' | 'medium' | 'high' | 'critical';
piiCount: number;
processingTimeMs: number;
error?: string;
usedFallback?: boolean;
}
/**
* Hauptklasse für die PII-Anonymisierung
*/
export class Anonymizer {
private config: AnonymizerConfig;
private placeholderCounters: Map<PiiType, number>;
constructor(config: Partial<AnonymizerConfig> = {}) {
this.config = { ...DEFAULT_CONFIG, ...config };
this.placeholderCounters = new Map();
}
/**
* Anonymisiert einen Text durch PII-Erkennung und Ersetzung
*/
async anonymize(text: string): Promise<AnonymizeResult> {
const startTime = Date.now();
// Schneller Vorab-Check
if (!mightContainPii(text)) {
return {
success: true,
anonymizedText: text,
mapping: {},
sensitivityLevel: 'low',
piiCount: 0,
processingTimeMs: Date.now() - startTime,
};
}
try {
// Versuche LLM-basierte Erkennung
const result = await this.anonymizeWithLlm(text);
const processingTimeMs = Date.now() - startTime;
return {
success: true,
anonymizedText: result.anonymizedText,
mapping: this.buildReverseMapping(result.piiFound),
sensitivityLevel: calculateSensitivityLevel(result.piiFound),
piiCount: result.piiFound.length,
processingTimeMs,
};
} catch (error) {
// Fallback: Pattern-basierte Erkennung
if (this.config.fallbackEnabled) {
const fallbackResult = this.anonymizeWithPatterns(text);
const processingTimeMs = Date.now() - startTime;
return {
success: true,
anonymizedText: fallbackResult.anonymizedText,
mapping: this.buildReverseMapping(fallbackResult.piiFound),
sensitivityLevel: calculateSensitivityLevel(fallbackResult.piiFound),
piiCount: fallbackResult.piiFound.length,
processingTimeMs,
usedFallback: true,
};
}
return {
success: false,
anonymizedText: text,
mapping: {},
sensitivityLevel: 'low',
piiCount: 0,
processingTimeMs: Date.now() - startTime,
error: error instanceof Error ? error.message : String(error),
};
}
}
/**
* LLM-basierte PII-Erkennung via Ollama
*/
private async anonymizeWithLlm(text: string): Promise<PiiDetectionResult> {
const prompt = generatePrompt(text, this.config.model);
const response = await fetch(`${this.config.ollamaUrl}/api/chat`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: this.config.model,
messages: [
{ role: 'system', content: prompt.system },
{ role: 'user', content: prompt.user },
],
stream: false,
options: {
temperature: prompt.config.temperature,
num_predict: prompt.config.maxTokens,
},
}),
});
if (!response.ok) {
throw new Error(`Ollama API error: ${response.status} ${response.statusText}`);
}
const data = await response.json() as { message?: { content?: string } };
const content = data.message?.content || '';
return this.parseLlmResponse(content, text);
}
/**
* Parst die LLM-Antwort und extrahiert PII-Daten
*/
private parseLlmResponse(response: string, originalText: string): PiiDetectionResult {
const cleaned = cleanJsonResponse(response);
let parsed: unknown;
try {
parsed = JSON.parse(cleaned);
} catch {
// Versuche, JSON aus dem Text zu extrahieren
const jsonMatch = cleaned.match(/\{[\s\S]*\}/);
if (jsonMatch) {
try {
parsed = JSON.parse(jsonMatch[0]);
} catch (e) {
throw new Error(`Failed to parse LLM response: ${e}`);
}
} else {
throw new Error('No valid JSON found in LLM response');
}
}
if (!validatePiiResult(parsed)) {
throw new Error('Invalid PII result structure');
}
// Normalisiere PII-Liste
const piiFound: PiiInstance[] = parsed.pii_found.map((item: unknown, index: number) => {
const pii = item as Record<string, string>;
return {
type: pii.type as PiiType,
original: pii.original,
replacement: pii.replacement || generatePlaceholder(pii.type as PiiType, index + 1),
};
});
return {
piiFound,
anonymizedText: parsed.anonymized_text,
originalText,
};
}
/**
* Fallback: Pattern-basierte PII-Erkennung
* Wird verwendet, wenn LLM nicht verfügbar ist
*/
private anonymizeWithPatterns(text: string): PiiDetectionResult {
const piiFound: PiiInstance[] = [];
const replacements: Array<{ original: string; replacement: string }> = [];
// Pattern-basierte Erkennung für bekannte PII-Typen
const patterns: Array<{ type: PiiType; regex: RegExp }> = [
{ type: 'email', regex: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g },
{ type: 'phone', regex: /(?:\+49|0)[\s\-/]?\d{1,4}[\s\-/]?\d{1,4}[\s\-/]?\d{1,4}[\s\-/]?\d{0,4}\b/g },
{ type: 'iban', regex: /\b[A-Z]{2}\d{2}(?:[\s]?\d{4}){4,6}\b/g },
{ type: 'birthdate', regex: /\b(?:0?[1-9]|[12][0-9]|3[01])[\.\/\-](?:0?[1-9]|1[0-2])[\.\/\-](?:19|20)\d{2}\b/g },
{ type: 'credit_card', regex: /\b(?:\d{4}[\s-]?){3,4}\d{1,4}\b/g },
];
for (const { type, regex } of patterns) {
const matches = text.matchAll(regex);
let counter = 1;
for (const match of matches) {
const original = match[0];
// Vermeide Duplikate
if (replacements.some(r => r.original === original)) continue;
const placeholder = generatePlaceholder(type, counter++);
piiFound.push({ type, original, replacement: placeholder });
replacements.push({ original, replacement: placeholder });
}
}
const anonymizedText = replaceInText(text, replacements);
return {
piiFound,
anonymizedText,
originalText: text,
};
}
/**
* Baut ein Reverse-Mapping für die Re-Identifizierung
*/
private buildReverseMapping(piiFound: PiiInstance[]): ReverseMapping {
const mapping: ReverseMapping = {};
for (const pii of piiFound) {
mapping[pii.replacement] = pii;
}
return mapping;
}
/**
* Setzt den Placeholder-Zähler zurück
*/
resetCounters(): void {
this.placeholderCounters.clear();
}
}
/**
* Convenience-Funktion für einfache Anonymisierung
*/
export async function anonymizeText(
text: string,
config?: Partial<AnonymizerConfig>
): Promise<AnonymizeResult> {
const anonymizer = new Anonymizer(config);
return anonymizer.anonymize(text);
}
anonymizer/src/index.ts
+220
View File
@@ -0,0 +1,220 @@
/**
* Haupt-Export für den Anonymizer Service
* Einfache API für Integration in das Backend
*/
import { Anonymizer, anonymizeText, AnonymizeResult, AnonymizerConfig } from './anonymizer.js';
import {
reidentify,
ReidentifyResult,
ReidentifyOptions,
serializeMapping,
deserializeMapping,
validateMapping,
mergeMappings,
extractPlaceholders,
getMappingStats,
} from './reverser.js';
import {
PiiType,
PiiInstance,
PiiDetectionResult,
ReverseMapping,
PII_TYPE_METADATA,
PII_PATTERNS,
} from './pii-types.js';
import {
generatePrompt,
generateShortPrompt,
PromptConfig,
} from './prompts/pii-detection.js';
import {
generatePlaceholder,
replaceInText,
cleanJsonResponse,
calculateSensitivityLevel,
maskPiiForLog,
mightContainPii,
truncateText,
} from './utils/text.js';
// ==================== Haupt-API ====================
/**
* Einzige Funktion zum Anonymisieren von Text
* Wrapper für einfachen Zugriff aus dem Backend
*/
export async function anonymize(
text: string,
config?: Partial<AnonymizerConfig>
): Promise<AnonymizeResult> {
return anonymizeText(text, config);
}
/**
* Einzige Funktion zum Re-Identifizieren von Text
* Wrapper für einfachen Zugriff aus dem Backend
*/
export function reIdentify(
anonymizedText: string,
mapping: ReverseMapping,
options?: Partial<ReidentifyOptions>
): ReidentifyResult {
return reidentify(anonymizedText, mapping, options);
}
// ==================== Erweiterte API ====================
export interface PrivacyGatewayAnonymizer {
// Kern-Funktionen
anonymize(text: string): Promise<AnonymizeResult>;
reIdentify(text: string, mapping: ReverseMapping, options?: Partial<ReidentifyOptions>): ReidentifyResult;
// Hilfs-Funktionen
validateMapping(mapping: ReverseMapping): { valid: boolean; errors: string[] };
getMappingInfo(mapping: ReverseMapping): ReturnType<typeof getMappingStats>;
serialize(mapping: ReverseMapping): string;
deserialize(serialized: string): ReverseMapping;
}
/**
* Erstellt eine konfigurierte Anonymizer-Instanz
* Für fortgeschrittene Nutzung mit mehreren Konfigurationen
*/
export function createAnonymizer(config?: Partial<AnonymizerConfig>): PrivacyGatewayAnonymizer {
const instance = new Anonymizer(config);
return {
anonymize: (text: string) => instance.anonymize(text),
reIdentify: (text: string, mapping: ReverseMapping, options?: Partial<ReidentifyOptions>) =>
reidentify(text, mapping, options),
validateMapping: (mapping: ReverseMapping) => validateMapping(mapping),
getMappingInfo: (mapping: ReverseMapping) => getMappingStats(mapping),
serialize: (mapping: ReverseMapping) => serializeMapping(mapping),
deserialize: (serialized: string) => deserializeMapping(serialized),
};
}
// ==================== Integration für Backend ====================
/**
* Standard-Konfiguration für das Backend
*/
export const DEFAULT_BACKEND_CONFIG: AnonymizerConfig = {
ollamaUrl: process.env.OLLAMA_URL || 'http://192.168.2.122:11434',
model: process.env.OLLAMA_MODEL || 'llama3.2',
timeoutMs: parseInt(process.env.OLLAMA_TIMEOUT || '30000', 10),
maxRetries: 2,
fallbackEnabled: true,
};
/**
* Singleton-Instanz für das Backend
*/
let globalAnonymizer: PrivacyGatewayAnonymizer | null = null;
/**
* Gibt die globale Anonymizer-Instanz zurück
* Lazy-Initialisierung
*/
export function getAnonymizer(): PrivacyGatewayAnonymizer {
if (!globalAnonymizer) {
globalAnonymizer = createAnonymizer(DEFAULT_BACKEND_CONFIG);
}
return globalAnonymizer;
}
/**
* Setzt die globale Konfiguration zurück
* Nützlich für Tests oder Konfigurations-Updates
*/
export function resetAnonymizer(): void {
globalAnonymizer = null;
}
// ==================== Schnelle Hilfsfunktionen ====================
/**
* Prüft ob ein Text PII enthalten könnte (schneller Vorab-Check)
*/
export function quickCheck(text: string): boolean {
return mightContainPii(text);
}
/**
* Gibt Informationen über unterstützte PII-Typen zurück
*/
export function getSupportedPiiTypes(): Array<{
type: PiiType;
label: string;
description: string;
example: string;
sensitiveLevel: string;
}> {
return Object.values(PII_TYPE_METADATA).map(meta => ({
type: meta.type,
label: meta.label,
description: meta.description,
example: meta.example,
sensitiveLevel: meta.sensitiveLevel,
}));
}
/**
* Maskiert einen PII-Wert für Logging
*/
export function maskForLog(value: string, type: PiiType): string {
return maskPiiForLog(value, type);
}
// ==================== Type-Exports ====================
export {
// Haupt-Klassen
Anonymizer,
// Re-Identifizierung
reidentify,
serializeMapping,
deserializeMapping,
validateMapping,
mergeMappings,
extractPlaceholders,
getMappingStats,
// Prompts
generatePrompt,
generateShortPrompt,
// Utils
generatePlaceholder,
replaceInText,
cleanJsonResponse,
calculateSensitivityLevel,
truncateText,
// Types
AnonymizerConfig,
AnonymizeResult,
ReidentifyResult,
ReidentifyOptions,
PromptConfig,
PiiType,
PiiInstance,
PiiDetectionResult,
ReverseMapping,
PII_TYPE_METADATA,
PII_PATTERNS,
};
// ==================== Default Export ====================
export default {
anonymize,
reIdentify,
createAnonymizer,
getAnonymizer,
quickCheck,
getSupportedPiiTypes,
maskForLog,
};
anonymizer/src/pii-types.ts
+146
View File
@@ -0,0 +1,146 @@
/**
* PII-Typen Definitionen für das Privacy Gateway
* Definiert alle erkennbaren persönlichen Informationen und ihre Metadaten
*/
export type PiiType =
| 'name_person'
| 'name_company'
| 'address'
| 'email'
| 'phone'
| 'birthdate'
| 'account_number'
| 'iban'
| 'employee_id'
| 'credit_card';
export interface PiiInstance {
type: PiiType;
original: string;
replacement: string;
startIndex?: number;
endIndex?: number;
confidence?: number;
}
export interface PiiDetectionResult {
piiFound: PiiInstance[];
anonymizedText: string;
originalText: string;
}
export interface ReplacementMap {
[placeholder: string]: string;
}
export interface ReverseMapping {
[placeholder: string]: PiiInstance;
}
// Metadaten für jeden PII-Typ
export interface PiiTypeMetadata {
type: PiiType;
label: string;
description: string;
example: string;
prefix: string;
sensitiveLevel: 'low' | 'medium' | 'high' | 'critical';
}
export const PII_TYPE_METADATA: Record<PiiType, PiiTypeMetadata> = {
name_person: {
type: 'name_person',
label: 'Personenname',
description: 'Vor- und Nachname einer natürlichen Person',
example: 'Max Mustermann',
prefix: 'NAME',
sensitiveLevel: 'high',
},
name_company: {
type: 'name_company',
label: 'Firmenname',
description: 'Name eines Unternehmens oder Organisation',
example: 'Musterfirma GmbH',
prefix: 'COMPANY',
sensitiveLevel: 'medium',
},
address: {
type: 'address',
label: 'Adresse',
description: 'Straße, Hausnummer, PLZ und Ort',
example: 'Musterstraße 42, 12345 Berlin',
prefix: 'ADDRESS',
sensitiveLevel: 'high',
},
email: {
type: 'email',
label: 'E-Mail',
description: 'E-Mail-Adresse',
example: 'max@mustermann.de',
prefix: 'EMAIL',
sensitiveLevel: 'high',
},
phone: {
type: 'phone',
label: 'Telefonnummer',
description: 'Deutsche Telefonnummern (Mobil/Festnetz)',
example: '+49 170 12345678',
prefix: 'PHONE',
sensitiveLevel: 'medium',
},
birthdate: {
type: 'birthdate',
label: 'Geburtsdatum',
description: 'Geburtsdatum in verschiedenen Formaten',
example: '01.01.1990',
prefix: 'BIRTHDATE',
sensitiveLevel: 'high',
},
account_number: {
type: 'account_number',
label: 'Kontonummer',
description: 'Bankkontonummer (nicht IBAN)',
example: '1234567890',
prefix: 'ACCOUNT',
sensitiveLevel: 'critical',
},
iban: {
type: 'iban',
label: 'IBAN',
description: 'Internationale Bankkontonummer',
example: 'DE89 3704 0044 0532 0130 00',
prefix: 'IBAN',
sensitiveLevel: 'critical',
},
employee_id: {
type: 'employee_id',
label: 'Personalnummer',
description: 'Mitarbeiter- oder Personalnummer',
example: 'EMP-12345',
prefix: 'EMPID',
sensitiveLevel: 'medium',
},
credit_card: {
type: 'credit_card',
label: 'Kreditkarte',
description: 'Kreditkartennummer',
example: '4111 1111 1111 1111',
prefix: 'CC',
sensitiveLevel: 'critical',
},
};
// Regex-Patterns für Vorab-Validierung (zusätzlich zum LLM)
export const PII_PATTERNS: Record<PiiType, RegExp> = {
email: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g,
phone: /(?:\+49|0)[\s\-/]?[\d\s\-/]{7,15}\b/g,
iban: /\b[A-Z]{2}\d{2}[\s]?[\d]{4}[\s]?[\d]{4}[\s]?[\d]{4}[\s]?[\d]{4}[\s]?[\d]{0,4}\b/g,
credit_card: /\b(?:\d{4}[\s-]?){3,4}\d{1,4}\b/g,
birthdate: /\b(?:0?[1-9]|[12][0-9]|3[01])[\.\/\-](?:0?[1-9]|1[0-2])[\.\/\-](?:19|20)\d{2}\b/g,
account_number: /\b\d{8,12}\b/g,
employee_id: /\b(?:EMP|PN|MA)[\-\s]?\d{4,10}\b/gi,
name_person: /\b[A-ZÄÖÜ][a-zäöüß]+\s+[A-ZÄÖÜ][a-zäöüß]+\b/g,
name_company: /\b[A-ZÄÖÜ][a-zäöüß]+(?:\s+(?:GmbH|AG|KG|OHG|e\.?V\.?|UG|Ltd|Inc|Corp))\b/gi,
address: /\b[A-ZÄÖÜ][a-zäöüß]+(?:straße|str|weg|platz|allee|gasse)\s+\d+[\s,]*\d{5}\s+[A-ZÄÖÜ][a-zäöüß]+\b/gi,
};
anonymizer/src/prompts/pii-detection.ts
+172
View File
@@ -0,0 +1,172 @@
/**
* Prompt-Templates für PII-Erkennung
* Optimiert für lokale LLMs (Ollama)
*/
export interface PromptConfig {
model: string;
temperature: number;
maxTokens: number;
systemPrompt: string;
userPromptTemplate: string;
}
// System-Prompt: Definiert die Rolle und Regeln
export const SYSTEM_PROMPT = `Du bist ein spezialisierter Datenschutz-Assistent für PII-Erkennung (Personally Identifiable Information).
Deine Aufgabe ist es, persönliche Daten in Texten zu identifizieren und zu anonymisieren.
REGELN:
1. Identifiziere ALLE persönlichen Informationen im Text
2. Ersetze jede PII durch einen neutralen Platzhalter im Format [TYP_NUMMER]
3. Antworte NUR mit gültigem JSON - kein Markdown, keine Erklärungen
4. Sei gründlich: Es ist besser zu viel zu erkennen als zu wenig
5. Bei Namen: Unterscheide zwischen Personen- und Firmennamen
6. Bei Adressen: Erkenne komplette Adressen (Straße + Ort)
7. Bei Telefonnummern: Erkenne deutsche Formate (+49, 0...)
8. Bei Datumsangaben: Nur Geburtsdaten (nicht Termine)
PII-TYPEN:
- name_person: Vor- und Nachname einer Person (z.B. "Max Mustermann")
- name_company: Firmenname (z.B. "Musterfirma GmbH")
- address: Vollständige Adresse (z.B. "Musterstraße 1, 12345 Berlin")
- email: E-Mail-Adresse (z.B. "max@beispiel.de")
- phone: Telefonnummer (z.B. "+49 170 12345678")
- birthdate: Geburtsdatum (z.B. "15.03.1985")
- account_number: Kontonummer (z.B. "1234567890")
- iban: IBAN (z.B. "DE89 3704 0044 0532 0130 00")
- employee_id: Personalnummer (z.B. "EMP-12345")
- credit_card: Kreditkartennummer (z.B. "4111 1111 1111 1111")
WICHTIG: Der anonymisierte Text muss natürlich lesbar bleiben, nur die sensitiven Daten werden ersetzt.`;
// User-Prompt Template mit Platzhalter für den zu analysierenden Text
export const USER_PROMPT_TEMPLATE = `Analysiere den folgenden Text und identifiziere ALLE persönlichen Informationen (PII).
Für jede gefundene PII gib zurück:
- type: Der PII-Typ (siehe System-Regeln)
- original: Der originale Wert im Text
- replacement: Ein eindeutiger Platzhalter [TYP_INDEX]
ANTWORTE NUR mit diesem JSON-Format:
{
"pii_found": [
{
"type": "name_person",
"original": "Max Mustermann",
"replacement": "[NAME_1]"
}
],
"anonymized_text": "Hallo [NAME_1], ..."
}
Zu analysierender Text:
---
{{TEXT}}
---`;
// Erweiterte Version mit Beispielen für bessere Few-Shot Performance
export const USER_PROMPT_TEMPLATE_FEW_SHOT = `Analysiere den folgenden Text und identifiziere ALLE persönlichen Informationen (PII).
BEISPIELE:
Eingabe: "Herr Klaus Müller von der Firma Schmidt & Co GmbH wohnt in der Hauptstraße 42, 10115 Berlin. Erreichbar unter klaus.mueller@email.de oder 030-12345678."
Ausgabe:
{
"pii_found": [
{"type": "name_person", "original": "Klaus Müller", "replacement": "[NAME_1]"},
{"type": "name_company", "original": "Schmidt & Co GmbH", "replacement": "[COMPANY_1]"},
{"type": "address", "original": "Hauptstraße 42, 10115 Berlin", "replacement": "[ADDRESS_1]"},
{"type": "email", "original": "klaus.mueller@email.de", "replacement": "[EMAIL_1]"},
{"type": "phone", "original": "030-12345678", "replacement": "[PHONE_1]"}
],
"anonymized_text": "Herr [NAME_1] von der Firma [COMPANY_1] wohnt in der [ADDRESS_1]. Erreichbar unter [EMAIL_1] oder [PHONE_1]."
}
Eingabe: "Überweisung an Max Mustermann, IBAN: DE89 3704 0044 0532 0130 00, Geburtsdatum: 15.03.1985"
Ausgabe:
{
"pii_found": [
{"type": "name_person", "original": "Max Mustermann", "replacement": "[NAME_1]"},
{"type": "iban", "original": "DE89 3704 0044 0532 0130 00", "replacement": "[IBAN_1]"},
{"type": "birthdate", "original": "15.03.1985", "replacement": "[BIRTHDATE_1]"}
],
"anonymized_text": "Überweisung an [NAME_1], IBAN: [IBAN_1], Geburtsdatum: [BIRTHDATE_1]"
}
ANWENDUNG:
Eingabe: {{TEXT}}
Ausgabe:`;
// Konfiguration für verschiedene Modelle
export const MODEL_CONFIGS: Record<string, PromptConfig> = {
'llama3.2': {
model: 'llama3.2',
temperature: 0.1,
maxTokens: 4096,
systemPrompt: SYSTEM_PROMPT,
userPromptTemplate: USER_PROMPT_TEMPLATE,
},
'mistral': {
model: 'mistral',
temperature: 0.1,
maxTokens: 4096,
systemPrompt: SYSTEM_PROMPT,
userPromptTemplate: USER_PROMPT_TEMPLATE,
},
'qwen2.5': {
model: 'qwen2.5',
temperature: 0.1,
maxTokens: 4096,
systemPrompt: SYSTEM_PROMPT,
userPromptTemplate: USER_PROMPT_TEMPLATE_FEW_SHOT,
},
'gemma2': {
model: 'gemma2',
temperature: 0.1,
maxTokens: 4096,
systemPrompt: SYSTEM_PROMPT,
userPromptTemplate: USER_PROMPT_TEMPLATE,
},
'phi4': {
model: 'phi4',
temperature: 0.1,
maxTokens: 4096,
systemPrompt: SYSTEM_PROMPT,
userPromptTemplate: USER_PROMPT_TEMPLATE_FEW_SHOT,
},
'default': {
model: 'llama3.2',
temperature: 0.1,
maxTokens: 4096,
systemPrompt: SYSTEM_PROMPT,
userPromptTemplate: USER_PROMPT_TEMPLATE,
},
};
// Hilfsfunktion zum Generieren des Prompts
export function generatePrompt(
text: string,
modelName: string = 'default'
): { system: string; user: string; config: PromptConfig } {
const config = MODEL_CONFIGS[modelName] || MODEL_CONFIGS.default;
const userPrompt = config.userPromptTemplate.replace('{{TEXT}}', text);
return {
system: config.systemPrompt,
user: userPrompt,
config,
};
}
// Optimierter Prompt für sehr kurze Texte (schnellere Verarbeitung)
export const SHORT_TEXT_SYSTEM_PROMPT = `Du bist ein PII-Erkennungs-Assistent. Erkenne alle persönlichen Daten.
Antworte nur mit JSON: {"pii_found":[],"anonymized_text":""}
Typen: name_person,name_company,address,email,phone,birthdate,account_number,iban,employee_id,credit_card`;
export function generateShortPrompt(text: string): { system: string; user: string } {
return {
system: SHORT_TEXT_SYSTEM_PROMPT,
user: `Text: "${text}"\nJSON:`,
};
}
anonymizer/src/reverser.ts
+261
View File
@@ -0,0 +1,261 @@
/**
* Re-Identifizierung (De-Anonymisierung)
* Stellt anonymisierte Texte wieder her
*/
import { PiiInstance, ReverseMapping, PII_TYPE_METADATA, PiiType } from './pii-types.js';
import { extractTypeFromPlaceholder, extractIndexFromPlaceholder } from './utils/text.js';
export interface ReidentifyResult {
success: boolean;
reidentifiedText: string;
replacementsMade: number;
errors: string[];
}
export interface ReidentifyOptions {
strictMode: boolean; // Bei true: Fehler wenn Platzhalter nicht gefunden
partialMatch: boolean; // Bei true: Auch teilweise Übereinstimmungen
caseSensitive: boolean; // Bei false: Groß-/Kleinschreibung ignorieren
}
export const DEFAULT_REIDENTIFY_OPTIONS: ReidentifyOptions = {
strictMode: false,
partialMatch: false,
caseSensitive: true,
};
/**
* Re-Identifiziert einen anonymisierten Text
* Ersetzt Platzhalter durch Originalwerte
*/
export function reidentify(
anonymizedText: string,
mapping: ReverseMapping,
options: Partial<ReidentifyOptions> = {}
): ReidentifyResult {
const opts = { ...DEFAULT_REIDENTIFY_OPTIONS, ...options };
const errors: string[] = [];
let reidentifiedText = anonymizedText;
let replacementsMade = 0;
// Sortiere Platzhalter nach Länge (absteigend), damit längere zuerst ersetzt werden
const placeholders = Object.keys(mapping).sort((a, b) => b.length - a.length);
for (const placeholder of placeholders) {
const pii = mapping[placeholder];
if (!pii || !pii.original) {
errors.push(`Missing original value for placeholder: ${placeholder}`);
if (opts.strictMode) {
return {
success: false,
reidentifiedText: anonymizedText,
replacementsMade,
errors,
};
}
continue;
}
// Ersetze Platzhalter durch Original
const escapedPlaceholder = placeholder.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const regex = new RegExp(
escapedPlaceholder,
opts.caseSensitive ? 'g' : 'gi'
);
const matches = reidentifiedText.match(regex);
if (matches) {
reidentifiedText = reidentifiedText.replace(regex, pii.original);
replacementsMade += matches.length;
} else if (opts.strictMode) {
errors.push(`Placeholder not found in text: ${placeholder}`);
}
}
return {
success: errors.length === 0 || !opts.strictMode,
reidentifiedText,
replacementsMade,
errors,
};
}
/**
* Inkrementelle Re-Identifizierung
* Fügt neue Ersetzungen zu bestehendem Text hinzu
*/
export function reidentifyIncremental(
currentText: string,
additionalMapping: ReverseMapping,
options?: Partial<ReidentifyOptions>
): ReidentifyResult {
return reidentify(currentText, additionalMapping, options);
}
/**
* Erstellt eine serialisierbare Version des Mappings
* Für Speicherung oder Übertragung
*/
export function serializeMapping(mapping: ReverseMapping): string {
// Entferne nicht-serialisierbare Felder
const serializable: Record<string, { type: string; original: string; replacement: string }> = {};
for (const [placeholder, pii] of Object.entries(mapping)) {
serializable[placeholder] = {
type: pii.type,
original: pii.original,
replacement: pii.replacement,
};
}
return JSON.stringify(serializable, null, 2);
}
/**
* Deserialisiert ein Mapping aus einem String
*/
export function deserializeMapping(serialized: string): ReverseMapping {
const parsed = JSON.parse(serialized) as Record<string, { type: string; original: string; replacement: string }>;
const mapping: ReverseMapping = {};
for (const [placeholder, data] of Object.entries(parsed)) {
mapping[placeholder] = {
type: data.type as PiiType,
original: data.original,
replacement: data.replacement,
};
}
return mapping;
}
/**
* Validiert ein Mapping
* Prüft auf Konsistenz und Vollständigkeit
*/
export function validateMapping(mapping: ReverseMapping): { valid: boolean; errors: string[] } {
const errors: string[] = [];
for (const [placeholder, pii] of Object.entries(mapping)) {
// Prüfe ob Platzhalter-Format korrekt
if (!placeholder.match(/^\[[A-Z_]+_\d+\]$/)) {
errors.push(`Invalid placeholder format: ${placeholder}`);
}
// Prüfe ob PII-Typ bekannt
if (!PII_TYPE_METADATA[pii.type]) {
errors.push(`Unknown PII type "${pii.type}" for placeholder ${placeholder}`);
}
// Prüfe ob Original-Wert vorhanden
if (!pii.original || pii.original.trim() === '') {
errors.push(`Missing or empty original value for placeholder ${placeholder}`);
}
// Prüfe ob Platzhalter mit PII.replacement übereinstimmt
if (pii.replacement !== placeholder) {
errors.push(`Mismatch: placeholder "${placeholder}" vs replacement "${pii.replacement}"`);
}
}
return { valid: errors.length === 0, errors };
}
/**
* Mergt mehrere Mappings zu einem
* Bei Konflikten: spätere Mappings überschreiben frühere
*/
export function mergeMappings(...mappings: ReverseMapping[]): ReverseMapping {
const merged: ReverseMapping = {};
for (const mapping of mappings) {
for (const [placeholder, pii] of Object.entries(mapping)) {
merged[placeholder] = pii;
}
}
return merged;
}
/**
* Extrahiert Platzhalter aus einem anonymisierten Text
*/
export function extractPlaceholders(text: string): string[] {
const matches = text.match(/\[[A-Z_]+_\d+\]/g);
return matches ? [...new Set(matches)] : [];
}
/**
* Erstellt ein partielles Mapping basierend auf gefundenen Platzhaltern
* Nützlich wenn man nur bestimmte PII-Typen re-identifizieren möchte
*/
export function filterMappingByTypes(
mapping: ReverseMapping,
types: PiiType[]
): ReverseMapping {
const filtered: ReverseMapping = {};
for (const [placeholder, pii] of Object.entries(mapping)) {
if (types.includes(pii.type)) {
filtered[placeholder] = pii;
}
}
return filtered;
}
/**
* Erstellt ein partielles Mapping basierend auf Placeholder-Präfix
*/
export function filterMappingByPlaceholder(
mapping: ReverseMapping,
prefix: string
): ReverseMapping {
const filtered: ReverseMapping = {};
for (const [placeholder, pii] of Object.entries(mapping)) {
if (placeholder.startsWith(`[${prefix}_`)) {
filtered[placeholder] = pii;
}
}
return filtered;
}
/**
* Berechnet Statistiken über ein Mapping
*/
export function getMappingStats(mapping: ReverseMapping): {
totalPlaceholders: number;
byType: Record<string, number>;
sensitiveLevel: 'low' | 'medium' | 'high' | 'critical';
} {
const byType: Record<string, number> = {};
let criticalCount = 0;
let highCount = 0;
for (const pii of Object.values(mapping)) {
byType[pii.type] = (byType[pii.type] || 0) + 1;
if (pii.type === 'credit_card' || pii.type === 'iban') {
criticalCount++;
} else if (['name_person', 'address', 'birthdate', 'email'].includes(pii.type)) {
highCount++;
}
}
const totalPlaceholders = Object.keys(mapping).length;
let sensitiveLevel: 'low' | 'medium' | 'high' | 'critical' = 'low';
if (criticalCount > 0) sensitiveLevel = 'critical';
else if (highCount >= 3 || totalPlaceholders >= 5) sensitiveLevel = 'high';
else if (highCount > 0 || totalPlaceholders >= 2) sensitiveLevel = 'medium';
return {
totalPlaceholders,
byType,
sensitiveLevel,
};
}
anonymizer/src/test/test-cases.ts
+370
View File
@@ -0,0 +1,370 @@
/**
* Testfälle für das PII-Anonymisierungs-System
* Umfassende Testabdeckung für verschiedene Szenarien
*/
import { PiiDetectionResult, ReverseMapping } from '../pii-types.js';
export interface TestCase {
name: string;
description: string;
input: string;
expectedPii: Array<{
type: string;
original: string;
}>;
shouldAnonymize: boolean;
category: 'basic' | 'advanced' | 'edge' | 'realworld';
}
export interface TestResult {
testCase: TestCase;
success: boolean;
anonymizedText?: string;
detectedPii?: PiiDetectionResult['piiFound'];
reidentifiedText?: string;
errors: string[];
}
// Basis-Testfälle
export const BASIC_TEST_CASES: TestCase[] = [
{
name: 'einfache_email',
description: 'Einfache E-Mail-Adresse',
input: 'Kontaktieren Sie mich unter max.mustermann@beispiel.de',
expectedPii: [
{ type: 'email', original: 'max.mustermann@beispiel.de' },
],
shouldAnonymize: true,
category: 'basic',
},
{
name: 'deutsche_telefonnummer',
description: 'Deutsche Telefonnummer mit Vorwahl',
input: 'Rufen Sie mich an: 030-12345678 oder +49 170 12345678',
expectedPii: [
{ type: 'phone', original: '030-12345678' },
{ type: 'phone', original: '+49 170 12345678' },
],
shouldAnonymize: true,
category: 'basic',
},
{
name: 'personenname',
description: 'Vollständiger Personenname',
input: 'Herr Dr. Klaus Müller wird Sie betreuen.',
expectedPii: [
{ type: 'name_person', original: 'Klaus Müller' },
],
shouldAnonymize: true,
category: 'basic',
},
{
name: 'vollstaendige_adresse',
description: 'Vollständige deutsche Adresse',
input: 'Meine Adresse lautet: Musterstraße 42, 10115 Berlin',
expectedPii: [
{ type: 'address', original: 'Musterstraße 42, 10115 Berlin' },
],
shouldAnonymize: true,
category: 'basic',
},
{
name: 'iban',
description: 'Deutsche IBAN',
input: 'Bitte überweisen Sie auf DE89 3704 0044 0532 0130 00',
expectedPii: [
{ type: 'iban', original: 'DE89 3704 0044 0532 0130 00' },
],
shouldAnonymize: true,
category: 'basic',
},
{
name: 'geburtsdatum',
description: 'Geburtsdatum verschiedener Formate',
input: 'Geboren am 15.03.1985, zuvor am 15/03/1985',
expectedPii: [
{ type: 'birthdate', original: '15.03.1985' },
],
shouldAnonymize: true,
category: 'basic',
},
];
// Erweiterte Testfälle
export const ADVANCED_TEST_CASES: TestCase[] = [
{
name: 'kombination_mehrere_pii',
description: 'Mehrere PII-Typen in einem Text',
input: 'Hallo, ich bin Anna Schmidt von der Firma Tech Solutions GmbH. \
Sie erreichen mich unter anna.schmidt@tech-solutions.de oder +49 89 12345678. \
Meine Adresse: Hauptstraße 1, 80331 München. IBAN: DE12 3456 7890 1234 5678 90',
expectedPii: [
{ type: 'name_person', original: 'Anna Schmidt' },
{ type: 'name_company', original: 'Tech Solutions GmbH' },
{ type: 'email', original: 'anna.schmidt@tech-solutions.de' },
{ type: 'phone', original: '+49 89 12345678' },
{ type: 'address', original: 'Hauptstraße 1, 80331 München' },
{ type: 'iban', original: 'DE12 3456 7890 1234 5678 90' },
],
shouldAnonymize: true,
category: 'advanced',
},
{
name: 'kreditkarte',
description: 'Kreditkartennummer',
input: 'Meine Karte: 4111 1111 1111 1111 läuft ab 12/25',
expectedPii: [
{ type: 'credit_card', original: '4111 1111 1111 1111' },
],
shouldAnonymize: true,
category: 'advanced',
},
{
name: 'personalnummer',
description: 'Personalnummer verschiedener Formate',
input: 'Meine Personalnummer ist EMP-12345 oder MA-98765',
expectedPii: [
{ type: 'employee_id', original: 'EMP-12345' },
],
shouldAnonymize: true,
category: 'advanced',
},
{
name: 'kontonummer',
description: 'Alte Kontonummer (nicht IBAN)',
input: 'Bitte überweisen Sie auf Konto 1234567890, BLZ 70090100',
expectedPii: [
{ type: 'account_number', original: '1234567890' },
],
shouldAnonymize: true,
category: 'advanced',
},
];
// Edge Cases
export const EDGE_TEST_CASES: TestCase[] = [
{
name: 'keine_pii',
description: 'Text ohne erkennbare PII',
input: 'Das Wetter ist heute wunderschön und ich plane einen Spaziergang.',
expectedPii: [],
shouldAnonymize: false,
category: 'edge',
},
{
name: 'leerer_text',
description: 'Leerer Eingabetext',
input: '',
expectedPii: [],
shouldAnonymize: false,
category: 'edge',
},
{
name: 'nur_whitespace',
description: 'Nur Leerzeichen',
input: ' \n\t ',
expectedPii: [],
shouldAnonymize: false,
category: 'edge',
},
{
name: 'sehr_langer_text',
description: 'Sehr langer Text mit verstreuten PII',
input: 'Lorem ipsum '.repeat(100) + ' max.mustermann@beispiel.de ' + 'dolor sit '.repeat(100),
expectedPii: [
{ type: 'email', original: 'max.mustermann@beispiel.de' },
],
shouldAnonymize: true,
category: 'edge',
},
{
name: 'gleiche_pii_mehrfach',
description: 'Selbe E-Mail mehrfach im Text',
input: 'Schreiben Sie an kontakt@firma.de oder kontakt@firma.de',
expectedPii: [
{ type: 'email', original: 'kontakt@firma.de' },
],
shouldAnonymize: true,
category: 'edge',
},
{
name: 'aehnliche_worte',
description: 'Wörter die wie PII aussehen aber keine sind',
input: 'Das Passwort lautet test@123 und die Adresse ist localhost',
expectedPii: [],
shouldAnonymize: false,
category: 'edge',
},
];
// Real-World Szenarien
export const REALWORLD_TEST_CASES: TestCase[] = [
{
name: 'support_ticket',
description: 'Typischer Support-Chat mit Kundendaten',
input: `Kunde: Hallo, ich habe ein Problem mit meinem Konto.
Support: Guten Tag, wie kann ich helfen?
Kunde: Mein Name ist Sabine Weber, ich wohne in Berlin und meine Mail ist s.weber@web.de
Support: Danke Frau Weber, ich schaue mir das an.`,
expectedPii: [
{ type: 'name_person', original: 'Sabine Weber' },
{ type: 'email', original: 's.weber@web.de' },
],
shouldAnonymize: true,
category: 'realworld',
},
{
name: 'rechnung_kontext',
description: 'Rechnungsinformationen',
input: `Rechnung Nr. 2024-001 an
Herrn Peter Schmidt
Musterweg 15
20095 Hamburg
Betrag: 1.234,56 EUR
IBAN: DE44 2001 0020 0123 4567 89`,
expectedPii: [
{ type: 'name_person', original: 'Peter Schmidt' },
{ type: 'address', original: 'Musterweg 15, 20095 Hamburg' },
{ type: 'iban', original: 'DE44 2001 0020 0123 4567 89' },
],
shouldAnonymize: true,
category: 'realworld',
},
{
name: 'terminplanung',
description: 'Terminplanung mit Kontaktdaten',
input: `Hallo Frau Dr. Angela Müller,
wir haben Ihren Termin am 15.03.2024 um 14:30 Uhr bestätigt.
Bei Fragen erreichen Sie uns unter:
- E-Mail: praxis@mueller-med.de
- Telefon: 089 / 12345678
Mit freundlichen Grüßen
Praxis Dr. Müller`,
expectedPii: [
{ type: 'name_person', original: 'Angela Müller' },
{ type: 'email', original: 'praxis@mueller-med.de' },
{ type: 'phone', original: '089 / 12345678' },
],
shouldAnonymize: true,
category: 'realworld',
},
];
// Alle Testfälle zusammen
export const ALL_TEST_CASES: TestCase[] = [
...BASIC_TEST_CASES,
...ADVANCED_TEST_CASES,
...EDGE_TEST_CASES,
...REALWORLD_TEST_CASES,
];
/**
* Validiert ein Anonymisierungsergebnis gegen erwartete Werte
*/
export function validateResult(
result: PiiDetectionResult,
testCase: TestCase
): { valid: boolean; errors: string[] } {
const errors: string[] = [];
// Prüfe ob anonymisiert wurde wenn erwartet
if (testCase.shouldAnonymize && result.anonymizedText === testCase.input) {
errors.push('Text wurde nicht anonymisiert');
}
// Prüfe ob alle erwarteten PII gefunden wurden
for (const expected of testCase.expectedPii) {
const found = result.piiFound.some(
p => p.type === expected.type && p.original === expected.original
);
if (!found) {
errors.push(`Erwartete PII nicht gefunden: ${expected.type} = "${expected.original}"`);
}
}
// Prüfe ob zu viele PII gefunden wurden (False Positives)
if (result.piiFound.length > testCase.expectedPii.length) {
const extra = result.piiFound.filter(
p => !testCase.expectedPii.some(e => e.original === p.original)
);
errors.push(`Unerwartete PII gefunden: ${extra.map(p => `${p.type}="${p.original}"`).join(', ')}`);
}
// Prüfe Re-Identifizierung
const reidentified = result.anonymizedText;
for (const pii of result.piiFound) {
if (reidentified.includes(pii.original)) {
errors.push(`Originalwert "${pii.original}" noch im anonymisierten Text enthalten`);
}
}
return { valid: errors.length === 0, errors };
}
/**
* Führt einen Testfall aus
*/
export async function runTestCase(
testCase: TestCase,
anonymizeFn: (text: string) => Promise<PiiDetectionResult>,
reidentifyFn: (text: string, mapping: ReverseMapping) => string
): Promise<TestResult> {
const errors: string[] = [];
try {
// Anonymisiere
const result = await anonymizeFn(testCase.input);
// Validiere
const validation = validateResult(result, testCase);
errors.push(...validation.errors);
// Re-Identifiziere
const mapping: ReverseMapping = {};
for (const pii of result.piiFound) {
mapping[pii.replacement] = pii;
}
const reidentifiedText = reidentifyFn(result.anonymizedText, mapping);
// Prüfe ob Re-Identifizierung erfolgreich
if (reidentifiedText !== testCase.input) {
errors.push('Re-Identifizierung ergab nicht den Originaltext');
}
return {
testCase,
success: errors.length === 0,
anonymizedText: result.anonymizedText,
detectedPii: result.piiFound,
reidentifiedText,
errors,
};
} catch (error) {
errors.push(`Exception: ${error instanceof Error ? error.message : String(error)}`);
return {
testCase,
success: false,
errors,
};
}
}
// Beispiel-Mapping für manuelle Tests
export const SAMPLE_MAPPING: ReverseMapping = {
'[NAME_1]': { type: 'name_person', original: 'Max Mustermann', replacement: '[NAME_1]' },
'[EMAIL_1]': { type: 'email', original: 'max@beispiel.de', replacement: '[EMAIL_1]' },
'[PHONE_1]': { type: 'phone', original: '030-12345678', replacement: '[PHONE_1]' },
'[ADDRESS_1]': { type: 'address', original: 'Musterstraße 1, 12345 Berlin', replacement: '[ADDRESS_1]' },
'[IBAN_1]': { type: 'iban', original: 'DE89 3704 0044 0532 0130 00', replacement: '[IBAN_1]' },
'[COMPANY_1]': { type: 'name_company', original: 'Musterfirma GmbH', replacement: '[COMPANY_1]' },
};
export const SAMPLE_ANONYMIZED_TEXT =
'Hallo [NAME_1], kontaktiere mich unter [EMAIL_1] oder [PHONE_1]. \
Ich wohne in [ADDRESS_1]. Überweise an [IBAN_1]. Arbeite bei [COMPANY_1].';
anonymizer/src/utils/text.ts
+158
View File
@@ -0,0 +1,158 @@
/**
* Hilfsfunktionen für Textverarbeitung und PII-Handling
*/
import { PiiInstance, PiiType, PII_TYPE_METADATA } from '../pii-types.js';
/**
* Generiert einen eindeutigen Platzhalter für einen PII-Typ
*/
export function generatePlaceholder(type: PiiType, index: number): string {
const metadata = PII_TYPE_METADATA[type];
return `[${metadata.prefix}_${index}]`;
}
/**
* Extrahiert den Typ aus einem Platzhalter
* z.B. "[NAME_1]" -> "name_person"
*/
export function extractTypeFromPlaceholder(placeholder: string): PiiType | null {
const match = placeholder.match(/\[([A-Z_]+)_\d+\]/);
if (!match) return null;
const prefix = match[1];
// Reverse lookup vom Prefix zum Typ
for (const [type, metadata] of Object.entries(PII_TYPE_METADATA)) {
if (metadata.prefix === prefix) {
return type as PiiType;
}
}
return null;
}
/**
* Extrahiert den Index aus einem Platzhalter
* z.B. "[NAME_1]" -> 1
*/
export function extractIndexFromPlaceholder(placeholder: string): number {
const match = placeholder.match(/\[[A-Z_]+_(\d+)\]/);
return match ? parseInt(match[1], 10) : 0;
}
/**
* Ersetzt alle Vorkommen eines Strings in einem Text
* Beachtet: Längere Strings zuerst ersetzen, um Teil-Überschneidungen zu vermeiden
*/
export function replaceInText(
text: string,
replacements: Array<{ original: string; replacement: string }>
): string {
// Sortiere nach Länge (absteigend), damit längere Strings zuerst ersetzt werden
const sorted = [...replacements].sort((a, b) => b.original.length - a.original.length);
let result = text;
for (const { original, replacement } of sorted) {
// Escape spezielle Regex-Zeichen im Original
const escaped = original.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
// Globale Ersetzung (case-sensitive für Präzision)
result = result.replace(new RegExp(escaped, 'g'), replacement);
}
return result;
}
/**
* Bereinigt JSON-String von Markdown-Code-Blocks und Whitespace
*/
export function cleanJsonResponse(response: string): string {
return response
.replace(/```json\n?/g, '')
.replace(/```\n?/g, '')
.replace(/^\s+|\s+$/g, '');
}
/**
* Validiert und repariert ein PII-Ergebnis
* Stellt sicher, dass alle Felder vorhanden sind
*/
export function validatePiiResult(result: unknown): result is { pii_found: unknown[]; anonymized_text: string } {
if (typeof result !== 'object' || result === null) return false;
const r = result as Record<string, unknown>;
return Array.isArray(r.pii_found) && typeof r.anonymized_text === 'string';
}
/**
* Berechnet die Sensitivitätsstufe eines Textes
* Basierend auf der Anzahl und Art der erkannten PII
*/
export function calculateSensitivityLevel(piiList: PiiInstance[]): 'low' | 'medium' | 'high' | 'critical' {
if (piiList.length === 0) return 'low';
const criticalCount = piiList.filter(p => p.type === 'credit_card' || p.type === 'iban').length;
const highCount = piiList.filter(p =>
p.type === 'name_person' ||
p.type === 'address' ||
p.type === 'birthdate' ||
p.type === 'email'
).length;
if (criticalCount > 0) return 'critical';
if (highCount >= 3 || piiList.length >= 5) return 'high';
if (highCount > 0 || piiList.length >= 2) return 'medium';
return 'low';
}
/**
* Maskiert einen PII-Wert für Logging/Anzeige
* z.B. "max.mustermann@email.de" -> "m***@e***.de"
*/
export function maskPiiForLog(value: string, type: PiiType): string {
if (value.length <= 4) return '****';
if (type === 'email') {
const [local, domain] = value.split('@');
if (!domain) return value.substring(0, 2) + '***';
const [domainName, tld] = domain.split('.');
return `${local.substring(0, 1)}***@${domainName?.substring(0, 1) ?? ''}***.${tld ?? ''}`;
}
if (type === 'phone' || type === 'credit_card' || type === 'iban') {
return value.substring(0, 4) + ' **** **** ' + value.substring(value.length - 4);
}
// Default: Erste 2 und letzte 2 Zeichen
return value.substring(0, 2) + '***' + value.substring(value.length - 2);
}
/**
* Zählt die Vorkommen eines Substrings (case-sensitive)
*/
export function countOccurrences(text: string, search: string): number {
if (!search) return 0;
const matches = text.match(new RegExp(search.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'g'));
return matches?.length ?? 0;
}
/**
* Prüft, ob ein Text potenziell PII enthält (schneller Vorab-Check)
*/
export function mightContainPii(text: string): boolean {
// Schnelle Heuristik: Enthält E-Mail, viele Zahlen oder bestimmte Schlüsselwörter?
const hasEmail = text.includes('@');
const hasPhone = /\b(?:\+49|0)[\d\s\-/]{7,}\b/.test(text);
const hasDate = /\b\d{1,2}[\.\/\-]\d{1,2}[\.\/\-]\d{2,4}\b/.test(text);
const hasIban = /\b[A-Z]{2}\d{2}\b/.test(text);
return hasEmail || hasPhone || hasDate || hasIban;
}
/**
* Truncates text for display purposes
*/
export function truncateText(text: string, maxLength: number = 100): string {
if (text.length <= maxLength) return text;
return text.substring(0, maxLength - 3) + '...';
}
anonymizer/tsconfig.json
+26
View File
@@ -0,0 +1,26 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "bundler",
"lib": ["ES2022"],
"outDir": "./dist",
"rootDir": "./src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"declaration": true,
"declarationMap": true,
"sourceMap": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"isolatedModules": true,
"verbatimModuleSyntax": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
backend/.env.example
+25
View File
@@ -0,0 +1,25 @@
# Database
DB_HOST=localhost
DB_PORT=5432
DB_NAME=privacy_gateway
DB_USER=postgres
DB_PASSWORD=your_password
# Redis
REDIS_HOST=localhost
REDIS_PORT=6379
# Ollama (Local Anonymizer)
OLLAMA_HOST=localhost
OLLAMA_PORT=11434
OLLAMA_TARGET_HOST=localhost
OLLAMA_TARGET_PORT=11434
# Models
ANONYMIZATION_MODEL=llama3.2
CHAT_MODEL=llama3.2
# Server
PORT=3000
NODE_ENV=development
LOG_LEVEL=info
backend/Dockerfile
+36
View File
@@ -0,0 +1,36 @@
# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY tsconfig.json ./
COPY src ./src
RUN npm run build
# Production stage
FROM node:20-alpine AS production
WORKDIR /app
RUN apk add --no-cache ca-certificates
ENV NODE_ENV=production
ENV PORT=3000
COPY package*.json ./
RUN npm ci --only=production && npm cache clean --force
COPY --from=builder /app/dist ./dist
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1
USER node
CMD ["node", "dist/server.js"]
backend/README.md
+74
View File
@@ -0,0 +1,74 @@
# Privacy Gateway Backend
Node.js/Express API mit Ollama-Proxy und Session-Management für den Privacy Gateway.
## Features
- **Session-Management**: Erstellen, Lesen, Löschen von Chat-Sessions
- **PII-Anonymisierung**: Automatische Erkennung und Maskierung sensibler Daten
- **Ollama-Proxy**: Transparente Weiterleitung zu externen KI-Modellen
- **Streaming**: SSE-basierte Antwort-Streams
- **Caching**: Redis-basierte Performance-Optimierung
- **PostgreSQL**: Persistente Datenspeicherung
## Schnelleinstieg
```bash
# 1. Umgebungsvariablen konfigurieren
cp .env.example .env
# Bearbeite .env mit deinen Daten
# 2. Mit Docker Compose starten
docker-compose up -d
# 3. Ollama-Modell herunterladen
docker-compose exec anonymizer ollama pull llama3.2
```
## API-Endpunkte
| Methode | Endpunkt | Beschreibung |
|---------|----------|--------------|
| GET | `/api/sessions` | Alle Sessions abrufen |
| POST | `/api/sessions` | Neue Session erstellen |
| GET | `/api/sessions/:id` | Session mit Messages |
| DELETE | `/api/sessions/:id` | Session löschen |
| POST | `/api/sessions/:id/chat` | Chat-Nachricht senden |
| GET | `/api/models` | Verfügbare Modelle |
| GET | `/health` | Health Check |
| GET | `/ready` | Readiness Check |
## Entwicklung
```bash
# Lokale Installation
npm install
npm run dev
# Build
npm run build
npm start
```
## Architektur
```
Client → Express API → [Anonymizer (Ollama)] → Externe KI
↓
PostgreSQL + Redis
```
## PII-Typen
- `PERSON`: Personennamen
- `EMAIL`: E-Mail-Adressen
- `PHONE`: Telefonnummern
- `ADDRESS`: Adressen
- `ORG`: Organisationen
- `ID`: Identifikationsnummern
- `DATE`: Persönliche Daten
- `FINANCIAL`: Bank-/Kreditkarten-Daten
## Lizenz
Proprietär - Täger IT & Gebäude-Systeme
backend/docker-compose.yml
+98
View File
@@ -0,0 +1,98 @@
version: '3.8'
services:
backend:
build:
context: .
dockerfile: Dockerfile
container_name: privacy-gateway-backend
restart: unless-stopped
ports:
- "${PORT:-3000}:3000"
environment:
- NODE_ENV=${NODE_ENV:-production}
- PORT=3000
- DB_HOST=postgres
- DB_PORT=5432
- DB_NAME=${DB_NAME:-privacy_gateway}
- DB_USER=${DB_USER:-postgres}
- DB_PASSWORD=${DB_PASSWORD}
- REDIS_HOST=redis
- REDIS_PORT=6379
- OLLAMA_HOST=${OLLAMA_HOST:-anonymizer}
- OLLAMA_PORT=${OLLAMA_PORT:-11434}
- OLLAMA_TARGET_HOST=${OLLAMA_TARGET_HOST:-ollama}
- OLLAMA_TARGET_PORT=${OLLAMA_TARGET_PORT:-11434}
- ANONYMIZATION_MODEL=${ANONYMIZATION_MODEL:-llama3.2}
- CHAT_MODEL=${CHAT_MODEL:-llama3.2}
- LOG_LEVEL=${LOG_LEVEL:-info}
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
anonymizer:
condition: service_started
networks:
- privacy-gateway
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
postgres:
image: postgres:16-alpine
container_name: privacy-gateway-postgres
restart: unless-stopped
environment:
- POSTGRES_DB=${DB_NAME:-privacy_gateway}
- POSTGRES_USER=${DB_USER:-postgres}
- POSTGRES_PASSWORD=${DB_PASSWORD}
volumes:
- postgres_data:/var/lib/postgresql/data
networks:
- privacy-gateway
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${DB_USER:-postgres}"]
interval: 5s
timeout: 5s
retries: 5
redis:
image: redis:7-alpine
container_name: privacy-gateway-redis
restart: unless-stopped
volumes:
- redis_data:/data
networks:
- privacy-gateway
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 5
anonymizer:
image: ollama/ollama:latest
container_name: privacy-gateway-anonymizer
restart: unless-stopped
environment:
- OLLAMA_HOST=0.0.0.0
- OLLAMA_PORT=11434
volumes:
- anonymizer_models:/root/.ollama
networks:
- privacy-gateway
# Remove GPU access for broader compatibility
command: serve
volumes:
postgres_data:
redis_data:
anonymizer_models:
networks:
privacy-gateway:
driver: bridge
backend/package.json
+42
View File
@@ -0,0 +1,42 @@
{
"name": "privacy-gateway-backend",
"version": "1.0.0",
"description": "Privacy Gateway Backend - Ollama Proxy mit Anonymisierung",
"main": "dist/server.js",
"scripts": {
"build": "tsc",
"start": "node dist/server.js",
"dev": "tsx src/server.ts",
"watch": "tsx watch src/server.ts",
"lint": "eslint src --ext .ts",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"express": "^4.18.2",
"cors": "^2.8.5",
"helmet": "^7.1.0",
"dotenv": "^16.3.1",
"pg": "^8.11.3",
"redis": "^4.6.12",
"uuid": "^9.0.1",
"express-validator": "^7.0.1",
"morgan": "^1.10.0",
"winston": "^3.11.0"
},
"devDependencies": {
"@types/express": "^4.17.21",
"@types/cors": "^2.8.17",
"@types/node": "^20.10.5",
"@types/pg": "^8.10.9",
"@types/uuid": "^9.0.7",
"@types/morgan": "^1.9.9",
"typescript": "^5.3.3",
"tsx": "^4.7.0",
"eslint": "^8.56.0",
"@typescript-eslint/eslint-plugin": "^6.15.0",
"@typescript-eslint/parser": "^6.15.0"
},
"engines": {
"node": ">=18.0.0"
}
}
backend/src/db/index.ts
+236
View File
@@ -0,0 +1,236 @@
import { Pool, PoolConfig, QueryResult } from 'pg';
import { Logger } from 'winston';
import { createLogger } from '../utils/logger';
import { Session, Message, PIIMapping } from '../types';
const logger: Logger = createLogger('Database');
const poolConfig: PoolConfig = {
host: process.env.DB_HOST || 'localhost',
port: parseInt(process.env.DB_PORT || '5432', 10),
database: process.env.DB_NAME || 'privacy_gateway',
user: process.env.DB_USER || 'postgres',
password: process.env.DB_PASSWORD || '',
max: 20,
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 5000,
};
const pool = new Pool(poolConfig);
pool.on('connect', () => {
logger.info('New database connection established');
});
pool.on('error', (err: Error) => {
logger.error('Unexpected database error', { error: err.message });
});
export const query = async <T = unknown>(
text: string,
params?: unknown[]
): Promise<QueryResult<T>> => {
const start = Date.now();
try {
const result = await pool.query(text, params);
const duration = Date.now() - start;
logger.debug('Query executed', { duration, rows: result.rowCount });
return result;
} catch (error) {
logger.error('Query failed', { error: (error as Error).message, text });
throw error;
}
};
export const getPool = (): Pool => pool;
export const closePool = async (): Promise<void> => {
await pool.end();
logger.info('Database pool closed');
};
export const initDB = async (): Promise<void> => {
const client = await pool.connect();
try {
await client.query('BEGIN');
await client.query(`
CREATE TABLE IF NOT EXISTS sessions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(255) NOT NULL,
model VARCHAR(100) NOT NULL DEFAULT 'llama3.2',
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
metadata JSONB
)
`);
await client.query(`
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
session_id UUID NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
role VARCHAR(20) NOT NULL CHECK (role IN ('user', 'assistant', 'system')),
original_content TEXT NOT NULL,
anonymized_content TEXT NOT NULL,
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
)
`);
await client.query(`
CREATE TABLE IF NOT EXISTS pii_mappings (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
session_id UUID NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
message_id UUID NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
pii_type VARCHAR(50) NOT NULL,
original_value TEXT NOT NULL,
anonymized_value TEXT NOT NULL,
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
)
`);
await client.query(`
CREATE INDEX IF NOT EXISTS idx_messages_session_id ON messages(session_id)
`);
await client.query(`
CREATE INDEX IF NOT EXISTS idx_pii_mappings_session_id ON pii_mappings(session_id)
`);
await client.query(`
CREATE INDEX IF NOT EXISTS idx_pii_mappings_message_id ON pii_mappings(message_id)
`);
await client.query('COMMIT');
logger.info('Database initialized successfully');
} catch (error) {
await client.query('ROLLBACK');
logger.error('Database initialization failed', { error: (error as Error).message });
throw error;
} finally {
client.release();
}
};
export const SessionQueries = {
create: async (name: string, model: string, metadata?: Record<string, unknown>): Promise<Session> => {
const result = await query<Session>(
'INSERT INTO sessions (name, model, metadata) VALUES ($1, $2, $3) RETURNING *',
[name, model, metadata ? JSON.stringify(metadata) : null]
);
return result.rows[0];
},
findAll: async (): Promise<Session[]> => {
const result = await query<Session>(
'SELECT * FROM sessions ORDER BY updated_at DESC'
);
return result.rows;
},
findById: async (id: string): Promise<Session | null> => {
const result = await query<Session>(
'SELECT * FROM sessions WHERE id = $1',
[id]
);
return result.rows[0] || null;
},
update: async (id: string, updates: Partial<Session>): Promise<Session | null> => {
const setClauses: string[] = [];
const values: unknown[] = [];
let paramIndex = 1;
if (updates.name) {
setClauses.push(`name = $${paramIndex++}`);
values.push(updates.name);
}
if (updates.model) {
setClauses.push(`model = $${paramIndex++}`);
values.push(updates.model);
}
if (updates.metadata) {
setClauses.push(`metadata = $${paramIndex++}`);
values.push(JSON.stringify(updates.metadata));
}
setClauses.push(`updated_at = CURRENT_TIMESTAMP`);
values.push(id);
const result = await query<Session>(
`UPDATE sessions SET ${setClauses.join(', ')} WHERE id = $${paramIndex} RETURNING *`,
values
);
return result.rows[0] || null;
},
delete: async (id: string): Promise<boolean> => {
const result = await query(
'DELETE FROM sessions WHERE id = $1',
[id]
);
return (result.rowCount || 0) > 0;
},
};
export const MessageQueries = {
create: async (
sessionId: string,
role: string,
originalContent: string,
anonymizedContent: string
): Promise<Message> => {
const result = await query<Message>(
'INSERT INTO messages (session_id, role, original_content, anonymized_content) VALUES ($1, $2, $3, $4) RETURNING *',
[sessionId, role, originalContent, anonymizedContent]
);
return result.rows[0];
},
findBySessionId: async (sessionId: string): Promise<Message[]> => {
const result = await query<Message>(
'SELECT * FROM messages WHERE session_id = $1 ORDER BY created_at ASC',
[sessionId]
);
return result.rows;
},
findById: async (id: string): Promise<Message | null> => {
const result = await query<Message>(
'SELECT * FROM messages WHERE id = $1',
[id]
);
return result.rows[0] || null;
},
};
export const PIIMappingQueries = {
create: async (
sessionId: string,
messageId: string,
piiType: string,
originalValue: string,
anonymizedValue: string
): Promise<PIIMapping> => {
const result = await query<PIIMapping>(
'INSERT INTO pii_mappings (session_id, message_id, pii_type, original_value, anonymized_value) VALUES ($1, $2, $3, $4, $5) RETURNING *',
[sessionId, messageId, piiType, originalValue, anonymizedValue]
);
return result.rows[0];
},
findBySessionId: async (sessionId: string): Promise<PIIMapping[]> => {
const result = await query<PIIMapping>(
'SELECT * FROM pii_mappings WHERE session_id = $1',
[sessionId]
);
return result.rows;
},
findByMessageId: async (messageId: string): Promise<PIIMapping[]> => {
const result = await query<PIIMapping>(
'SELECT * FROM pii_mappings WHERE message_id = $1',
[messageId]
);
return result.rows;
},
};
backend/src/routes/chat.ts
+194
View File
@@ -0,0 +1,194 @@
import { Router, Request, Response } from 'express';
import { body, param, validationResult } from 'express-validator';
import { SessionQueries, MessageQueries, PIIMappingQueries } from '../db';
import { anonymizeText, deanonymizeText } from '../services/anonymizer';
import { chat, chatStream } from '../services/ollama';
import { cacheDelete, cacheClearPattern } from '../services/redis';
import { createLogger } from '../utils/logger';
import { ChatRequest, OllamaMessage, OllamaChatResponse } from '../types';
const router = Router();
const logger = createLogger('ChatRoute');
const CHAT_MODEL = process.env.CHAT_MODEL || 'llama3.2';
const CACHE_PREFIX = 'session:';
const handleValidationErrors = (req: Request, res: Response, next: () => void) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
res.status(400).json({
success: false,
errors: errors.array(),
});
return;
}
next();
};
router.post(
'/:id/chat',
[
param('id').isUUID().withMessage('Invalid session ID'),
body('message').isString().trim().notEmpty().withMessage('Message is required'),
body('stream').optional().isBoolean(),
handleValidationErrors,
],
async (req: Request, res: Response) => {
const { id } = req.params;
const { message, stream }: ChatRequest = req.body;
try {
logger.debug('Processing chat request', { sessionId: id, stream });
const session = await SessionQueries.findById(id);
if (!session) {
res.status(404).json({
success: false,
error: 'Session not found',
});
return;
}
const existingMessages = await MessageQueries.findBySessionId(id);
const history: OllamaMessage[] = existingMessages.map((msg) => ({
role: msg.role as 'user' | 'assistant' | 'system',
content: msg.anonymized_content,
}));
logger.debug('Anonymizing user message');
const anonymizationResult = await anonymizeText(message);
const userMessage = await MessageQueries.create(
id,
'user',
message,
anonymizationResult.anonymizedText
);
for (const mapping of anonymizationResult.mappings) {
await PIIMappingQueries.create(
id,
userMessage.id,
mapping.pii_type,
mapping.original_value,
mapping.anonymized_value
);
}
if (stream) {
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
const messages: OllamaMessage[] = [
...history,
{ role: 'user', content: anonymizationResult.anonymizedText },
];
let responseContent = '';
let hasError = false;
try {
await chatStream(
session.model || CHAT_MODEL,
messages,
(chunk: OllamaChatResponse) => {
if (chunk.message?.content) {
responseContent += chunk.message.content;
res.write(
`data: ${JSON.stringify({
content: chunk.message.content,
done: chunk.done,
})}\n\n`
);
}
if (chunk.done) {
res.write('data: [DONE]\n\n');
}
},
(error: Error) => {
logger.error('Stream error', { error: error.message });
hasError = true;
res.write(
`data: ${JSON.stringify({
error: error.message,
done: true,
})}\n\n`
);
}
);
if (!hasError && responseContent) {
const allMappings = await PIIMappingQueries.findBySessionId(id);
const deanonymizedResponse = await deanonymizeText(
responseContent,
allMappings
);
await MessageQueries.create(id, 'assistant', deanonymizedResponse, responseContent);
await cacheDelete(`${CACHE_PREFIX}${id}`);
await cacheClearPattern(`${CACHE_PREFIX}all`);
}
} catch (error) {
logger.error('Streaming chat failed', { error: (error as Error).message });
res.write(
`data: ${JSON.stringify({
error: 'Streaming failed',
done: true,
})}\n\n`
);
}
res.end();
return;
}
const messages: OllamaMessage[] = [
...history,
{ role: 'user', content: anonymizationResult.anonymizedText },
];
logger.debug('Sending request to Ollama', { model: session.model || CHAT_MODEL });
const ollamaResponse = await chat(session.model || CHAT_MODEL, messages);
const allMappings = await PIIMappingQueries.findBySessionId(id);
const deanonymizedResponse = await deanonymizeText(
ollamaResponse.message.content,
allMappings
);
const assistantMessage = await MessageQueries.create(
id,
'assistant',
deanonymizedResponse,
ollamaResponse.message.content
);
await cacheDelete(`${CACHE_PREFIX}${id}`);
await cacheClearPattern(`${CACHE_PREFIX}all`);
res.json({
success: true,
data: {
session_id: id,
user_message: userMessage,
assistant_message: assistantMessage,
usage: {
prompt_tokens: ollamaResponse.prompt_eval_count,
completion_tokens: ollamaResponse.eval_count,
total_tokens: (ollamaResponse.prompt_eval_count || 0) + (ollamaResponse.eval_count || 0),
},
},
});
} catch (error) {
logger.error('Chat request failed', { error: (error as Error).message });
res.status(500).json({
success: false,
error: 'Chat request failed',
});
}
}
);
export default router;
backend/src/routes/models.ts
+47
View File
@@ -0,0 +1,47 @@
import { Router, Request, Response } from 'express';
import { listModels } from '../services/ollama';
import { cacheGet, cacheSet } from '../services/redis';
import { createLogger } from '../utils/logger';
import { OllamaModel } from '../types';
const router = Router();
const logger = createLogger('ModelsRoute');
const CACHE_KEY = 'ollama:models';
const CACHE_TTL = 300;
router.get('/', async (_req: Request, res: Response) => {
try {
logger.debug('Fetching available models');
const cached = await cacheGet<OllamaModel[]>(CACHE_KEY);
if (cached) {
logger.debug('Returning cached models');
res.json({
success: true,
data: cached,
cached: true,
});
return;
}
const response = await listModels();
const models = response.models || [];
await cacheSet(CACHE_KEY, models, CACHE_TTL);
res.json({
success: true,
data: models,
});
} catch (error) {
logger.error('Failed to fetch models', { error: (error as Error).message });
res.status(500).json({
success: false,
error: 'Failed to fetch models',
details: (error as Error).message,
});
}
});
export default router;
backend/src/routes/sessions.ts
+179
View File
@@ -0,0 +1,179 @@
import { Router, Request, Response } from 'express';
import { body, param, validationResult } from 'express-validator';
import { SessionQueries, MessageQueries, PIIMappingQueries } from '../db';
import { cacheGet, cacheSet, cacheDelete, cacheClearPattern } from '../services/redis';
import { createLogger } from '../utils/logger';
import { CreateSessionRequest, Session } from '../types';
const router = Router();
const logger = createLogger('SessionsRoute');
const CACHE_PREFIX = 'session:';
const CACHE_TTL = 3600;
const handleValidationErrors = (req: Request, res: Response, next: () => void) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
res.status(400).json({
success: false,
errors: errors.array(),
});
return;
}
next();
};
router.get('/', async (_req: Request, res: Response) => {
try {
logger.debug('Fetching all sessions');
const cached = await cacheGet<Session[]>(`${CACHE_PREFIX}all`);
if (cached) {
logger.debug('Returning cached sessions');
res.json({
success: true,
data: cached,
cached: true,
});
return;
}
const sessions = await SessionQueries.findAll();
await cacheSet(`${CACHE_PREFIX}all`, sessions, CACHE_TTL);
res.json({
success: true,
data: sessions,
});
} catch (error) {
logger.error('Failed to fetch sessions', { error: (error as Error).message });
res.status(500).json({
success: false,
error: 'Failed to fetch sessions',
});
}
});
router.post(
'/',
[
body('name').isString().trim().notEmpty().withMessage('Name is required'),
body('model').optional().isString().trim(),
body('metadata').optional().isObject(),
handleValidationErrors,
],
async (req: Request, res: Response) => {
try {
const { name, model, metadata }: CreateSessionRequest = req.body;
logger.debug('Creating new session', { name, model });
const newSession = await SessionQueries.create(
name,
model || process.env.CHAT_MODEL || 'llama3.2',
metadata
);
await cacheClearPattern(`${CACHE_PREFIX}all`);
res.status(201).json({
success: true,
data: newSession,
});
} catch (error) {
logger.error('Failed to create session', { error: (error as Error).message });
res.status(500).json({
success: false,
error: 'Failed to create session',
});
}
}
);
router.get(
'/:id',
[param('id').isUUID().withMessage('Invalid session ID'), handleValidationErrors],
async (req: Request, res: Response) => {
try {
const { id } = req.params;
logger.debug('Fetching session', { id });
const cached = await cacheGet<Session & { messages: unknown[] }>(`${CACHE_PREFIX}${id}`);
if (cached) {
logger.debug('Returning cached session');
res.json({
success: true,
data: cached,
cached: true,
});
return;
}
const session = await SessionQueries.findById(id);
if (!session) {
res.status(404).json({
success: false,
error: 'Session not found',
});
return;
}
const messages = await MessageQueries.findBySessionId(id);
const result = {
...session,
messages,
};
await cacheSet(`${CACHE_PREFIX}${id}`, result, CACHE_TTL);
res.json({
success: true,
data: result,
});
} catch (error) {
logger.error('Failed to fetch session', { error: (error as Error).message });
res.status(500).json({
success: false,
error: 'Failed to fetch session',
});
}
}
);
router.delete(
'/:id',
[param('id').isUUID().withMessage('Invalid session ID'), handleValidationErrors],
async (req: Request, res: Response) => {
try {
const { id } = req.params;
logger.debug('Deleting session', { id });
const deleted = await SessionQueries.delete(id);
if (!deleted) {
res.status(404).json({
success: false,
error: 'Session not found',
});
return;
}
await cacheDelete(`${CACHE_PREFIX}${id}`);
await cacheClearPattern(`${CACHE_PREFIX}all`);
res.json({
success: true,
message: 'Session deleted successfully',
});
} catch (error) {
logger.error('Failed to delete session', { error: (error as Error).message });
res.status(500).json({
success: false,
error: 'Failed to delete session',
});
}
}
);
export default router;
backend/src/server.ts
+149
View File
@@ -0,0 +1,149 @@
import express, { Request, Response, NextFunction } from 'express';
import cors from 'cors';
import helmet from 'helmet';
import morgan from 'morgan';
import dotenv from 'dotenv';
import { createLogger } from './utils/logger';
import { initDB, closePool } from './db';
import { connectRedis, disconnectRedis } from './services/redis';
import { checkHealth as checkAnonymizerHealth } from './services/anonymizer';
import sessionsRouter from './routes/sessions';
import chatRouter from './routes/chat';
import modelsRouter from './routes/models';
dotenv.config();
const logger = createLogger('Server');
const app = express();
const PORT = parseInt(process.env.PORT || '3000', 10);
app.use(helmet());
app.use(cors({
origin: process.env.CORS_ORIGIN || '*',
methods: ['GET', 'POST', 'DELETE', 'OPTIONS'],
allowedHeaders: ['Content-Type', 'Authorization'],
}));
app.use(express.json({ limit: '10mb' }));
app.use(morgan('combined', {
stream: {
write: (message: string) => logger.info(message.trim()),
},
}));
app.use('/api/sessions', sessionsRouter);
app.use('/api/sessions', chatRouter);
app.use('/api/models', modelsRouter);
app.get('/health', async (_req: Request, res: Response) => {
const dbHealthy = true;
const redisHealthy = true;
const anonymizerHealthy = await checkAnonymizerHealth();
const isHealthy = dbHealthy && redisHealthy;
res.status(isHealthy ? 200 : 503).json({
status: isHealthy ? 'healthy' : 'unhealthy',
timestamp: new Date().toISOString(),
services: {
database: dbHealthy ? 'connected' : 'disconnected',
redis: redisHealthy ? 'connected' : 'disconnected',
anonymizer: anonymizerHealthy ? 'available' : 'unavailable',
},
version: process.env.npm_package_version || '1.0.0',
});
});
app.get('/ready', async (_req: Request, res: Response) => {
try {
const anonymizerHealthy = await checkAnonymizerHealth();
if (anonymizerHealthy) {
res.status(200).json({
ready: true,
timestamp: new Date().toISOString(),
});
} else {
res.status(503).json({
ready: false,
error: 'Anonymizer service unavailable',
});
}
} catch (error) {
res.status(503).json({
ready: false,
error: 'Health check failed',
});
}
});
app.use('*', (_req: Request, res: Response) => {
res.status(404).json({
success: false,
error: 'Endpoint not found',
});
});
app.use((err: Error, _req: Request, res: Response, _next: NextFunction) => {
logger.error('Unhandled error', { error: err.message, stack: err.stack });
res.status(500).json({
success: false,
error: 'Internal server error',
});
});
const shutdown = async (signal: string) => {
logger.info(`Received ${signal}, shutting down gracefully...`);
try {
await disconnectRedis();
await closePool();
logger.info('Cleanup complete, exiting');
process.exit(0);
} catch (error) {
logger.error('Error during shutdown', { error: (error as Error).message });
process.exit(1);
}
};
process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));
process.on('uncaughtException', (error) => {
logger.error('Uncaught exception', { error: error.message, stack: error.stack });
shutdown('uncaughtException');
});
process.on('unhandledRejection', (reason) => {
logger.error('Unhandled rejection', { reason });
});
const startServer = async () => {
try {
logger.info('Starting Privacy Gateway Backend...');
await initDB();
logger.info('Database initialized');
await connectRedis();
logger.info('Redis connected');
const anonymizerHealthy = await checkAnonymizerHealth();
if (anonymizerHealthy) {
logger.info('Anonymizer service is available');
} else {
logger.warn('Anonymizer service is not available - chat functionality may be limited');
}
app.listen(PORT, () => {
logger.info(`Server running on port ${PORT}`);
logger.info(`Environment: ${process.env.NODE_ENV || 'development'}`);
});
} catch (error) {
logger.error('Failed to start server', { error: (error as Error).message });
process.exit(1);
}
};
startServer();
backend/src/services/anonymizer.ts
+223
View File
@@ -0,0 +1,223 @@
import http from 'http';
import { createLogger } from '../utils/logger';
import { AnonymizationResult, OllamaChatResponse, OllamaMessage } from '../types';
const logger = createLogger('Anonymizer');
const ANON_HOST = process.env.OLLAMA_HOST || 'localhost';
const ANON_PORT = parseInt(process.env.OLLAMA_PORT || '11434', 10);
const ANON_MODEL = process.env.ANONYMIZATION_MODEL || 'llama3.2';
interface RequestOptions {
hostname: string;
port: number;
path: string;
method: string;
headers?: Record<string, string>;
}
const makeRequest = <T>(
options: RequestOptions,
postData?: string
): Promise<T> => {
return new Promise((resolve, reject) => {
const req = http.request(
{
hostname: options.hostname,
port: options.port,
path: options.path,
method: options.method,
headers: {
'Content-Type': 'application/json',
...options.headers,
},
},
(res) => {
let data = '';
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
if (res.statusCode && res.statusCode >= 200 && res.statusCode < 300) {
try {
resolve(JSON.parse(data) as T);
} catch {
resolve(data as unknown as T);
}
} else {
reject(new Error(`HTTP ${res.statusCode}: ${data}`));
}
});
}
);
req.on('error', (err) => {
reject(err);
});
req.setTimeout(30000, () => {
req.destroy();
reject(new Error('Request timeout'));
});
if (postData) {
req.write(postData);
}
req.end();
});
};
const ANONYMIZATION_PROMPT = `Du bist ein PII (Personally Identifiable Information) Anonymisierungssystem.
Deine Aufgabe ist es, sensible Informationen in Texten zu erkennen und durch Platzhalter zu ersetzen.
Ersetze folgende PII-Typen:
- PERSON: Namen von Personen (z.B. "Max Mustermann", "Anna Müller")
- EMAIL: E-Mail-Adressen
- PHONE: Telefonnummern
- ADDRESS: Adressen (Straße, Hausnummer, PLZ, Ort)
- ORG: Organisationen, Firmen, Behörden
- ID: Ausweisnummern, Kundennummern, IDs
- DATE: Geburtsdaten, spezifische persönliche Daten
- FINANCIAL: IBAN, Kreditkartennummern, Kontonummern
Format: Ersetze durch [PERSONTYPE_1], [EMAIL_1], [PHONE_1], etc.
Antworte ausschließlich mit einem JSON-Objekt im folgenden Format:
{
"anonymizedText": "Der anonymisierte Text mit Platzhaltern",
"mappings": [
{
"pii_type": "PERSON",
"original_value": "Max Mustermann",
"anonymized_value": "[PERSON_1]"
}
]
}
Falls keine PII gefunden wird:
{
"anonymizedText": "Originaltext ohne Änderungen",
"mappings": []
}
Hier ist der zu anonymisierende Text:
`;
export const anonymizeText = async (text: string): Promise<AnonymizationResult> => {
logger.debug('Anonymizing text', { textLength: text.length });
const messages: OllamaMessage[] = [
{
role: 'system',
content: 'Du bist ein präzises PII-Anonymisierungssystem. Antworte immer mit gültigem JSON.',
},
{
role: 'user',
content: ANONYMIZATION_PROMPT + text,
},
];
const requestBody = {
model: ANON_MODEL,
messages,
stream: false,
format: 'json',
options: {
temperature: 0.0,
num_predict: 2048,
},
};
try {
const response = await makeRequest<OllamaChatResponse>(
{
hostname: ANON_HOST,
port: ANON_PORT,
path: '/api/chat',
method: 'POST',
},
JSON.stringify(requestBody)
);
const content = response.message?.content || '{}';
let result: AnonymizationResult;
try {
result = JSON.parse(content) as AnonymizationResult;
} catch (parseError) {
logger.warn('Failed to parse anonymization response as JSON, attempting extraction', {
content: content.substring(0, 200),
});
const jsonMatch = content.match(/\{[\s\S]*\}/);
if (jsonMatch) {
result = JSON.parse(jsonMatch[0]) as AnonymizationResult;
} else {
throw new Error('No JSON found in response');
}
}
if (!result.anonymizedText) {
result.anonymizedText = text;
}
if (!result.mappings) {
result.mappings = [];
}
logger.debug('Anonymization complete', {
mappingsCount: result.mappings.length,
piiTypes: result.mappings.map(m => m.pii_type),
});
return result;
} catch (error) {
logger.error('Anonymization failed', { error: (error as Error).message });
return {
anonymizedText: text,
mappings: [],
};
}
};
export const deanonymizeText = async (
text: string,
mappings: Array<{ pii_type: string; original_value: string; anonymized_value: string }>
): Promise<string> => {
logger.debug('Deanonymizing text', { textLength: text.length, mappingsCount: mappings.length });
let deanonymized = text;
const sortedMappings = [...mappings].sort((a, b) =>
b.anonymized_value.length - a.anonymized_value.length
);
for (const mapping of sortedMappings) {
const regex = new RegExp(
mapping.anonymized_value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'),
'g'
);
deanonymized = deanonymized.replace(regex, mapping.original_value);
}
logger.debug('Deanonymization complete');
return deanonymized;
};
export const checkHealth = async (): Promise<boolean> => {
try {
await makeRequest<unknown>({
hostname: ANON_HOST,
port: ANON_PORT,
path: '/api/tags',
method: 'GET',
});
return true;
} catch {
return false;
}
};
backend/src/services/ollama.ts
+230
View File
@@ -0,0 +1,230 @@
import http from 'http';
import { createLogger } from '../utils/logger';
import { OllamaChatRequest, OllamaChatResponse, OllamaListResponse, OllamaMessage } from '../types';
const logger = createLogger('Ollama');
const OLLAMA_HOST = process.env.OLLAMA_TARGET_HOST || process.env.OLLAMA_HOST || 'localhost';
const OLLAMA_PORT = parseInt(process.env.OLLAMA_TARGET_PORT || process.env.OLLAMA_PORT || '11434', 10);
interface RequestOptions {
hostname: string;
port: number;
path: string;
method: string;
headers?: Record<string, string>;
}
const makeRequest = <T>(
options: RequestOptions,
postData?: string
): Promise<T> => {
return new Promise((resolve, reject) => {
const req = http.request(
{
hostname: options.hostname,
port: options.port,
path: options.path,
method: options.method,
headers: {
'Content-Type': 'application/json',
...options.headers,
},
},
(res) => {
let data = '';
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
if (res.statusCode && res.statusCode >= 200 && res.statusCode < 300) {
try {
resolve(JSON.parse(data) as T);
} catch {
resolve(data as unknown as T);
}
} else {
reject(new Error(`HTTP ${res.statusCode}: ${data}`));
}
});
}
);
req.on('error', (err) => {
reject(err);
});
req.setTimeout(30000, () => {
req.destroy();
reject(new Error('Request timeout'));
});
if (postData) {
req.write(postData);
}
req.end();
});
};
export const listModels = async (): Promise<OllamaListResponse> => {
logger.debug('Fetching model list');
try {
const response = await makeRequest<OllamaListResponse>({
hostname: OLLAMA_HOST,
port: OLLAMA_PORT,
path: '/api/tags',
method: 'GET',
});
logger.debug(`Found ${response.models?.length || 0} models`);
return response;
} catch (error) {
logger.error('Failed to list models', { error: (error as Error).message });
throw error;
}
};
export const chat = async (
model: string,
messages: OllamaMessage[],
options?: { temperature?: number; top_p?: number; top_k?: number; num_predict?: number }
): Promise<OllamaChatResponse> => {
logger.debug('Sending chat request', { model, messageCount: messages.length });
const requestBody: OllamaChatRequest = {
model,
messages,
stream: false,
options: {
temperature: options?.temperature ?? 0.7,
top_p: options?.top_p ?? 0.9,
...options,
},
};
try {
const response = await makeRequest<OllamaChatResponse>({
hostname: OLLAMA_HOST,
port: OLLAMA_PORT,
path: '/api/chat',
method: 'POST',
}, JSON.stringify(requestBody));
logger.debug('Chat response received', {
done: response.done,
evalCount: response.eval_count,
});
return response;
} catch (error) {
logger.error('Chat request failed', { error: (error as Error).message });
throw error;
}
};
export const chatStream = (
model: string,
messages: OllamaMessage[],
onData: (chunk: OllamaChatResponse) => void,
onError: (error: Error) => void,
options?: { temperature?: number; top_p?: number; top_k?: number; num_predict?: number }
): Promise<void> => {
return new Promise((resolve, reject) => {
logger.debug('Starting streaming chat', { model, messageCount: messages.length });
const requestBody: OllamaChatRequest = {
model,
messages,
stream: true,
options: {
temperature: options?.temperature ?? 0.7,
top_p: options?.top_p ?? 0.9,
...options,
},
};
const req = http.request(
{
hostname: OLLAMA_HOST,
port: OLLAMA_PORT,
path: '/api/chat',
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
},
(res) => {
let buffer = '';
res.on('data', (chunk: Buffer) => {
buffer += chunk.toString();
const lines = buffer.split('\n');
buffer = lines.pop() || '';
for (const line of lines) {
if (line.trim()) {
try {
const parsed = JSON.parse(line) as OllamaChatResponse;
onData(parsed);
} catch (err) {
logger.warn('Failed to parse stream chunk', { line });
}
}
}
});
res.on('end', () => {
if (buffer.trim()) {
try {
const parsed = JSON.parse(buffer) as OllamaChatResponse;
onData(parsed);
} catch (err) {
logger.warn('Failed to parse final chunk', { buffer });
}
}
resolve();
});
res.on('error', (err) => {
onError(err);
reject(err);
});
}
);
req.on('error', (err) => {
logger.error('Stream request error', { error: err.message });
onError(err);
reject(err);
});
req.setTimeout(300000, () => {
req.destroy();
const timeoutError = new Error('Streaming request timeout');
onError(timeoutError);
reject(timeoutError);
});
req.write(JSON.stringify(requestBody));
req.end();
});
};
export const getModelInfo = async (modelName: string): Promise<unknown> => {
logger.debug('Fetching model info', { model: modelName });
try {
const response = await makeRequest<unknown>({
hostname: OLLAMA_HOST,
port: OLLAMA_PORT,
path: `/api/show`,
method: 'POST',
}, JSON.stringify({ name: modelName }));
return response;
} catch (error) {
logger.error('Failed to get model info', { error: (error as Error).message });
throw error;
}
};
backend/src/services/redis.ts
+80
View File
@@ -0,0 +1,80 @@
import { createClient, RedisClientType } from 'redis';
import { createLogger } from '../utils/logger';
const logger = createLogger('Redis');
const redisClient: RedisClientType = createClient({
socket: {
host: process.env.REDIS_HOST || 'localhost',
port: parseInt(process.env.REDIS_PORT || '6379', 10),
},
});
redisClient.on('error', (err) => {
logger.error('Redis client error', { error: err.message });
});
redisClient.on('connect', () => {
logger.info('Redis client connected');
});
let isConnected = false;
export const connectRedis = async (): Promise<void> => {
if (!isConnected) {
await redisClient.connect();
isConnected = true;
}
};
export const getRedisClient = (): RedisClientType => {
if (!isConnected) {
throw new Error('Redis client not connected. Call connectRedis() first.');
}
return redisClient;
};
export const disconnectRedis = async (): Promise<void> => {
if (isConnected) {
await redisClient.quit();
isConnected = false;
logger.info('Redis client disconnected');
}
};
export const cacheGet = async <T>(key: string): Promise<T | null> => {
try {
const value = await redisClient.get(key);
return value ? JSON.parse(value) : null;
} catch (error) {
logger.error('Cache get error', { error: (error as Error).message, key });
return null;
}
};
export const cacheSet = async <T>(key: string, value: T, ttlSeconds = 3600): Promise<void> => {
try {
await redisClient.setEx(key, ttlSeconds, JSON.stringify(value));
} catch (error) {
logger.error('Cache set error', { error: (error as Error).message, key });
}
};
export const cacheDelete = async (key: string): Promise<void> => {
try {
await redisClient.del(key);
} catch (error) {
logger.error('Cache delete error', { error: (error as Error).message, key });
}
};
export const cacheClearPattern = async (pattern: string): Promise<void> => {
try {
const keys = await redisClient.keys(pattern);
if (keys.length > 0) {
await redisClient.del(keys);
}
} catch (error) {
logger.error('Cache clear pattern error', { error: (error as Error).message, pattern });
}
};
backend/src/types/index.ts
+104
View File
@@ -0,0 +1,104 @@
export interface Session {
id: string;
name: string;
model: string;
created_at: Date;
updated_at: Date;
metadata?: Record<string, unknown>;
}
export interface Message {
id: string;
session_id: string;
role: 'user' | 'assistant' | 'system';
original_content: string;
anonymized_content: string;
created_at: Date;
}
export interface PIIMapping {
id: string;
session_id: string;
message_id: string;
pii_type: string;
original_value: string;
anonymized_value: string;
created_at: Date;
}
export interface CreateSessionRequest {
name: string;
model?: string;
metadata?: Record<string, unknown>;
}
export interface ChatRequest {
message: string;
stream?: boolean;
}
export interface ChatResponse {
session_id: string;
message: Message;
response?: string;
}
export interface OllamaMessage {
role: 'user' | 'assistant' | 'system';
content: string;
}
export interface OllamaChatRequest {
model: string;
messages: OllamaMessage[];
stream?: boolean;
options?: {
temperature?: number;
top_p?: number;
top_k?: number;
num_predict?: number;
};
}
export interface OllamaChatResponse {
model: string;
created_at: string;
message: OllamaMessage;
done: boolean;
done_reason?: string;
total_duration?: number;
load_duration?: number;
prompt_eval_count?: number;
prompt_eval_duration?: number;
eval_count?: number;
eval_duration?: number;
}
export interface OllamaModel {
name: string;
model: string;
modified_at: string;
size: number;
digest: string;
details?: {
parent_model?: string;
format?: string;
family?: string;
families?: string[];
parameter_size?: string;
quantization_level?: string;
};
}
export interface OllamaListResponse {
models: OllamaModel[];
}
export interface AnonymizationResult {
anonymizedText: string;
mappings: Array<{
pii_type: string;
original_value: string;
anonymized_value: string;
}>;
}
backend/src/utils/logger.ts
+39
View File
@@ -0,0 +1,39 @@
import winston from 'winston';
const { combine, timestamp, json, printf, colorize, errors } = winston.format;
const devFormat = printf(({ level, message, timestamp, ...metadata }) => {
let msg = `${timestamp} [${level}]: ${message}`;
if (Object.keys(metadata).length > 0) {
msg += ` ${JSON.stringify(metadata)}`;
}
return msg;
});
export const createLogger = (service: string): winston.Logger => {
const isDev = process.env.NODE_ENV !== 'production';
const logLevel = process.env.LOG_LEVEL || (isDev ? 'debug' : 'info');
return winston.createLogger({
level: logLevel,
defaultMeta: { service },
format: isDev
? combine(
colorize(),
timestamp({ format: 'HH:mm:ss' }),
errors({ stack: true }),
devFormat
)
: combine(
timestamp(),
json()
),
transports: [
new winston.transports.Console({
stderrLevels: ['error'],
}),
],
});
};
export const logger = createLogger('App');
backend/tsconfig.json
+23
View File
@@ -0,0 +1,23 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "commonjs",
"lib": ["ES2022"],
"outDir": "./dist",
"rootDir": "./src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"moduleResolution": "node",
"sourceMap": true,
"declaration": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
docker-compose.yml
+110
View File
@@ -0,0 +1,110 @@
version: '3.8'
services:
# PostgreSQL für Sessions und PII-Mappings
postgres:
image: postgres:15-alpine
container_name: pg-privacy-gateway
environment:
POSTGRES_DB: privacy_gateway
POSTGRES_USER: pguser
POSTGRES_PASSWORD: ${DB_PASSWORD:-pgsecret123}
volumes:
- postgres_data:/var/lib/postgresql/data
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
networks:
- privacy-net
healthcheck:
test: ["CMD-SHELL", "pg_isready -U pguser -d privacy_gateway"]
interval: 5s
timeout: 5s
retries: 5
# Anonymisierungs-Service (lokales LLM via Ollama)
ollama-anonymizer:
image: ollama/ollama:latest
container_name: ollama-anonymizer
volumes:
- ollama_models:/root/.ollama
environment:
- OLLAMA_KEEP_ALIVE=24h
networks:
- privacy-net
# GPU Support (optional)
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: 1
# capabilities: [gpu]
# Redis für Caching und Session-State
redis:
image: redis:7-alpine
container_name: redis-privacy-gateway
volumes:
- redis_data:/data
networks:
- privacy-net
# Backend API (wird vom Team entwickelt)
backend:
build:
context: ./backend
dockerfile: Dockerfile
container_name: privacy-gateway-api
environment:
- NODE_ENV=production
- DB_HOST=postgres
- DB_PORT=5432
- DB_NAME=privacy_gateway
- DB_USER=pguser
- DB_PASSWORD=${DB_PASSWORD:-pgsecret123}
- REDIS_HOST=redis
- REDIS_PORT=6379
- OLLAMA_HOST=ollama-anonymizer
- OLLAMA_PORT=11434
- ANONYMIZATION_MODEL=gemma4:latest
- CHAT_MODEL=${CHAT_MODEL:-llama3.2:latest}
- OLLAMA_TARGET_HOST=${OLLAMA_TARGET_HOST:-host.docker.internal}
- OLLAMA_TARGET_PORT=11434
ports:
- "${API_PORT:-3000}:3000"
volumes:
- ./backend/src:/app/src
- backend_uploads:/app/uploads
networks:
- privacy-net
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_started
ollama-anonymizer:
condition: service_started
# Frontend (wird vom Team entwickelt)
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
container_name: privacy-gateway-ui
environment:
- REACT_APP_API_URL=http://localhost:3000
ports:
- "${UI_PORT:-8080}:80"
networks:
- privacy-net
depends_on:
- backend
volumes:
postgres_data:
ollama_models:
redis_data:
backend_uploads:
networks:
privacy-net:
driver: bridge
frontend/.dockerignore
+11
View File
@@ -0,0 +1,11 @@
node_modules
dist
.git
.gitignore
README.md
.env
.env.local
.DS_Store
npm-debug.log*
yarn-debug.log*
yarn-error.log*
frontend/.env.example
+2
View File
@@ -0,0 +1,2 @@
# API URL fuer den Backend-Server
VITE_API_URL=http://localhost:3000
frontend/Dockerfile
+49
View File
@@ -0,0 +1,49 @@
# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
# Copy package files
COPY package*.json ./
# Install dependencies
RUN npm ci
# Copy source code
COPY . .
# Build the application
RUN npm run build
# Production stage
FROM nginx:alpine
# Copy built assets from builder
COPY --from=builder /app/dist /usr/share/nginx/html
# Copy nginx config for SPA
RUN echo 'server { \
listen 80; \
server_name localhost; \
root /usr/share/nginx/html; \
index index.html; \
\
location / { \
try_files $uri $uri/ /index.html; \
} \
\
location /api { \
proxy_pass http://backend:3000; \
proxy_http_version 1.1; \
proxy_set_header Upgrade $http_upgrade; \
proxy_set_header Connection "upgrade"; \
proxy_set_header Host $host; \
proxy_set_header X-Real-IP $remote_addr; \
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; \
proxy_set_header X-Forwarded-Proto $scheme; \
} \
}' > /etc/nginx/conf.d/default.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
frontend/README.md
+53
View File
@@ -0,0 +1,53 @@
# Privacy Gateway Frontend
React-basierte Chat-Oberfläche für das Privacy Gateway.
## Features
- ✨ Multi-Window Chat-Sessions (wie OpenWebUI)
- 🎨 Dark Mode Support
- 📝 Markdown-Rendering mit Syntax-Highlighting
- 👁️ Markdown-Live-Preview beim Schreiben
- 📱 Responsive Design
- 🔄 Streaming-Response Support
## Tech Stack
- React 18 + TypeScript
- Vite
- Tailwind CSS
- React Markdown + Syntax Highlighter
- Lucide Icons
## Setup
```bash
# Dependencies installieren
npm install
# Development Server starten
npm run dev
# Production Build
npm run build
```
## API-Endpunkte
Die Frontend-Anwendung kommuniziert mit dem Backend über:
- `GET /api/sessions` - Alle Sessions laden
- `POST /api/sessions` - Neue Session erstellen
- `GET /api/sessions/:id` - Einzelne Session laden
- `POST /api/sessions/:id/chat` - Nachricht senden (Stream)
- `DELETE /api/sessions/:id` - Session löschen
## Docker
```bash
# Image bauen
docker build -t privacy-gateway-frontend .
# Container starten
docker run -p 80:80 privacy-gateway-frontend
```
frontend/index.html
+14
View File
@@ -0,0 +1,14 @@
<!doctype html>
<html lang="de">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Privacy Gateway - Lokal gehosteter KI-Chat" />
<title>Privacy Gateway Chat</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
frontend/package.json
+30
View File
@@ -0,0 +1,30 @@
{
"name": "privacy-gateway-frontend",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc && vite build",
"preview": "vite preview"
},
"dependencies": {
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-markdown": "^9.0.1",
"react-syntax-highlighter": "^15.5.0",
"remark-gfm": "^4.0.0",
"lucide-react": "^0.344.0"
},
"devDependencies": {
"@types/react": "^18.2.55",
"@types/react-dom": "^18.2.19",
"@types/react-syntax-highlighter": "^15.5.11",
"@vitejs/plugin-react": "^4.2.1",
"autoprefixer": "^10.4.18",
"postcss": "^8.4.35",
"tailwindcss": "^3.4.1",
"typescript": "^5.2.2",
"vite": "^5.1.0"
}
}
frontend/postcss.config.js
+6
View File
@@ -0,0 +1,6 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}
frontend/src/App.tsx
+127
View File
@@ -0,0 +1,127 @@
import React, { useState, useEffect, useCallback } from 'react';
import { Sidebar } from './components/Sidebar';
import { ChatWindow } from './components/ChatWindow';
import { Session } from './types';
const API_URL = 'http://localhost:3000';
function App() {
const [sessions, setSessions] = useState<Session[]>([]);
const [activeSessionId, setActiveSessionId] = useState<string | null>(null);
const [isDarkMode, setIsDarkMode] = useState(true);
const [isLoading, setIsLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
// Load sessions
const loadSessions = useCallback(async () => {
try {
const response = await fetch(`${API_URL}/api/sessions`);
if (!response.ok) throw new Error('Failed to load sessions');
const data = await response.json();
setSessions(data.sessions || []);
} catch (err) {
setError('Verbindung zum Server fehlgeschlagen');
} finally {
setIsLoading(false);
}
}, []);
useEffect(() => {
loadSessions();
}, [loadSessions]);
// Create new session
const handleCreateSession = async () => {
try {
const response = await fetch(`${API_URL}/api/sessions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ title: `Chat ${sessions.length + 1}` }),
});
if (!response.ok) throw new Error('Failed to create session');
const data = await response.json();
setSessions((prev) => [data.session, ...prev]);
setActiveSessionId(data.session.id);
} catch (err) {
setError('Konnte keinen neuen Chat erstellen');
}
};
// Delete session
const handleDeleteSession = async (id: string) => {
try {
const response = await fetch(`${API_URL}/api/sessions/${id}`, {
method: 'DELETE',
});
if (!response.ok) throw new Error('Failed to delete session');
setSessions((prev) => prev.filter((s) => s.id !== id));
if (activeSessionId === id) {
setActiveSessionId(null);
}
} catch (err) {
setError('Konnte Chat nicht loeschen');
}
};
// Select session
const handleSelectSession = async (id: string) => {
try {
const response = await fetch(`${API_URL}/api/sessions/${id}`);
if (!response.ok) throw new Error('Failed to load session');
const data = await response.json();
setSessions((prev) =>
prev.map((s) => (s.id === id ? data.session : s))
);
setActiveSessionId(id);
} catch (err) {
setError('Konnte Chat nicht laden');
}
};
// Update session locally
const handleSessionUpdate = (updatedSession: Session) => {
setSessions((prev) =>
prev.map((s) => (s.id === updatedSession.id ? updatedSession : s))
);
};
const activeSession = sessions.find((s) => s.id === activeSessionId) || null;
if (isLoading) {
return (
<div
className={`h-screen flex items-center justify-center ${
isDarkMode ? 'bg-gray-900' : 'bg-gray-50'
}`}
>
<div className="animate-spin rounded-full h-12 w-12 border-b-2 border-blue-500"></div>
</div>
);
}
return (
<div
className={`h-screen flex ${
isDarkMode ? 'bg-gray-900' : 'bg-gray-50'
}`}
>
<Sidebar
sessions={sessions}
activeSessionId={activeSessionId}
onSessionSelect={handleSelectSession}
onSessionCreate={handleCreateSession}
onSessionDelete={handleDeleteSession}
isDarkMode={isDarkMode}
/>
<ChatWindow
session={activeSession}
isDarkMode={isDarkMode}
onToggleDarkMode={() => setIsDarkMode(!isDarkMode)}
onSessionUpdate={handleSessionUpdate}
/>
</div>
);
}
export default App;
frontend/src/components/ChatWindow.tsx
+262
View File
@@ -0,0 +1,262 @@
import React, { useState, useCallback } from 'react';
import { MessageList } from './MessageList';
import { InputArea } from './InputArea';
import { Session, Message } from '../types';
import { Moon, Sun, Bot } from 'lucide-react';
const API_URL = 'http://localhost:3000';
interface ChatWindowProps {
session: Session | null;
isDarkMode: boolean;
onToggleDarkMode: () => void;
onSessionUpdate: (session: Session) => void;
}
export const ChatWindow: React.FC<ChatWindowProps> = ({
session,
isDarkMode,
onToggleDarkMode,
onSessionUpdate,
}) => {
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const handleSendMessage = useCallback(
async (content: string) => {
if (!session) return;
setIsLoading(true);
setError(null);
// Optimistic user message
const userMessage: Message = {
id: `temp-${Date.now()}`,
role: 'user',
content,
timestamp: Date.now(),
};
const updatedMessages = [...session.messages, userMessage];
onSessionUpdate({
...session,
messages: updatedMessages,
updatedAt: Date.now(),
});
try {
const response = await fetch(`${API_URL}/api/sessions/${session.id}/chat`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({ message: content }),
});
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const reader = response.body?.getReader();
if (!reader) {
throw new Error('No response body');
}
let assistantContent = '';
const assistantMessage: Message = {
id: `assistant-${Date.now()}`,
role: 'assistant',
content: '',
timestamp: Date.now(),
};
const messagesWithAssistant = [...updatedMessages, assistantMessage];
onSessionUpdate({
...session,
messages: messagesWithAssistant,
updatedAt: Date.now(),
});
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value, { stream: true });
const lines = chunk.split('\n');
for (const line of lines) {
if (line.startsWith('data: ')) {
const data = line.slice(6);
if (data === '[DONE]') {
break;
}
try {
const parsed = JSON.parse(data);
if (parsed.content) {
assistantContent += parsed.content;
onSessionUpdate({
...session,
messages: [
...updatedMessages,
{ ...assistantMessage, content: assistantContent },
],
updatedAt: Date.now(),
});
}
} catch {
// Ignore malformed JSON
}
}
}
}
} catch (err) {
setError(err instanceof Error ? err.message : 'Ein Fehler ist aufgetreten');
// Revert to messages without the failed request
onSessionUpdate({
...session,
messages: session.messages,
updatedAt: Date.now(),
});
} finally {
setIsLoading(false);
}
},
[session, onSessionUpdate]
);
if (!session) {
return (
<div
className={`flex-1 flex flex-col items-center justify-center ${
isDarkMode ? 'bg-gray-900' : 'bg-gray-50'
}`}
>
<div
className={`text-center p-8 rounded-2xl ${
isDarkMode ? 'bg-gray-800' : 'bg-white'
} shadow-lg`}
>
<Bot size={64} className="mx-auto mb-4 text-blue-500" />
<h2
className={`text-2xl font-bold mb-4 ${
isDarkMode ? 'text-white' : 'text-gray-800'
}`}
>
Willkommen beim Privacy Gateway
</h2>
<p
className={`mb-6 ${
isDarkMode ? 'text-gray-400' : 'text-gray-500'
}`}
>
Waehle einen bestehenden Chat oder starte einen neuen.
</p>
<div
className={`text-sm ${
isDarkMode ? 'text-gray-500' : 'text-gray-400'
}`}
>
<p className="flex items-center justify-center gap-2 mb-2">
<span className="w-2 h-2 bg-green-500 rounded-full"></span>
Lokal gehostet
</p>
<p className="flex items-center justify-center gap-2 mb-2">
<span className="w-2 h-2 bg-blue-500 rounded-full"></span>
Datenschutz-fokussiert
</p>
<p className="flex items-center justify-center gap-2">
<span className="w-2 h-2 bg-purple-500 rounded-full"></span>
Open Source
</p>
</div>
</div>
</div>
);
}
return (
<div
className={`flex-1 flex flex-col ${
isDarkMode ? 'bg-gray-900' : 'bg-gray-50'
}`}
>
{/* Header */}
<div
className={`flex items-center justify-between px-4 py-3 border-b ${
isDarkMode
? 'bg-gray-800 border-gray-700'
: 'bg-white border-gray-200'
}`}
>
<div className="flex items-center gap-3">
<div>
<h1
className={`font-semibold ${
isDarkMode ? 'text-white' : 'text-gray-800'
}`}
>
{session.title || 'Neuer Chat'}
</h1>
<p
className={`text-xs ${
isDarkMode ? 'text-gray-500' : 'text-gray-400'
}`}
>
{session.messages.length} Nachrichten
</p>
</div>
</div>
<div className="flex items-center gap-2">
<button
onClick={onToggleDarkMode}
className={`p-2 rounded-lg transition-colors ${
isDarkMode
? 'hover:bg-gray-700 text-gray-400'
: 'hover:bg-gray-100 text-gray-600'
}`}
title={isDarkMode ? 'Hellmodus' : 'Dunkelmodus'}
>
{isDarkMode ? <Sun size={20} /> : <Moon size={20} />}
</button>
</div>
</div>
{/* Error Message */}
{error && (
<div
className={`px-4 py-2 text-sm text-center ${
isDarkMode
? 'bg-red-900/30 text-red-200 border-b border-red-800'
: 'bg-red-50 text-red-700 border-b border-red-200'
}`}
>
Fehler: {error}
<button
onClick={() => setError(null)}
className="ml-2 underline"
>
Schliessen
</button>
</div>
)}
{/* Messages */}
<MessageList
messages={session.messages}
isDarkMode={isDarkMode}
isLoading={isLoading}
/>
{/* Input */}
<InputArea
onSend={handleSendMessage}
isLoading={isLoading}
isDarkMode={isDarkMode}
disabled={!session}
/>
</div>
);
};
frontend/src/components/InputArea.tsx
+127
View File
@@ -0,0 +1,127 @@
import React, { useState, KeyboardEvent, useRef, useEffect } from 'react';
import { Send, Loader2, Eye, EyeOff } from 'lucide-react';
interface InputAreaProps {
onSend: (message: string) => void;
isLoading: boolean;
isDarkMode: boolean;
disabled?: boolean;
}
export const InputArea: React.FC<InputAreaProps> = ({
onSend,
isLoading,
isDarkMode,
disabled,
}) => {
const [input, setInput] = useState('');
const [showPreview, setShowPreview] = useState(false);
const textareaRef = useRef<HTMLTextAreaElement>(null);
useEffect(() => {
if (textareaRef.current) {
textareaRef.current.style.height = 'auto';
textareaRef.current.style.height = `${Math.min(textareaRef.current.scrollHeight, 200)}px`;
}
}, [input]);
const handleSend = () => {
if (input.trim() && !isLoading && !disabled) {
onSend(input.trim());
setInput('');
setShowPreview(false);
if (textareaRef.current) {
textareaRef.current.style.height = 'auto';
}
}
};
const handleKeyDown = (e: KeyboardEvent<HTMLTextAreaElement>) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
handleSend();
}
};
return (
<div className={`border-t ${isDarkMode ? 'border-gray-700 bg-gray-900' : 'border-gray-200 bg-white'}`}>
{showPreview && input.trim() && (
<div className={`mx-4 mt-4 p-4 rounded-lg border ${isDarkMode ? 'bg-gray-800 border-gray-700' : 'bg-gray-50 border-gray-200'}`}>
<div className={`text-xs font-medium mb-2 ${isDarkMode ? 'text-gray-400' : 'text-gray-500'}`}>
Markdown-Vorschau:
</div>
<div className={`prose prose-sm max-w-none dark:prose-invert max-h-48 overflow-y-auto ${isDarkMode ? 'text-gray-200' : 'text-gray-800'}`}>
{input.split('\n').map((line, i) => (
<div key={i}>{line || '\u00A0'}</div>
))}
</div>
</div>
)}
<div className="p-4">
<div
className={`flex items-end gap-2 rounded-lg border-2 p-3 transition-colors ${
isDarkMode
? 'bg-gray-800 border-gray-700 focus-within:border-blue-500'
: 'bg-white border-gray-300 focus-within:border-blue-500'
}`}
>
<textarea
ref={textareaRef}
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={handleKeyDown}
placeholder={disabled ? 'Waehle einen Chat aus...' : 'Nachricht schreiben...'}
disabled={disabled || isLoading}
rows={1}
className={`flex-1 resize-none bg-transparent outline-none max-h-[200px] ${
isDarkMode ? 'text-gray-100 placeholder-gray-500' : 'text-gray-800 placeholder-gray-400'
}`}
style={{ minHeight: '24px' }}
/>
<div className="flex items-center gap-1">
<button
onClick={() => setShowPreview(!showPreview)}
disabled={!input.trim()}
className={`p-2 rounded-lg transition-colors ${
showPreview
? isDarkMode
? 'bg-blue-900 text-blue-300'
: 'bg-blue-100 text-blue-600'
: isDarkMode
? 'hover:bg-gray-700 text-gray-400 hover:text-gray-200'
: 'hover:bg-gray-100 text-gray-400 hover:text-gray-600'
} ${!input.trim() ? 'opacity-50 cursor-not-allowed' : ''}`}
title={showPreview ? 'Vorschau ausblenden' : 'Markdown-Vorschau'}
>
{showPreview ? <EyeOff size={18} /> : <Eye size={18} />}
</button>
<button
onClick={handleSend}
disabled={isLoading || !input.trim() || disabled}
className={`p-2 rounded-lg transition-colors ${
isLoading || !input.trim() || disabled
? isDarkMode
? 'bg-gray-700 text-gray-500'
: 'bg-gray-200 text-gray-400'
: isDarkMode
? 'bg-blue-600 hover:bg-blue-700 text-white'
: 'bg-blue-500 hover:bg-blue-600 text-white'
}`}
title="Senden (Enter)"
>
{isLoading ? <Loader2 size={18} className="animate-spin" /> : <Send size={18} />}
</button>
</div>
</div>
<div className={`mt-2 text-xs flex items-center justify-between ${isDarkMode ? 'text-gray-500' : 'text-gray-400'}`}>
<span>Shift+Enter fuer neue Zeile</span>
<span>{input.length} Zeichen</span>
</div>
</div>
</div>
);
};
frontend/src/components/MessageList.tsx
+215
View File
@@ -0,0 +1,215 @@
import React, { useEffect, useRef } from 'react';
import ReactMarkdown from 'react-markdown';
import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';
import { vscDarkPlus } from 'react-syntax-highlighter/dist/esm/styles/prism';
import { oneLight } from 'react-syntax-highlighter/dist/esm/styles/prism';
import remarkGfm from 'remark-gfm';
import { Message } from '../types';
import { User, Bot } from 'lucide-react';
interface MessageListProps {
messages: Message[];
isDarkMode: boolean;
isLoading?: boolean;
}
export const MessageList: React.FC<MessageListProps> = ({
messages,
isDarkMode,
isLoading,
}) => {
const messagesEndRef = useRef<HTMLDivElement>(null);
useEffect(() => {
messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
}, [messages]);
const formatTime = (timestamp: number) => {
return new Date(timestamp).toLocaleTimeString('de-DE', {
hour: '2-digit',
minute: '2-digit',
});
};
return (
<div className="flex-1 overflow-y-auto p-4 space-y-4">
{messages.length === 0 ? (
<div className={`text-center py-12 ${isDarkMode ? 'text-gray-400' : 'text-gray-500'}`}>
<Bot size={48} className="mx-auto mb-4 opacity-50" />
<p className="text-lg font-medium mb-2">Willkommen beim Privacy Gateway</p>
<p className="text-sm">Starte eine neue Unterhaltung oder waehle einen bestehenden Chat.</p>
</div>
) : (
messages.map((message) => (
<div
key={message.id}
className={`flex gap-3 ${message.role === 'user' ? 'flex-row-reverse' : ''}`}
>
<div
className={`flex-shrink-0 w-8 h-8 rounded-full flex items-center justify-center ${
message.role === 'user'
? isDarkMode
? 'bg-blue-600'
: 'bg-blue-500'
: isDarkMode
? 'bg-gray-700'
: 'bg-gray-200'
}`}
>
{message.role === 'user' ? (
<User size={18} className="text-white" />
) : (
<Bot size={18} className={isDarkMode ? 'text-gray-300' : 'text-gray-600'} />
)}
</div>
<div
className={`max-w-[80%] rounded-2xl px-4 py-3 ${
message.role === 'user'
? isDarkMode
? 'bg-blue-600 text-white'
: 'bg-blue-500 text-white'
: isDarkMode
? 'bg-gray-800 text-gray-100'
: 'bg-gray-100 text-gray-800'
}`}
>
<div className="prose prose-sm max-w-none dark:prose-invert">
<ReactMarkdown
remarkPlugins={[remarkGfm]}
components={{
code({ node, inline, className, children, ...props }: any) {
const match = /language-(\w+)/.exec(className || '');
const language = match ? match[1] : '';
return !inline && language ? (
<SyntaxHighlighter
style={isDarkMode ? vscDarkPlus : oneLight}
language={language}
PreTag="div"
{...props}
>
{String(children).replace(/\n$/, '')}
</SyntaxHighlighter>
) : (
<code className={`${isDarkMode ? 'bg-gray-700' : 'bg-gray-200'} px-1 py-0.5 rounded`} {...props}>
{children}
</code>
);
},
p({ children }) {
return <p className="mb-2 last:mb-0">{children}</p>;
},
ul({ children }) {
return <ul className="list-disc ml-4 mb-2">{children}</ul>;
},
ol({ children }) {
return <ol className="list-decimal ml-4 mb-2">{children}</ol>;
},
li({ children }) {
return <li className="mb-1">{children}</li>;
},
h1({ children }) {
return <h1 className="text-lg font-bold mb-2">{children}</h1>;
},
h2({ children }) {
return <h2 className="text-base font-bold mb-2">{children}</h2>;
},
h3({ children }) {
return <h3 className="text-sm font-bold mb-1">{children}</h3>;
},
blockquote({ children }) {
return (
<blockquote className={`border-l-4 pl-3 my-2 ${isDarkMode ? 'border-gray-600' : 'border-gray-300'}`}>
{children}
</blockquote>
);
},
a({ children, href }) {
return (
<a href={href} className="text-blue-400 hover:underline" target="_blank" rel="noopener noreferrer">
{children}
</a>
);
},
table({ children }) {
return (
<div className="overflow-x-auto my-2">
<table className="min-w-full border-collapse">
{children}
</table>
</div>
);
},
thead({ children }) {
return <thead className={`${isDarkMode ? 'bg-gray-700' : 'bg-gray-200'}`}>{children}</thead>;
},
th({ children }) {
return <th className="border px-3 py-2 text-left font-semibold">{children}</th>;
},
td({ children }) {
return <td className="border px-3 py-2">{children}</td>;
},
}}
>
{message.content}
</ReactMarkdown>
</div>
<div
className={`text-xs mt-2 ${
message.role === 'user'
? 'text-blue-200'
: isDarkMode
? 'text-gray-500'
: 'text-gray-400'
}`}
>
{formatTime(message.timestamp)}
</div>
</div>
</div>
))
)}
{isLoading && (
<div className="flex gap-3">
<div
className={`flex-shrink-0 w-8 h-8 rounded-full flex items-center justify-center ${
isDarkMode ? 'bg-gray-700' : 'bg-gray-200'
}`}
>
<Bot size={18} className={isDarkMode ? 'text-gray-300' : 'text-gray-600'} />
</div>
<div
className={`rounded-2xl px-4 py-3 ${
isDarkMode ? 'bg-gray-800' : 'bg-gray-100'
}`}
>
<div className="flex gap-1">
<span
className={`w-2 h-2 rounded-full animate-bounce ${
isDarkMode ? 'bg-gray-500' : 'bg-gray-400'
}`}
style={{ animationDelay: '0ms' }}
/>
<span
className={`w-2 h-2 rounded-full animate-bounce ${
isDarkMode ? 'bg-gray-500' : 'bg-gray-400'
}`}
style={{ animationDelay: '150ms' }}
/>
<span
className={`w-2 h-2 rounded-full animate-bounce ${
isDarkMode ? 'bg-gray-500' : 'bg-gray-400'
}`}
style={{ animationDelay: '300ms' }}
/>
</div>
</div>
</div>
)}
<div ref={messagesEndRef} />
</div>
);
};
frontend/src/components/Sidebar.tsx
+129
View File
@@ -0,0 +1,129 @@
import React from 'react';
import { Plus, Trash2, MessageSquare } from 'lucide-react';
import { Session } from '../types';
interface SidebarProps {
sessions: Session[];
activeSessionId: string | null;
onSessionSelect: (id: string) => void;
onSessionCreate: () => void;
onSessionDelete: (id: string) => void;
isDarkMode: boolean;
}
export const Sidebar: React.FC<SidebarProps> = ({
sessions,
activeSessionId,
onSessionSelect,
onSessionCreate,
onSessionDelete,
isDarkMode,
}) => {
const formatDate = (timestamp: number) => {
const date = new Date(timestamp);
return date.toLocaleDateString('de-DE', {
day: '2-digit',
month: '2-digit',
hour: '2-digit',
minute: '2-digit',
});
};
return (
<div className={`w-80 flex flex-col border-r ${isDarkMode ? 'bg-gray-900 border-gray-700' : 'bg-white border-gray-200'}`}>
<div className={`p-4 border-b ${isDarkMode ? 'border-gray-700' : 'border-gray-200'}`}>
<button
onClick={onSessionCreate}
className={`w-full flex items-center justify-center gap-2 px-4 py-2 rounded-lg font-medium transition-colors ${
isDarkMode
? 'bg-blue-600 hover:bg-blue-700 text-white'
: 'bg-blue-500 hover:bg-blue-600 text-white'
}`}
>
<Plus size={20} />
<span>Neuer Chat</span>
</button>
</div>
<div className="flex-1 overflow-y-auto">
{sessions.length === 0 ? (
<div className={`p-4 text-center text-sm ${isDarkMode ? 'text-gray-400' : 'text-gray-500'}`}>
Keine Chats vorhanden
</div>
) : (
<div className="p-2 space-y-1">
{sessions.map((session) => (
<div
key={session.id}
onClick={() => onSessionSelect(session.id)}
className={`group flex items-center gap-3 p-3 rounded-lg cursor-pointer transition-colors ${
activeSessionId === session.id
? isDarkMode
? 'bg-blue-900/30 border-l-2 border-blue-500'
: 'bg-blue-50 border-l-2 border-blue-500'
: isDarkMode
? 'hover:bg-gray-800'
: 'hover:bg-gray-100'
} ${isDarkMode ? 'border-gray-700' : 'border-gray-200'}`}
>
<MessageSquare
size={18}
className={`flex-shrink-0 ${
activeSessionId === session.id
? isDarkMode
? 'text-blue-400'
: 'text-blue-500'
: isDarkMode
? 'text-gray-500'
: 'text-gray-400'
}`}
/>
<div className="flex-1 min-w-0">
<p
className={`text-sm font-medium truncate ${
activeSessionId === session.id
? isDarkMode
? 'text-blue-100'
: 'text-blue-900'
: isDarkMode
? 'text-gray-200'
: 'text-gray-700'
}`}
>
{session.title || 'Neuer Chat'}
</p>
<p
className={`text-xs ${
isDarkMode ? 'text-gray-500' : 'text-gray-400'
}`}
>
{formatDate(session.updatedAt)}
</p>
</div>
<button
onClick={(e) => {
e.stopPropagation();
onSessionDelete(session.id);
}}
className={`opacity-0 group-hover:opacity-100 p-1 rounded transition-all ${
isDarkMode
? 'hover:bg-red-900/50 text-gray-500 hover:text-red-400'
: 'hover:bg-red-50 text-gray-400 hover:text-red-500'
}`}
title="Chat löschen"
>
<Trash2 size={16} />
</button>
</div>
))}
</div>
)}
</div>
<div className={`p-4 border-t text-xs ${isDarkMode ? 'border-gray-700 text-gray-500' : 'border-gray-200 text-gray-400'}`}>
<p>Privacy Gateway Chat</p>
<p>Alle Daten lokal gespeichert</p>
</div>
</div>
);
};
frontend/src/index.css
+72
View File
@@ -0,0 +1,72 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
:root {
font-family: Inter, system-ui, Avenir, Helvetica, Arial, sans-serif;
line-height: 1.5;
font-weight: 400;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
html, body, #root {
height: 100%;
overflow: hidden;
}
/* Custom scrollbar */
::-webkit-scrollbar {
width: 8px;
height: 8px;
}
::-webkit-scrollbar-track {
background: transparent;
}
::-webkit-scrollbar-thumb {
background: rgba(156, 163, 175, 0.5);
border-radius: 4px;
}
::-webkit-scrollbar-thumb:hover {
background: rgba(156, 163, 175, 0.7);
}
/* Code block styles */
pre {
border-radius: 0.5rem;
margin: 0.5rem 0 !important;
}
code {
font-family: 'Fira Code', 'Monaco', 'Consolas', monospace;
}
/* Smooth transitions */
* {
transition-property: background-color, border-color;
transition-timing-function: cubic-bezier(0.4, 0, 0.2, 1);
transition-duration: 150ms;
}
/* Animation */
@keyframes fadeIn {
from {
opacity: 0;
transform: translateY(10px);
}
to {
opacity: 1;
transform: translateY(0);
}
}
.fade-in {
animation: fadeIn 0.3s ease-out;
}
frontend/src/main.tsx
+10
View File
@@ -0,0 +1,10 @@
import React from 'react';
import ReactDOM from 'react-dom/client';
import App from './App';
import './index.css';
ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
<App />
</React.StrictMode>
);
frontend/src/types/index.ts
+26
View File
@@ -0,0 +1,26 @@
export interface Message {
id: string;
role: 'user' | 'assistant';
content: string;
timestamp: number;
}
export interface Session {
id: string;
title: string;
messages: Message[];
createdAt: number;
updatedAt: number;
}
export interface CreateSessionRequest {
title?: string;
}
export interface ChatRequest {
message: string;
}
export interface ChatResponse {
message: Message;
}
frontend/tailwind.config.js
+17
View File
@@ -0,0 +1,17 @@
/** @type {import('tailwindcss').Config} */
export default {
content: [
"./index.html",
"./src/**/*.{js,ts,jsx,tsx}",
],
darkMode: 'class',
theme: {
extend: {
fontFamily: {
sans: ['Inter', 'system-ui', 'Avenir', 'Helvetica', 'Arial', 'sans-serif'],
mono: ['Fira Code', 'Monaco', 'Consolas', 'monospace'],
},
},
},
plugins: [],
}
frontend/tsconfig.json
+25
View File
@@ -0,0 +1,25 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"baseUrl": ".",
"paths": {
"@/*": ["src/*"]
}
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
}
frontend/tsconfig.node.json
+11
View File
@@ -0,0 +1,11 @@
{
"compilerOptions": {
"composite": true,
"skipLibCheck": true,
"module": "ESNext",
"moduleResolution": "bundler",
"allowSyntheticDefaultImports": true,
"strict": true
},
"include": ["vite.config.ts"]
}
frontend/vite.config.ts
+30
View File
@@ -0,0 +1,30 @@
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
// https://vitejs.dev/config/
export default defineConfig({
plugins: [react()],
server: {
port: 5173,
host: true,
proxy: {
'/api': {
target: 'http://localhost:3000',
changeOrigin: true,
},
},
},
build: {
outDir: 'dist',
sourcemap: true,
rollupOptions: {
output: {
manualChunks: {
'vendor-react': ['react', 'react-dom'],
'vendor-markdown': ['react-markdown', 'react-syntax-highlighter', 'remark-gfm'],
'vendor-icons': ['lucide-react'],
},
},
},
},
});
init.sql
+69
View File
@@ -0,0 +1,69 @@
-- Privacy Gateway Datenbank-Schema
-- Sessions/Tabs für Multi-Chat
CREATE TABLE IF NOT EXISTS sessions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(255) NOT NULL,
model VARCHAR(100) DEFAULT 'llama3.2:latest',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
metadata JSONB DEFAULT '{}'
);
-- Nachrichten pro Session
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
session_id UUID REFERENCES sessions(id) ON DELETE CASCADE,
role VARCHAR(20) NOT NULL CHECK (role IN ('user', 'assistant', 'system')),
original_content TEXT NOT NULL,
anonymized_content TEXT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- PII-Mappings für Re-Identifizierung
CREATE TABLE IF NOT EXISTS pii_mappings (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
session_id UUID REFERENCES sessions(id) ON DELETE CASCADE,
message_id UUID REFERENCES messages(id) ON DELETE CASCADE,
pii_type VARCHAR(50) NOT NULL,
original_value TEXT NOT NULL,
anonymized_value TEXT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- PII-Typen-Enum
CREATE TYPE pii_type AS ENUM (
'name_person',
'name_company',
'address',
'email',
'phone',
'birthdate',
'account_number',
'iban',
'employee_id',
'credit_card',
'tax_id',
'custom'
);
-- Indexe für Performance
CREATE INDEX IF NOT EXISTS idx_messages_session ON messages(session_id);
CREATE INDEX IF NOT EXISTS idx_messages_created ON messages(created_at);
CREATE INDEX IF NOT EXISTS idx_pii_session ON pii_mappings(session_id);
CREATE INDEX IF NOT EXISTS idx_pii_message ON pii_mappings(message_id);
CREATE INDEX IF NOT EXISTS idx_pii_anonymized ON pii_mappings(anonymized_value);
-- Trigger für updated_at
CREATE OR REPLACE FUNCTION update_updated_at_column()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = CURRENT_TIMESTAMP;
RETURN NEW;
END;
$$ language 'plpgsql';
CREATE TRIGGER update_sessions_updated_at
BEFORE UPDATE ON sessions
FOR EACH ROW
EXECUTE FUNCTION update_updated_at_column();