GW_MC/DreamChat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: add pnpm workspace configuration for apps and packages

Browse Source
This commit is contained in:
GW_MC
2026-02-23 21:04:19 +08:00
parent ab02758382
commit 932f384f0d
31 changed files with 9081 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files

78
doc/architecture.md
View File

@@ -101,7 +101,7 @@ DreamChat is a character simulation platform built with a modular, extensible ar
#### 1. Auth Module
```typescript
// Dual authentication strategy
- KeycloakStrategy (OAuth2/OIDC)
- KeycloakStrategy (OAuth2/OIDC with group/role/attribute authorization)
- LocalStrategy (Password-based)
- JWT Guard for stateless auth
- Roles: USER, ADMIN
@@ -110,6 +110,13 @@ DreamChat is a character simulation platform built with a modular, extensible ar
- id, email, username
- passwordHash, keycloakSub
- role, isActive
// Keycloak Authorization
- Validates group membership (KEYCLOAK_REQUIRED_GROUP)
- Checks realm roles (KEYCLOAK_REQUIRED_ROLE)
- Checks client roles (KEYCLOAK_REQUIRED_CLIENT_ROLE)
- Validates user attributes (KEYCLOAK_REQUIRED_ATTRIBUTE)
- Auto-creates users on first login (if KEYCLOAK_AUTO_CREATE_USER=true)
```
#### 2. Character Module
@@ -119,14 +126,23 @@ DreamChat is a character simulation platform built with a modular, extensible ar
- CharacterRepository (Prisma)
- DTOs: CreateCharacterDto, UpdateCharacterDto, CharacterResponseDto
Entities:
Prisma Models:
- Character
- id, name, avatar
- id, name, avatarUrl
- personalityPrompt: string
- attributes: JSON (complex attribute system)
- backstory: string
- knowledgeSources: CharacterKnowledge[]
- vectorMemories: VectorMemory[]
- createdBy: User
- createdAt, updatedAt
- CharacterKnowledge
- id, name, sourceType (file/url/manual)
- sourceName, mimeType, fileSize
- rawContent: string
- status (pending/processing/completed/failed)
- processingInfo: JSON
- vectorMemories: VectorMemory[]
- characterId: Character
```
#### 3. Chat Module (MVP)
@@ -169,21 +185,32 @@ WebSocket Events:
- HuggingFaceAPIProvider: Uses HuggingFace Inference API
- VectorStoreService (uses Prisma with pgvector extension)
- addDocument(conversationId, content, metadata)
- similaritySearch(conversationId, query, k=5)
- addDocument(targetId, content, metadata, memoryType)
- similaritySearch(targetId, query, k=5, memoryType)
- Supports both conversation and character memory
- Uses raw Prisma queries with pgvector operators
- MemoryManager
- buildContext(conversationId, currentMessage): string
- buildConversationContext(conversationId, currentMessage): string
- buildCharacterContext(characterId, query): string
- summarizeOldMessages(conversationId): Promise<void>
- retrieveRelevantMemories(conversationId, query): Document[]
- retrieveRelevantMemories(targetId, query, memoryType): Document[]
- CharacterKnowledgeService
- importKnowledge(characterId, file/url)
- chunkAndEmbed(knowledgeId, content)
- processKnowledgeSource(knowledgeId)
- searchCharacterKnowledge(characterId, query)
Prisma Model:
- VectorMemory
- id
- conversationId (relation)
- content: String
- embedding: Unsupported("vector") // pgvector type
- embedding: Unsupported("vector")
- memoryType: enum ('conversation' | 'character')
- conversationId?: Conversation
- characterId?: Character
- knowledgeId?: CharacterKnowledge
- metadata: Json?
- createdAt: DateTime
```
@@ -235,6 +262,23 @@ class DataPreprocessor {
chunk(text: string, maxChunkSize: number): string[];
extractEntities(text: string): Entity[];
}
// Character Knowledge Import Service
class CharacterKnowledgeService {
// Import file/URL as character knowledge
importKnowledge(characterId: string, file: File): Promise<CharacterKnowledge>;
importFromUrl(characterId: string, url: string): Promise<CharacterKnowledge>;
// Process and embed knowledge
processKnowledge(knowledgeId: string): Promise<void>;
chunkAndEmbed(knowledgeId: string, content: string): Promise<void>;
// Search character knowledge
searchKnowledge(characterId: string, query: string): Promise<VectorMemory[]>;
// Get knowledge context for LLM
buildKnowledgeContext(characterId: string, query: string): string;
}
```
### Frontend (React + Vite)
@@ -438,9 +482,17 @@ dreamchat/
│ └── typescript/
│
├── prisma/ # Database schema (shared)
│ ├── schema.prisma
│ ├── schema.prisma # Main schema with imports
│ ├── migrations/
│ └── seed.ts
│ ├── seed.ts
│ └── models/ # Individual model files
│ ├── user.prisma
│ ├── character.prisma
│ ├── conversation.prisma
│ ├── message.prisma
│ ├── vectorMemory.prisma
│ ├── importDocument.prisma
│ └── storyBranch.prisma
│
├── docker-compose.yml
├── pnpm-workspace.yaml

327
doc/database-schema.md
View File

@@ -23,35 +23,41 @@ CREATE EXTENSION IF NOT EXISTS "pgvector";
│ username │ │ name │ │ user_id (FK) │
│ password_hash │ │ avatar_url │ │ title │
│ keycloak_sub │ │ personality │ │ created_at │
│ role │ │ backstory │ │ updated_at │
│ created_at │ │ attributes │ └────────┬────────┘
│ updated_at │ │ created_at │ │
└─────────────────┘ │ updated_at │ │
└─────────────────┘ │
┌─────────────────┐ │
│import_documents │ │
├─────────────────┤ │
│ id (PK) │ │
│ user_id (FK) │ │
│ source_type │ ┌─────────────────┐ │
│ source_name │ │ messages │◄───────────────┘
│ content │ ├─────────────────┤
│ metadata │ │ id (PK) │
│ vector_id │ │ conversation_id │
│ created_at │ │ role │
└─────────────────┘ │ content │
│ tokens_used │
│ model │
┌─────────────────┐ │ metadata │
│vector_memories │ │ created_at │
├─────────────────┤ └─────────────────┘
│ id (PK) │
│ conversation_id │ ┌─────────────────┐
│ content │ │ story_branches │ (Phase 2)
│ embedding │ ├─────────────────┤
│ metadata │ │ id (PK) │
│ created_at │ │ conversation_id │
└─────────────────┘ │ parent_id (FK) │
│ role │ │ attributes │ │ updated_at │
│ created_at │ │ created_at │ └────────┬────────┘
│ updated_at │ │ updated_at │ │
└─────────────────┘ └────────┬────────┘ │
│ │
┌────────┴────────┐ │
│character_knowledge│ │
├─────────────────┤ │
│ id (PK) │◄───────────────┤
│ character_id │ │
│ name │ │
│ source_type │ │
│ raw_content │ │
│ status │ │
└────────┬────────┘ │
│ │
┌─────────────────┐ ┌────────┴────────┐ ┌────────┴────────┐
│import_documents │ │ vector_memories│ │ messages │
├─────────────────┤ ├─────────────────┤ ├─────────────────┤
│ id (PK) │ │ id (PK) │ │ id (PK) │
│ user_id (FK) │ │ content │ │ conversation_id │
│ source_type │ │ embedding │ │ role │
│ source_name │ │ memory_type │ │ content │
│ content │ │ conversation_id │ │ tokens_used │
│ status │ │ character_id │ │ model │
└─────────────────┘ │ knowledge_id │ │ metadata │
│ created_at │ │ created_at │
└─────────────────┘ └─────────────────┘
│
┌────────┴────────┐
│ story_branches │ (Phase 2)
├─────────────────┤
│ id (PK) │
│ conversation_id │
│ parent_id (FK) │
│ content │
│ direction │
│ metadata │
@@ -89,7 +95,7 @@ CREATE INDEX idx_users_keycloak_sub ON users(keycloak_sub);
### 2. characters
Character definitions with complex attribute system (JSONB for flexibility).
Character definitions with complex attribute system. Character knowledge is stored separately in `character_knowledge` with embeddings.
```sql
CREATE TABLE characters (
@@ -101,9 +107,6 @@ CREATE TABLE characters (
-- Core personality prompt sent to LLM
personality_prompt TEXT NOT NULL,
-- Backstory context for the character
backstory TEXT,
-- Complex attribute system (structured JSON)
-- Example: {"traits": ["brave", "witty"], "age": 25, "species": "human"}
attributes JSONB DEFAULT '{}',
@@ -181,19 +184,53 @@ CREATE INDEX idx_messages_created_at ON messages(created_at);
CREATE INDEX idx_messages_conversation_created ON messages(conversation_id, created_at);
```
### 5. vector_memories
### 5. character_knowledge
Vector embeddings for conversation memory using pgvector. Stores chunked content for semantic search.
Multiple knowledge sources for characters. Each source is chunked and stored with embeddings in `vector_memories`.
```sql
-- Create vector extension first
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TYPE import_source_type AS ENUM ('file', 'url', 'manual');
CREATE TYPE import_status AS ENUM ('pending', 'processing', 'completed', 'failed');
CREATE TABLE character_knowledge (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
character_id UUID NOT NULL REFERENCES characters(id) ON DELETE CASCADE,
-- Knowledge source info
name VARCHAR(255) NOT NULL, -- Display name
source_type import_source_type NOT NULL,
source_name VARCHAR(255) NOT NULL, -- Original filename or URL
mime_type VARCHAR(100),
file_size BIGINT,
-- Raw content (before chunking)
raw_content TEXT,
-- Processing status
status import_status DEFAULT 'pending',
processing_info JSONB, -- chunks count, errors, etc.
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE INDEX idx_character_knowledge_character ON character_knowledge(character_id);
CREATE INDEX idx_character_knowledge_status ON character_knowledge(status);
```
### 6. vector_memories
Unified vector embeddings storage for:
- **Character knowledge** - Background info, imported documents (linked to `character_knowledge`)
- **Conversation history** - Chat context (linked to `conversations`)
```sql
CREATE TYPE memory_type AS ENUM ('conversation', 'character');
CREATE TABLE vector_memories (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
conversation_id UUID NOT NULL REFERENCES conversations(id) ON DELETE CASCADE,
-- The text content
-- The text chunk
content TEXT NOT NULL,
-- Vector embedding (configurable dimension based on model)
@@ -201,14 +238,21 @@ CREATE TABLE vector_memories (
-- Must match the EMBEDDING_DIMENSION env var
embedding VECTOR({{EMBEDDING_DIMENSION}}),
-- Metadata for filtering
metadata JSONB DEFAULT '{
"chunk_index": 0,
"source": "conversation",
"timestamp": null
}',
-- Memory type determines the context
memory_type memory_type DEFAULT 'conversation',
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
-- Metadata (chunk_index, source_info, etc.)
metadata JSONB,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
-- Polymorphic relations (at least one must be set)
-- For conversation context
conversation_id UUID REFERENCES conversations(id) ON DELETE CASCADE,
-- For character knowledge
character_id UUID REFERENCES characters(id) ON DELETE CASCADE,
knowledge_id UUID REFERENCES character_knowledge(id) ON DELETE CASCADE
);
-- HNSW index for efficient similarity search
@@ -216,22 +260,22 @@ CREATE TABLE vector_memories (
-- CREATE INDEX idx_vector_memories_embedding ON vector_memories
-- USING hnsw (embedding vector_cosine_ops);
CREATE INDEX idx_vector_memories_conversation ON vector_memories(conversation_id);
CREATE INDEX idx_vector_memories_conversation ON vector_memories(conversation_id) WHERE conversation_id IS NOT NULL;
CREATE INDEX idx_vector_memories_character ON vector_memories(character_id) WHERE character_id IS NOT NULL;
CREATE INDEX idx_vector_memories_knowledge ON vector_memories(knowledge_id) WHERE knowledge_id IS NOT NULL;
CREATE INDEX idx_vector_memories_type ON vector_memories(memory_type);
```
### 6. import_documents
### 7. import_documents
Raw imported documents from files or web scraping.
General-purpose imported documents (not linked to characters). For character knowledge, use `character_knowledge`.
```sql
CREATE TYPE import_source_type AS ENUM ('file', 'url');
CREATE TYPE import_status AS ENUM ('pending', 'processing', 'completed', 'failed');
CREATE TABLE import_documents (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
source_type import_source_type NOT NULL,
source_type import_source_type NOT NULL, -- file, url, manual
source_name VARCHAR(255) NOT NULL, -- filename or URL
-- Mime type for files
@@ -258,7 +302,7 @@ CREATE INDEX idx_import_documents_user_id ON import_documents(user_id);
CREATE INDEX idx_import_documents_status ON import_documents(status);
```
### 7. story_branches (Phase 2)
### 8. story_branches (Phase 2)
Tree structure for branching narratives.
@@ -291,7 +335,7 @@ CREATE INDEX idx_story_branches_conversation ON story_branches(conversation_id);
CREATE INDEX idx_story_branches_parent ON story_branches(parent_id);
```
### 8. conversation_participants (Phase 3 - Multi-Character)
### 9. conversation_participants (Phase 3 - Multi-Character)
Supports multiple characters in a single conversation.
@@ -313,12 +357,73 @@ CREATE TABLE conversation_participants (
CREATE INDEX idx_participants_conversation ON conversation_participants(conversation_id);
```
### Enums
```sql
-- User roles
CREATE TYPE user_role AS ENUM ('USER', 'ADMIN');
-- Message roles
CREATE TYPE message_role AS ENUM ('user', 'assistant', 'system');
-- Import/knowledge source types
CREATE TYPE import_source_type AS ENUM ('file', 'url', 'manual');
-- Processing status
CREATE TYPE import_status AS ENUM ('pending', 'processing', 'completed', 'failed');
-- Vector memory types
CREATE TYPE memory_type AS ENUM ('conversation', 'character');
```
## Prisma Schema (Reference)
Prisma schema uses the [multi-file schema](https://www.prisma.io/docs/orm/prisma-schema/overview/location) feature. Models are organized in `prisma/models/` folder and imported into `schema.prisma`.
### Schema Structure
```
prisma/
├── schema.prisma # Main schema file with imports
├── seed.ts # Database seeding
└── models/
├── user.prisma # User model + UserRole enum
├── character.prisma # Character + CharacterKnowledge models
├── conversation.prisma # Conversation + ConversationParticipant
├── message.prisma # Message model + MessageRole enum
├── vectorMemory.prisma # VectorMemory + MemoryType enum
├── importDocument.prisma # ImportDocument model
└── storyBranch.prisma # StoryBranch model
```
### Main Schema (schema.prisma)
```prisma
// schema.prisma
generator client {
provider = "prisma-client-js"
}
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}
// Import all models from the models folder
import { User } from "./models/user"
import { Character, CharacterKnowledge } from "./models/character"
import { Conversation, ConversationParticipant } from "./models/conversation"
import { Message } from "./models/message"
import { VectorMemory } from "./models/vectorMemory"
import { ImportDocument } from "./models/importDocument"
import { StoryBranch } from "./models/storyBranch"
```
### Full Schema Definition
```prisma
// schema.prisma
// models/user.prisma
generator client {
provider = "prisma-client-js"
@@ -344,6 +449,7 @@ enum MessageRole {
enum ImportSourceType {
file
url
manual
}
enum ImportStatus {
@@ -353,6 +459,11 @@ enum ImportStatus {
failed
}
enum MemoryType {
conversation
character
}
// Models
model User {
id String @id @default(uuid())
@@ -378,7 +489,6 @@ model Character {
name String
avatarUrl String?
personalityPrompt String
backstory String?
attributes Json @default("{}")
config Json @default("{}")
isPublic Boolean @default(false)
@@ -386,30 +496,53 @@ model Character {
updatedAt DateTime @updatedAt
userId String
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
conversations Conversation[]
knowledgeSources CharacterKnowledge[]
vectorMemories VectorMemory[]
@@index([userId])
@@index([name])
}
model Conversation {
id String @id @default(uuid())
title String?
messageCount Int @default(0)
totalTokens Int @default(0)
settings Json @default("{}")
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
model CharacterKnowledge {
id String @id @default(uuid())
name String
sourceType ImportSourceType
sourceName String
mimeType String?
fileSize BigInt?
rawContent String?
status ImportStatus @default(pending)
processingInfo Json?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
userId String
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
characterId String
character Character @relation(fields: [characterId], references: [id], onDelete: Cascade)
messages Message[]
characterId String
character Character @relation(fields: [characterId], references: [id], onDelete: Cascade)
vectorMemories VectorMemory[]
@@index([characterId])
@@index([status])
}
model Conversation {
id String @id @default(uuid())
title String?
messageCount Int @default(0)
totalTokens Int @default(0)
settings Json @default("{}")
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
userId String
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
characterId String
character Character @relation(fields: [characterId], references: [id], onDelete: Cascade)
messages Message[]
vectorMemories VectorMemory[]
storyBranches StoryBranch[]
participants ConversationParticipant[]
storyBranches StoryBranch[]
participants ConversationParticipant[]
@@index([userId])
@@index([characterId])
@@ -436,14 +569,24 @@ model Message {
model VectorMemory {
id String @id @default(uuid())
content String
embedding Unsupported("vector")? // pgvector extension
embedding Unsupported("vector")?
memoryType MemoryType @default(conversation)
metadata Json?
createdAt DateTime @default(now())
conversationId String
conversation Conversation @relation(fields: [conversationId], references: [id], onDelete: Cascade)
conversationId String?
conversation Conversation? @relation(fields: [conversationId], references: [id], onDelete: Cascade)
characterId String?
character Character? @relation(fields: [characterId], references: [id], onDelete: Cascade)
knowledgeId String?
knowledge CharacterKnowledge? @relation(fields: [knowledgeId], references: [id], onDelete: Cascade)
@@index([conversationId])
@@index([characterId])
@@index([knowledgeId])
@@index([memoryType])
}
model ImportDocument {
@@ -546,11 +689,16 @@ export class CharacterRepository {
```typescript
// Similarity search using pgvector with Prisma
async similaritySearch(
conversationId: string,
targetId: string,
queryEmbedding: number[],
memoryType: MemoryType,
k: number = 5
) {
// Using raw query for pgvector-specific operations
// Build the where clause based on memory type
const whereClause = memoryType === 'conversation'
? { conversationId: targetId, memoryType }
: { characterId: targetId, memoryType };
const results = await this.prisma.$queryRaw`
SELECT
id,
@@ -558,7 +706,7 @@ async similaritySearch(
metadata,
embedding <=> ${queryEmbedding}::vector as distance
FROM "VectorMemory"
WHERE "conversationId" = ${conversationId}
WHERE ${whereClause}
ORDER BY embedding <=> ${queryEmbedding}::vector
LIMIT ${k}
`;
@@ -566,20 +714,23 @@ async similaritySearch(
return results;
}
// Alternative: using cosine similarity
async similaritySearchCosine(
conversationId: string,
// Search character knowledge
async searchCharacterKnowledge(
characterId: string,
queryEmbedding: number[],
k: number = 5
) {
const results = await this.prisma.$queryRaw`
SELECT
id,
content,
metadata,
1 - (embedding <=> ${queryEmbedding}::vector) as similarity
FROM "VectorMemory"
WHERE "conversationId" = ${conversationId}
vm.id,
vm.content,
vm.metadata,
ck.name as source_name,
1 - (vm.embedding <=> ${queryEmbedding}::vector) as similarity
FROM "VectorMemory" vm
JOIN "CharacterKnowledge" ck ON vm."knowledgeId" = ck.id
WHERE vm."characterId" = ${characterId}
AND vm."memoryType" = 'character'
ORDER BY similarity DESC
LIMIT ${k}
`;

211
doc/deployment.md
View File

@@ -112,24 +112,6 @@ services:
networks:
- dreamchat-network
keycloak:
image: quay.io/keycloak/keycloak:23.0
restart: unless-stopped
command: start-dev
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
KC_DB: postgres
KC_DB_URL: jdbc:postgresql://db:5432/keycloak
KC_DB_USERNAME: postgres
KC_DB_PASSWORD: postgres
ports:
- "8080:8080"
depends_on:
- db
networks:
- dreamchat-network
volumes:
postgres-data:
@@ -138,6 +120,8 @@ networks:
driver: bridge
```
**Note:** Keycloak is configured as an external service. Set `KEYCLOAK_URL` in your environment to point to your external Keycloak instance.
### .devcontainer/Dockerfile
```dockerfile
@@ -274,15 +258,15 @@ services:
timeout: 10s
retries: 3
# Frontend
# Frontend (static file server)
# Note: External reverse proxy expected for SSL and routing
frontend:
build:
context: .
dockerfile: apps/frontend/Dockerfile
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "3001:3000"
environment:
- VITE_API_URL=/api
- VITE_WS_URL=/ws
@@ -319,23 +303,6 @@ services:
networks:
- dreamchat-network
# Nginx Reverse Proxy (optional)
nginx:
image: nginx:alpine
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./nginx/ssl:/etc/nginx/ssl:ro
- model-cache:/model-cache:ro
depends_on:
- backend
- frontend
networks:
- dreamchat-network
volumes:
postgres-data:
redis-data:
@@ -400,7 +367,7 @@ KEYCLOAK_CLIENT_SECRET=your_keycloak_secret
```dockerfile
# apps/backend/Dockerfile
FROM node:20-alpine AS base
FROM node:24-alpine AS base
RUN npm install -g pnpm@8
FROM base AS dependencies
@@ -456,7 +423,7 @@ CMD ["node", "dist/main.js"]
```dockerfile
# apps/frontend/Dockerfile
FROM node:20-alpine AS base
FROM node:24-alpine AS base
RUN npm install -g pnpm@8
FROM base AS dependencies
@@ -486,46 +453,59 @@ RUN pnpm --filter @dreamchat/shared build
# Build frontend
RUN pnpm --filter @dreamchat/frontend build
# Production with Nginx
FROM nginx:alpine
# Production stage - using serve for static files
# External reverse proxy (nginx/traefik/etc.) expected
FROM node:24-alpine AS production
WORKDIR /app
# Install serve
RUN npm install -g serve
# Copy built assets
COPY --from=build /app/apps/frontend/dist /usr/share/nginx/html
COPY --from=build /app/apps/frontend/dist ./dist
# Copy nginx config
COPY apps/frontend/nginx.conf /etc/nginx/conf.d/default.conf
# Create non-root user
RUN addgroup -g 1001 -S nodejs
RUN adduser -S nodejs -u 1001
USER nodejs
EXPOSE 80
EXPOSE 3000
CMD ["nginx", "-g", "daemon off;"]
# Serve static files
# Note: External reverse proxy should handle:
# - SSL/TLS termination
# - Path routing (/api -> backend, / -> frontend)
# - WebSocket proxying
CMD ["serve", "-s", "dist", "-l", "3000"]
```
### frontend/nginx.conf
### External Reverse Proxy Configuration
The frontend container serves static files on port 3000. An external reverse proxy is expected to handle:
- **SSL/TLS termination**
- **Path routing**:
- `/api/*` → Backend (port 3000)
- `/ws` → Backend WebSocket (port 3000)
- `/*` → Frontend (port 3001)
- **Static file caching**
Example nginx configuration:
```nginx
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html;
listen 443 ssl;
server_name dreamchat.example.com;
# Gzip compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
ssl_certificate /path/to/cert.pem;
ssl_certificate_key /path/to/key.pem;
# API proxy
location /api {
proxy_pass http://backend:3000/api;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_cache_bypass $http_upgrade;
}
# WebSocket proxy
@@ -534,45 +514,110 @@ server {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Static files
# Frontend static files
location / {
try_files $uri $uri/ /index.html;
add_header Cache-Control "public, max-age=31536000, immutable";
}
# Health check
location /health {
access_log off;
return 200 "healthy\n";
add_header Content-Type text/plain;
proxy_pass http://frontend:3000;
proxy_http_version 1.1;
proxy_set_header Host $host;
}
}
```
## Keycloak Configuration
## Keycloak Configuration (External)
### Initial Setup
Keycloak is configured as an external service with support for group/role/attribute-based authorization and auto-user creation.
1. Access Keycloak admin console: `http://localhost:8080/admin`
2. Login with admin credentials
3. Create new realm: `dreamchat`
4. Create client: `dreamchat-backend`
### Prerequisites
1. Have a running Keycloak instance (self-hosted or managed)
2. Configure the following environment variables in `.env`:
```bash
# Basic Keycloak settings
KEYCLOAK_ENABLED=true
KEYCLOAK_URL=http://your-keycloak-server:8080
KEYCLOAK_REALM=dreamchat
KEYCLOAK_CLIENT_ID=dreamchat-backend
KEYCLOAK_CLIENT_SECRET=your_keycloak_secret
# Authorization settings (optional but recommended)
KEYCLOAK_REQUIRED_GROUP=dreamchat-users
KEYCLOAK_REQUIRED_ROLE=dreamchat-access
KEYCLOAK_REQUIRED_CLIENT_ROLE=user
KEYCLOAK_REQUIRED_ATTRIBUTE=approved:true
# Auto-create users
KEYCLOAK_AUTO_CREATE_USER=true
KEYCLOAK_DEFAULT_USER_ROLE=USER
```
### Keycloak Realm Setup
1. Access your Keycloak admin console
2. Create new realm: `dreamchat`
3. Create client: `dreamchat-backend`
- Client authentication: ON
- Authorization: ON
- Valid redirect URIs: `http://localhost:3000/*`
- Web origins: `http://localhost:3000`
5. Create client: `dreamchat-frontend`
4. Create client: `dreamchat-frontend`
- Client authentication: OFF
- Valid redirect URIs: `http://localhost:5173/*`
- Web origins: `http://localhost:5173`
### Authorization Configuration
You can restrict access based on:
**1. Group Membership**
```bash
KEYCLOAK_REQUIRED_GROUP=dreamchat-users
```
Users must be members of this Keycloak group to access the application.
**2. Realm Role**
```bash
KEYCLOAK_REQUIRED_ROLE=dreamchat-access
```
Users must have this realm-level role.
**3. Client Role**
```bash
KEYCLOAK_REQUIRED_CLIENT_ROLE=user
```
Users must have this role for the `dreamchat-backend` client.
**4. User Attribute**
```bash
KEYCLOAK_REQUIRED_ATTRIBUTE=department:engineering
# or
KEYCLOAK_REQUIRED_ATTRIBUTE=approved:true
```
Users must have this attribute with the specified value.
### User Auto-Creation
When `KEYCLOAK_AUTO_CREATE_USER=true`:
- Users are automatically created in the database on first Keycloak login
- Username is derived from Keycloak preferred_username
- Email is taken from Keycloak email claim
- Role is set to `KEYCLOAK_DEFAULT_USER_ROLE` (default: USER)
- The `keycloakSub` field links the local user to Keycloak
When `KEYCLOAK_AUTO_CREATE_USER=false`:
- Only existing local users can log in via Keycloak
- The `keycloakSub` must match between Keycloak and local user
### Example Keycloak Group/Role Setup
1. Create a group: `dreamchat-users`
2. Create a realm role: `dreamchat-access`
3. Assign the group and/or role to users who should have access
4. Configure `KEYCLOAK_REQUIRED_GROUP` and/or `KEYCLOAK_REQUIRED_ROLE`
### realm-export.json (Optional)
```json

12
doc/implementation-plan.md
View File

@@ -29,7 +29,7 @@ This document outlines the phased implementation approach for DreamChat.
- [ ] Initialize NestJS app in `apps/backend`
- [ ] Configure Prisma ORM with PostgreSQL
- [ ] Install and configure pgvector extension
- [ ] Define Prisma schema for all entities
- [ ] Define Prisma schema using multi-file structure (`prisma/models/`)
- [ ] Configure Jest for unit testing
- [ ] Set up Swagger/OpenAPI
- [ ] Implement basic logging (Winston/Pino)
@@ -204,10 +204,15 @@ HUGGINGFACE_API_KEY=hf_...
#### Backend Tasks
- [ ] Import adapter interface
- [ ] Text file adapter
- [ ] Text file adapter (txt, md)
- [ ] File upload endpoint
- [ ] Data preprocessor (cleaning)
- [ ] Basic text chunking
- [ ] CharacterKnowledge Prisma model
- [ ] CharacterKnowledgeService
- [ ] Import file/URL as knowledge
- [ ] Chunk and embed content
- [ ] Store in VectorMemory with type='character'
#### Frontend Tasks
- [ ] Import page
@@ -274,7 +279,8 @@ HUGGINGFACE_API_KEY=hf_...
- [ ] Predefined scraper: AO3
- [ ] Predefined scraper: FanFiction.net
- [ ] URL validation and scraper selection
- [ ] URL import endpoint
- [ ] URL import endpoint (stores as CharacterKnowledge)
- [ ] PDF and Markdown support for character knowledge
#### Frontend Tasks
- [ ] URL import component

6
doc/monorepo-guide.md
View File

@@ -379,7 +379,7 @@ import {
@WebSocketGateway({ namespace: 'chat' })
export class ChatGateway implements OnGatewayConnection {
@SubscribeMessage(WebSocketEventType.JOIN_CONVERSATION)
async handleJoin(
@MessageBody() payload: JoinConversationPayload,
@@ -496,7 +496,7 @@ export function useChat(conversationId: string) {
const sendMessage = useCallback((content: string) => {
if (!socket) return;
const payload: SendMessagePayload = {
conversationId,
content,
@@ -616,7 +616,7 @@ services:
```dockerfile
# apps/backend/Dockerfile
FROM node:20-alpine AS base
FROM node:24-alpine AS base
RUN npm install -g pnpm
FROM base AS dependencies