DreamChat/doc/deployment.md

# DreamChat Deployment Guide

## Overview

This document covers the deployment configuration including Docker Compose setup, DevContainer configuration, and production deployment procedures.

## DevContainer Setup

### .devcontainer/devcontainer.json

```json
{
  "name": "DreamChat Development",
  "dockerComposeFile": "docker-compose.yml",
  "service": "app",
  "workspaceFolder": "/workspace",
  "features": {
    "ghcr.io/devcontainers/features/node:1": {
      "version": "20"
    },
    "ghcr.io/devcontainers/features/docker-in-docker:2": {}
  },
  "customizations": {
    "vscode": {
      "extensions": [
        "dbaeumer.vscode-eslint",
        "esbenp.prettier-vscode",
        "bradlc.vscode-tailwindcss",
        "ms-vscode.vscode-typescript-next",
        "nestjs.vscode-nestjs",
        "prisma.prisma"
      ],
      "settings": {
        "editor.formatOnSave": true,
        "editor.defaultFormatter": "esbenp.prettier-vscode",
        "typescript.preferences.importModuleSpecifier": "relative"
      }
    }
  },
  "forwardPorts": [3000, 5173, 5432, 8080],
  "portsAttributes": {
    "3000": {
      "label": "Backend API",
      "onAutoForward": "notify"
    },
    "5173": {
      "label": "Frontend Dev Server",
      "onAutoForward": "notify"
    },
    "5432": {
      "label": "PostgreSQL",
      "onAutoForward": "silent"
    },
    "8080": {
      "label": "Keycloak",
      "onAutoForward": "notify"
    }
  },
  "postCreateCommand": "bash .devcontainer/post-create.sh",
  "remoteUser": "node",
  "mounts": [
    "source=${localWorkspaceFolderBasename}-node_modules,target=${containerWorkspaceFolder}/node_modules,type=volume",
    "source=${localWorkspaceFolderBasename}-pnpm-store,target=/home/node/.local/share/pnpm/store,type=volume"
  ]
}
```

### .devcontainer/docker-compose.yml

```yaml
version: '3.8'

services:
  app:
    build:
      context: ..
      dockerfile: .devcontainer/Dockerfile
    volumes:
      - ..:/workspace:cached
      - /var/run/docker.sock:/var/run/docker.sock
    command: sleep infinity
    environment:
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/dreamchat
      - REDIS_URL=redis://redis:6379
      - KEYCLOAK_URL=http://keycloak:8080
    depends_on:
      - db
      - redis
      - keycloak
    networks:
      - dreamchat-network

  db:
    image: ankane/pgvector:latest
    restart: unless-stopped
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: dreamchat
    volumes:
      - postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    networks:
      - dreamchat-network

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    ports:
      - "6379:6379"
    networks:
      - dreamchat-network

volumes:
  postgres-data:

networks:
  dreamchat-network:
    driver: bridge
```

**Note:** Keycloak is configured as an external service. Set `KEYCLOAK_URL` in your environment to point to your external Keycloak instance.

### .devcontainer/Dockerfile

```dockerfile
FROM mcr.microsoft.com/devcontainers/typescript-node:20

# Install additional tools
RUN apt-get update && apt-get install -y \
    postgresql-client \
    redis-tools \
    && rm -rf /var/lib/apt/lists/*

# Install pnpm
RUN npm install -g pnpm@8

# Set working directory
WORKDIR /workspace

# Install global packages
RUN npm install -g @nestjs/cli@latest

# Create non-root user
USER node
```

### .devcontainer/post-create.sh

```bash
#!/bin/bash
set -e

echo "🚀 Setting up DreamChat monorepo development environment..."

# Install pnpm globally
npm install -g pnpm@8

# Install all dependencies (uses pnpm workspaces)
echo "📦 Installing dependencies..."
cd /workspace
pnpm install

# Build shared packages first
echo "📦 Building shared packages..."
pnpm --filter @dreamchat/shared build

# Generate Prisma client
pnpm db:generate

# Copy environment files if they don't exist
if [ ! -f /workspace/apps/backend/.env ]; then
    echo "⚙️  Creating backend .env file..."
    cat > /workspace/apps/backend/.env << EOF
NODE_ENV=development
PORT=3000
DATABASE_URL=postgresql://postgres:postgres@db:5432/dreamchat
JWT_SECRET=dev-jwt-secret-change-in-production
JWT_EXPIRES_IN=1h
JWT_REFRESH_EXPIRES_IN=7d
LLM_PROVIDER=openrouter
LLM_API_KEY=your-openrouter-api-key
LLM_MODEL=openai/gpt-4o
KEYCLOAK_URL=http://localhost:8080
KEYCLOAK_REALM=dreamchat
KEYCLOAK_CLIENT_ID=dreamchat-backend
EOF
fi

if [ ! -f /workspace/apps/frontend/.env ]; then
    echo "⚙️  Creating frontend .env file..."
    cat > /workspace/apps/frontend/.env << EOF
VITE_API_URL=http://localhost:3000/api
VITE_WS_URL=ws://localhost:3000
VITE_KEYCLOAK_URL=http://localhost:8080
VITE_KEYCLOAK_REALM=dreamchat
VITE_KEYCLOAK_CLIENT_ID=dreamchat-frontend
EOF
fi

echo "✅ Development environment setup complete!"
echo ""
echo "Next steps:"
echo "1. Start all apps: pnpm dev"
echo "2. Or start individually:"
echo "   - Backend: pnpm --filter @dreamchat/backend dev"
echo "   - Frontend: pnpm --filter @dreamchat/frontend dev"
echo "3. Access Keycloak admin at http://localhost:8080 (admin/admin)"
```

## Docker Compose Production

### docker-compose.yml (Root)

```yaml
version: '3.8'

services:
  # Backend API
  backend:
    build:
      context: .
      dockerfile: apps/backend/Dockerfile
    restart: unless-stopped
    environment:
      - NODE_ENV=production
      - PORT=3000
      - DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD}@db:5432/dreamchat
      - JWT_SECRET=${JWT_SECRET}
      - JWT_EXPIRES_IN=${JWT_EXPIRES_IN:-1h}
      - JWT_REFRESH_EXPIRES_IN=${JWT_REFRESH_EXPIRES_IN:-7d}
      - LLM_PROVIDER=${LLM_PROVIDER}
      - LLM_API_KEY=${LLM_API_KEY}
      - LLM_MODEL=${LLM_MODEL}
      - EMBEDDING_PROVIDER=${EMBEDDING_PROVIDER:-local}
      - EMBEDDING_MODEL=${EMBEDDING_MODEL:-Xenova/all-MiniLM-L6-v2}
      - EMBEDDING_DIMENSION=${EMBEDDING_DIMENSION:-384}
      - EMBEDDING_DEVICE=${EMBEDDING_DEVICE:-cpu}
      - HUGGINGFACE_API_KEY=${HUGGINGFACE_API_KEY}
      - KEYCLOAK_URL=${KEYCLOAK_URL}
      - KEYCLOAK_REALM=${KEYCLOAK_REALM}
      - KEYCLOAK_CLIENT_ID=${KEYCLOAK_CLIENT_ID}
      - KEYCLOAK_CLIENT_SECRET=${KEYCLOAK_CLIENT_SECRET}
    ports:
      - "3000:3000"
    depends_on:
      db:
        condition: service_healthy
    volumes:
      - backend-logs:/app/logs
      - model-cache:/app/models
    networks:
      - dreamchat-network
    healthcheck:
      test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  # Frontend (static file server)
  # Note: External reverse proxy expected for SSL and routing
  frontend:
    build:
      context: .
      dockerfile: apps/frontend/Dockerfile
    restart: unless-stopped
    ports:
      - "3001:3000"
    environment:
      - VITE_API_URL=/api
      - VITE_WS_URL=/ws
    depends_on:
      - backend
    networks:
      - dreamchat-network

  # Database
  db:
    image: ankane/pgvector:latest
    restart: unless-stopped
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: dreamchat
    volumes:
      - postgres-data:/var/lib/postgresql/data
      - ./init-scripts:/docker-entrypoint-initdb.d
    networks:
      - dreamchat-network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5

  # Redis (optional, for session storage and caching)
  redis:
    image: redis:7-alpine
    restart: unless-stopped
    volumes:
      - redis-data:/data
    networks:
      - dreamchat-network

volumes:
  postgres-data:
  redis-data:
  backend-logs:
  model-cache:  # Persist downloaded embedding models

networks:
  dreamchat-network:
    driver: bridge
```

### Embedding Model Cache (Optional)

To avoid re-downloading models on every restart, models are cached in a Docker volume:

```yaml
# Add to docker-compose volumes
volumes:
  model-cache:
```

Models are downloaded on first use and cached at `/app/models` in the backend container.

Common models and their sizes:
- `Xenova/all-MiniLM-L6-v2`: ~80MB (384 dimensions)
- `Xenova/all-mpnet-base-v2`: ~420MB (768 dimensions)
- `BAAI/bge-small-en`: ~130MB (384 dimensions)

### .env.example

```bash
# Database
POSTGRES_PASSWORD=your_secure_password_here

# JWT
JWT_SECRET=your_jwt_secret_key_min_32_chars
JWT_EXPIRES_IN=1h
JWT_REFRESH_EXPIRES_IN=7d

# LLM Configuration
LLM_PROVIDER=openrouter
LLM_API_KEY=sk-or-v1-...
LLM_MODEL=openai/gpt-4o

# Embedding Configuration (Local HuggingFace by default)
EMBEDDING_PROVIDER=local
EMBEDDING_MODEL=Xenova/all-MiniLM-L6-v2
EMBEDDING_DIMENSION=384
EMBEDDING_DEVICE=cpu

# HuggingFace API (optional - if not using local embeddings)
# HUGGINGFACE_API_KEY=hf_...

# Keycloak (optional - for external auth)
KEYCLOAK_URL=http://keycloak:8080
KEYCLOAK_REALM=dreamchat
KEYCLOAK_CLIENT_ID=dreamchat-backend
KEYCLOAK_CLIENT_SECRET=your_keycloak_secret
```

## Backend Dockerfile

```dockerfile
# apps/backend/Dockerfile
FROM node:24-alpine AS base
RUN npm install -g pnpm@8

FROM base AS dependencies
WORKDIR /app

# Copy workspace configuration
COPY pnpm-workspace.yaml package.json ./
COPY apps/backend/package.json ./apps/backend/
COPY packages/shared/package.json ./packages/shared/

# Install dependencies
RUN pnpm install --frozen-lockfile

FROM base AS build
WORKDIR /app
COPY --from=dependencies /app/node_modules ./node_modules
COPY --from=dependencies /app/apps/backend/node_modules ./apps/backend/node_modules
COPY --from=dependencies /app/packages/shared/node_modules ./packages/shared/node_modules

# Copy source code
COPY packages/shared ./packages/shared
COPY apps/backend ./apps/backend

# Build shared packages first
RUN pnpm --filter @dreamchat/shared build

# Build backend
RUN pnpm --filter @dreamchat/backend build

FROM base AS production
WORKDIR /app

# Copy only production dependencies
COPY --from=dependencies /app/node_modules ./node_modules
COPY --from=dependencies /app/apps/backend/node_modules ./apps/backend/node_modules
COPY --from=build /app/apps/backend/dist ./dist
COPY --from=build /app/packages/shared/dist ./node_modules/@dreamchat/shared/dist

# Create logs directory
RUN mkdir -p /app/logs

# Non-root user
RUN addgroup -g 1001 -S nodejs
RUN adduser -S nodejs -u 1001
USER nodejs

EXPOSE 3000

CMD ["node", "dist/main.js"]
```

## Frontend Dockerfile

```dockerfile
# apps/frontend/Dockerfile
FROM node:24-alpine AS base
RUN npm install -g pnpm@8

FROM base AS dependencies
WORKDIR /app

# Copy workspace configuration
COPY pnpm-workspace.yaml package.json ./
COPY apps/frontend/package.json ./apps/frontend/
COPY packages/shared/package.json ./packages/shared/

# Install dependencies
RUN pnpm install --frozen-lockfile

FROM base AS build
WORKDIR /app
COPY --from=dependencies /app/node_modules ./node_modules
COPY --from=dependencies /app/apps/frontend/node_modules ./apps/frontend/node_modules
COPY --from=dependencies /app/packages/shared/node_modules ./packages/shared/node_modules

# Copy source code
COPY packages/shared ./packages/shared
COPY apps/frontend ./apps/frontend

# Build shared packages first
RUN pnpm --filter @dreamchat/shared build

# Build frontend
RUN pnpm --filter @dreamchat/frontend build

# Production stage - using serve for static files
# External reverse proxy (nginx/traefik/etc.) expected
FROM node:24-alpine AS production
WORKDIR /app

# Install serve
RUN npm install -g serve

# Copy built assets
COPY --from=build /app/apps/frontend/dist ./dist

# Create non-root user
RUN addgroup -g 1001 -S nodejs
RUN adduser -S nodejs -u 1001
USER nodejs

EXPOSE 3000

# Serve static files
# Note: External reverse proxy should handle:
# - SSL/TLS termination
# - Path routing (/api -> backend, / -> frontend)
# - WebSocket proxying
CMD ["serve", "-s", "dist", "-l", "3000"]
```

### External Reverse Proxy Configuration

The frontend container serves static files on port 3000. An external reverse proxy is expected to handle:

- **SSL/TLS termination**
- **Path routing**:
  - `/api/*` → Backend (port 3000)
  - `/ws` → Backend WebSocket (port 3000)
  - `/*` → Frontend (port 3001)
- **Static file caching**

Example nginx configuration:

```nginx
server {
    listen 443 ssl;
    server_name dreamchat.example.com;

    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    # API proxy
    location /api {
        proxy_pass http://backend:3000/api;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    # WebSocket proxy
    location /ws {
        proxy_pass http://backend:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }

    # Frontend static files
    location / {
        proxy_pass http://frontend:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
    }
}
```

## Keycloak Configuration (External)

Keycloak is configured as an external service with support for group/role/attribute-based authorization and auto-user creation.

### Prerequisites

1. Have a running Keycloak instance (self-hosted or managed)
2. Configure the following environment variables in `.env`:

```bash
# Basic Keycloak settings
KEYCLOAK_ENABLED=true
KEYCLOAK_URL=http://your-keycloak-server:8080
KEYCLOAK_REALM=dreamchat
KEYCLOAK_CLIENT_ID=dreamchat-backend
KEYCLOAK_CLIENT_SECRET=your_keycloak_secret

# Authorization settings (optional but recommended)
KEYCLOAK_REQUIRED_GROUP=dreamchat-users
KEYCLOAK_REQUIRED_ROLE=dreamchat-access
KEYCLOAK_REQUIRED_CLIENT_ROLE=user
KEYCLOAK_REQUIRED_ATTRIBUTE=approved:true

# Auto-create users
KEYCLOAK_AUTO_CREATE_USER=true
KEYCLOAK_DEFAULT_USER_ROLE=USER
```

### Keycloak Realm Setup

1. Access your Keycloak admin console
2. Create new realm: `dreamchat`
3. Create client: `dreamchat-backend`
   - Client authentication: ON
   - Authorization: ON
   - Valid redirect URIs: `http://localhost:3000/*`
   - Web origins: `http://localhost:3000`

4. Create client: `dreamchat-frontend`
   - Client authentication: OFF
   - Valid redirect URIs: `http://localhost:5173/*`
   - Web origins: `http://localhost:5173`

### Authorization Configuration

You can restrict access based on:

**1. Group Membership**
```bash
KEYCLOAK_REQUIRED_GROUP=dreamchat-users
```
Users must be members of this Keycloak group to access the application.

**2. Realm Role**
```bash
KEYCLOAK_REQUIRED_ROLE=dreamchat-access
```
Users must have this realm-level role.

**3. Client Role**
```bash
KEYCLOAK_REQUIRED_CLIENT_ROLE=user
```
Users must have this role for the `dreamchat-backend` client.

**4. User Attribute**
```bash
KEYCLOAK_REQUIRED_ATTRIBUTE=department:engineering
# or
KEYCLOAK_REQUIRED_ATTRIBUTE=approved:true
```
Users must have this attribute with the specified value.

### User Auto-Creation

When `KEYCLOAK_AUTO_CREATE_USER=true`:
- Users are automatically created in the database on first Keycloak login
- Username is derived from Keycloak preferred_username
- Email is taken from Keycloak email claim
- Role is set to `KEYCLOAK_DEFAULT_USER_ROLE` (default: USER)
- The `keycloakSub` field links the local user to Keycloak

When `KEYCLOAK_AUTO_CREATE_USER=false`:
- Only existing local users can log in via Keycloak
- The `keycloakSub` must match between Keycloak and local user

### Example Keycloak Group/Role Setup

1. Create a group: `dreamchat-users`
2. Create a realm role: `dreamchat-access`
3. Assign the group and/or role to users who should have access
4. Configure `KEYCLOAK_REQUIRED_GROUP` and/or `KEYCLOAK_REQUIRED_ROLE`

### realm-export.json (Optional)

```json
{
  "realm": "dreamchat",
  "enabled": true,
  "clients": [
    {
      "clientId": "dreamchat-backend",
      "enabled": true,
      "clientAuthenticatorType": "client-secret",
      "secret": "**********",
      "redirectUris": ["http://localhost:3000/*"],
      "webOrigins": ["http://localhost:3000"],
      "protocol": "openid-connect"
    },
    {
      "clientId": "dreamchat-frontend",
      "enabled": true,
      "publicClient": true,
      "redirectUris": ["http://localhost:5173/*"],
      "webOrigins": ["http://localhost:5173"],
      "protocol": "openid-connect"
    }
  ]
}
```

## Deployment Procedures

### Local Development

```bash
# Using DevContainer
1. Open project in VS Code
2. Click "Reopen in Container"
3. Wait for setup to complete
4. Start services:
   - Terminal 1: pnpm --filter @dreamchat/backend dev
   - Terminal 2: pnpm --filter @dreamchat/frontend dev
   - Or run all: pnpm dev
```

### Production Deployment

```bash
# 1. Clone repository
git clone https://github.com/yourusername/dreamchat.git
cd dreamchat

# 2. Create environment file
cp .env.example .env
# Edit .env with production values

# 3. Build and start
docker-compose up -d --build

# 4. Run database migrations
docker-compose exec backend pnpm db:migrate

# 5. Check health
curl http://localhost:3000/health

# 6. View logs
docker-compose logs -f backend
```

### Backup and Restore

```bash
# Backup database
docker-compose exec db pg_dump -U postgres -Fc dreamchat > backup_$(date +%Y%m%d).dump

# Restore database
docker-compose exec -T db pg_restore -U postgres -d dreamchat < backup_20240223.dump

# Backup volumes
docker run --rm -v dreamchat_postgres-data:/data -v $(pwd):/backup alpine tar czf /backup/postgres-backup.tar.gz -C /data .
```

### Updates

```bash
# Pull latest changes
git pull origin main

# Rebuild and restart
docker-compose down
docker-compose up -d --build

# Run migrations
docker-compose exec backend npx prisma migrate deploy
```

## Monitoring and Logging

### Health Checks

```bash
# Backend health
curl http://localhost:3000/health

# Database health
docker-compose exec db pg_isready -U postgres

# Full stack
docker-compose ps
```

### Log Management

```bash
# View all logs
docker-compose logs

# View specific service
docker-compose logs -f backend

# View last 100 lines
docker-compose logs --tail=100 backend
```

## Security Considerations

1. **Change default passwords** in production
2. **Use HTTPS** with valid SSL certificates
3. **Enable firewall** rules for required ports only
4. **Regular updates** of base images and dependencies
5. **Secrets management** - use Docker secrets or external vault
6. **Network isolation** - separate networks for different services

## Troubleshooting

### Common Issues

1. **Database connection failed**
   - Check DATABASE_URL environment variable
   - Ensure db service is healthy: `docker-compose ps`

2. **WebSocket not connecting**
   - Verify VITE_WS_URL matches your domain
   - Check Nginx proxy configuration

3. **Keycloak authentication issues**
   - Verify client configuration in Keycloak admin
   - Check redirect URIs match exactly

4. **File upload fails**
   - Check file size limits in Nginx config
   - Verify disk space in backend container