DreamChat/README.md

# DreamChat

A character simulation and interactive storytelling platform with AI-powered conversations.

## Features

- **Character Simulation**: Create custom characters with personalities, attributes, and backstories
- **Interactive Dialogue**: Real-time chat with AI characters using WebSocket streaming
- **Story Generation**: Branching narratives with tree-view visualization
- **Vector Memory**: Context-aware conversations using local embeddings
- **Data Import**: Import character data from files (TXT, PDF, MD) or web sources
- **Multi-Character Chat**: Group conversations with multiple characters (Phase 3)

## Tech Stack

- **Backend**: NestJS + TypeScript + Prisma + PostgreSQL (pgvector)
- **Frontend**: React + Vite + TypeScript + Tailwind CSS
- **Package Manager**: pnpm workspaces (monorepo)
- **AI/LLM**: OpenRouter with flexible provider support
- **Embeddings**: Local HuggingFace models (@xenova/transformers)
- **Auth**: Password-based (with optional external Keycloak SSO supporting group/role/attribute authorization)
- **DevOps**: Docker + DevContainer (external reverse proxy expected)

## Quick Start

### Using DevContainer (Recommended)

1. Open project in VS Code
2. Install [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
3. Press `F1` → "Dev Containers: Reopen in Container"
4. Wait for setup to complete
5. Start development:
   ```bash
   pnpm dev
   ```

### Manual Setup

Prerequisites:
- Node.js 20+
- pnpm 8+
- PostgreSQL 15+ with pgvector extension
- Redis (optional)

1. Install dependencies:
   ```bash
   pnpm install
   ```

2. Build shared packages:
   ```bash
   pnpm --filter @dreamchat/shared build
   ```

3. Setup database:
   ```bash
   pnpm db:generate
   pnpm db:migrate
   pnpm db:seed
   ```

4. Create environment files:
   ```bash
   cp .env.example apps/backend/.env
   cp .env.example apps/frontend/.env
   # Edit both files with your configuration
   ```

5. Start development:
   ```bash
   pnpm dev
   ```

## Project Structure

```
dreamchat/
├── apps/
│   ├── backend/           # NestJS API
│   └── frontend/          # React + Vite SPA
├── packages/
│   └── shared/            # Shared types & WebSocket definitions
├── prisma/
│   ├── schema.prisma      # Main schema (imports from models/)
│   ├── seed.ts            # Seed data
│   └── models/            # Individual model files
│       ├── user.prisma
│       ├── character.prisma
│       └── ...
├── .devcontainer/         # DevContainer configuration
├── docker-compose.yml     # Production Docker Compose
└── doc/                   # Project documentation
```

## Documentation

See the `doc/` folder for comprehensive documentation:

- [architecture.md](doc/architecture.md) - System architecture and design
- [monorepo-guide.md](doc/monorepo-guide.md) - pnpm workspace setup
- [database-schema.md](doc/database-schema.md) - Database schema (Prisma)
- [api-spec.md](doc/api-spec.md) - REST API & WebSocket specifications
- [implementation-plan.md](doc/implementation-plan.md) - Phased roadmap
- [frontend-guide.md](doc/frontend-guide.md) - Frontend architecture
- [deployment.md](doc/deployment.md) - Deployment guide

## Development Commands

```bash
# Install dependencies
pnpm install

# Start all apps in development mode
pnpm dev

# Build all packages
pnpm build

# Run tests
pnpm test

# Database commands
pnpm db:generate      # Generate Prisma client
pnpm db:migrate       # Run migrations
pnpm db:studio        # Open Prisma Studio
pnpm db:seed          # Seed database

# Lint
pnpm lint

# Clean
pnpm clean
```

## Environment Variables

### Backend (`apps/backend/.env`)

```bash
NODE_ENV=development
PORT=3000
DATABASE_URL=postgresql://postgres:postgres@db:5432/dreamchat
JWT_SECRET=your-secret-key
LLM_PROVIDER=openrouter
LLM_API_KEY=your-api-key
LLM_MODEL=openai/gpt-4o
EMBEDDING_PROVIDER=local
EMBEDDING_MODEL=Xenova/all-MiniLM-L6-v2

# Keycloak (optional)
KEYCLOAK_ENABLED=false
KEYCLOAK_URL=http://your-keycloak-server:8080
KEYCLOAK_REALM=dreamchat
KEYCLOAK_CLIENT_ID=dreamchat-backend
KEYCLOAK_CLIENT_SECRET=your-secret

# Keycloak Authorization (optional)
KEYCLOAK_REQUIRED_GROUP=dreamchat-users
KEYCLOAK_REQUIRED_ROLE=dreamchat-access
KEYCLOAK_AUTO_CREATE_USER=true
```

### Frontend (`apps/frontend/.env`)

```bash
VITE_API_URL=http://localhost:3000/api
VITE_WS_URL=ws://localhost:3000
```

## Deployment

See [deployment.md](doc/deployment.md) for production deployment instructions.

Quick deployment with Docker:

```bash
# Copy and edit environment file
cp .env.example .env

# Build and start services
docker-compose up -d --build

# Run migrations
docker-compose exec backend pnpm db:migrate
```

**Note:** An external reverse proxy (nginx, Traefik, etc.) is expected for SSL termination and routing. See [deployment.md](doc/deployment.md) for configuration examples.

## Troubleshooting

### Memory Issues (JavaScript heap out of memory)

The local embedding model uses TensorFlow.js which can consume significant memory. If you encounter OOM errors:

1. **Increase Node.js memory limit:**
   ```bash
   # In apps/backend/.env
   NODE_OPTIONS=--max-old-space-size=768
   
   # Or when starting manually
   node --max-old-space-size=768 dist/main
   ```

2. **Use low-memory mode (for 512MB RAM VPS):**
   ```bash
   cd apps/backend
   pnpm run start:low-memory
   ```

3. **Use quantized model (4x smaller memory):**
   ```bash
   # In apps/backend/.env
   EMBEDDING_QUANTIZED=true
   ```

4. **Limit import size:** The system automatically:
   - Limits content to ~30KB per import
   - Limits to 30 chunks maximum
   - Processes chunks sequentially with delays
   - Adds GC pauses between chunks

### System Requirements

- **Minimum:** 512MB RAM (with low-memory configuration)
- **Recommended:** 1GB+ RAM for comfortable operation
- **Import processing:** Large imports may take longer on low-memory systems

## License

MIT