feat: Implement knowledge import feature for characters

- Added KnowledgeImport page for importing character knowledge from URLs.
- Integrated URL validation and error handling for unsupported websites.
- Created API endpoints for importing content from URLs and retrieving character knowledge.
- Enhanced VectorStoreService with logging and error handling for vector memory storage.
- Updated frontend to display knowledge sources and manage them effectively.
- Added support for fetching recent character knowledge as a fallback in similarity searches.
- Updated OpenAPI documentation to reflect new import functionality.
This commit is contained in:
GW_MC
2026-02-24 14:29:26 +00:00
parent 8714d6bd22
commit e033d67ec1
30 changed files with 2018 additions and 204 deletions

View File

@@ -186,6 +186,45 @@ docker-compose exec backend pnpm db:migrate
**Note:** An external reverse proxy (nginx, Traefik, etc.) is expected for SSL termination and routing. See [deployment.md](doc/deployment.md) for configuration examples.
## Troubleshooting
### Memory Issues (JavaScript heap out of memory)
The local embedding model uses TensorFlow.js which can consume significant memory. If you encounter OOM errors:
1. **Increase Node.js memory limit:**
```bash
# In apps/backend/.env
NODE_OPTIONS=--max-old-space-size=768
# Or when starting manually
node --max-old-space-size=768 dist/main
```
2. **Use low-memory mode (for 512MB RAM VPS):**
```bash
cd apps/backend
pnpm run start:low-memory
```
3. **Use quantized model (4x smaller memory):**
```bash
# In apps/backend/.env
EMBEDDING_QUANTIZED=true
```
4. **Limit import size:** The system automatically:
- Limits content to ~30KB per import
- Limits to 30 chunks maximum
- Processes chunks sequentially with delays
- Adds GC pauses between chunks
### System Requirements
- **Minimum:** 512MB RAM (with low-memory configuration)
- **Recommended:** 1GB+ RAM for comfortable operation
- **Import processing:** Large imports may take longer on low-memory systems
## License
MIT