feat: Implement knowledge import feature for characters
- Added KnowledgeImport page for importing character knowledge from URLs. - Integrated URL validation and error handling for unsupported websites. - Created API endpoints for importing content from URLs and retrieving character knowledge. - Enhanced VectorStoreService with logging and error handling for vector memory storage. - Updated frontend to display knowledge sources and manage them effectively. - Added support for fetching recent character knowledge as a fallback in similarity searches. - Updated OpenAPI documentation to reflect new import functionality.
This commit is contained in:
39
README.md
39
README.md
@@ -186,6 +186,45 @@ docker-compose exec backend pnpm db:migrate
|
||||
|
||||
**Note:** An external reverse proxy (nginx, Traefik, etc.) is expected for SSL termination and routing. See [deployment.md](doc/deployment.md) for configuration examples.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Memory Issues (JavaScript heap out of memory)
|
||||
|
||||
The local embedding model uses TensorFlow.js which can consume significant memory. If you encounter OOM errors:
|
||||
|
||||
1. **Increase Node.js memory limit:**
|
||||
```bash
|
||||
# In apps/backend/.env
|
||||
NODE_OPTIONS=--max-old-space-size=768
|
||||
|
||||
# Or when starting manually
|
||||
node --max-old-space-size=768 dist/main
|
||||
```
|
||||
|
||||
2. **Use low-memory mode (for 512MB RAM VPS):**
|
||||
```bash
|
||||
cd apps/backend
|
||||
pnpm run start:low-memory
|
||||
```
|
||||
|
||||
3. **Use quantized model (4x smaller memory):**
|
||||
```bash
|
||||
# In apps/backend/.env
|
||||
EMBEDDING_QUANTIZED=true
|
||||
```
|
||||
|
||||
4. **Limit import size:** The system automatically:
|
||||
- Limits content to ~30KB per import
|
||||
- Limits to 30 chunks maximum
|
||||
- Processes chunks sequentially with delays
|
||||
- Adds GC pauses between chunks
|
||||
|
||||
### System Requirements
|
||||
|
||||
- **Minimum:** 512MB RAM (with low-memory configuration)
|
||||
- **Recommended:** 1GB+ RAM for comfortable operation
|
||||
- **Import processing:** Large imports may take longer on low-memory systems
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
Reference in New Issue
Block a user