Skip to content
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,11 @@
dist
node_modules

.eliza
.elizadb
.env
.elizadb*
.env
.cursor
coverage
cypress/screenshots
docs
231 changes: 231 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,25 @@ The plugin provides these actions that your agent can use:

- "Process the document at /path/to/file.pdf"
- "Remember this: The sky is blue"
- Send files as attachments in your message - they'll be automatically processed!

2. **SEARCH_KNOWLEDGE** - Search the knowledge base
- "Search your knowledge for quantum computing"

## 📎 Attachment Processing

The knowledge plugin now intelligently handles attachments:

- **Direct file attachments** - Just attach files to your message
- **URL attachments** - Share links to documents and they'll be downloaded and processed
- **Multiple attachments** - Process many files at once
- **Smart detection** - The agent automatically detects when you want to save attachments

Examples:
- "Save these documents" + [attach PDFs]
- "Add this to your knowledge" + [attach text file]
- "Learn from this website" + [URL attachment]

## 🌐 Web Interface

The plugin includes a web interface for managing documents! Access it at:
Expand All @@ -78,6 +93,14 @@ The plugin includes a web interface for managing documents! Access it at:
http://localhost:3000/api/agents/[your-agent-id]/plugins/knowledge/display
```

Features:
- 📋 List all documents with metadata
- 🔍 Search through your knowledge base
- 📊 Visual graph of document relationships
- ⬆️ Upload new documents
- 🗑️ Delete existing documents
- 🔄 Update document metadata

---

## ⚠️ Advanced Configuration (Developers Only)
Expand Down Expand Up @@ -157,6 +180,8 @@ MAX_OUTPUT_TOKENS=4096
- `GET /api/agents/{agentId}/plugins/knowledge/documents` - List documents
- `GET /api/agents/{agentId}/plugins/knowledge/documents/{id}` - Get specific document
- `DELETE /api/agents/{agentId}/plugins/knowledge/documents/{id}` - Delete document
- `PUT /api/agents/{agentId}/plugins/knowledge/documents/{id}` - Update document metadata
- `POST /api/agents/{agentId}/plugins/knowledge/search` - Search knowledge base
- `GET /api/agents/{agentId}/plugins/knowledge/display` - Web interface

### Programmatic Usage
Expand All @@ -174,10 +199,216 @@ const result = await knowledgeService.addKnowledge({
roomId: 'room-id',
entityId: 'entity-id',
});

// Update document metadata
await runtime.updateMemory({
id: documentId,
metadata: {
type: MemoryType.DOCUMENT,
tags: ['updated', 'important'],
source: 'manual-update',
},
});

// Delete a document
await knowledgeService.deleteMemory(documentId);
```

</details>

## 📝 License

See the ElizaOS license for details.

### Advanced Features

The knowledge plugin now includes several advanced features for enterprise-grade knowledge management:

#### 🔍 Advanced Search

Search with filters, sorting, and pagination:

```typescript
const results = await knowledgeService.advancedSearch({
query: 'machine learning',
filters: {
contentType: ['application/pdf', 'text/markdown'],
tags: ['ai', 'research'],
dateRange: {
start: new Date('2024-01-01'),
end: new Date('2024-12-31'),
},
minSimilarity: 0.7,
},
sort: {
field: 'similarity', // or 'createdAt', 'updatedAt', 'title'
order: 'desc',
},
limit: 20,
offset: 0,
includeMetadata: true,
});
```

Natural language search examples:
- "Search for pdf documents about AI from last week"
- "Find recent markdown files sorted by relevant"
- "Look for documents with blockchain tags from today"

#### 📦 Batch Operations

Process multiple documents efficiently:

```typescript
const result = await knowledgeService.batchOperation({
operation: 'add', // or 'update', 'delete'
items: [
{ data: { content: 'Doc 1', contentType: 'text/plain', ... } },
{ data: { content: 'Doc 2', contentType: 'text/plain', ... } },
// ... more items
],
});

console.log(`Processed: ${result.successful} successful, ${result.failed} failed`);
```

#### 📊 Analytics & Insights

Get comprehensive analytics about your knowledge base:

```typescript
const analytics = await knowledgeService.getAnalytics();

// Returns:
// {
// totalDocuments: 150,
// totalFragments: 450,
// storageSize: 5242880, // bytes
// contentTypes: {
// 'application/pdf': 80,
// 'text/plain': 50,
// 'text/markdown': 20,
// },
// queryStats: {
// totalQueries: 1000,
// averageResponseTime: 250, // ms
// topQueries: [
// { query: 'AI research', count: 50 },
// { query: 'blockchain', count: 30 },
// ],
// },
// usageByDate: [...],
// }
```

#### 📤 Export & Import

Export your knowledge base in multiple formats:

```typescript
// Export to JSON
const jsonExport = await knowledgeService.exportKnowledge({
format: 'json',
includeMetadata: true,
documentIds: ['id1', 'id2'], // optional filter
dateRange: { start: new Date('2024-01-01') }, // optional filter
});

// Export to CSV
const csvExport = await knowledgeService.exportKnowledge({
format: 'csv',
});

// Export to Markdown
const markdownExport = await knowledgeService.exportKnowledge({
format: 'markdown',
includeMetadata: false,
});
```

Import knowledge from various formats:

```typescript
// Import from JSON
const importResult = await knowledgeService.importKnowledge(jsonData, {
format: 'json',
validateBeforeImport: true,
overwriteExisting: false,
});

// Import from CSV
const csvResult = await knowledgeService.importKnowledge(csvData, {
format: 'csv',
batchSize: 100,
});
```

#### 🎯 Action Chaining

The SEARCH_KNOWLEDGE action now returns structured data that can be used by other actions:

```typescript
// SEARCH_KNOWLEDGE returns:
{
data: {
query: 'machine learning',
results: [...], // KnowledgeItem[]
count: 5,
},
text: 'Found 5 results...',
}

// This data can be consumed by other actions like:
// - ANALYZE_KNOWLEDGE: Analyze search results
// - SUMMARIZE_KNOWLEDGE: Create summaries
// - FILTER_KNOWLEDGE: Further filter results
```

#### ⚙️ Configuration Options

New configuration options for advanced features:

```env
# Search Configuration
SEARCH_MATCH_THRESHOLD=0.7 # Minimum similarity score (0-1)
SEARCH_RESULT_COUNT=20 # Default number of results

# Feature Flags
ENABLE_VERSIONING=true # Track document versions
ENABLE_ANALYTICS=true # Enable analytics tracking

# Performance
BATCH_PROCESSING_SIZE=5 # Items processed in parallel
```

### Available Actions

The plugin provides these enhanced actions:

1. **PROCESS_KNOWLEDGE** - Add documents, text, URLs, or attachments
2. **SEARCH_KNOWLEDGE** - Basic knowledge search with action chaining support
3. **ADVANCED_KNOWLEDGE_SEARCH** - Advanced search with filters and sorting
4. **KNOWLEDGE_ANALYTICS** - Get analytics and insights
5. **EXPORT_KNOWLEDGE** - Export knowledge base to various formats

</details>

## 🧪 Testing

The plugin includes comprehensive test coverage:

```bash
# Run all tests
npm test

# Run unit tests
npm run test:unit

# Run E2E tests (including advanced features)
npm test

# Run Cypress UI tests
npm run test:cypress:open
```

## 🤝 Contributing
Loading