Compare commits
10 Commits
e597617266
...
470dd8da00
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
470dd8da00 | ||
|
|
aa2a2d99ba | ||
|
|
ee7b4cc74f | ||
|
|
b6b451779d | ||
|
|
905981cd1e | ||
|
|
b1a576f61e | ||
|
|
8163f906cc | ||
|
|
0e7aa3e7f2 | ||
|
|
fdc7a8588d | ||
|
|
a5929134da |
50
.claude/agents/coder.md
Normal file
50
.claude/agents/coder.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
name: coder
|
||||
description: "Use this agent when new Go features need to be implemented or existing Go code needs to be modified. This agent writes maintainable, idiomatic Go code that adheres to all project requirements. Examples:\n\n<example>\nContext: The user wants a new agent or command.\nuser: 'Füge einen neuen /status Command zum Discord-Bot hinzu'\nassistant: 'Ich starte den coder Agenten für die Implementierung.'\n<commentary>\nNeue Funktionalität in Go → coder Agent.\n</commentary>\n</example>\n\n<example>\nContext: The user wants to refactor existing code.\nuser: 'Extrahiere die Email-Logik in ein eigenes Package'\nassistant: 'Ich nutze den coder Agenten für das Refactoring.'\n<commentary>\nCode-Änderung in Go → coder Agent.\n</commentary>\n</example>"
|
||||
color: green
|
||||
---
|
||||
|
||||
You are an experienced Go developer. You implement features, fix bugs and refactor code — clean, idiomatic and maintainable.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Read `CLAUDE.md` and `doc/architecture.md` — understand architecture and conventions
|
||||
2. Read affected source files before making changes
|
||||
3. Implement according to the quality criteria below
|
||||
4. Verify: does the code compile? (`go build ./...`)
|
||||
5. Update `CLAUDE.md` if architecture or interfaces have changed
|
||||
6. Brief summary: what was implemented, which files were changed
|
||||
|
||||
## Quality Criteria
|
||||
|
||||
### Maintainability
|
||||
- Functions have a single clear responsibility (Single Responsibility)
|
||||
- Explicit error handling: every `error` return value is handled
|
||||
- No global variables except where it matches existing project patterns
|
||||
|
||||
### Readability
|
||||
- Comments for non-self-explanatory code (Why, not What)
|
||||
- Exported functions have GoDoc comments
|
||||
- Names are self-explanatory and consistent with existing code
|
||||
|
||||
### Go Idioms
|
||||
- Wrap errors with `fmt.Errorf("context: %w", err)`
|
||||
- New packages and interfaces only when clearly justified
|
||||
- No `panic()` in production code except for programming errors
|
||||
|
||||
### Security
|
||||
- No sensitive data (passwords, tokens) in logs
|
||||
- Input validation at system boundaries (external inputs, API calls)
|
||||
|
||||
## Project-Specific Notes
|
||||
|
||||
- **`config.Cfg`** is a global variable — in tests, `config.LoadConfig()` must be called or `Cfg` set directly
|
||||
- **Defer-first pattern**: Discord handlers send `InteractionResponseDeferredChannelMessageWithSource` immediately, then compute — never wait >3s
|
||||
- **Agent interface**: All agents implement `Handle(Request) Response` (see `internal/agents/agent.go`)
|
||||
- **Deployment**: Binary is cross-compiled locally (`CGO_ENABLED=0 GOOS=linux GOARCH=amd64`) — no CGO allowed
|
||||
|
||||
## Constraints
|
||||
|
||||
- No new external dependencies without explicit request
|
||||
- Tests are written by the `tester` agent — you focus on production code
|
||||
- After architecture changes, `CLAUDE.md` must be up to date
|
||||
57
.claude/agents/software-architect.md
Normal file
57
.claude/agents/software-architect.md
Normal file
@@ -0,0 +1,57 @@
|
||||
---
|
||||
name: software-architect
|
||||
description: "Use this agent to verify or enforce software architecture, review structural decisions, or ensure new code fits the existing design. Invoke after larger changes, when adding new files/packages, or when the user asks for an architecture review. Examples:\n\n<example>\nContext: A new feature was implemented and the user wants to verify it fits the architecture.\nuser: 'Prüf ob der neue Code zur Architektur passt'\nassistant: 'Ich starte den software-architect Agenten für eine Architekturprüfung.'\n<commentary>\nArchitekturprüfung → software-architect Agent.\n</commentary>\n</example>\n\n<example>\nContext: The user plans a larger refactoring.\nuser: 'Ich will die Email-Logik umstrukturieren'\nassistant: 'Lass mich den software-architect Agenten fragen, ob das zur Architektur passt.'\n<commentary>\nStrukturelle Entscheidung → software-architect Agent.\n</commentary>\n</example>"
|
||||
color: blue
|
||||
---
|
||||
|
||||
You are the software architect for this project. You oversee the software structure, make architecture decisions and ensure the code remains consistent, maintainable and extensible.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Architecture Review
|
||||
1. Read `CLAUDE.md` and `doc/architecture.md` — understand target architecture
|
||||
2. Read all relevant Go source files
|
||||
3. Check responsibilities: is code in the right package/file?
|
||||
4. Check new files/packages: are they justified?
|
||||
5. Create findings (format below)
|
||||
|
||||
### Structural Decisions for New Features
|
||||
1. Evaluate where new code belongs (package, file, function)
|
||||
2. Check whether a new package is justified (rule of thumb: from a clearly delimited domain or >300 lines)
|
||||
3. Give concrete recommendations with justification
|
||||
|
||||
### Maintain CLAUDE.md
|
||||
After architecture changes, update `CLAUDE.md`:
|
||||
- Architecture section must reflect current state
|
||||
- Document new packages/binaries
|
||||
- Remove outdated sections
|
||||
|
||||
## Architecture Principles
|
||||
|
||||
1. **Simplicity over abstraction**: Interfaces and abstractions only where they provide real value
|
||||
2. **Package cohesion**: A package has a clearly delimited responsibility
|
||||
3. **No dependency creep**: New external dependencies need good reason
|
||||
4. **Continue existing patterns**: New code follows the style of existing code
|
||||
|
||||
## Findings Format
|
||||
|
||||
```
|
||||
## Architecture Findings
|
||||
|
||||
### Compliant
|
||||
- [What is good]
|
||||
|
||||
### Violations
|
||||
- [What violates the architecture, with concrete location and justification]
|
||||
|
||||
### Recommendations
|
||||
- [Concrete measures, prioritized]
|
||||
|
||||
### CLAUDE.md Status
|
||||
- [Is the documentation up to date? What is missing?]
|
||||
```
|
||||
|
||||
## Constraints
|
||||
|
||||
- You give recommendations and findings — production code is written by the `coder` agent
|
||||
- You only modify `CLAUDE.md`, no source files
|
||||
48
.claude/agents/tester.md
Normal file
48
.claude/agents/tester.md
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
name: tester
|
||||
description: "Use this agent when new Go code has been written or modified and needs unit tests, or when existing tests need review and improvement. Examples:\n\n<example>\nContext: A new function was added.\nuser: 'Ich habe eine neue Funktion in brain/ingest.go hinzugefügt'\nassistant: 'Ich starte den tester Agenten für Unit-Tests.'\n<commentary>\nNeuer Go-Code → tester Agent für Tests.\n</commentary>\n</example>\n\n<example>\nContext: The user wants a quality check.\nuser: 'Kannst du die Testabdeckung für den Task-Agent prüfen?'\nassistant: 'Ich starte den tester Agenten für eine Testüberprüfung.'\n<commentary>\nQualitätssicherung → tester Agent.\n</commentary>\n</example>"
|
||||
color: red
|
||||
---
|
||||
|
||||
You are an experienced Go developer specialized in writing high-quality unit tests. You know Go's `testing` package, table-driven tests and best practices for testing logic that has external dependencies (Qdrant, LocalAI, IMAP, Discord).
|
||||
|
||||
## Your Tasks
|
||||
|
||||
1. **Analyze target code**: Understand what the function/method does before writing tests
|
||||
2. **Write comprehensive tests** using Go's standard `testing` package:
|
||||
- Table-driven tests (`[]struct{ name, input, expected }`) for multiple cases
|
||||
- Cover happy paths, edge cases and error conditions
|
||||
- Test boundary values (empty strings, nil, zero values)
|
||||
3. **Isolate external dependencies**: Test functions that require Qdrant, LocalAI or IMAP so that pure logic (chunking, ID generation, formatting) is testable without external services
|
||||
4. **Ensure test quality**:
|
||||
- Tests must be deterministic and independent of each other
|
||||
- Use `t.Helper()` in helper functions
|
||||
- Use `t.Cleanup()` for resource teardown
|
||||
- No `time.Sleep` — use channels or sync primitives
|
||||
5. **Follow Go conventions**:
|
||||
- Test files as `*_test.go`
|
||||
- Test functions as `TestXxx`
|
||||
- `t.Errorf` for non-fatal, `t.Fatalf` for fatal errors
|
||||
- No external test frameworks — stdlib only
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Read the code to be tested
|
||||
2. Identify testable units
|
||||
3. List test cases: success, failure, edge cases
|
||||
4. Write test file with clear, self-explanatory test names
|
||||
5. Verify imports and types
|
||||
6. Self-review: no test that trivially always passes
|
||||
7. Summary: what was tested, which coverage gaps remain
|
||||
|
||||
## Project-Specific Notes
|
||||
|
||||
- **`config.Cfg`** must be initialized in tests — either call `config.LoadConfig()` or set `config.Cfg` directly with test values
|
||||
- **Existing tests as reference**: `internal/brain/ingest_test.go`, `internal/agents/task/store_test.go`, `internal/agents/agent_test.go`, `internal/config/config_test.go`
|
||||
- **External services** (Qdrant, LocalAI, IMAP) are not available in tests — only test pure logic (chunking, ID generation, formatting, parsing)
|
||||
|
||||
## Constraints
|
||||
|
||||
- Go stdlib only — no external test frameworks (no testify, gomock, etc.)
|
||||
- Tests must run without external services (`go test ./...`)
|
||||
- Logic that strictly requires external services: make testable with interface wrappers and pass that as a recommendation to the `coder` agent
|
||||
5
.dockerignore
Normal file
5
.dockerignore
Normal file
@@ -0,0 +1,5 @@
|
||||
bin/
|
||||
*.exe
|
||||
deploy.env
|
||||
.git
|
||||
.claude
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -1,4 +1,5 @@
|
||||
bin/
|
||||
config.yml
|
||||
deploy.env
|
||||
ask
|
||||
discord
|
||||
|
||||
317
CLAUDE.md
317
CLAUDE.md
@@ -1,50 +1,267 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
**my-brain-importer** is a personal RAG (Retrieval-Augmented Generation) system written in Go. It ingests Markdown notes and image descriptions into a Qdrant vector database and answers questions using a local LLM via LocalAI.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Build all binaries (Linux + Windows cross-compile)
|
||||
bash build.sh
|
||||
|
||||
# Run directly without building
|
||||
go run ./cmd/ingest/
|
||||
go run ./cmd/ask/ "your question here"
|
||||
|
||||
# Build individual binaries
|
||||
go build ./cmd/ingest/
|
||||
go build ./cmd/ask/
|
||||
|
||||
# Run tests
|
||||
go test ./...
|
||||
|
||||
# Tidy dependencies
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
Binaries are output to `./bin/`. The `config.yml` file must exist in the working directory at runtime.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two CLI tools share a common internal library:
|
||||
|
||||
**`cmd/ingest/`** → `internal/brain/ingest.go` + `internal/brain/ingest_json.go`
|
||||
- Markdown mode: recursively finds `.md` files, splits by `# `/`## ` headings, chunks long sections (max 800 chars) by paragraphs, embeds in batches of 10, upserts to Qdrant
|
||||
- JSON mode (when arg ends in `.json`): imports image description records with `file_path`, `file_name`, `description` fields
|
||||
|
||||
**`cmd/ask/`** → `internal/brain/ask.go`
|
||||
- Embeds the question, searches Qdrant (top-k, score threshold 0.5), deduplicates by text content, streams LLM response constrained to retrieved context
|
||||
|
||||
**`internal/config/config.go`** initializes all clients: gRPC connection to Qdrant and OpenAI-compatible HTTP clients for embeddings and chat (both point to LocalAI).
|
||||
|
||||
## Key Patterns
|
||||
|
||||
- **Deterministic IDs**: SHA256 of `source:text` — upserting the same content is always idempotent
|
||||
- **Excluded directories**: `05_Agents` and `.git` are skipped during markdown ingest
|
||||
- **config.yml** must be present in the working directory; defines Qdrant host/port/api_key, embedding model + dimensions, chat model, `brain_root` path, and `top_k`
|
||||
- External services: Qdrant (gRPC port 6334) and LocalAI (HTTP, OpenAI-compatible API)
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
**my-brain-importer** is a personal AI assistant and RAG (Retrieval-Augmented Generation) system written in Go. It ingests Markdown notes into a Qdrant vector database, answers questions using a local LLM (LocalAI), and is primarily controlled via Discord. A background daemon sends proactive email summaries and a daily morning briefing.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Build all binaries (Linux + Windows cross-compile)
|
||||
bash build.sh
|
||||
|
||||
# Primary entry point: Discord Bot (includes daemon)
|
||||
go run ./cmd/discord/
|
||||
|
||||
# CLI tools
|
||||
go run ./cmd/ingest/ # Markdown importieren
|
||||
go run ./cmd/ingest/ bild.json # JSON importieren
|
||||
go run ./cmd/ask/ "your question here" # Frage stellen
|
||||
|
||||
# Test: IMAP-Verbindung
|
||||
go run ./cmd/mailtest/
|
||||
|
||||
# Test: LLM-Zusammenfassung ohne IMAP
|
||||
go run ./cmd/mailtest/ -llm-only
|
||||
|
||||
# Run tests
|
||||
go test ./...
|
||||
|
||||
# Run a single test
|
||||
go test ./internal/brain/ -run TestChunk -v
|
||||
|
||||
# Debug-Logging (LLM-Prompts + Antworten)
|
||||
DEBUG=1 go run ./cmd/discord/
|
||||
|
||||
# Tidy dependencies
|
||||
go mod tidy
|
||||
|
||||
# Integration test (prüft Erreichbarkeit aller externen Dienste + Unit-Tests)
|
||||
bash test-integration.sh
|
||||
|
||||
# Deployment auf Remote-Server via Docker
|
||||
cp deploy.env.example deploy.env # einmalig: Credentials eintragen
|
||||
bash deploy.sh # rsync source + docker compose build + up
|
||||
```
|
||||
|
||||
Binaries are output to `./bin/`. The `config.yml` file must exist in the working directory at runtime.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Discord (primäres Interface)
|
||||
↓ Slash-Commands + @Mention
|
||||
cmd/discord/main.go
|
||||
├── internal/agents/research/ → brain.AskQuery() + Konversationsverlauf
|
||||
├── internal/agents/memory/ → brain.RunIngest(), brain.IngestChatMessage()
|
||||
├── internal/agents/task/ → tasks.json (atomisches JSON, DueDate + Priority)
|
||||
└── internal/agents/tool/email/ → IMAP + LLM-Triage + Zusammenfassung + Move to Archive + Cleanup
|
||||
↓
|
||||
[Daemon-Goroutine] startDaemon()
|
||||
├── IMAP IDLE (pro Account) → Echtzeit-Email-Benachrichtigung + LLM-Triage
|
||||
├── Morgen-Briefing (täglich 8h) → Tasks + Emails kombiniert
|
||||
├── Archiv-Aufräumen (täglich 2h) → CleanupArchiveFolders() nach retention_days
|
||||
└── Nacht-Ingest (täglich 23h) → brain.IngestEmailFolder() für alle Archiv-Ordner
|
||||
|
||||
cmd/ingest/ + cmd/ask/ (CLI-Tools, direkt nutzbar)
|
||||
↓
|
||||
internal/brain/ (Core RAG-Logik)
|
||||
↓
|
||||
Qdrant (gRPC) + LocalAI (HTTP, OpenAI-kompatibel)
|
||||
```
|
||||
|
||||
### Packages
|
||||
|
||||
| Package | Zweck |
|
||||
|---------|-------|
|
||||
| `cmd/discord/` | Discord Bot + Daemon (primärer Einstiegspunkt) |
|
||||
| `cmd/ask/` | CLI-Tool: Fragen stellen |
|
||||
| `cmd/ingest/` | CLI-Tool: Markdown/JSON importieren |
|
||||
| `cmd/mailtest/` | Testprogramm: IMAP + LLM-Test |
|
||||
| `internal/brain/` | Core RAG: Embeddings, Qdrant-Suche, LLM-Streaming |
|
||||
| `internal/config/` | Konfiguration + Client-Initialisierung (globale `Cfg`) |
|
||||
| `internal/agents/` | Agent-Interface (`Request`/`Response`/`HistoryMessage`) |
|
||||
| `internal/agents/research/` | Research-Agent: Wissensdatenbank-Abfragen (mit History) |
|
||||
| `internal/agents/memory/` | Memory-Agent: Ingest + Chat-Speicherung |
|
||||
| `internal/agents/task/` | Task-Agent: Aufgabenverwaltung (tasks.json) |
|
||||
| `internal/agents/tool/` | Tool-Dispatcher |
|
||||
| `internal/agents/tool/email/` | IMAP-Client + LLM-Triage + Email-Analyse + IDLE-Watcher + Move to Archive + CleanupOldEmails |
|
||||
| `internal/agents/tool/rss/` | RSS-Feed-Watcher: Feeds fetchen, Artikel in Qdrant importieren, Daemon-Integration |
|
||||
| `internal/triage/` | RAG-basiertes Lernen: Triage-Entscheidungen in Qdrant speichern + suchen (eigenes Package um Import-Zyklus brain↔email zu vermeiden) |
|
||||
|
||||
### Discord Commands
|
||||
|
||||
| Slash-Command | @Mention | Funktion |
|
||||
|---------------|----------|---------|
|
||||
| `/ask`, `/research` | `@bot <frage>` | Wissensdatenbank abfragen (mit Chat-Gedächtnis) |
|
||||
| `/deepask` | – | Tiefe Recherche mit Multi-Step Reasoning (max 3 iterative RAG-Suchen) |
|
||||
| `/asknobrain` | – | Direkt an LLM (kein RAG) |
|
||||
| `/memory store` | `@bot remember <text>` | Text speichern |
|
||||
| `/memory ingest` | `@bot ingest` | Markdown neu einlesen |
|
||||
| `/memory url <url>` | – | URL-Inhalt in Wissensdatenbank importieren |
|
||||
| `/memory profile <text>` | – | Fakt zum Kerngedächtnis hinzufügen (wird in jeden LLM-Prompt eingebaut) |
|
||||
| `/memory profile-show` | – | Kerngedächtnis anzeigen |
|
||||
| `/knowledge list` | – | Gespeicherte Quellen auflisten |
|
||||
| `/knowledge delete <source>` | – | Quelle aus Wissensdatenbank löschen |
|
||||
| `/task add <text> [faellig] [prioritaet]` | `@bot task add <text> [--due YYYY-MM-DD] [--priority hoch]` | Task hinzufügen |
|
||||
| `/task list/done/delete` | `@bot task <aktion>` | Aufgaben verwalten |
|
||||
| `/email summary/unread/remind` | `@bot email <aktion>` | Email-Analyse |
|
||||
| `/email ingest [ordner]` | `@bot email ingest [ordner]` | Emails aus IMAP-Ordner in Wissensdatenbank importieren (Standard: Archiv) |
|
||||
| `/email move <ordner>` | `@bot email move <ordner>` | Ungelesene Emails in Archivordner verschieben (Choices dynamisch aus `archive_folders`) |
|
||||
| `/status` | – | Bot-Gesundheit: alle Dienste + offene Tasks |
|
||||
| `/clear` | `@bot clear` | Gesprächsverlauf für diesen Channel löschen |
|
||||
| `/remember` | – | Alias für `/memory store` |
|
||||
| `/ingest` | – | Alias für `/memory ingest` |
|
||||
| *(PDF-Anhang)* | `@bot` + PDF-Datei | PDF direkt per Discord importieren |
|
||||
|
||||
## Key Patterns
|
||||
|
||||
- **Deterministic IDs**: SHA256 of `source:text` — upserting the same content is always idempotent
|
||||
- **Excluded directories**: `05_Agents` and `.git` are skipped during markdown ingest
|
||||
- **config.yml** must be present in the working directory at runtime
|
||||
- **Agent Interface**: alle Agenten implementieren `Handle(Request) Response`
|
||||
- **Defer-first Pattern**: Discord-Handlers senden sofort Defer, dann berechnen — nie >3s warten
|
||||
- **LLM-Fallback**: Email-Zusammenfassung zeigt Rohliste wenn LLM nicht erreichbar
|
||||
- **Daemon**: läuft als Goroutine im Discord-Bot-Prozess (`startDaemon()`)
|
||||
- **config.Cfg**: globale Variable — bei Tests muss `config.LoadConfig()` aufgerufen oder Cfg direkt gesetzt werden
|
||||
- **Konversationsverlauf**: Pro Discord-Channel werden die letzten 10 Frage-Antwort-Paare in-memory gehalten und als History an `brain.AskQuery()` übergeben — Reset via `/clear` oder `@bot clear`
|
||||
- **Task-Felder**: `DueDate *time.Time` und `Priority string` (hoch/mittel/niedrig) — rückwärtskompatibel (omitempty)
|
||||
- **Email processed_folder**: Nach Zusammenfassung werden ungelesene Emails in konfigurierten IMAP-Ordner verschoben (leer = deaktiviert)
|
||||
- **Morgen-Briefing**: `dailyBriefing()` kombiniert offene Tasks (mit Fälligkeits-Highlighting) + ungelesene Emails täglich um 8:00
|
||||
- **Archiv-Cleanup**: `email.CleanupArchiveFolders()` läuft täglich um `cleanup_hour` (Standard: 2:00) — iteriert alle Accounts/`archive_folders`, löscht Emails älter als `retention_days` via IMAP `\Deleted` + `EXPUNGE`. `retention_days: 0` = dauerhaft behalten (No-op).
|
||||
- **Email-Triage**: `email.triageUnread()` klassifiziert ungelesene Emails sequentiell (eine nach der anderen) als wichtig/unwichtig via LLM. Wichtige Emails werden in `triage_important_folder`, unwichtige in `triage_unimportant_folder` verschoben. Jede Entscheidung wird in Qdrant gespeichert (`type: email_triage`). Bei nächster Klassifizierung sucht `triage.SearchSimilar()` ähnliche Entscheidungen (Score ≥ 0.7) als Few-Shot-Kontext — das Modell lernt aus der Geschichte. Triage läuft vor `SummarizeUnreadAccount()`.
|
||||
- **Nacht-Ingest**: `nightlyIngest()` läuft täglich um `ingest_hour` (Standard: 23:00) — importiert alle Emails aller Archiv-Ordner in Qdrant via `brain.IngestEmailFolder()`.
|
||||
- **User-Permissions**: `discord.allowed_users: ["user-id1", "user-id2"]` — wenn gesetzt, dürfen nur diese Discord-User-IDs den Bot nutzen. Leer = keine Einschränkung.
|
||||
- **URL-Ingest**: `brain.IngestURL(url)` — fetcht URL, extrahiert sichtbaren HTML-Text (skippt script/style/nav/footer), chunked und importiert in Qdrant. Via `/memory url <url>`.
|
||||
- **PDF-Ingest**: `brain.IngestPDF(path, source)` — extrahiert Text aus PDF via `github.com/ledongthuc/pdf`, chunked und importiert. Trigger: PDF-Anhang an @Bot-Mention.
|
||||
- **Knowledge Management**: `brain.ListSources()` (Qdrant Scroll) + `brain.DeleteBySource()` (Qdrant Filter-Delete). Via `/knowledge list` und `/knowledge delete <source>`.
|
||||
- **Core Memory**: `brain_root/core_memory.md` — persistente Fakten über den Nutzer. `brain.LoadCoreMemory()` wird in `AskQuery()` in den System-Prompt eingefügt. Via `/memory profile <text>` und `/memory profile-show`.
|
||||
- **RSS-Watcher**: `rss.Watcher` — fetcht alle `rss_feeds` aus Config, importiert neue Artikel via `brain.IngestText()`. Läuft als Goroutine im Daemon.
|
||||
- **IngestText**: `brain.IngestText(text, source, type)` — generische Ingest-Funktion für beliebige Texte (kein Datei-I/O nötig).
|
||||
- **Dynamische /email move Choices**: `patchEmailMoveChoices()` wird in `main()` nach `config.LoadConfig()` aufgerufen und ersetzt die statischen Discord-Choices mit konfigurierten `archive_folders`. Fallback auf Legacy-Hardcoding (`2Jahre`/`5Jahre`/`Archiv`) wenn keine `archive_folders` konfiguriert.
|
||||
- **Archive folder resolution**: `resolveArchiveFolder(name)` in `tool/agent.go` sucht case-insensitiv in `acc.ArchiveFolders` (Name oder IMAPFolder), dann Legacy-Fallback. Gilt für Slash-Commands und `@bot email move <name>`.
|
||||
- **IMAP IDLE**: `email.IdleWatcher` pro Account — Echtzeit-Benachrichtigung bei neuen Emails, kein Polling mehr. Race-sichere Implementierung mit `atomic.Uint32` für `numMsgs`. Automatischer Reconnect nach 60s bei Fehler.
|
||||
- **Mehrere Email-Accounts**: `config.AllEmailAccounts()` gibt alle Accounts zurück — zuerst `email_accounts:` (Liste), Fallback auf Legacy `email:` Block. Alle Email-Funktionen iterieren über alle Accounts.
|
||||
- **`/status`**: Ruft `diag.RunAll()` auf + Task-Zähler — zeigt Echtzeit-Status aller externen Dienste
|
||||
|
||||
## config.yml – Alle Felder
|
||||
|
||||
```yaml
|
||||
# Einzelner Email-Account (Legacy, abwärtskompatibel):
|
||||
email:
|
||||
host: imap.strato.de
|
||||
port: 143
|
||||
user: user@example.de
|
||||
password: "..."
|
||||
starttls: true # oder tls: true für implizites TLS (Port 993)
|
||||
folder: INBOX # optional, Standard: INBOX
|
||||
processed_folder: "Processed" # nach Zusammenfassung verschieben (leer = deaktiviert)
|
||||
triage_important_folder: "Wichtig" # LLM-Triage: wichtige Emails hierhin (leer = in INBOX lassen)
|
||||
triage_unimportant_folder: "Unwichtig" # LLM-Triage: unwichtige Emails hierhin (leer = deaktiviert)
|
||||
model: "" # optional: eigenes LLM-Modell für Email-Analyse
|
||||
archive_folders: # optional: Archivordner mit automatischer Bereinigung
|
||||
- name: "Archiv"
|
||||
imap_folder: "Archiv"
|
||||
retention_days: 0 # 0 = dauerhaft behalten (kein Cleanup)
|
||||
- name: "5Jahre"
|
||||
imap_folder: "5Jahre"
|
||||
retention_days: 1825
|
||||
- name: "2Jahre"
|
||||
imap_folder: "2Jahre"
|
||||
retention_days: 730
|
||||
|
||||
# Mehrere Email-Accounts (hat Vorrang vor email:):
|
||||
email_accounts:
|
||||
- name: "Privat"
|
||||
host: imap.strato.de
|
||||
port: 143
|
||||
starttls: true
|
||||
user: privat@example.de
|
||||
password: "..."
|
||||
processed_folder: "Processed"
|
||||
archive_folders: # optional, wie im email:-Block oben
|
||||
- name: "Archiv"
|
||||
imap_folder: "Archiv"
|
||||
retention_days: 0
|
||||
- name: "Arbeit"
|
||||
host: imap.firma.de
|
||||
port: 993
|
||||
tls: true
|
||||
user: jacek@firma.de
|
||||
password: "..."
|
||||
|
||||
daemon:
|
||||
channel_id: "123456789" # Discord-Channel für Daemon-Nachrichten
|
||||
email_interval_min: 30 # (veraltet, IDLE ersetzt Polling)
|
||||
task_reminder_hour: 8 # Uhrzeit des Morgen-Briefings (Standard: 8)
|
||||
cleanup_hour: 2 # Uhrzeit des täglichen Archiv-Aufräumens (Standard: 2)
|
||||
ingest_hour: 23 # Uhrzeit des nächtlichen Email-Ingests (Standard: 23)
|
||||
|
||||
# Discord User-Permissions (optional):
|
||||
discord:
|
||||
token: "..."
|
||||
guild_id: "..." # optional: nur für diese Guild registrieren
|
||||
allowed_users: # optional: leer = alle dürfen den Bot nutzen
|
||||
- "123456789" # Discord User-ID
|
||||
|
||||
# RSS-Feeds (optional):
|
||||
rss_feeds:
|
||||
- url: "https://example.com/feed.xml"
|
||||
interval_hours: 24 # Polling-Intervall (Standard: 24h)
|
||||
- url: "https://news.example.de/rss"
|
||||
interval_hours: 6
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
Deployment läuft über Docker auf dem Remote-Server. `deploy.sh` cross-compiliert das Binary **lokal** (`CGO_ENABLED=0 GOOS=linux GOARCH=amd64`), überträgt Binary + Dockerfile + docker-compose.yml + config.yml per `sshpass`/`scp` und startet den Container auf dem Server via `docker compose up -d --build`. Kein Go-Toolchain auf dem Server nötig.
|
||||
|
||||
```bash
|
||||
# deploy.env (nicht in Git):
|
||||
DEPLOY_HOST=192.168.1.118
|
||||
DEPLOY_USER=christoph
|
||||
DEPLOY_PASS=...
|
||||
DEPLOY_DIR=/home/christoph/brain-bot
|
||||
|
||||
# Deploy: lokal bauen + scp + docker compose up
|
||||
bash deploy.sh
|
||||
|
||||
# Status/Logs auf dem Server:
|
||||
ssh user@host 'cd /home/christoph/brain-bot && docker compose ps'
|
||||
ssh user@host 'cd /home/christoph/brain-bot && docker compose logs -f'
|
||||
```
|
||||
|
||||
Docker-Setup: Minimales Runtime-Image (`alpine:3.21`), `Dockerfile` kopiert nur das fertige Binary. `docker-compose.yml` mountet `config.yml`, `tasks.json` und `brain_data/`.
|
||||
|
||||
### Systemd (DEPRECATED)
|
||||
|
||||
Der alte Systemd-Ansatz wird nicht mehr aktiv genutzt. `deploy.sh` deaktiviert einen vorhandenen systemd-Service automatisch bei der Migration zu Docker.
|
||||
|
||||
```bash
|
||||
# Einmalig auf dem Server (falls noch nicht migriert):
|
||||
sudo systemctl stop brain-bot
|
||||
sudo systemctl disable brain-bot
|
||||
|
||||
# Manueller Fallback ohne Docker:
|
||||
scp bin/discord-bot user@host:/home/christoph/brain-bot/
|
||||
ssh user@host 'sudo systemctl restart brain-bot'
|
||||
```
|
||||
|
||||
## Model Limitations
|
||||
|
||||
Das konfigurierte Modell (`Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-GGUF`) hat folgende Grenzen:
|
||||
|
||||
- **Kontextfenster**: Begrenzt — bei sehr langen Email-Listen oder vielen Chunks kann die Antwort abgeschnitten werden (`MaxTokens: 600`)
|
||||
- **Latenz**: Lokales Modell auf NAS — Antwortzeiten variieren (5–60s je nach Last)
|
||||
- **Encoding**: Betreffzeilen in `windows-1252` (Strato) werden nicht dekodiert — das LLM interpretiert sie trotzdem meist korrekt
|
||||
- **Halluzinationen**: Das Modell kann bei unklarem Kontext eigenes Wissen einmischen — ist im System-Prompt mit "Aus meinem Wissen:" markiert
|
||||
- **Streaming-Timeout**: Kein expliziter Timeout auf LLM-Calls — bei Hänger wird Discord-Interaktion erst nach 15min ungültig
|
||||
|
||||
## External Services
|
||||
|
||||
- **Qdrant** (`192.168.1.4:6334`) — Vektordatenbank, gRPC
|
||||
- **LocalAI** (`192.168.1.118:8080`) — lokales LLM, OpenAI-kompatibles API
|
||||
- **Strato IMAP** (`imap.strato.de:143`, STARTTLS) — Email-Abruf
|
||||
- **Discord** — primäres Interface (Bot-Token in `config.yml`)
|
||||
|
||||
6
Dockerfile
Normal file
6
Dockerfile
Normal file
@@ -0,0 +1,6 @@
|
||||
# Minimales Runtime-Image – Binary wird lokal cross-compiliert und per scp übertragen
|
||||
FROM alpine:3.21
|
||||
RUN apk add --no-cache ca-certificates tzdata
|
||||
WORKDIR /app
|
||||
COPY discord-bot .
|
||||
CMD ["./discord-bot"]
|
||||
383
README.md
383
README.md
@@ -1,43 +1,340 @@
|
||||
# my-brain-importer
|
||||
|
||||
Persönlicher Wissens-Agent für den AI_Brain. Importiert Markdown-Notizen und Bildbeschreibungen in eine Qdrant-Vektordatenbank und beantwortet Fragen darüber mit einem lokalen LLM.
|
||||
|
||||
## Voraussetzungen
|
||||
|
||||
- Go 1.22+
|
||||
- LocalAI läuft auf `embedding.url` mit dem konfigurierten Embedding-Modell geladen
|
||||
- LocalAI läuft auf `chat.url` mit dem konfigurierten Chat-Modell geladen
|
||||
- Qdrant läuft auf dem NAS (Port 6334 gRPC, Port 6333 Dashboard)
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
bash build.sh
|
||||
```
|
||||
|
||||
Erzeugt `bin/ingest`, `bin/ingest.exe`, `bin/ask`, `bin/ask.exe`.
|
||||
|
||||
## Nutzung
|
||||
|
||||
```bash
|
||||
# Markdown-Dateien aus brain_root importieren
|
||||
./bin/ingest
|
||||
|
||||
# Alternatives Verzeichnis angeben
|
||||
./bin/ingest /pfad/zum/verzeichnis
|
||||
|
||||
# Bildbeschreibungen aus JSON importieren
|
||||
./bin/ingest image_descriptions.json
|
||||
|
||||
# Frage stellen
|
||||
./bin/ask "Was sind meine Reisepläne für Norwegen?"
|
||||
./bin/ask "Erzähl mir über Veronica Bellmore"
|
||||
```
|
||||
|
||||
## Brain aktualisieren
|
||||
|
||||
Kein Löschen der Datenbank nötig — einfach `./bin/ingest` erneut ausführen:
|
||||
- Bestehende Chunks → gleiche SHA256-ID → Qdrant überschreibt
|
||||
- Neue Dateien → neue IDs → werden hinzugefügt
|
||||
|
||||
Architektur und Konfiguration: [doc/architecture.md](doc/architecture.md)
|
||||
# Brain-Bot
|
||||
|
||||
Persönlicher KI-Assistent und RAG-System in Go. Speichert Wissen in einer Qdrant-Vektordatenbank und beantwortet Fragen über ein lokales LLM. Primäres Interface: Discord-Bot. Läuft als systemd-Dienst auf einem Home-Server.
|
||||
|
||||
## Features
|
||||
|
||||
- **Discord-Bot** mit Slash-Commands und @Mention
|
||||
- **RAG** über Markdown-Notizen, Emails, URLs, PDFs und RSS-Artikel
|
||||
- **Email-Management**: IMAP IDLE, Triage (wichtig/unwichtig via LLM), Archiv-Cleanup
|
||||
- **Task-Verwaltung** mit Fälligkeit und Priorität
|
||||
- **Morgen-Briefing** täglich um 8:00 (Tasks + ungelesene Emails)
|
||||
- **Core Memory**: persistente Nutzerfakten, automatisch in jeden LLM-Prompt eingebaut
|
||||
- **RSS-Watcher**: automatisches Importieren von Feed-Artikeln
|
||||
- **User-Permissions**: optionale Einschränkung auf bestimmte Discord-User-IDs
|
||||
|
||||
---
|
||||
|
||||
## Voraussetzungen
|
||||
|
||||
| Dienst | Adresse | Zweck |
|
||||
|--------|---------|-------|
|
||||
| Qdrant | `192.168.1.4:6334` (gRPC) | Vektordatenbank |
|
||||
| LocalAI | `192.168.1.118:8080` | Embeddings + Chat (OpenAI-kompatibel) |
|
||||
| IMAP-Server | konfigurierbar | Email-Abruf (STARTTLS oder TLS) |
|
||||
| Discord | Bot-Token | Primäres Interface |
|
||||
|
||||
---
|
||||
|
||||
## Schnellstart
|
||||
|
||||
```bash
|
||||
# Einmalig: config.yml anlegen
|
||||
cp config.yml.example config.yml # Credentials eintragen
|
||||
|
||||
# Bot starten
|
||||
go run ./cmd/discord/
|
||||
|
||||
# Oder: CLI-Tools
|
||||
go run ./cmd/ask/ "Was sind meine TODOs?"
|
||||
go run ./cmd/ingest/ # Markdown aus brain_root importieren
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Discord-Commands
|
||||
|
||||
### Wissen abfragen
|
||||
|
||||
| Command | Beschreibung |
|
||||
|---------|-------------|
|
||||
| `/ask <frage>` | Wissensdatenbank abfragen (mit Gesprächsgedächtnis) |
|
||||
| `/research <frage>` | Alias für `/ask` |
|
||||
| `/asknobrain <frage>` | Direkt ans LLM, kein RAG |
|
||||
| `/clear` | Gesprächsverlauf dieses Channels löschen |
|
||||
|
||||
### Wissen speichern
|
||||
|
||||
| Command | Beschreibung |
|
||||
|---------|-------------|
|
||||
| `/memory store <text>` | Text direkt in Wissensdatenbank speichern |
|
||||
| `/memory ingest` | Alle Markdown-Dateien aus `brain_root` importieren |
|
||||
| `/memory url <url>` | Webseite fetchen und importieren |
|
||||
| `/memory profile <text>` | Fakt zum Kerngedächtnis hinzufügen |
|
||||
| `/memory profile-show` | Kerngedächtnis anzeigen |
|
||||
| `/remember <text>` | Alias für `/memory store` |
|
||||
| `/ingest` | Alias für `/memory ingest` |
|
||||
| *(PDF-Anhang)* | PDF an @Bot schicken → automatisch importiert |
|
||||
|
||||
### Wissensdatenbank verwalten
|
||||
|
||||
| Command | Beschreibung |
|
||||
|---------|-------------|
|
||||
| `/knowledge list` | Alle gespeicherten Quellen auflisten |
|
||||
| `/knowledge delete <source>` | Quelle und alle ihre Chunks löschen |
|
||||
|
||||
### Tasks
|
||||
|
||||
| Command | Beschreibung |
|
||||
|---------|-------------|
|
||||
| `/task add <text> [--due YYYY-MM-DD] [--priority hoch\|mittel\|niedrig]` | Task anlegen |
|
||||
| `/task list` | Alle offenen Tasks anzeigen |
|
||||
| `/task done <id>` | Task als erledigt markieren |
|
||||
| `/task delete <id>` | Task löschen |
|
||||
|
||||
### Email
|
||||
|
||||
| Command | Beschreibung |
|
||||
|---------|-------------|
|
||||
| `/email summary` | Letzte Emails zusammenfassen |
|
||||
| `/email unread` | Ungelesene Emails zusammenfassen |
|
||||
| `/email remind` | Termine und Deadlines aus Emails extrahieren |
|
||||
| `/email ingest [ordner]` | Emails eines IMAP-Ordners in Qdrant importieren |
|
||||
| `/email move <ordner>` | Emails interaktiv in Archivordner verschieben |
|
||||
| `/email triage` | Letzte 10 Emails als wichtig/unwichtig klassifizieren |
|
||||
|
||||
### Sonstiges
|
||||
|
||||
| Command | Beschreibung |
|
||||
|---------|-------------|
|
||||
| `/status` | Verbindungen prüfen, offene Tasks zählen |
|
||||
|
||||
### @Mention
|
||||
|
||||
```
|
||||
@Brain <frage>
|
||||
@Brain task add <text> [--due YYYY-MM-DD] [--priority hoch]
|
||||
@Brain task list / done <id> / delete <id>
|
||||
@Brain email summary / unread / remind / ingest [ordner] / move <ordner>
|
||||
@Brain remember <text>
|
||||
@Brain clear
|
||||
```
|
||||
|
||||
PDF-Datei an @Brain anhängen → wird automatisch importiert.
|
||||
|
||||
---
|
||||
|
||||
## Konfiguration (`config.yml`)
|
||||
|
||||
```yaml
|
||||
# Qdrant-Vektordatenbank
|
||||
qdrant:
|
||||
host: "192.168.1.4"
|
||||
port: "6334"
|
||||
api_key: "geheimespasswort"
|
||||
collection: "jacek-brain"
|
||||
|
||||
# Embedding-Modell (LocalAI)
|
||||
embedding:
|
||||
url: "http://192.168.1.118:8080/v1"
|
||||
model: "qwen3-embedding-4b"
|
||||
dimensions: 2560 # muss exakt zum Modell passen
|
||||
|
||||
# Chat-Modell (LocalAI)
|
||||
chat:
|
||||
url: "http://192.168.1.118:8080/v1"
|
||||
model: "Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-GGUF"
|
||||
|
||||
# Discord-Bot
|
||||
discord:
|
||||
token: "Bot-Token"
|
||||
guild_id: "" # leer = global (bis 1h Verzögerung); Guild-ID = sofort
|
||||
allowed_users: # optional: leer = alle erlaubt
|
||||
- "123456789012345678" # Discord User-ID
|
||||
|
||||
# Wissensbasis-Verzeichnis (Markdown-Dateien)
|
||||
brain_root: "/mnt/c/Users/jacek/AI_Brain"
|
||||
top_k: 5 # Anzahl der Suchergebnisse pro Anfrage
|
||||
score_threshold: 0.55 # Minimale Relevanz (0.0–1.0)
|
||||
|
||||
# Task-Speicher
|
||||
tasks:
|
||||
store_path: "./tasks.json"
|
||||
|
||||
# Daemon-Einstellungen
|
||||
daemon:
|
||||
channel_id: "1234567890" # Discord-Channel für proaktive Nachrichten
|
||||
task_reminder_hour: 8 # Morgen-Briefing Uhrzeit (0-23, Standard: 8)
|
||||
cleanup_hour: 2 # Archiv-Aufräumen Uhrzeit (0-23, Standard: 2)
|
||||
ingest_hour: 23 # Email-Ingest Uhrzeit (0-23, Standard: 23)
|
||||
|
||||
# RSS-Feeds (optional)
|
||||
rss_feeds:
|
||||
- url: "https://example.com/feed.xml"
|
||||
interval_hours: 24
|
||||
- url: "https://heise.de/rss/heise.rdf"
|
||||
interval_hours: 6
|
||||
|
||||
# Email — einzelner Account (Legacy)
|
||||
email:
|
||||
host: "imap.strato.de"
|
||||
port: 143
|
||||
user: "user@example.de"
|
||||
password: "passwort"
|
||||
starttls: true # oder tls: true für Port 993
|
||||
folder: "INBOX"
|
||||
processed_folder: "" # nach Zusammenfassung verschieben (leer = deaktiviert)
|
||||
model: "" # eigenes LLM-Modell für Email-Analyse (leer = chat.model)
|
||||
triage_important_folder: "Wichtig"
|
||||
triage_unimportant_folder: "Unwichtig"
|
||||
archive_folders:
|
||||
- name: "Archiv"
|
||||
imap_folder: "Archiv"
|
||||
retention_days: 0 # 0 = dauerhaft behalten
|
||||
- name: "5Jahre"
|
||||
imap_folder: "5Jahre"
|
||||
retention_days: 1825
|
||||
- name: "2Jahre"
|
||||
imap_folder: "2Jahre"
|
||||
retention_days: 730
|
||||
|
||||
# Email — mehrere Accounts (hat Vorrang vor email:)
|
||||
email_accounts:
|
||||
- name: "Privat"
|
||||
host: "imap.strato.de"
|
||||
port: 143
|
||||
starttls: true
|
||||
user: "privat@example.de"
|
||||
password: "passwort"
|
||||
processed_folder: "Processed"
|
||||
triage_important_folder: "Wichtig"
|
||||
triage_unimportant_folder: "Unwichtig"
|
||||
archive_folders:
|
||||
- name: "Archiv"
|
||||
imap_folder: "Archiv"
|
||||
retention_days: 0
|
||||
- name: "Arbeit"
|
||||
host: "imap.firma.de"
|
||||
port: 993
|
||||
tls: true
|
||||
user: "jacek@firma.de"
|
||||
password: "passwort"
|
||||
```
|
||||
|
||||
> **Wichtig:** Wenn `embedding.model` oder `dimensions` geändert wird, muss die Qdrant-Collection neu erstellt werden (im Dashboard löschen, dann `/memory ingest` erneut ausführen).
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
```bash
|
||||
# deploy.env anlegen (einmalig)
|
||||
cp deploy.env.example deploy.env
|
||||
# Credentials eintragen: DEPLOY_HOST, DEPLOY_USER, DEPLOY_PASS, DEPLOY_DIR, SERVICE_NAME
|
||||
|
||||
# Deploy
|
||||
bash deploy.sh # build + scp + systemctl restart
|
||||
```
|
||||
|
||||
Das Script baut das Linux-Binary, überträgt es per `sshpass`/`scp` und startet den systemd-Service neu.
|
||||
|
||||
```bash
|
||||
# Systemd-Service (einmalig auf dem Server einrichten)
|
||||
sudo systemctl unmask brain-bot # falls masked
|
||||
sudo systemctl enable brain-bot
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Entwicklung
|
||||
|
||||
```bash
|
||||
# Tests ausführen
|
||||
go test ./...
|
||||
|
||||
# Bot lokal starten
|
||||
go run ./cmd/discord/
|
||||
|
||||
# Debug-Logging (LLM-Prompts + Antworten)
|
||||
DEBUG=1 go run ./cmd/discord/
|
||||
|
||||
# Abhängigkeiten aufräumen
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
### Projektstruktur
|
||||
|
||||
```
|
||||
cmd/
|
||||
discord/main.go Discord-Bot + Daemon (primärer Einstiegspunkt)
|
||||
ask/main.go CLI: Fragen stellen
|
||||
ingest/main.go CLI: Markdown/JSON importieren
|
||||
mailtest/main.go CLI: IMAP + LLM testen
|
||||
|
||||
internal/
|
||||
config/
|
||||
config.go Konfigurationsstruktur, Client-Factories
|
||||
config_test.go Config-Tests
|
||||
brain/
|
||||
ask.go RAG-Suche + LLM-Antwort (AskQuery, ChatDirect)
|
||||
ingest.go Markdown-Import, Chunking, IngestText
|
||||
ingest_json.go JSON-Import (Bildbeschreibungen)
|
||||
ingest_email.go IMAP-Ordner → Qdrant
|
||||
ingest_url.go URL fetchen → Qdrant
|
||||
ingest_pdf.go PDF-Text → Qdrant
|
||||
knowledge.go ListSources, DeleteBySource
|
||||
core_memory.go Kerngedächtnis (core_memory.md)
|
||||
agents/
|
||||
agent.go Agent-Interface (Request/Response/HistoryMessage)
|
||||
actions.go Typsichere Action-Konstanten
|
||||
memory/agent.go Memory-Agent (store, ingest, url, profile)
|
||||
research/agent.go Research-Agent (RAG-Suche mit History)
|
||||
task/
|
||||
agent.go Task-Agent (add, list, done, delete)
|
||||
store.go Atomarer JSON-Speicher (tasks.json)
|
||||
tool/
|
||||
agent.go Tool-Dispatcher + ResolveArchiveFolder
|
||||
email/
|
||||
client.go IMAP-Client (Verbinden, Fetch, Move, Delete)
|
||||
summary.go Email-Zusammenfassung + LLM-Triage
|
||||
idle.go IMAP IDLE-Watcher (Echtzeit-Benachrichtigung)
|
||||
rss/
|
||||
watcher.go RSS-Feed-Watcher (gofeed + brain.IngestText)
|
||||
triage/
|
||||
triage.go RAG-basiertes Triage-Lernen (eigenes Package)
|
||||
diag/
|
||||
diag.go Verbindungsdiagnose (/status)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Wie es funktioniert
|
||||
|
||||
### RAG-Pipeline
|
||||
|
||||
1. Nutzer stellt Frage via Discord
|
||||
2. Frage wird in Vektor umgewandelt (Embedding-Modell)
|
||||
3. Qdrant liefert die `top_k` ähnlichsten Chunks (Score ≥ `score_threshold`)
|
||||
4. Core Memory + Chunks + Gesprächshistory werden in LLM-Prompt eingefügt
|
||||
5. LLM generiert Antwort (Streaming)
|
||||
|
||||
### Email-Triage
|
||||
|
||||
1. IMAP IDLE meldet neue Email in Echtzeit
|
||||
2. `triageUnread()` klassifiziert jede Email via LLM (wichtig/unwichtig)
|
||||
3. Ähnliche frühere Entscheidungen aus Qdrant werden als Few-Shot-Kontext mitgegeben
|
||||
4. Jede Entscheidung wird in Qdrant gespeichert (Typ `email_triage`) → Modell lernt
|
||||
5. Wichtige Emails bleiben in INBOX, unwichtige werden in `triage_unimportant_folder` verschoben
|
||||
|
||||
### Core Memory
|
||||
|
||||
- Datei: `brain_root/core_memory.md`
|
||||
- Inhalt: Eine Zeile pro Fakt (Markdown-Liste)
|
||||
- Wird bei **jeder** `AskQuery()`-Anfrage automatisch in den System-Prompt eingebaut
|
||||
- Kein Neustart nötig (wird bei jedem Call frisch geladen)
|
||||
|
||||
### Deterministische IDs
|
||||
|
||||
Alle Qdrant-Punkte haben SHA256-basierte IDs (`source:text`). Derselbe Chunk kann beliebig oft importiert werden — Qdrant überschreibt denselben Punkt (Upsert), keine Duplikate.
|
||||
|
||||
---
|
||||
|
||||
## Externe Dienste
|
||||
|
||||
| Dienst | Standard-Adresse | Protokoll |
|
||||
|--------|-----------------|-----------|
|
||||
| Qdrant | `192.168.1.4:6334` | gRPC |
|
||||
| LocalAI (Embedding) | `192.168.1.118:8080` | HTTP/OpenAI |
|
||||
| LocalAI (Chat) | `192.168.1.118:8080` | HTTP/OpenAI (Streaming) |
|
||||
| Strato IMAP | `imap.strato.de:143` | STARTTLS |
|
||||
| Discord | `discord.com` | WebSocket |
|
||||
|
||||
2
build.sh
2
build.sh
@@ -27,4 +27,4 @@ echo "Fertig. Nutzung:"
|
||||
echo " $OUT_DIR/ingest # Markdown importieren"
|
||||
echo " $OUT_DIR/ingest bild.json # JSON importieren"
|
||||
echo " $OUT_DIR/ask \"Was sind meine Pläne?\""
|
||||
echo " $OUT_DIR/discord-bot # Discord-Bot starten"
|
||||
echo " $OUT_DIR/discord-bot # Discord-Bot + Daemon starten"
|
||||
|
||||
38
cmd/agenttest/main.go
Normal file
38
cmd/agenttest/main.go
Normal file
@@ -0,0 +1,38 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"my-brain-importer/internal/agents"
|
||||
"my-brain-importer/internal/agents/task"
|
||||
"my-brain-importer/internal/agents/tool/email"
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
func main() {
|
||||
config.LoadConfig()
|
||||
|
||||
fmt.Println("=== Task-Agent Test ===")
|
||||
a := task.New()
|
||||
|
||||
r := a.Handle(agents.Request{Action: agents.ActionAdd, Args: []string{"Synology DSM Update durchfuehren"}})
|
||||
fmt.Println("ADD:", r.Text)
|
||||
|
||||
r = a.Handle(agents.Request{Action: agents.ActionAdd, Args: []string{"Zahnarzt Termin bestaetigen"}})
|
||||
fmt.Println("ADD:", r.Text)
|
||||
|
||||
r = a.Handle(agents.Request{Action: agents.ActionList})
|
||||
fmt.Println("\nLIST:\n" + r.Text)
|
||||
|
||||
fmt.Println("\n=== Email-Agent Test mit Testdaten ===")
|
||||
testMails := []email.Message{
|
||||
{Subject: "KRITISCH: Synology Update verfuegbar", From: "noreply@synology.com", Date: "2026-03-19 14:09"},
|
||||
{Subject: "Rechnung Maerz 2026", From: "buchhaltung@strom.de", Date: "2026-03-18 09:00"},
|
||||
{Subject: "Newsletter GoLang Weekly", From: "newsletter@golangweekly.com", Date: "2026-03-17 18:00"},
|
||||
}
|
||||
result, err := email.SummarizeMessages(testMails, "Fasse zusammen und priorisiere nach Dringlichkeit.")
|
||||
if err != nil {
|
||||
fmt.Println("LLM-Fehler:", err)
|
||||
} else {
|
||||
fmt.Println(result)
|
||||
}
|
||||
}
|
||||
1637
cmd/discord/main.go
1637
cmd/discord/main.go
File diff suppressed because it is too large
Load Diff
110
cmd/mailtest/main.go
Normal file
110
cmd/mailtest/main.go
Normal file
@@ -0,0 +1,110 @@
|
||||
// mailtest – IMAP-Verbindungstest und LLM-Zusammenfassungstest
|
||||
//
|
||||
// Flags:
|
||||
//
|
||||
// -llm-only Überspringt IMAP, testet LLM mit eingebetteten Testdaten
|
||||
// -unread Holt nur ungelesene Emails (statt letzte 5)
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"log"
|
||||
"log/slog"
|
||||
"os"
|
||||
|
||||
"my-brain-importer/internal/agents/tool/email"
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// testEmails sind eingebettete Beispieldaten für den LLM-Test ohne IMAP.
|
||||
var testEmails = []email.Message{
|
||||
{Subject: "KRITISCH: Server down in Produktion", From: "monitoring@company.com", Date: "2026-03-19 09:00"},
|
||||
{Subject: "Meeting morgen 10 Uhr – Projektreview Q1", From: "chef@company.com", Date: "2026-03-19 08:30"},
|
||||
{Subject: "Rechnung Nr. 2026-042 fällig bis 25.03.", From: "buchhaltung@lieferant.de", Date: "2026-03-18 14:00"},
|
||||
{Subject: "Re: Urlaubsantrag genehmigt", From: "hr@company.com", Date: "2026-03-18 11:00"},
|
||||
{Subject: "Newsletter: Neue Features in Go 1.26", From: "newsletter@golangweekly.com", Date: "2026-03-17 18:00"},
|
||||
{Subject: "Amazon: Ihre Bestellung wurde versendet", From: "no-reply@amazon.de", Date: "2026-03-17 10:00"},
|
||||
{Subject: "Erinnerung: Zahnarzttermin 21.03. um 15:00", From: "praxis@zahnarzt.de", Date: "2026-03-16 09:00"},
|
||||
}
|
||||
|
||||
func main() {
|
||||
llmOnly := flag.Bool("llm-only", false, "Überspringe IMAP, teste LLM mit eingebetteten Testdaten")
|
||||
unread := flag.Bool("unread", false, "Hole nur ungelesene Emails")
|
||||
verbose := flag.Bool("v", false, "Verbose Logging (Debug-Level)")
|
||||
flag.Parse()
|
||||
|
||||
config.LoadConfig()
|
||||
|
||||
// Logging konfigurieren
|
||||
logLevel := slog.LevelInfo
|
||||
if *verbose {
|
||||
logLevel = slog.LevelDebug
|
||||
}
|
||||
slog.SetDefault(slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: logLevel})))
|
||||
|
||||
if *llmOnly {
|
||||
runLLMTest()
|
||||
return
|
||||
}
|
||||
|
||||
cfg := config.Cfg.Email
|
||||
fmt.Printf("🔌 Verbinde mit %s:%d (TLS=%v StartTLS=%v) als %s ...\n",
|
||||
cfg.Host, cfg.Port, cfg.TLS, cfg.StartTLS, cfg.User)
|
||||
|
||||
cl, err := email.Connect()
|
||||
if err != nil {
|
||||
log.Fatalf("❌ Verbindung fehlgeschlagen: %v", err)
|
||||
}
|
||||
defer cl.Close()
|
||||
fmt.Println("✅ Login erfolgreich!")
|
||||
|
||||
var msgs []email.Message
|
||||
if *unread {
|
||||
fmt.Println("📥 Hole ungelesene Emails...")
|
||||
msgs, err = cl.FetchUnread()
|
||||
} else {
|
||||
fmt.Println("📥 Hole die letzten 5 Emails...")
|
||||
msgs, err = cl.FetchRecent(5)
|
||||
}
|
||||
if err != nil {
|
||||
log.Fatalf("❌ Fetch fehlgeschlagen: %v", err)
|
||||
}
|
||||
if len(msgs) == 0 {
|
||||
fmt.Println("📭 Keine Emails gefunden.")
|
||||
return
|
||||
}
|
||||
for i, m := range msgs {
|
||||
fmt.Printf("[%d] %s | Von: %s | %s\n", i+1, m.Date, m.From, m.Subject)
|
||||
}
|
||||
|
||||
fmt.Printf("\n🤖 Teste LLM-Zusammenfassung (Modell: %s)...\n", effectiveModel())
|
||||
summary, err := email.Summarize()
|
||||
if err != nil {
|
||||
log.Fatalf("❌ Fehler: %v", err)
|
||||
}
|
||||
fmt.Println(summary)
|
||||
}
|
||||
|
||||
func runLLMTest() {
|
||||
fmt.Printf("🧪 LLM-Testmodus mit %d eingebetteten Testdaten\n", len(testEmails))
|
||||
fmt.Printf(" Modell: %s\n\n", effectiveModel())
|
||||
for i, m := range testEmails {
|
||||
fmt.Printf("[%d] %s | %s | %s\n", i+1, m.Date, m.From, m.Subject)
|
||||
}
|
||||
|
||||
fmt.Println("\n🤖 Starte LLM-Zusammenfassung...")
|
||||
result, err := email.SummarizeMessages(testEmails, "Fasse diese Emails zusammen. Priorisiere nach Dringlichkeit und Wichtigkeit.")
|
||||
if err != nil {
|
||||
log.Fatalf("❌ LLM-Fehler: %v", err)
|
||||
}
|
||||
fmt.Println("\n--- Ergebnis ---")
|
||||
fmt.Println(result)
|
||||
}
|
||||
|
||||
func effectiveModel() string {
|
||||
if config.Cfg.Email.Model != "" {
|
||||
return config.Cfg.Email.Model
|
||||
}
|
||||
return config.Cfg.Chat.Model
|
||||
}
|
||||
12
deploy.env.example
Normal file
12
deploy.env.example
Normal file
@@ -0,0 +1,12 @@
|
||||
# Deploy-Konfiguration – Kopieren als deploy.env und anpassen
|
||||
# deploy.env wird NICHT in Git eingecheckt
|
||||
|
||||
DEPLOY_HOST=192.168.1.118
|
||||
DEPLOY_USER=todo
|
||||
DEPLOY_PASS=geheim
|
||||
|
||||
# Zielverzeichnis auf dem Server
|
||||
DEPLOY_DIR=/home/todo/brain-bot
|
||||
|
||||
# Docker-Image-Name (optional)
|
||||
IMAGE_NAME=brain-bot
|
||||
83
deploy.sh
Executable file
83
deploy.sh
Executable file
@@ -0,0 +1,83 @@
|
||||
#!/usr/bin/env bash
|
||||
# deploy.sh – Baut Binary lokal (cross-compile) und deployt es per scp auf den Server
|
||||
set -euo pipefail
|
||||
|
||||
# Credentials aus deploy.env laden
|
||||
ENV_FILE="$(dirname "$0")/deploy.env"
|
||||
if [[ ! -f "$ENV_FILE" ]]; then
|
||||
echo "❌ deploy.env nicht gefunden. Kopiere deploy.env.example nach deploy.env und passe sie an."
|
||||
exit 1
|
||||
fi
|
||||
source "$ENV_FILE"
|
||||
|
||||
: "${DEPLOY_HOST:?DEPLOY_HOST fehlt in deploy.env}"
|
||||
: "${DEPLOY_USER:?DEPLOY_USER fehlt in deploy.env}"
|
||||
: "${DEPLOY_PASS:?DEPLOY_PASS fehlt in deploy.env}"
|
||||
: "${DEPLOY_DIR:=/home/${DEPLOY_USER}/brain-bot}"
|
||||
|
||||
# Sudo-Passwort: standardmäßig gleich wie SSH-Passwort
|
||||
SUDO_PASS="${SUDO_PASS:-$DEPLOY_PASS}"
|
||||
|
||||
SSH_OPTS="-o StrictHostKeyChecking=no -o BatchMode=no"
|
||||
|
||||
if ! command -v sshpass &>/dev/null; then
|
||||
echo "❌ sshpass nicht installiert: sudo apt install sshpass"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ssh_cmd() {
|
||||
sshpass -p "$DEPLOY_PASS" ssh $SSH_OPTS "${DEPLOY_USER}@${DEPLOY_HOST}" "$@"
|
||||
}
|
||||
|
||||
scp_cmd() {
|
||||
sshpass -p "$DEPLOY_PASS" scp $SSH_OPTS "$@"
|
||||
}
|
||||
|
||||
sudo_cmd() {
|
||||
ssh_cmd "echo '${SUDO_PASS}' | sudo -S -p '' $*"
|
||||
}
|
||||
|
||||
# ── Lokaler Cross-Compile-Build ─────────────────────────────────────────────
|
||||
echo "🔨 Baue Linux/amd64-Binary lokal..."
|
||||
mkdir -p bin
|
||||
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/discord-bot ./cmd/discord/
|
||||
echo "✅ Binary gebaut: bin/discord-bot"
|
||||
|
||||
# ── Systemd-Service deaktivieren (einmalige Migration) ──────────────────────
|
||||
if ssh_cmd "systemctl is-enabled --quiet brain-bot 2>/dev/null"; then
|
||||
echo "🔄 Migriere von systemd zu Docker..."
|
||||
sudo_cmd "systemctl stop brain-bot" 2>/dev/null || true
|
||||
sudo_cmd "systemctl disable brain-bot" 2>/dev/null || true
|
||||
echo "✅ Systemd-Service deaktiviert"
|
||||
fi
|
||||
|
||||
# ── Alten Container stoppen ─────────────────────────────────────────────────
|
||||
echo "⏹️ Stoppe laufenden Container..."
|
||||
ssh_cmd "cd '${DEPLOY_DIR}' && docker compose down 2>/dev/null" || true
|
||||
echo "✅ Container gestoppt"
|
||||
|
||||
# ── Zielverzeichnis anlegen ─────────────────────────────────────────────────
|
||||
echo "📁 Stelle Zielverzeichnis sicher: ${DEPLOY_DIR}..."
|
||||
ssh_cmd "mkdir -p '${DEPLOY_DIR}'"
|
||||
|
||||
# ── Dateien übertragen ───────────────────────────────────────────────────────
|
||||
echo "📦 Übertrage Binary..."
|
||||
scp_cmd bin/discord-bot "${DEPLOY_USER}@${DEPLOY_HOST}:${DEPLOY_DIR}/discord-bot"
|
||||
|
||||
echo "📋 Übertrage Dockerfile + docker-compose.yml..."
|
||||
scp_cmd Dockerfile docker-compose.yml "${DEPLOY_USER}@${DEPLOY_HOST}:${DEPLOY_DIR}/"
|
||||
|
||||
echo "📋 Übertrage config.yml..."
|
||||
scp_cmd config.yml "${DEPLOY_USER}@${DEPLOY_HOST}:${DEPLOY_DIR}/config.yml"
|
||||
|
||||
# tasks.json anlegen falls nicht vorhanden (Volume-Mount braucht existierende Datei)
|
||||
ssh_cmd "test -f '${DEPLOY_DIR}/tasks.json' || echo '[]' > '${DEPLOY_DIR}/tasks.json'"
|
||||
|
||||
# ── Docker-Image auf Server bauen und starten ────────────────────────────────
|
||||
echo "🚀 Baue Image und starte Container..."
|
||||
ssh_cmd "cd '${DEPLOY_DIR}' && docker compose up -d --build"
|
||||
|
||||
echo ""
|
||||
echo "✅ Deployment abgeschlossen!"
|
||||
echo " Status: ssh ${DEPLOY_USER}@${DEPLOY_HOST} 'cd ${DEPLOY_DIR} && docker compose ps'"
|
||||
echo " Logs: ssh ${DEPLOY_USER}@${DEPLOY_HOST} 'cd ${DEPLOY_DIR} && docker compose logs -f'"
|
||||
@@ -1,60 +1,234 @@
|
||||
# Architektur
|
||||
|
||||
```
|
||||
AI_Brain/
|
||||
*.md Dateien
|
||||
│
|
||||
▼
|
||||
bin/ingest Embeddings via LocalAI
|
||||
│
|
||||
▼
|
||||
Qdrant (NAS) ◄──── bin/ask ──► LM Studio (Chat)
|
||||
```
|
||||
|
||||
- **Embeddings**: LocalAI unter `embedding.url` (Modell konfigurierbar)
|
||||
- **Vektordatenbank**: Qdrant auf dem NAS
|
||||
- **Chat-Completion**: LocalAI unter `chat.url` (Modell konfigurierbar)
|
||||
|
||||
## Projektstruktur
|
||||
|
||||
```
|
||||
AI-Agent/
|
||||
cmd/
|
||||
ingest/main.go Entry Point für ingest-Binary
|
||||
ask/main.go Entry Point für ask-Binary
|
||||
internal/
|
||||
config/config.go Config-Struct, Clients, Verbindungen
|
||||
brain/
|
||||
ingest.go Markdown-Import, Chunking
|
||||
ingest_json.go JSON-Import (Bildbeschreibungen)
|
||||
ask.go Suche + LLM-Antwort
|
||||
bin/ Kompilierte Binaries (von build.sh erzeugt)
|
||||
config.yml Alle Einstellungen
|
||||
build.sh Baut beide Binaries
|
||||
```
|
||||
|
||||
## Konfiguration
|
||||
|
||||
Alle Einstellungen in `config.yml` (muss im Arbeitsverzeichnis liegen):
|
||||
|
||||
```yaml
|
||||
qdrant:
|
||||
host: "192.168.1.4"
|
||||
port: "6334"
|
||||
api_key: "..."
|
||||
collection: "jacek-brain"
|
||||
|
||||
embedding:
|
||||
url: "http://192.168.1.118:8080/v1"
|
||||
model: "qwen3-embedding-4b"
|
||||
dimensions: 2560 # muss zum Modell passen
|
||||
|
||||
chat:
|
||||
url: "http://192.168.1.118:8080/v1"
|
||||
model: "qwen3.5-4b-claude-4.6-opus-reasoning-distilled"
|
||||
|
||||
brain_root: "/mnt/c/Users/jacek/AI_Brain"
|
||||
top_k: 3
|
||||
```
|
||||
|
||||
> **Wichtig:** Wenn du `embedding.model` oder `dimensions` änderst, muss die Qdrant-Collection neu erstellt werden (im Dashboard löschen, dann `ingest` erneut ausführen).
|
||||
# Architektur
|
||||
|
||||
## Übersicht
|
||||
|
||||
```
|
||||
Discord (primäres Interface)
|
||||
↓ Slash-Commands + @Mention + PDF-Anhänge
|
||||
cmd/discord/main.go
|
||||
├── Research-Agent → brain.AskQuery() + Konversationsverlauf pro Channel
|
||||
├── Memory-Agent → brain.RunIngest(), IngestChatMessage(), IngestURL(), CoreMemory
|
||||
├── Task-Agent → tasks.json (atomares JSON, DueDate + Priority)
|
||||
├── Tool-Agent → Dispatcher für Email-Aktionen
|
||||
└── Daemon-Goroutinen:
|
||||
├── IMAP IDLE (pro Account) → Echtzeit-Triage + Discord-Benachrichtigung
|
||||
├── RSS-Watcher → Artikel-Import in Qdrant
|
||||
├── Morgen-Briefing (08:00) → Tasks + Emails kombiniert
|
||||
├── Archiv-Cleanup (02:00) → CleanupArchiveFolders() nach retention_days
|
||||
└── Nacht-Ingest (23:00) → brain.IngestEmailFolder() für alle Archive
|
||||
↓
|
||||
Qdrant (gRPC, 192.168.1.4:6334) LocalAI (HTTP, 192.168.1.118:8080)
|
||||
Vektordatenbank Embedding-Modell + Chat-Modell
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Packages
|
||||
|
||||
### `cmd/discord/`
|
||||
Primärer Einstiegspunkt. Registriert Discord Slash-Commands, verarbeitet Interaktionen und @Mentions, startet den Daemon.
|
||||
|
||||
**Wichtige Funktionen:**
|
||||
- `main()` — Config laden, Discord verbinden, Commands registrieren, Daemon starten
|
||||
- `onInteraction()` — Slash-Command-Handler mit Berechtigungsprüfung
|
||||
- `onMessage()` — @Mention-Handler inkl. PDF-Anhang-Erkennung
|
||||
- `routeMessage()` — Leitet @Mention-Text an passenden Agenten weiter
|
||||
- `startDaemon()` — Startet IMAP IDLE, RSS-Watcher, tägliche Timer
|
||||
- `dailyBriefing()` — Morgen-Briefing (Tasks + Emails)
|
||||
- `nightlyIngest()` — Archiv-Ordner in Qdrant importieren
|
||||
- `patchEmailMoveChoices()` — Discord-Choices dynamisch aus Config befüllen
|
||||
- `isAllowed(userID)` — User-Berechtigungsprüfung
|
||||
|
||||
### `internal/config/`
|
||||
Konfigurationsstruktur (`Config`), Client-Factories und `AllEmailAccounts()`.
|
||||
|
||||
```go
|
||||
type Config struct {
|
||||
Qdrant, Embedding, Chat // Externe Dienste
|
||||
Discord // Token, GuildID, AllowedUsers
|
||||
Email / EmailAccounts // IMAP (Legacy/Multi-Account)
|
||||
Tasks // JSON-Pfad
|
||||
Daemon // Timer-Uhrzeiten, Channel-ID
|
||||
BrainRoot, TopK, ScoreThreshold
|
||||
RSSFeeds // RSS-Feed-URLs + Intervalle
|
||||
}
|
||||
```
|
||||
|
||||
- `LoadConfig()` — liest `config.yml`, validiert Pflichtfelder
|
||||
- `AllEmailAccounts()` — gibt alle Accounts zurück (Multi-Account-Vorrang über Legacy)
|
||||
- `NewQdrantConn()`, `NewEmbeddingClient()`, `NewChatClient()` — Client-Factories
|
||||
|
||||
### `internal/brain/`
|
||||
Core RAG-Logik. Alle Funktionen sind zustandslos (keine globalen Verbindungen).
|
||||
|
||||
| Datei | Inhalt |
|
||||
|-------|--------|
|
||||
| `ask.go` | `AskQuery()` (Suche + LLM), `ChatDirect()`, `searchKnowledge()` |
|
||||
| `ingest.go` | `RunIngest()` (Markdown), `IngestChatMessage()`, `IngestText()`, Chunking |
|
||||
| `ingest_json.go` | JSON-Import (Bildbeschreibungen) |
|
||||
| `ingest_email.go` | `IngestEmailFolder()` — IMAP-Ordner → Qdrant |
|
||||
| `ingest_url.go` | `IngestURL()` — HTTP-Fetch + HTML-Text-Extraktion → Qdrant |
|
||||
| `ingest_pdf.go` | `IngestPDF()` — PDF-Text-Extraktion → Qdrant |
|
||||
| `knowledge.go` | `ListSources()` (Scroll), `DeleteBySource()` (Filter-Delete) |
|
||||
| `core_memory.go` | `LoadCoreMemory()`, `AppendCoreMemory()` — `brain_root/core_memory.md` |
|
||||
|
||||
**ID-Schema:** `SHA256(source + ":" + text)[:16]` als Hex — deterministische Upserts, keine Duplikate.
|
||||
|
||||
**Chunking-Strategie:**
|
||||
1. Text nach Markdown-Überschriften (`#`, `##`, `###`) aufteilen
|
||||
2. Abschnitte > 800 Zeichen nach Paragraphen (`\n\n`) aufteilen
|
||||
3. Minimum-Länge 20 Zeichen
|
||||
|
||||
**Core Memory in AskQuery:**
|
||||
```
|
||||
System-Prompt = Basis-Prompt
|
||||
+ "\n\n## Fakten über den Nutzer:\n" + core_memory.md (wenn nicht leer)
|
||||
```
|
||||
|
||||
### `internal/agents/`
|
||||
|
||||
**`agent.go`** — Gemeinsame Interfaces:
|
||||
```go
|
||||
type Agent interface { Handle(Request) Response }
|
||||
type Request struct { Action, Args, Author, Source, History }
|
||||
type Response struct { Text, RawAnswer string; Error error }
|
||||
type HistoryMessage struct { Role, Content string }
|
||||
```
|
||||
|
||||
**`actions.go`** — Alle Action-Konstanten (typsicher, keine Magic Strings).
|
||||
|
||||
**`memory/agent.go`** — Delegiert an `brain.*`:
|
||||
- `store` → `brain.IngestChatMessage()`
|
||||
- `ingest` → `brain.RunIngest()`
|
||||
|
||||
**`research/agent.go`** — Ruft `brain.AskQuery()` auf, formatiert Antwort mit Quellenangaben.
|
||||
|
||||
**`task/agent.go`** + **`task/store.go`** — Task-CRUD über atomares `tasks.json`:
|
||||
- Felder: `ID` (UUID), `Text`, `Done bool`, `DueDate *time.Time`, `Priority string`
|
||||
- Atomares Schreiben: temp-Datei → rename (kein Datenverlust bei Absturz)
|
||||
|
||||
### `internal/agents/tool/`
|
||||
|
||||
**`agent.go`** — Dispatcht Email-Actions an `email`-Package-Funktionen. `ResolveArchiveFolder()` für case-insensitive Ordnersuche.
|
||||
|
||||
**`email/client.go`** — IMAP-Client-Wrapper:
|
||||
- `ConnectAccount()` — Verbindung (STARTTLS oder implizites TLS)
|
||||
- `FetchUnread()`, `FetchWithBody()`, `FetchRecentForSelect()`
|
||||
- `MoveMessages()`, `MoveOldEmails()`, `DeleteByAge()`
|
||||
|
||||
**`email/summary.go`** — Email-Zusammenfassung + Triage:
|
||||
- `SummarizeUnread()` — Alle Accounts, LLM-Zusammenfassung
|
||||
- `triageUnread()` — LLM-Klassifikation + Qdrant-Lernen via `triage` Package
|
||||
- `CleanupArchiveFolders()` — Alte Emails löschen nach `retention_days`
|
||||
|
||||
**`email/idle.go`** — IMAP IDLE-Watcher:
|
||||
- `IdleWatcher` — pro Account, race-sicher mit `atomic.Uint32`
|
||||
- Automatischer Reconnect nach 60s bei Fehler
|
||||
- Callback bei neuen Emails → `triageUnread()` + Discord-Nachricht
|
||||
|
||||
**`rss/watcher.go`** — RSS-Feed-Watcher:
|
||||
- `IngestFeed(url)` — gofeed Parser → Artikel als Text → `brain.IngestText()`
|
||||
- `Watcher.Run(ctx)` — Goroutine, pollt alle Feeds im minimalen Intervall
|
||||
- `IngestAllFeeds()` / `FormatResults()` — Batch-Import + Discord-Formatting
|
||||
|
||||
### `internal/triage/`
|
||||
Eigenes Package um Import-Zyklen zu vermeiden (`brain` ↔ `email` ↔ `triage`).
|
||||
- `StoreTriage()` — Triage-Entscheidung in Qdrant speichern (Typ `email_triage`)
|
||||
- `SearchSimilar()` — Ähnliche frühere Entscheidungen finden (Score ≥ 0.7) als Few-Shot-Kontext
|
||||
|
||||
### `internal/diag/`
|
||||
- `RunAll()` — Prüft Qdrant, LocalAI (Embedding + Chat), IMAP-Verbindungen
|
||||
- `Format()` / `Log()` — Ausgabe für Discord und Konsole
|
||||
|
||||
---
|
||||
|
||||
## Datenflüsse
|
||||
|
||||
### Slash-Command: `/ask <frage>`
|
||||
```
|
||||
Discord → onInteraction → researchAgent.Handle(ActionQuery)
|
||||
→ research/agent.go → brain.AskQuery(frage, history)
|
||||
→ searchKnowledge() → Qdrant Search
|
||||
→ LoadCoreMemory()
|
||||
→ LLM Stream (LocalAI)
|
||||
→ Discord Edit (Deferred Response)
|
||||
```
|
||||
|
||||
### @Bot + PDF-Anhang
|
||||
```
|
||||
Discord → onMessage → Attachment .pdf erkannt
|
||||
→ http.Get(att.URL) → temp-Datei
|
||||
→ brain.IngestPDF(tmpFile, att.Filename)
|
||||
→ ledongthuc/pdf → Text extrahieren
|
||||
→ ingestChunks() → Qdrant Upsert
|
||||
→ Discord Reply
|
||||
```
|
||||
|
||||
### IMAP IDLE → Neue Email
|
||||
```
|
||||
email.IdleWatcher.Run()
|
||||
→ IMAP IDLE Command (Server-Push)
|
||||
→ triageUnread()
|
||||
→ triage.SearchSimilar() (Few-Shot aus Qdrant)
|
||||
→ LLM: wichtig / unwichtig?
|
||||
→ triage.StoreTriage() (Entscheidung → Qdrant)
|
||||
→ email.MoveMessages() (IMAP Move)
|
||||
→ SummarizeUnreadAccount() (LLM-Zusammenfassung)
|
||||
→ Discord ChannelMessageSend
|
||||
```
|
||||
|
||||
### RSS-Watcher
|
||||
```
|
||||
rss.Watcher.Run(ctx)
|
||||
→ gofeed.Parser.ParseURL(feedURL)
|
||||
→ buildArticleText(item) (Titel + Datum + URL + Beschreibung)
|
||||
→ brain.IngestText()
|
||||
→ splitLongSection()
|
||||
→ ingestChunks() → Qdrant Upsert
|
||||
→ Discord ChannelMessageSend (Zusammenfassung)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Konfigurationsvalidierung
|
||||
|
||||
`validateConfig()` prüft beim Start:
|
||||
- `qdrant.host`, `qdrant.port`
|
||||
- `embedding.url`, `embedding.model`
|
||||
- `chat.url`, `chat.model`
|
||||
|
||||
Fehlt eines dieser Felder → fataler Fehler, Bot startet nicht.
|
||||
|
||||
Discord-Token wird separat in `main()` geprüft.
|
||||
|
||||
---
|
||||
|
||||
## Deployment-Architektur
|
||||
|
||||
```
|
||||
Entwickler-PC (WSL2)
|
||||
│ bash deploy.sh
|
||||
│ 1. CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build (lokal)
|
||||
│ 2. sshpass + scp (Binary + Dockerfile + docker-compose.yml + config.yml)
|
||||
│ 3. docker compose up -d --build (remote)
|
||||
▼
|
||||
Home-Server (192.168.1.118)
|
||||
├── Docker: brain-bot Container (alpine:3.21 + Binary)
|
||||
│ Volumes: config.yml, tasks.json, brain_data/
|
||||
│
|
||||
├── LocalAI (Port 8080) — Embeddings + Chat
|
||||
└── Qdrant (auf 192.168.1.4, Port 6334) — Vektordatenbank
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Bekannte Grenzen
|
||||
|
||||
| Grenze | Details |
|
||||
|--------|---------|
|
||||
| LLM-Kontextfenster | `MaxTokens: 600` — lange Email-Listen werden abgeschnitten |
|
||||
| LLM-Latenz | 5–60s je nach Modell + NAS-Last |
|
||||
| IMAP-Encoding | Strato: `windows-1252` Betreffzeilen werden nicht dekodiert |
|
||||
| Streaming-Timeout | Kein expliziter LLM-Timeout — Discord-Interaktion läuft nach 15min ab |
|
||||
| PDF-Extraktion | Nur Text-PDFs; gescannte PDFs (nur Bilder) liefern keinen Text |
|
||||
| Discord Message-Limit | 2000 Zeichen — lange Antworten werden abgeschnitten |
|
||||
|
||||
14
docker-compose.yml
Normal file
14
docker-compose.yml
Normal file
@@ -0,0 +1,14 @@
|
||||
services:
|
||||
bot:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
image: brain-bot:latest
|
||||
container_name: brain-bot
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- TZ=Europe/Berlin
|
||||
volumes:
|
||||
- ./config.yml:/app/config.yml:ro
|
||||
- ./tasks.json:/app/tasks.json
|
||||
- ./brain_data:/app/brain_data
|
||||
15
go.mod
15
go.mod
@@ -1,8 +1,10 @@
|
||||
module my-brain-importer
|
||||
|
||||
go 1.22.2
|
||||
go 1.24.1
|
||||
|
||||
require (
|
||||
github.com/bwmarrin/discordgo v0.29.0
|
||||
github.com/emersion/go-imap/v2 v2.0.0-beta.8
|
||||
github.com/qdrant/go-client v1.12.0
|
||||
github.com/sashabaranov/go-openai v1.37.0
|
||||
google.golang.org/grpc v1.71.0
|
||||
@@ -10,8 +12,17 @@ require (
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/bwmarrin/discordgo v0.29.0 // indirect
|
||||
github.com/PuerkitoBio/goquery v1.8.0 // indirect
|
||||
github.com/andybalholm/cascadia v1.3.1 // indirect
|
||||
github.com/emersion/go-message v0.18.2 // indirect
|
||||
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 // indirect
|
||||
github.com/gorilla/websocket v1.4.2 // indirect
|
||||
github.com/json-iterator/go v1.1.12 // indirect
|
||||
github.com/ledongthuc/pdf v0.0.0-20250511090121-5959a4027728 // indirect
|
||||
github.com/mmcdole/gofeed v1.3.0 // indirect
|
||||
github.com/mmcdole/goxpp v1.1.1-0.20240225020742-a0c311522b23 // indirect
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
||||
github.com/modern-go/reflect2 v1.0.2 // indirect
|
||||
golang.org/x/crypto v0.32.0 // indirect
|
||||
golang.org/x/net v0.34.0 // indirect
|
||||
golang.org/x/sys v0.29.0 // indirect
|
||||
|
||||
58
go.sum
58
go.sum
@@ -1,5 +1,17 @@
|
||||
github.com/PuerkitoBio/goquery v1.8.0 h1:PJTF7AmFCFKk1N6V6jmKfrNH9tV5pNE6lZMkG0gta/U=
|
||||
github.com/PuerkitoBio/goquery v1.8.0/go.mod h1:ypIiRMtY7COPGk+I/YbZLbxsxn9g5ejnI2HSMtkjZvI=
|
||||
github.com/andybalholm/cascadia v1.3.1 h1:nhxRkql1kdYCc8Snf7D5/D3spOX+dBgjA6u8x004T2c=
|
||||
github.com/andybalholm/cascadia v1.3.1/go.mod h1:R4bJ1UQfqADjvDa4P6HZHLh/3OxWWEqc0Sk8XGwHqvA=
|
||||
github.com/bwmarrin/discordgo v0.29.0 h1:FmWeXFaKUwrcL3Cx65c20bTRW+vOb6k8AnaP+EgjDno=
|
||||
github.com/bwmarrin/discordgo v0.29.0/go.mod h1:NJZpH+1AfhIcyQsPeuBKsUtYrRnjkyu0kIVMCHkZtRY=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/emersion/go-imap/v2 v2.0.0-beta.8 h1:5IXZK1E33DyeP526320J3RS7eFlCYGFgtbrfapqDPug=
|
||||
github.com/emersion/go-imap/v2 v2.0.0-beta.8/go.mod h1:dhoFe2Q0PwLrMD7oZw8ODuaD0vLYPe5uj2wcOMnvh48=
|
||||
github.com/emersion/go-message v0.18.2 h1:rl55SQdjd9oJcIoQNhubD2Acs1E6IzlZISRTK7x/Lpg=
|
||||
github.com/emersion/go-message v0.18.2/go.mod h1:XpJyL70LwRvq2a8rVbHXikPgKj8+aI0kGdHlg16ibYA=
|
||||
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 h1:oP4q0fw+fOSWn3DfFi4EXdT+B+gTtzx8GC9xsc26Znk=
|
||||
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6/go.mod h1:iL2twTeMvZnrg54ZoPDNfJaJaqy0xIQFuBdrLsmspwQ=
|
||||
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
|
||||
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
@@ -8,14 +20,32 @@ github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek
|
||||
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
|
||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc=
|
||||
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
|
||||
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
|
||||
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
|
||||
github.com/ledongthuc/pdf v0.0.0-20250511090121-5959a4027728 h1:QwWKgMY28TAXaDl+ExRDqGQltzXqN/xypdKP86niVn8=
|
||||
github.com/ledongthuc/pdf v0.0.0-20250511090121-5959a4027728/go.mod h1:1fEHWurg7pvf5SG6XNE5Q8UZmOwex51Mkx3SLhrW5B4=
|
||||
github.com/mmcdole/gofeed v1.3.0 h1:5yn+HeqlcvjMeAI4gu6T+crm7d0anY85+M+v6fIFNG4=
|
||||
github.com/mmcdole/gofeed v1.3.0/go.mod h1:9TGv2LcJhdXePDzxiuMnukhV2/zb6VtnZt1mS+SjkLE=
|
||||
github.com/mmcdole/goxpp v1.1.1-0.20240225020742-a0c311522b23 h1:Zr92CAlFhy2gL+V1F+EyIuzbQNbSgP4xhTODZtrXUtk=
|
||||
github.com/mmcdole/goxpp v1.1.1-0.20240225020742-a0c311522b23/go.mod h1:v+25+lT2ViuQ7mVxcncQ8ch1URund48oH+jhjiwEgS8=
|
||||
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
|
||||
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/qdrant/go-client v1.12.0 h1:KqsIKDAw5iQmxDzRjbzRjhvQ+Igyr7Y84vDCinf1T4M=
|
||||
github.com/qdrant/go-client v1.12.0/go.mod h1:zFa6t5Y3Oqecoa0aSsGWhMqQWq3x3kTPvm0sMf5qplw=
|
||||
github.com/sashabaranov/go-openai v1.37.0 h1:hQQowgYm4OXJ1Z/wTrE+XZaO20BYsL0R3uRPSpfNZkY=
|
||||
github.com/sashabaranov/go-openai v1.37.0/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
|
||||
go.opentelemetry.io/otel v1.34.0 h1:zRLXxLCgL1WyKsPVrgbSdMN4c0FMkDAskSTQP+0hdUY=
|
||||
@@ -28,20 +58,48 @@ go.opentelemetry.io/otel/sdk/metric v1.34.0 h1:5CeK9ujjbFVL5c1PhLuStg1wxA7vQv7ce
|
||||
go.opentelemetry.io/otel/sdk/metric v1.34.0/go.mod h1:jQ/r8Ze28zRKoNRdkjCZxfs6YvBTG1+YIqyFVFYec5w=
|
||||
go.opentelemetry.io/otel/trace v1.34.0 h1:+ouXS2V8Rd4hp4580a8q23bg0azF2nI8cqLYnC8mh/k=
|
||||
go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
|
||||
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
|
||||
golang.org/x/crypto v0.32.0 h1:euUpcYgM8WcP71gNpTqQCn6rC2t6ULUPiOzfWaXVVfc=
|
||||
golang.org/x/crypto v0.32.0/go.mod h1:ZnnJkOaASj8g0AjIduWNlq2NRxL0PlBrbKVyZ6V/Ugc=
|
||||
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
||||
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
|
||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||
golang.org/x/net v0.0.0-20210916014120-12bc252f5db8/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
|
||||
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
|
||||
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
|
||||
golang.org/x/net v0.34.0 h1:Mb7Mrk043xzHgnRM88suvJFwzVrRfHEHJEl5/71CKw0=
|
||||
golang.org/x/net v0.34.0/go.mod h1:di0qlW3YNM5oh6GqDGQr92MyTozJPmybPK4Ev/Gm31k=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
|
||||
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
|
||||
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
|
||||
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
|
||||
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250115164207-1a7da9e5054f h1:OxYkA3wjPsZyBylwymxSHa7ViiW1Sml4ToBrncvFehI=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250115164207-1a7da9e5054f/go.mod h1:+2Yz8+CLJbIfL9z73EW45avw8Lmge3xVElCP9zEKi50=
|
||||
google.golang.org/grpc v1.71.0 h1:kF77BGdPTQ4/JZWMlb9VpJ5pa25aqvVqogsxNHHdeBg=
|
||||
|
||||
40
internal/agents/actions.go
Normal file
40
internal/agents/actions.go
Normal file
@@ -0,0 +1,40 @@
|
||||
// actions.go – Typsichere Konstanten für Agent-Actions
|
||||
package agents
|
||||
|
||||
const (
|
||||
// Research
|
||||
ActionQuery = "query"
|
||||
ActionDeepAsk = "deepask"
|
||||
|
||||
// Memory
|
||||
ActionStore = "store"
|
||||
ActionIngest = "ingest"
|
||||
|
||||
// Task
|
||||
ActionAdd = "add"
|
||||
ActionList = "list"
|
||||
ActionDone = "done"
|
||||
ActionDelete = "delete"
|
||||
|
||||
// Memory
|
||||
ActionIngestURL = "url"
|
||||
ActionIngestPDF = "pdf"
|
||||
ActionProfile = "profile"
|
||||
ActionProfileShow = "profile-show"
|
||||
|
||||
// Knowledge
|
||||
ActionKnowledgeList = "list"
|
||||
ActionKnowledgeDelete = "delete"
|
||||
|
||||
// Tool/Email
|
||||
ActionEmail = "email"
|
||||
ActionEmailSummary = "summary"
|
||||
ActionEmailUnread = "unread"
|
||||
ActionEmailRemind = "remind"
|
||||
ActionEmailIngest = "ingest"
|
||||
ActionEmailMove = "move"
|
||||
ActionEmailTriage = "triage"
|
||||
ActionEmailTriageHistory = "triage-history"
|
||||
ActionEmailTriageCorrect = "triage-correct"
|
||||
ActionEmailTriageSearch = "triage-search"
|
||||
)
|
||||
29
internal/agents/agent.go
Normal file
29
internal/agents/agent.go
Normal file
@@ -0,0 +1,29 @@
|
||||
// agent.go – Gemeinsames Interface für alle Agenten
|
||||
package agents
|
||||
|
||||
// HistoryMessage repräsentiert eine vorherige Konversationsnachricht.
|
||||
type HistoryMessage struct {
|
||||
Role string // "user" oder "assistant"
|
||||
Content string
|
||||
}
|
||||
|
||||
// Request enthält die Eingabe für einen Agenten.
|
||||
type Request struct {
|
||||
Action string // z.B. "store", "list", "done", "summary"
|
||||
Args []string // Argumente für die Aktion
|
||||
Author string // Discord-Username (für Kontext)
|
||||
Source string // Herkunft (z.B. "discord/#channelID")
|
||||
History []HistoryMessage // Konversationsverlauf (für Chat-Gedächtnis)
|
||||
}
|
||||
|
||||
// Response enthält die Ausgabe eines Agenten.
|
||||
type Response struct {
|
||||
Text string // Formattierte Antwort
|
||||
Error error // Fehler, falls aufgetreten
|
||||
RawAnswer string // Unformatierte LLM-Antwort (für Konversationsverlauf)
|
||||
}
|
||||
|
||||
// Agent ist das gemeinsame Interface für alle Agenten.
|
||||
type Agent interface {
|
||||
Handle(req Request) Response
|
||||
}
|
||||
43
internal/agents/agent_test.go
Normal file
43
internal/agents/agent_test.go
Normal file
@@ -0,0 +1,43 @@
|
||||
package agents
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestHistoryMessage_Fields(t *testing.T) {
|
||||
h := HistoryMessage{Role: "user", Content: "Hallo"}
|
||||
if h.Role != "user" {
|
||||
t.Errorf("Role: %q", h.Role)
|
||||
}
|
||||
if h.Content != "Hallo" {
|
||||
t.Errorf("Content: %q", h.Content)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRequest_HistoryAppend(t *testing.T) {
|
||||
req := Request{
|
||||
Action: ActionQuery,
|
||||
Args: []string{"Frage"},
|
||||
History: []HistoryMessage{
|
||||
{Role: "user", Content: "Vorherige Frage"},
|
||||
{Role: "assistant", Content: "Vorherige Antwort"},
|
||||
},
|
||||
}
|
||||
if len(req.History) != 2 {
|
||||
t.Errorf("History len: %d", len(req.History))
|
||||
}
|
||||
if req.History[0].Role != "user" {
|
||||
t.Errorf("erstes Element: %q", req.History[0].Role)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponse_RawAnswer(t *testing.T) {
|
||||
resp := Response{
|
||||
Text: "**Formatierte** Antwort",
|
||||
RawAnswer: "Formatierte Antwort",
|
||||
}
|
||||
if resp.RawAnswer == "" {
|
||||
t.Error("RawAnswer sollte gesetzt sein")
|
||||
}
|
||||
if resp.Text == resp.RawAnswer {
|
||||
t.Error("Text und RawAnswer sollten sich unterscheiden")
|
||||
}
|
||||
}
|
||||
56
internal/agents/memory/agent.go
Normal file
56
internal/agents/memory/agent.go
Normal file
@@ -0,0 +1,56 @@
|
||||
// memory/agent.go – Memory-Agent: wraps brain.RunIngest und brain.IngestChatMessage
|
||||
package memory
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"my-brain-importer/internal/agents"
|
||||
"my-brain-importer/internal/brain"
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// Agent verwaltet das Einspeichern von Wissen.
|
||||
type Agent struct{}
|
||||
|
||||
func New() *Agent { return &Agent{} }
|
||||
|
||||
// Handle unterstützt zwei Aktionen:
|
||||
// - "store": Speichert Text als Chat-Nachricht
|
||||
// - "ingest": Startet den Markdown-Ingest aus brain_root
|
||||
func (a *Agent) Handle(req agents.Request) agents.Response {
|
||||
switch req.Action {
|
||||
case agents.ActionStore:
|
||||
return a.store(req)
|
||||
case agents.ActionIngest:
|
||||
return a.ingest(req)
|
||||
default:
|
||||
return agents.Response{Text: "❌ Unbekannte Memory-Aktion. Verfügbar: store, ingest"}
|
||||
}
|
||||
}
|
||||
|
||||
func (a *Agent) store(req agents.Request) agents.Response {
|
||||
if len(req.Args) == 0 {
|
||||
return agents.Response{Text: "❌ Kein Text zum Speichern angegeben."}
|
||||
}
|
||||
text := strings.Join(req.Args, " ")
|
||||
author := req.Author
|
||||
if author == "" {
|
||||
author = "unknown"
|
||||
}
|
||||
source := req.Source
|
||||
if source == "" {
|
||||
source = "agent"
|
||||
}
|
||||
|
||||
err := brain.IngestChatMessage(text, author, source)
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler beim Speichern: %v", err)}
|
||||
}
|
||||
return agents.Response{Text: fmt.Sprintf("✅ Gespeichert: _%s_", text)}
|
||||
}
|
||||
|
||||
func (a *Agent) ingest(_ agents.Request) agents.Response {
|
||||
brain.RunIngest(config.Cfg.BrainRoot)
|
||||
return agents.Response{Text: fmt.Sprintf("✅ Ingest abgeschlossen! Quelle: `%s`", config.Cfg.BrainRoot)}
|
||||
}
|
||||
71
internal/agents/research/agent.go
Normal file
71
internal/agents/research/agent.go
Normal file
@@ -0,0 +1,71 @@
|
||||
// research/agent.go – Research-Agent: wraps brain.AskQuery
|
||||
package research
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"my-brain-importer/internal/agents"
|
||||
"my-brain-importer/internal/brain"
|
||||
)
|
||||
|
||||
// Agent beantwortet Fragen mit der Wissensdatenbank.
|
||||
type Agent struct{}
|
||||
|
||||
func New() *Agent { return &Agent{} }
|
||||
|
||||
func (a *Agent) Handle(req agents.Request) agents.Response {
|
||||
if len(req.Args) == 0 {
|
||||
return agents.Response{Text: "❌ Keine Frage angegeben."}
|
||||
}
|
||||
question := strings.Join(req.Args, " ")
|
||||
|
||||
if req.Action == agents.ActionDeepAsk {
|
||||
return a.handleDeepAsk(question, req.History)
|
||||
}
|
||||
|
||||
answer, chunks, err := brain.AskQuery(question, req.History)
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler: %v", err)}
|
||||
}
|
||||
if len(chunks) == 0 {
|
||||
return agents.Response{Text: "❌ Keine relevanten Informationen in der Datenbank gefunden.\nFüge mehr Daten mit `/ingest` hinzu."}
|
||||
}
|
||||
|
||||
var sb strings.Builder
|
||||
fmt.Fprintf(&sb, "💬 **Antwort auf:** _%s_\n\n", question)
|
||||
sb.WriteString(answer)
|
||||
sb.WriteString("\n\n📚 **Quellen:**\n")
|
||||
for _, chunk := range chunks {
|
||||
fmt.Fprintf(&sb, "• %.1f%% – %s\n", chunk.Score*100, chunk.Source)
|
||||
}
|
||||
return agents.Response{Text: sb.String(), RawAnswer: answer}
|
||||
}
|
||||
|
||||
// handleDeepAsk führt eine tiefe Recherche mit Multi-Step Reasoning durch.
|
||||
func (a *Agent) handleDeepAsk(question string, history []agents.HistoryMessage) agents.Response {
|
||||
answer, chunks, err := brain.DeepAskQuery(question, history)
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler: %v", err)}
|
||||
}
|
||||
if len(chunks) == 0 {
|
||||
return agents.Response{Text: "❌ Keine relevanten Informationen in der Datenbank gefunden.\nFüge mehr Daten mit `/ingest` hinzu."}
|
||||
}
|
||||
|
||||
// Quellen deduplizieren
|
||||
seenSources := make(map[string]float32)
|
||||
for _, chunk := range chunks {
|
||||
if existing, ok := seenSources[chunk.Source]; !ok || chunk.Score > existing {
|
||||
seenSources[chunk.Source] = chunk.Score
|
||||
}
|
||||
}
|
||||
|
||||
var sb strings.Builder
|
||||
fmt.Fprintf(&sb, "🔬 **Tiefe Recherche:** _%s_\n\n", question)
|
||||
sb.WriteString(answer)
|
||||
fmt.Fprintf(&sb, "\n\n📚 **Quellen** (%d Chunks aus %d Quellen):\n", len(chunks), len(seenSources))
|
||||
for source, score := range seenSources {
|
||||
fmt.Fprintf(&sb, "• %.1f%% – %s\n", score*100, source)
|
||||
}
|
||||
return agents.Response{Text: sb.String(), RawAnswer: answer}
|
||||
}
|
||||
209
internal/agents/task/agent.go
Normal file
209
internal/agents/task/agent.go
Normal file
@@ -0,0 +1,209 @@
|
||||
// task/agent.go – Task-Agent: add/list/done/delete
|
||||
package task
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"my-brain-importer/internal/agents"
|
||||
)
|
||||
|
||||
// Agent verwaltet Aufgaben über tasks.json.
|
||||
type Agent struct {
|
||||
store *Store
|
||||
}
|
||||
|
||||
// New erstellt einen neuen Task-Agenten.
|
||||
func New() *Agent {
|
||||
return &Agent{store: NewStore()}
|
||||
}
|
||||
|
||||
// Handle unterstützt: add, list, done, delete
|
||||
func (a *Agent) Handle(req agents.Request) agents.Response {
|
||||
switch req.Action {
|
||||
case agents.ActionAdd:
|
||||
return a.add(req)
|
||||
case agents.ActionList:
|
||||
return a.list()
|
||||
case agents.ActionDone:
|
||||
return a.done(req)
|
||||
case agents.ActionDelete:
|
||||
return a.del(req)
|
||||
default:
|
||||
return agents.Response{Text: "❌ Unbekannte Task-Aktion. Verfügbar: add, list, done, delete"}
|
||||
}
|
||||
}
|
||||
|
||||
// parseAddArgs parst Text, --due YYYY-MM-DD und --priority WERT aus den Args.
|
||||
func parseAddArgs(args []string) (text, priority string, dueDate *time.Time) {
|
||||
var textParts []string
|
||||
for i := 0; i < len(args); i++ {
|
||||
switch args[i] {
|
||||
case "--due", "-d":
|
||||
i++
|
||||
if i < len(args) {
|
||||
t, err := time.Parse("2006-01-02", args[i])
|
||||
if err == nil {
|
||||
dueDate = &t
|
||||
}
|
||||
}
|
||||
case "--priority", "-p":
|
||||
i++
|
||||
if i < len(args) {
|
||||
priority = strings.ToLower(args[i])
|
||||
}
|
||||
default:
|
||||
textParts = append(textParts, args[i])
|
||||
}
|
||||
}
|
||||
text = strings.Join(textParts, " ")
|
||||
return
|
||||
}
|
||||
|
||||
func (a *Agent) add(req agents.Request) agents.Response {
|
||||
if len(req.Args) == 0 {
|
||||
return agents.Response{Text: "❌ Kein Task-Text angegeben."}
|
||||
}
|
||||
text, priority, dueDate := parseAddArgs(req.Args)
|
||||
if text == "" {
|
||||
return agents.Response{Text: "❌ Kein Task-Text angegeben."}
|
||||
}
|
||||
t, err := a.store.Add(text, priority, dueDate)
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler: %v", err)}
|
||||
}
|
||||
shortID := t.ID
|
||||
if len(shortID) > 6 {
|
||||
shortID = shortID[len(shortID)-6:]
|
||||
}
|
||||
var extras []string
|
||||
if t.Priority != "" {
|
||||
extras = append(extras, "Priorität: "+t.Priority)
|
||||
}
|
||||
if t.DueDate != nil {
|
||||
extras = append(extras, "Fällig: "+t.DueDate.Format("02.01.2006"))
|
||||
}
|
||||
info := ""
|
||||
if len(extras) > 0 {
|
||||
info = " (" + strings.Join(extras, ", ") + ")"
|
||||
}
|
||||
return agents.Response{Text: fmt.Sprintf("✅ Task hinzugefügt: `%s`%s (ID: `%s`)", t.Text, info, shortID)}
|
||||
}
|
||||
|
||||
func (a *Agent) list() agents.Response {
|
||||
tasks, err := a.store.Load()
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler: %v", err)}
|
||||
}
|
||||
if len(tasks) == 0 {
|
||||
return agents.Response{Text: "📋 Keine Tasks vorhanden."}
|
||||
}
|
||||
|
||||
today := time.Now().Truncate(24 * time.Hour)
|
||||
tomorrow := today.Add(24 * time.Hour)
|
||||
|
||||
var sb strings.Builder
|
||||
sb.WriteString("📋 **Task-Liste:**\n\n")
|
||||
openCount := 0
|
||||
for _, t := range tasks {
|
||||
status := "⬜"
|
||||
if t.Done {
|
||||
status = "✅"
|
||||
} else {
|
||||
openCount++
|
||||
}
|
||||
shortID := t.ID
|
||||
if len(shortID) > 6 {
|
||||
shortID = shortID[len(shortID)-6:]
|
||||
}
|
||||
|
||||
var meta []string
|
||||
if t.Priority != "" {
|
||||
switch t.Priority {
|
||||
case "hoch":
|
||||
meta = append(meta, "🔴 hoch")
|
||||
case "mittel":
|
||||
meta = append(meta, "🟡 mittel")
|
||||
case "niedrig":
|
||||
meta = append(meta, "🟢 niedrig")
|
||||
default:
|
||||
meta = append(meta, t.Priority)
|
||||
}
|
||||
}
|
||||
if t.DueDate != nil && !t.Done {
|
||||
due := t.DueDate.Truncate(24 * time.Hour)
|
||||
switch {
|
||||
case due.Before(today):
|
||||
meta = append(meta, "⏰ **ÜBERFÄLLIG** "+t.DueDate.Format("02.01."))
|
||||
case due.Equal(today):
|
||||
meta = append(meta, "⏰ heute fällig")
|
||||
case due.Equal(tomorrow):
|
||||
meta = append(meta, "📅 morgen fällig")
|
||||
default:
|
||||
meta = append(meta, "📅 "+t.DueDate.Format("02.01.2006"))
|
||||
}
|
||||
}
|
||||
|
||||
line := fmt.Sprintf("%s `%s` – %s", status, shortID, t.Text)
|
||||
if len(meta) > 0 {
|
||||
line += " · " + strings.Join(meta, " · ")
|
||||
}
|
||||
sb.WriteString(line + "\n")
|
||||
}
|
||||
fmt.Fprintf(&sb, "\n*%d offen, %d gesamt*", openCount, len(tasks))
|
||||
return agents.Response{Text: sb.String()}
|
||||
}
|
||||
|
||||
func (a *Agent) done(req agents.Request) agents.Response {
|
||||
if len(req.Args) == 0 {
|
||||
return agents.Response{Text: "❌ Keine Task-ID angegeben."}
|
||||
}
|
||||
id := req.Args[0]
|
||||
tasks, err := a.store.Load()
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler: %v", err)}
|
||||
}
|
||||
fullID := resolveID(tasks, id)
|
||||
if fullID == "" {
|
||||
return agents.Response{Text: fmt.Sprintf("❌ Task `%s` nicht gefunden.", id)}
|
||||
}
|
||||
if err := a.store.MarkDone(fullID); err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler: %v", err)}
|
||||
}
|
||||
return agents.Response{Text: fmt.Sprintf("✅ Task `%s` als erledigt markiert.", id)}
|
||||
}
|
||||
|
||||
func (a *Agent) del(req agents.Request) agents.Response {
|
||||
if len(req.Args) == 0 {
|
||||
return agents.Response{Text: "❌ Keine Task-ID angegeben."}
|
||||
}
|
||||
id := req.Args[0]
|
||||
tasks, err := a.store.Load()
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler: %v", err)}
|
||||
}
|
||||
fullID := resolveID(tasks, id)
|
||||
if fullID == "" {
|
||||
return agents.Response{Text: fmt.Sprintf("❌ Task `%s` nicht gefunden.", id)}
|
||||
}
|
||||
if err := a.store.Delete(fullID); err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Fehler: %v", err)}
|
||||
}
|
||||
return agents.Response{Text: fmt.Sprintf("🗑️ Task `%s` gelöscht.", id)}
|
||||
}
|
||||
|
||||
// resolveID findet eine vollständige ID aus einer vollständigen oder kurzen (letzte 6 Zeichen).
|
||||
func resolveID(tasks []Task, id string) string {
|
||||
for _, t := range tasks {
|
||||
if t.ID == id {
|
||||
return t.ID
|
||||
}
|
||||
}
|
||||
for _, t := range tasks {
|
||||
if len(t.ID) >= 6 && t.ID[len(t.ID)-6:] == id {
|
||||
return t.ID
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
236
internal/agents/task/agent_test.go
Normal file
236
internal/agents/task/agent_test.go
Normal file
@@ -0,0 +1,236 @@
|
||||
package task
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"my-brain-importer/internal/agents"
|
||||
)
|
||||
|
||||
func newTestAgent(t *testing.T) (*Agent, func()) {
|
||||
t.Helper()
|
||||
f, err := os.CreateTemp("", "tasks_agent_*.json")
|
||||
if err != nil {
|
||||
t.Fatalf("temp-Datei: %v", err)
|
||||
}
|
||||
name := f.Name()
|
||||
f.Close()
|
||||
os.Remove(name) // Datei entfernen → loadLocked gibt leeres Slice zurück
|
||||
a := &Agent{store: &Store{path: name}}
|
||||
return a, func() { os.Remove(name) }
|
||||
}
|
||||
|
||||
// ── parseAddArgs ─────────────────────────────────────────────────────────────
|
||||
|
||||
func TestParseAddArgs_TextOnly(t *testing.T) {
|
||||
text, prio, due := parseAddArgs([]string{"Arzttermin", "buchen"})
|
||||
if text != "Arzttermin buchen" {
|
||||
t.Errorf("text: %q", text)
|
||||
}
|
||||
if prio != "" {
|
||||
t.Errorf("prio sollte leer sein: %q", prio)
|
||||
}
|
||||
if due != nil {
|
||||
t.Error("due sollte nil sein")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAddArgs_AllFlags(t *testing.T) {
|
||||
text, prio, due := parseAddArgs([]string{"Zahnarzt", "--due", "2026-12-01", "--priority", "hoch"})
|
||||
if text != "Zahnarzt" {
|
||||
t.Errorf("text: %q", text)
|
||||
}
|
||||
if prio != "hoch" {
|
||||
t.Errorf("prio: %q", prio)
|
||||
}
|
||||
if due == nil {
|
||||
t.Fatal("due sollte gesetzt sein")
|
||||
}
|
||||
if due.Format("2006-01-02") != "2026-12-01" {
|
||||
t.Errorf("due: %v", due)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAddArgs_ShortFlags(t *testing.T) {
|
||||
text, prio, due := parseAddArgs([]string{"Meeting", "-p", "mittel", "-d", "2026-06-15"})
|
||||
if text != "Meeting" {
|
||||
t.Errorf("text: %q", text)
|
||||
}
|
||||
if prio != "mittel" {
|
||||
t.Errorf("prio: %q", prio)
|
||||
}
|
||||
if due == nil || due.Format("2006-01-02") != "2026-06-15" {
|
||||
t.Errorf("due: %v", due)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAddArgs_InvalidDate(t *testing.T) {
|
||||
_, _, due := parseAddArgs([]string{"Task", "--due", "kein-datum"})
|
||||
if due != nil {
|
||||
t.Error("ungültiges Datum sollte nil ergeben")
|
||||
}
|
||||
}
|
||||
|
||||
// ── Agent.Handle ─────────────────────────────────────────────────────────────
|
||||
|
||||
func TestAgent_Add(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
resp := a.Handle(agents.Request{Action: agents.ActionAdd, Args: []string{"Mein Task"}})
|
||||
if resp.Error != nil {
|
||||
t.Fatalf("Add: %v", resp.Error)
|
||||
}
|
||||
if !strings.Contains(resp.Text, "Mein Task") {
|
||||
t.Errorf("Antwort enthält keinen Task-Text: %q", resp.Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_Add_WithDueAndPriority(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
resp := a.Handle(agents.Request{
|
||||
Action: agents.ActionAdd,
|
||||
Args: []string{"Frist", "--due", "2026-12-31", "--priority", "hoch"},
|
||||
})
|
||||
if resp.Error != nil {
|
||||
t.Fatalf("Add: %v", resp.Error)
|
||||
}
|
||||
if !strings.Contains(resp.Text, "Frist") {
|
||||
t.Errorf("Task-Text fehlt: %q", resp.Text)
|
||||
}
|
||||
if !strings.Contains(resp.Text, "hoch") {
|
||||
t.Errorf("Priorität fehlt: %q", resp.Text)
|
||||
}
|
||||
if !strings.Contains(resp.Text, "31.12.2026") {
|
||||
t.Errorf("Datum fehlt: %q", resp.Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_Add_NoArgs(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
resp := a.Handle(agents.Request{Action: agents.ActionAdd, Args: []string{}})
|
||||
if !strings.Contains(resp.Text, "❌") {
|
||||
t.Errorf("erwartet Fehlermeldung, got: %q", resp.Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_List_Empty(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
resp := a.Handle(agents.Request{Action: agents.ActionList})
|
||||
if !strings.Contains(resp.Text, "Keine Tasks") {
|
||||
t.Errorf("erwartet 'Keine Tasks': %q", resp.Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_List_ShowsDueDate(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
due := time.Now().Add(48 * time.Hour) // übermorgen
|
||||
a.store.Add("Übermorgen", "", &due)
|
||||
|
||||
resp := a.Handle(agents.Request{Action: agents.ActionList})
|
||||
if !strings.Contains(resp.Text, "📅") {
|
||||
t.Errorf("Datum-Icon fehlt in Liste: %q", resp.Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_List_ShowsOverdue(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
past := time.Now().Add(-48 * time.Hour) // vorgestern
|
||||
a.store.Add("Überfällig", "", &past)
|
||||
|
||||
resp := a.Handle(agents.Request{Action: agents.ActionList})
|
||||
if !strings.Contains(resp.Text, "ÜBERFÄLLIG") {
|
||||
t.Errorf("ÜBERFÄLLIG fehlt: %q", resp.Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_Done(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
addResp := a.Handle(agents.Request{Action: agents.ActionAdd, Args: []string{"Erledigen"}})
|
||||
// Short-ID aus Antwort extrahieren
|
||||
tasks, _ := a.store.Load()
|
||||
if len(tasks) == 0 {
|
||||
t.Fatal("kein Task angelegt")
|
||||
}
|
||||
id := tasks[0].ID
|
||||
shortID := id[len(id)-6:]
|
||||
|
||||
_ = addResp
|
||||
resp := a.Handle(agents.Request{Action: agents.ActionDone, Args: []string{shortID}})
|
||||
if resp.Error != nil {
|
||||
t.Fatalf("Done: %v", resp.Error)
|
||||
}
|
||||
if !strings.Contains(resp.Text, "✅") {
|
||||
t.Errorf("erwartet ✅: %q", resp.Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_Delete(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
a.Handle(agents.Request{Action: agents.ActionAdd, Args: []string{"Löschen"}})
|
||||
tasks, _ := a.store.Load()
|
||||
id := tasks[0].ID
|
||||
shortID := id[len(id)-6:]
|
||||
|
||||
resp := a.Handle(agents.Request{Action: agents.ActionDelete, Args: []string{shortID}})
|
||||
if resp.Error != nil {
|
||||
t.Fatalf("Delete: %v", resp.Error)
|
||||
}
|
||||
if !strings.Contains(resp.Text, "🗑️") {
|
||||
t.Errorf("erwartet 🗑️: %q", resp.Text)
|
||||
}
|
||||
|
||||
tasks, _ = a.store.Load()
|
||||
if len(tasks) != 0 {
|
||||
t.Errorf("Task sollte gelöscht sein, noch %d vorhanden", len(tasks))
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_UnknownAction(t *testing.T) {
|
||||
a, cleanup := newTestAgent(t)
|
||||
defer cleanup()
|
||||
|
||||
resp := a.Handle(agents.Request{Action: "unknown"})
|
||||
if !strings.Contains(resp.Text, "❌") {
|
||||
t.Errorf("erwartet Fehlermeldung: %q", resp.Text)
|
||||
}
|
||||
}
|
||||
|
||||
// ── resolveID ────────────────────────────────────────────────────────────────
|
||||
|
||||
func TestResolveID_FullMatch(t *testing.T) {
|
||||
tasks := []Task{{ID: "123456789"}}
|
||||
if got := resolveID(tasks, "123456789"); got != "123456789" {
|
||||
t.Errorf("got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveID_ShortMatch(t *testing.T) {
|
||||
tasks := []Task{{ID: "123456789"}}
|
||||
if got := resolveID(tasks, "456789"); got != "123456789" {
|
||||
t.Errorf("got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveID_NotFound(t *testing.T) {
|
||||
tasks := []Task{{ID: "123456789"}}
|
||||
if got := resolveID(tasks, "000000"); got != "" {
|
||||
t.Errorf("erwartet leer, got %q", got)
|
||||
}
|
||||
}
|
||||
176
internal/agents/task/store.go
Normal file
176
internal/agents/task/store.go
Normal file
@@ -0,0 +1,176 @@
|
||||
// task/store.go – JSON-Persistenz für Tasks (atomisches Schreiben)
|
||||
package task
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// Task repräsentiert eine Aufgabe.
|
||||
type Task struct {
|
||||
ID string `json:"id"`
|
||||
Text string `json:"text"`
|
||||
Done bool `json:"done"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
DoneAt *time.Time `json:"done_at,omitempty"`
|
||||
DueDate *time.Time `json:"due_date,omitempty"`
|
||||
Priority string `json:"priority,omitempty"` // "hoch", "mittel", "niedrig"
|
||||
}
|
||||
|
||||
// Store verwaltet tasks.json mit atomischen Schreiboperationen.
|
||||
type Store struct {
|
||||
mu sync.Mutex
|
||||
path string
|
||||
}
|
||||
|
||||
// NewStore erstellt einen Store mit dem Pfad aus der Config.
|
||||
func NewStore() *Store {
|
||||
path := config.Cfg.Tasks.StorePath
|
||||
if path == "" {
|
||||
path = "./tasks.json"
|
||||
}
|
||||
return &Store{path: path}
|
||||
}
|
||||
|
||||
// Load liest alle Tasks aus der JSON-Datei.
|
||||
func (s *Store) Load() ([]Task, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
data, err := os.ReadFile(s.path)
|
||||
if os.IsNotExist(err) {
|
||||
return []Task{}, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("tasks lesen: %w", err)
|
||||
}
|
||||
|
||||
var tasks []Task
|
||||
if err := json.Unmarshal(data, &tasks); err != nil {
|
||||
return nil, fmt.Errorf("tasks parsen: %w", err)
|
||||
}
|
||||
return tasks, nil
|
||||
}
|
||||
|
||||
// save schreibt alle Tasks atomisch. Muss unter mu aufgerufen werden.
|
||||
func (s *Store) save(tasks []Task) error {
|
||||
data, err := json.MarshalIndent(tasks, "", " ")
|
||||
if err != nil {
|
||||
return fmt.Errorf("tasks serialisieren: %w", err)
|
||||
}
|
||||
tmp := s.path + ".tmp"
|
||||
if err := os.WriteFile(tmp, data, 0644); err != nil {
|
||||
return fmt.Errorf("temp-datei schreiben: %w", err)
|
||||
}
|
||||
return os.Rename(tmp, s.path)
|
||||
}
|
||||
|
||||
// Add fügt einen neuen Task hinzu.
|
||||
func (s *Store) Add(text, priority string, dueDate *time.Time) (Task, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
tasks, err := s.loadLocked()
|
||||
if err != nil {
|
||||
return Task{}, err
|
||||
}
|
||||
|
||||
t := Task{
|
||||
ID: fmt.Sprintf("%d", time.Now().UnixNano()),
|
||||
Text: text,
|
||||
Done: false,
|
||||
CreatedAt: time.Now(),
|
||||
Priority: priority,
|
||||
DueDate: dueDate,
|
||||
}
|
||||
tasks = append(tasks, t)
|
||||
|
||||
if err := s.save(tasks); err != nil {
|
||||
return Task{}, err
|
||||
}
|
||||
return t, nil
|
||||
}
|
||||
|
||||
// MarkDone markiert einen Task als erledigt.
|
||||
func (s *Store) MarkDone(id string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
tasks, err := s.loadLocked()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
found := false
|
||||
now := time.Now()
|
||||
for i, t := range tasks {
|
||||
if t.ID == id {
|
||||
tasks[i].Done = true
|
||||
tasks[i].DoneAt = &now
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return fmt.Errorf("task %q nicht gefunden", id)
|
||||
}
|
||||
return s.save(tasks)
|
||||
}
|
||||
|
||||
// Delete löscht einen Task.
|
||||
func (s *Store) Delete(id string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
tasks, err := s.loadLocked()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
newTasks := tasks[:0]
|
||||
for _, t := range tasks {
|
||||
if t.ID != id {
|
||||
newTasks = append(newTasks, t)
|
||||
}
|
||||
}
|
||||
if len(newTasks) == len(tasks) {
|
||||
return fmt.Errorf("task %q nicht gefunden", id)
|
||||
}
|
||||
return s.save(newTasks)
|
||||
}
|
||||
|
||||
// OpenTasks gibt alle offenen Tasks zurück.
|
||||
func (s *Store) OpenTasks() ([]Task, error) {
|
||||
tasks, err := s.Load()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var open []Task
|
||||
for _, t := range tasks {
|
||||
if !t.Done {
|
||||
open = append(open, t)
|
||||
}
|
||||
}
|
||||
return open, nil
|
||||
}
|
||||
|
||||
// loadLocked liest ohne eigenes Lock (muss unter s.mu aufgerufen werden).
|
||||
func (s *Store) loadLocked() ([]Task, error) {
|
||||
data, err := os.ReadFile(s.path)
|
||||
if os.IsNotExist(err) {
|
||||
return []Task{}, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("tasks lesen: %w", err)
|
||||
}
|
||||
var tasks []Task
|
||||
if err := json.Unmarshal(data, &tasks); err != nil {
|
||||
return nil, fmt.Errorf("tasks parsen: %w", err)
|
||||
}
|
||||
return tasks, nil
|
||||
}
|
||||
169
internal/agents/task/store_test.go
Normal file
169
internal/agents/task/store_test.go
Normal file
@@ -0,0 +1,169 @@
|
||||
package task
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func newTestStore(t *testing.T) (*Store, func()) {
|
||||
t.Helper()
|
||||
f, err := os.CreateTemp("", "tasks_test_*.json")
|
||||
if err != nil {
|
||||
t.Fatalf("temp-Datei erstellen: %v", err)
|
||||
}
|
||||
name := f.Name()
|
||||
f.Close()
|
||||
os.Remove(name) // Datei entfernen → loadLocked gibt leeres Slice zurück
|
||||
s := &Store{path: name}
|
||||
return s, func() { os.Remove(name) }
|
||||
}
|
||||
|
||||
func TestStore_AddAndLoad(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
task, err := s.Add("Test Task", "hoch", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("Add: %v", err)
|
||||
}
|
||||
if task.Text != "Test Task" {
|
||||
t.Errorf("Text: got %q, want %q", task.Text, "Test Task")
|
||||
}
|
||||
if task.Priority != "hoch" {
|
||||
t.Errorf("Priority: got %q, want %q", task.Priority, "hoch")
|
||||
}
|
||||
if task.Done {
|
||||
t.Error("neuer Task sollte nicht Done sein")
|
||||
}
|
||||
|
||||
tasks, err := s.Load()
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
if len(tasks) != 1 {
|
||||
t.Fatalf("len: got %d, want 1", len(tasks))
|
||||
}
|
||||
}
|
||||
|
||||
func TestStore_AddWithDueDate(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
due := time.Date(2026, 12, 31, 0, 0, 0, 0, time.UTC)
|
||||
task, err := s.Add("Frist-Task", "mittel", &due)
|
||||
if err != nil {
|
||||
t.Fatalf("Add: %v", err)
|
||||
}
|
||||
if task.DueDate == nil {
|
||||
t.Fatal("DueDate sollte gesetzt sein")
|
||||
}
|
||||
if !task.DueDate.Equal(due) {
|
||||
t.Errorf("DueDate: got %v, want %v", task.DueDate, due)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStore_MarkDone(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
task, _ := s.Add("Erledigbarer Task", "", nil)
|
||||
if err := s.MarkDone(task.ID); err != nil {
|
||||
t.Fatalf("MarkDone: %v", err)
|
||||
}
|
||||
|
||||
tasks, _ := s.Load()
|
||||
if !tasks[0].Done {
|
||||
t.Error("Task sollte Done=true sein")
|
||||
}
|
||||
if tasks[0].DoneAt == nil {
|
||||
t.Error("DoneAt sollte gesetzt sein")
|
||||
}
|
||||
}
|
||||
|
||||
func TestStore_MarkDone_NotFound(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
err := s.MarkDone("nichtexistent")
|
||||
if err == nil {
|
||||
t.Error("erwartet Fehler für unbekannte ID")
|
||||
}
|
||||
}
|
||||
|
||||
func TestStore_Delete(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
t1, _ := s.Add("Task 1", "", nil)
|
||||
s.Add("Task 2", "", nil)
|
||||
|
||||
if err := s.Delete(t1.ID); err != nil {
|
||||
t.Fatalf("Delete: %v", err)
|
||||
}
|
||||
|
||||
tasks, _ := s.Load()
|
||||
if len(tasks) != 1 {
|
||||
t.Fatalf("nach Delete: got %d, want 1", len(tasks))
|
||||
}
|
||||
if tasks[0].Text != "Task 2" {
|
||||
t.Errorf("falscher Task verblieben: %q", tasks[0].Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStore_Delete_NotFound(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
err := s.Delete("nichtexistent")
|
||||
if err == nil {
|
||||
t.Error("erwartet Fehler für unbekannte ID")
|
||||
}
|
||||
}
|
||||
|
||||
func TestStore_OpenTasks(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
t1, _ := s.Add("Offen", "", nil)
|
||||
s.Add("Auch offen", "", nil)
|
||||
s.MarkDone(t1.ID)
|
||||
|
||||
open, err := s.OpenTasks()
|
||||
if err != nil {
|
||||
t.Fatalf("OpenTasks: %v", err)
|
||||
}
|
||||
if len(open) != 1 {
|
||||
t.Fatalf("OpenTasks: got %d, want 1", len(open))
|
||||
}
|
||||
if open[0].Text != "Auch offen" {
|
||||
t.Errorf("falscher offener Task: %q", open[0].Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStore_EmptyFile(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
tasks, err := s.Load()
|
||||
if err != nil {
|
||||
t.Fatalf("Load auf leerer Datei: %v", err)
|
||||
}
|
||||
if len(tasks) != 0 {
|
||||
t.Errorf("erwartet leer, got %d", len(tasks))
|
||||
}
|
||||
}
|
||||
|
||||
func TestStore_Idempotent_MultipleAdds(t *testing.T) {
|
||||
s, cleanup := newTestStore(t)
|
||||
defer cleanup()
|
||||
|
||||
s.Add("A", "", nil)
|
||||
s.Add("B", "niedrig", nil)
|
||||
s.Add("C", "hoch", nil)
|
||||
|
||||
tasks, _ := s.Load()
|
||||
if len(tasks) != 3 {
|
||||
t.Fatalf("erwartet 3 Tasks, got %d", len(tasks))
|
||||
}
|
||||
}
|
||||
272
internal/agents/tool/agent.go
Normal file
272
internal/agents/tool/agent.go
Normal file
@@ -0,0 +1,272 @@
|
||||
// tool/agent.go – Tool-Agent: Dispatcher für externe Tools (Email, ...)
|
||||
package tool
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"my-brain-importer/internal/agents"
|
||||
"my-brain-importer/internal/agents/tool/email"
|
||||
"my-brain-importer/internal/brain"
|
||||
"my-brain-importer/internal/config"
|
||||
"my-brain-importer/internal/triage"
|
||||
)
|
||||
|
||||
// Agent verteilt Tool-Anfragen an spezialisierte Sub-Agenten.
|
||||
type Agent struct{}
|
||||
|
||||
func New() *Agent { return &Agent{} }
|
||||
|
||||
// Handle unterstützt: email
|
||||
func (a *Agent) Handle(req agents.Request) agents.Response {
|
||||
switch req.Action {
|
||||
case agents.ActionEmail:
|
||||
return a.handleEmail(req)
|
||||
default:
|
||||
return agents.Response{Text: "❌ Unbekannte Tool-Aktion. Verfügbar: email"}
|
||||
}
|
||||
}
|
||||
|
||||
func (a *Agent) handleEmail(req agents.Request) agents.Response {
|
||||
subAction := agents.ActionEmailSummary
|
||||
if len(req.Args) > 0 {
|
||||
subAction = req.Args[0]
|
||||
}
|
||||
|
||||
var (
|
||||
result string
|
||||
err error
|
||||
)
|
||||
|
||||
switch subAction {
|
||||
case agents.ActionEmailSummary:
|
||||
result, err = email.Summarize()
|
||||
case agents.ActionEmailUnread:
|
||||
result, err = email.SummarizeUnread()
|
||||
case agents.ActionEmailRemind:
|
||||
result, err = email.ExtractReminders()
|
||||
case agents.ActionEmailIngest:
|
||||
return a.handleEmailIngest(req)
|
||||
case agents.ActionEmailMove:
|
||||
return a.handleEmailMove(req)
|
||||
case agents.ActionEmailTriage:
|
||||
return a.handleEmailTriage()
|
||||
case agents.ActionEmailTriageHistory:
|
||||
return a.handleTriageHistory(req)
|
||||
case agents.ActionEmailTriageCorrect:
|
||||
return a.handleTriageCorrect(req)
|
||||
case agents.ActionEmailTriageSearch:
|
||||
return a.handleTriageSearch(req)
|
||||
default:
|
||||
return agents.Response{Text: fmt.Sprintf("❌ Unbekannte Email-Aktion `%s`. Verfügbar: summary, unread, remind, ingest, move, triage, triage-history, triage-correct, triage-search", subAction)}
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Email-Fehler: %v", err)}
|
||||
}
|
||||
return agents.Response{Text: "📧 **Email-Analyse:**\n\n" + result}
|
||||
}
|
||||
|
||||
// handleEmailIngest importiert Emails aus einem IMAP-Ordner in Qdrant.
|
||||
// Args[1] = Ordnername (Standard: "Archiv")
|
||||
func (a *Agent) handleEmailIngest(req agents.Request) agents.Response {
|
||||
folder := "Archiv"
|
||||
if len(req.Args) > 1 && req.Args[1] != "" {
|
||||
folder = req.Args[1]
|
||||
}
|
||||
|
||||
accounts := config.AllEmailAccounts()
|
||||
if len(accounts) == 0 {
|
||||
return agents.Response{Text: "❌ Kein Email-Account konfiguriert."}
|
||||
}
|
||||
|
||||
total := 0
|
||||
var errs []string
|
||||
for _, acc := range accounts {
|
||||
n, err := brain.IngestEmailFolder(acc, folder, 500)
|
||||
if err != nil {
|
||||
errs = append(errs, fmt.Sprintf("%s: %v", acc.Name, err))
|
||||
continue
|
||||
}
|
||||
total += n
|
||||
}
|
||||
|
||||
if len(errs) > 0 && total == 0 {
|
||||
return agents.Response{Text: fmt.Sprintf("❌ Email-Ingest fehlgeschlagen:\n%s", joinLines(errs))}
|
||||
}
|
||||
|
||||
msg := fmt.Sprintf("✅ **Email-Ingest abgeschlossen:** %d Emails aus `%s` in die Wissensdatenbank importiert.", total, folder)
|
||||
if len(errs) > 0 {
|
||||
msg += "\n⚠️ Fehler bei einigen Accounts:\n" + joinLines(errs)
|
||||
}
|
||||
return agents.Response{Text: msg}
|
||||
}
|
||||
|
||||
// handleEmailMove verschiebt alle ungelesenen Emails in einen konfigurierten Archivordner.
|
||||
// Args[1] = Zielordner-Name (aus archive_folders in config oder Legacy: 2Jahre/5Jahre/Archiv)
|
||||
func (a *Agent) handleEmailMove(req agents.Request) agents.Response {
|
||||
if len(req.Args) < 2 || req.Args[1] == "" {
|
||||
return agents.Response{Text: "❌ Zielordner fehlt. " + buildMoveFoldersHint()}
|
||||
}
|
||||
dest := req.Args[1]
|
||||
|
||||
imapFolder, ok := resolveArchiveFolder(dest)
|
||||
if !ok {
|
||||
return agents.Response{Text: fmt.Sprintf("❌ Unbekannter Ordner `%s`. %s", dest, buildMoveFoldersHint())}
|
||||
}
|
||||
|
||||
accounts := config.AllEmailAccounts()
|
||||
if len(accounts) == 0 {
|
||||
return agents.Response{Text: "❌ Kein Email-Account konfiguriert."}
|
||||
}
|
||||
|
||||
total := 0
|
||||
var errs []string
|
||||
for _, acc := range accounts {
|
||||
n, err := email.MoveUnread(acc, imapFolder)
|
||||
if err != nil {
|
||||
errs = append(errs, fmt.Sprintf("%s: %v", acc.Name, err))
|
||||
continue
|
||||
}
|
||||
total += n
|
||||
}
|
||||
|
||||
if len(errs) > 0 && total == 0 {
|
||||
return agents.Response{Text: fmt.Sprintf("❌ Verschieben fehlgeschlagen:\n%s", joinLines(errs))}
|
||||
}
|
||||
if total == 0 {
|
||||
return agents.Response{Text: fmt.Sprintf("📭 Keine ungelesenen Emails zum Verschieben nach `%s`.", imapFolder)}
|
||||
}
|
||||
|
||||
msg := fmt.Sprintf("✅ **%d Email(s)** nach `%s` verschoben.", total, imapFolder)
|
||||
if len(errs) > 0 {
|
||||
msg += "\n⚠️ Fehler:\n" + joinLines(errs)
|
||||
}
|
||||
return agents.Response{Text: msg}
|
||||
}
|
||||
|
||||
// handleEmailTriage klassifiziert die letzten 10 Emails aller Accounts und verschiebt sie.
|
||||
func (a *Agent) handleEmailTriage() agents.Response {
|
||||
result, err := email.TriageRecentAllAccounts(10)
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Triage fehlgeschlagen: %v", err)}
|
||||
}
|
||||
return agents.Response{Text: "🗂️ **Email-Triage (letzte 10 Emails):**\n\n" + result}
|
||||
}
|
||||
|
||||
// handleTriageHistory zeigt die letzten N Triage-Entscheidungen.
|
||||
func (a *Agent) handleTriageHistory(req agents.Request) agents.Response {
|
||||
limit := uint32(10)
|
||||
if len(req.Args) > 1 && req.Args[1] != "" {
|
||||
if n, err := strconv.ParseUint(req.Args[1], 10, 32); err == nil && n > 0 {
|
||||
limit = uint32(n)
|
||||
}
|
||||
}
|
||||
|
||||
results, err := triage.ListRecent(limit)
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Triage-History fehlgeschlagen: %v", err)}
|
||||
}
|
||||
if len(results) == 0 {
|
||||
return agents.Response{Text: "📭 Keine Triage-Entscheidungen gespeichert."}
|
||||
}
|
||||
|
||||
var sb strings.Builder
|
||||
fmt.Fprintf(&sb, "🗂️ **Triage-History (%d Einträge):**\n\n", len(results))
|
||||
for i, r := range results {
|
||||
fmt.Fprintf(&sb, "**%d.** %s\n", i+1, r.Text)
|
||||
}
|
||||
return agents.Response{Text: sb.String()}
|
||||
}
|
||||
|
||||
// handleTriageCorrect korrigiert eine Triage-Entscheidung (wichtig↔unwichtig).
|
||||
func (a *Agent) handleTriageCorrect(req agents.Request) agents.Response {
|
||||
if len(req.Args) < 2 || req.Args[1] == "" {
|
||||
return agents.Response{Text: "❌ Betreff fehlt. Beispiel: `/email triage-correct Newsletter`"}
|
||||
}
|
||||
query := strings.Join(req.Args[1:], " ")
|
||||
|
||||
msg, err := triage.CorrectDecision(query)
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Korrektur fehlgeschlagen: %v", err)}
|
||||
}
|
||||
return agents.Response{Text: "✅ " + msg}
|
||||
}
|
||||
|
||||
// handleTriageSearch sucht ähnliche Triage-Entscheidungen.
|
||||
func (a *Agent) handleTriageSearch(req agents.Request) agents.Response {
|
||||
if len(req.Args) < 2 || req.Args[1] == "" {
|
||||
return agents.Response{Text: "❌ Suchbegriff fehlt. Beispiel: `/email triage-search Newsletter`"}
|
||||
}
|
||||
query := strings.Join(req.Args[1:], " ")
|
||||
|
||||
results, err := triage.SearchExtended(query, 10)
|
||||
if err != nil {
|
||||
return agents.Response{Error: err, Text: fmt.Sprintf("❌ Triage-Suche fehlgeschlagen: %v", err)}
|
||||
}
|
||||
if len(results) == 0 {
|
||||
return agents.Response{Text: "📭 Keine ähnlichen Triage-Entscheidungen gefunden."}
|
||||
}
|
||||
|
||||
var sb strings.Builder
|
||||
fmt.Fprintf(&sb, "🔍 **Triage-Suche** (query: `%s`, %d Treffer):\n\n", query, len(results))
|
||||
for i, r := range results {
|
||||
fmt.Fprintf(&sb, "**%d.** [%.0f%%] %s\n", i+1, r.Score*100, r.Text)
|
||||
}
|
||||
return agents.Response{Text: sb.String()}
|
||||
}
|
||||
|
||||
// ResolveArchiveFolder ist die exportierte Version von resolveArchiveFolder für den Discord-Layer.
|
||||
func ResolveArchiveFolder(name string) (imapFolder string, ok bool) {
|
||||
return resolveArchiveFolder(name)
|
||||
}
|
||||
|
||||
// resolveArchiveFolder sucht den IMAP-Ordnernamen für einen Anzeigenamen aus der Config.
|
||||
// Fallback: Legacy-Hardcoding für 2Jahre/5Jahre/Archiv wenn keine archive_folders konfiguriert.
|
||||
func resolveArchiveFolder(name string) (imapFolder string, ok bool) {
|
||||
for _, acc := range config.AllEmailAccounts() {
|
||||
for _, af := range acc.ArchiveFolders {
|
||||
if strings.EqualFold(af.Name, name) || strings.EqualFold(af.IMAPFolder, name) {
|
||||
return af.IMAPFolder, true
|
||||
}
|
||||
}
|
||||
}
|
||||
// Legacy-Fallback für Konfigurationen ohne archive_folders
|
||||
legacy := map[string]string{
|
||||
"2jahre": "2Jahre",
|
||||
"5jahre": "5Jahre",
|
||||
"archiv": "Archiv",
|
||||
}
|
||||
if canonical, found := legacy[strings.ToLower(name)]; found {
|
||||
return canonical, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// buildMoveFoldersHint gibt eine Hinweis-Nachricht mit verfügbaren Archivordnern zurück.
|
||||
func buildMoveFoldersHint() string {
|
||||
seen := map[string]bool{}
|
||||
var names []string
|
||||
for _, acc := range config.AllEmailAccounts() {
|
||||
for _, af := range acc.ArchiveFolders {
|
||||
key := strings.ToLower(af.Name)
|
||||
if !seen[key] {
|
||||
seen[key] = true
|
||||
names = append(names, fmt.Sprintf("`%s`", af.Name))
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return "Verfügbar: `2Jahre`, `5Jahre`, `Archiv`"
|
||||
}
|
||||
return fmt.Sprintf("Verfügbar: %s", strings.Join(names, ", "))
|
||||
}
|
||||
|
||||
func joinLines(lines []string) string {
|
||||
result := ""
|
||||
for _, l := range lines {
|
||||
result += "• " + l + "\n"
|
||||
}
|
||||
return result
|
||||
}
|
||||
813
internal/agents/tool/email/client.go
Normal file
813
internal/agents/tool/email/client.go
Normal file
@@ -0,0 +1,813 @@
|
||||
// email/client.go – IMAP-Client für Email-Abfragen
|
||||
package email
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/tls"
|
||||
"encoding/base64"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"mime"
|
||||
"mime/multipart"
|
||||
"mime/quotedprintable"
|
||||
"net/mail"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
imap "github.com/emersion/go-imap/v2"
|
||||
"github.com/emersion/go-imap/v2/imapclient"
|
||||
"github.com/ledongthuc/pdf"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// Message repräsentiert eine Email (ohne Body für schnelle Übersichten).
|
||||
type Message struct {
|
||||
Subject string
|
||||
From string
|
||||
Date string
|
||||
}
|
||||
|
||||
// SelectMessage koppelt eine Message mit ihrer IMAP-Sequenznummer für UI-Zwecke.
|
||||
type SelectMessage struct {
|
||||
Message
|
||||
SeqNum uint32
|
||||
Unread bool // true = \Seen flag nicht gesetzt
|
||||
}
|
||||
|
||||
// MessageWithBody repräsentiert eine Email mit Text-Inhalt (für Datenbankimport).
|
||||
type MessageWithBody struct {
|
||||
Message
|
||||
Body string
|
||||
}
|
||||
|
||||
// Client wraps die IMAP-Verbindung.
|
||||
type Client struct {
|
||||
c *imapclient.Client
|
||||
folder string // INBOX-Ordner (leer = "INBOX")
|
||||
}
|
||||
|
||||
// Connect öffnet eine IMAP-Verbindung mit dem Legacy-Email-Block aus der Config.
|
||||
func Connect() (*Client, error) {
|
||||
cfg := config.Cfg.Email
|
||||
acc := config.EmailAccount{
|
||||
Host: cfg.Host,
|
||||
Port: cfg.Port,
|
||||
User: cfg.User,
|
||||
Password: cfg.Password,
|
||||
TLS: cfg.TLS,
|
||||
StartTLS: cfg.StartTLS,
|
||||
Folder: cfg.Folder,
|
||||
}
|
||||
return ConnectAccount(acc)
|
||||
}
|
||||
|
||||
// ConnectAccount öffnet eine IMAP-Verbindung für einen bestimmten EmailAccount.
|
||||
func ConnectAccount(acc config.EmailAccount) (*Client, error) {
|
||||
addr := fmt.Sprintf("%s:%d", acc.Host, acc.Port)
|
||||
|
||||
var (
|
||||
c *imapclient.Client
|
||||
err error
|
||||
)
|
||||
|
||||
switch {
|
||||
case acc.TLS:
|
||||
tlsCfg := &tls.Config{ServerName: acc.Host}
|
||||
c, err = imapclient.DialTLS(addr, &imapclient.Options{TLSConfig: tlsCfg})
|
||||
case acc.StartTLS:
|
||||
tlsCfg := &tls.Config{ServerName: acc.Host}
|
||||
c, err = imapclient.DialStartTLS(addr, &imapclient.Options{TLSConfig: tlsCfg})
|
||||
default:
|
||||
c, err = imapclient.DialInsecure(addr, nil)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP verbinden: %w", err)
|
||||
}
|
||||
|
||||
if err := c.Login(acc.User, acc.Password).Wait(); err != nil {
|
||||
c.Close()
|
||||
return nil, fmt.Errorf("IMAP login: %w", err)
|
||||
}
|
||||
|
||||
return &Client{c: c, folder: acc.Folder}, nil
|
||||
}
|
||||
|
||||
// Close schließt die Verbindung.
|
||||
func (cl *Client) Close() {
|
||||
cl.c.Logout().Wait()
|
||||
cl.c.Close()
|
||||
}
|
||||
|
||||
// EnsureFolder legt einen IMAP-Ordner an falls er nicht existiert.
|
||||
// Strato-kompatibel: ignoriert alle "already exists"-Varianten.
|
||||
func (cl *Client) EnsureFolder(folder string) error {
|
||||
err := cl.c.Create(folder, nil).Wait()
|
||||
if err == nil {
|
||||
slog.Info("IMAP: Ordner angelegt", "ordner", folder)
|
||||
return nil
|
||||
}
|
||||
errLower := strings.ToLower(err.Error())
|
||||
if strings.Contains(errLower, "already exists") ||
|
||||
strings.Contains(errLower, "alreadyexists") ||
|
||||
strings.Contains(errLower, "mailbox exists") ||
|
||||
strings.Contains(errLower, "exists") {
|
||||
return nil // Ordner existiert bereits — kein Fehler
|
||||
}
|
||||
slog.Error("IMAP: Ordner anlegen fehlgeschlagen", "ordner", folder, "fehler", err)
|
||||
return fmt.Errorf("IMAP create folder %s: %w", folder, err)
|
||||
}
|
||||
|
||||
// FetchRecent holt die letzten n Emails (Envelope-Daten, kein Body).
|
||||
func (cl *Client) FetchRecent(n uint32) ([]Message, error) {
|
||||
folder := cl.folder
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
selectData, err := cl.c.Select(folder, &imap.SelectOptions{ReadOnly: true}).Wait()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP select: %w", err)
|
||||
}
|
||||
if selectData.NumMessages == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
start := uint32(1)
|
||||
if selectData.NumMessages > n {
|
||||
start = selectData.NumMessages - n + 1
|
||||
}
|
||||
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddRange(start, selectData.NumMessages)
|
||||
|
||||
msgs, err := cl.c.Fetch(seqSet, &imap.FetchOptions{Envelope: true}).Collect()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP fetch: %w", err)
|
||||
}
|
||||
|
||||
return parseMessages(msgs), nil
|
||||
}
|
||||
|
||||
// FetchRecentFromFolder holt die letzten n Emails aus einem bestimmten IMAP-Ordner.
|
||||
func (cl *Client) FetchRecentFromFolder(folder string, n uint32) ([]Message, error) {
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
selectData, err := cl.c.Select(folder, &imap.SelectOptions{ReadOnly: true}).Wait()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP select %s: %w", folder, err)
|
||||
}
|
||||
if selectData.NumMessages == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
start := uint32(1)
|
||||
if selectData.NumMessages > n {
|
||||
start = selectData.NumMessages - n + 1
|
||||
}
|
||||
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddRange(start, selectData.NumMessages)
|
||||
|
||||
msgs, err := cl.c.Fetch(seqSet, &imap.FetchOptions{Envelope: true}).Collect()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP fetch %s: %w", folder, err)
|
||||
}
|
||||
|
||||
return parseMessages(msgs), nil
|
||||
}
|
||||
|
||||
// FetchUnread holt ungelesene Emails (Envelope-Daten, kein Body).
|
||||
func (cl *Client) FetchUnread() ([]Message, error) {
|
||||
folder := cl.folder
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
if _, err := cl.c.Select(folder, &imap.SelectOptions{ReadOnly: true}).Wait(); err != nil {
|
||||
return nil, fmt.Errorf("IMAP select: %w", err)
|
||||
}
|
||||
|
||||
searchData, err := cl.c.Search(&imap.SearchCriteria{
|
||||
NotFlag: []imap.Flag{imap.FlagSeen},
|
||||
}, nil).Wait()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP search: %w", err)
|
||||
}
|
||||
|
||||
seqNums := searchData.AllSeqNums()
|
||||
if len(seqNums) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddNum(seqNums...)
|
||||
|
||||
msgs, err := cl.c.Fetch(seqSet, &imap.FetchOptions{Envelope: true}).Collect()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP fetch: %w", err)
|
||||
}
|
||||
|
||||
return parseMessages(msgs), nil
|
||||
}
|
||||
|
||||
// FetchUnreadSeqNums holt ungelesene Emails und gibt zusätzlich die Sequenznummern zurück.
|
||||
// Selektiert den Ordner im Lese-Schreib-Modus (für nachfolgendes Verschieben).
|
||||
func (cl *Client) FetchUnreadSeqNums() ([]Message, []uint32, error) {
|
||||
folder := cl.folder
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
if _, err := cl.c.Select(folder, nil).Wait(); err != nil {
|
||||
return nil, nil, fmt.Errorf("IMAP select: %w", err)
|
||||
}
|
||||
|
||||
searchData, err := cl.c.Search(&imap.SearchCriteria{
|
||||
NotFlag: []imap.Flag{imap.FlagSeen},
|
||||
}, nil).Wait()
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("IMAP search: %w", err)
|
||||
}
|
||||
|
||||
seqNums := searchData.AllSeqNums()
|
||||
if len(seqNums) == 0 {
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddNum(seqNums...)
|
||||
|
||||
msgs, err := cl.c.Fetch(seqSet, &imap.FetchOptions{Envelope: true}).Collect()
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("IMAP fetch: %w", err)
|
||||
}
|
||||
|
||||
return parseMessages(msgs), seqNums, nil
|
||||
}
|
||||
|
||||
// MoveMessages verschiebt Nachrichten in einen anderen IMAP-Ordner.
|
||||
// Der Ordner muss im Lese-Schreib-Modus selektiert sein (via FetchUnreadSeqNums).
|
||||
func (cl *Client) MoveMessages(seqNums []uint32, destFolder string) error {
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddNum(seqNums...)
|
||||
if _, err := cl.c.Move(seqSet, destFolder).Wait(); err != nil {
|
||||
return fmt.Errorf("IMAP move: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// FetchUnreadForSelect gibt ungelesene Emails mit ihren Sequenznummern zurück.
|
||||
// Selektiert den Ordner im Lese-Schreib-Modus (für nachfolgendes Verschieben).
|
||||
func (cl *Client) FetchUnreadForSelect() ([]SelectMessage, error) {
|
||||
folder := cl.folder
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
if _, err := cl.c.Select(folder, nil).Wait(); err != nil {
|
||||
return nil, fmt.Errorf("IMAP select: %w", err)
|
||||
}
|
||||
|
||||
searchData, err := cl.c.Search(&imap.SearchCriteria{
|
||||
NotFlag: []imap.Flag{imap.FlagSeen},
|
||||
}, nil).Wait()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP search: %w", err)
|
||||
}
|
||||
|
||||
seqNums := searchData.AllSeqNums()
|
||||
if len(seqNums) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddNum(seqNums...)
|
||||
|
||||
rawMsgs, err := cl.c.Fetch(seqSet, &imap.FetchOptions{Envelope: true}).Collect()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP fetch: %w", err)
|
||||
}
|
||||
|
||||
seqToMsg := make(map[uint32]*imapclient.FetchMessageBuffer, len(rawMsgs))
|
||||
for _, m := range rawMsgs {
|
||||
if m.Envelope != nil {
|
||||
seqToMsg[m.SeqNum] = m
|
||||
}
|
||||
}
|
||||
|
||||
result := make([]SelectMessage, 0, len(seqNums))
|
||||
for _, sn := range seqNums {
|
||||
m, ok := seqToMsg[sn]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
result = append(result, SelectMessage{
|
||||
Message: parseMessage(m),
|
||||
SeqNum: sn,
|
||||
Unread: true,
|
||||
})
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// FetchRecentForSelect gibt die letzten n Emails mit Sequenznummern und Unread-Status zurück.
|
||||
// Selektiert den Ordner im Lese-Schreib-Modus (für nachfolgendes Verschieben).
|
||||
func (cl *Client) FetchRecentForSelect(n uint32) ([]SelectMessage, error) {
|
||||
folder := cl.folder
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
selectData, err := cl.c.Select(folder, nil).Wait()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP select: %w", err)
|
||||
}
|
||||
if selectData.NumMessages == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
start := uint32(1)
|
||||
if selectData.NumMessages > n {
|
||||
start = selectData.NumMessages - n + 1
|
||||
}
|
||||
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddRange(start, selectData.NumMessages)
|
||||
|
||||
rawMsgs, err := cl.c.Fetch(seqSet, &imap.FetchOptions{Envelope: true, Flags: true}).Collect()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP fetch: %w", err)
|
||||
}
|
||||
|
||||
result := make([]SelectMessage, 0, len(rawMsgs))
|
||||
for _, m := range rawMsgs {
|
||||
if m.Envelope == nil {
|
||||
continue
|
||||
}
|
||||
unread := true
|
||||
for _, f := range m.Flags {
|
||||
if f == imap.FlagSeen {
|
||||
unread = false
|
||||
break
|
||||
}
|
||||
}
|
||||
result = append(result, SelectMessage{
|
||||
Message: parseMessage(m),
|
||||
SeqNum: m.SeqNum,
|
||||
Unread: unread,
|
||||
})
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// MoveOldMessages verschiebt alle Emails im Ordner, die älter als olderThanDays Tage sind, nach destFolder.
|
||||
// Gibt die Anzahl verschobener Nachrichten zurück. olderThanDays <= 0 ist ein No-op.
|
||||
func (cl *Client) MoveOldMessages(folder, destFolder string, olderThanDays int) (int, error) {
|
||||
if olderThanDays <= 0 {
|
||||
return 0, nil
|
||||
}
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
cutoff := time.Now().AddDate(0, 0, -olderThanDays).Truncate(24 * time.Hour)
|
||||
|
||||
if _, err := cl.c.Select(folder, nil).Wait(); err != nil {
|
||||
return 0, fmt.Errorf("IMAP select %s: %w", folder, err)
|
||||
}
|
||||
|
||||
searchData, err := cl.c.Search(&imap.SearchCriteria{Before: cutoff}, nil).Wait()
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("IMAP search: %w", err)
|
||||
}
|
||||
|
||||
seqNums := searchData.AllSeqNums()
|
||||
if len(seqNums) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddNum(seqNums...)
|
||||
|
||||
if _, err := cl.c.Move(seqSet, destFolder).Wait(); err != nil {
|
||||
return 0, fmt.Errorf("IMAP move: %w", err)
|
||||
}
|
||||
|
||||
return len(seqNums), nil
|
||||
}
|
||||
|
||||
// MoveSpecificMessages selektiert den Inbox-Ordner und verschiebt die angegebenen Sequenznummern.
|
||||
func (cl *Client) MoveSpecificMessages(seqNums []uint32, destFolder string) error {
|
||||
folder := cl.folder
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
if _, err := cl.c.Select(folder, nil).Wait(); err != nil {
|
||||
return fmt.Errorf("IMAP select: %w", err)
|
||||
}
|
||||
return cl.MoveMessages(seqNums, destFolder)
|
||||
}
|
||||
|
||||
// CleanupOldEmails löscht Emails im Ordner, die älter als retentionDays sind.
|
||||
// Gibt die Anzahl gelöschter Nachrichten zurück. retentionDays <= 0 ist ein No-op.
|
||||
func (cl *Client) CleanupOldEmails(folder string, retentionDays int) (int, error) {
|
||||
if retentionDays <= 0 {
|
||||
return 0, nil
|
||||
}
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
cutoff := time.Now().AddDate(0, 0, -retentionDays).Truncate(24 * time.Hour)
|
||||
|
||||
if _, err := cl.c.Select(folder, nil).Wait(); err != nil {
|
||||
return 0, fmt.Errorf("IMAP select %s: %w", folder, err)
|
||||
}
|
||||
|
||||
searchData, err := cl.c.Search(&imap.SearchCriteria{Before: cutoff}, nil).Wait()
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("IMAP search: %w", err)
|
||||
}
|
||||
|
||||
seqNums := searchData.AllSeqNums()
|
||||
if len(seqNums) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddNum(seqNums...)
|
||||
|
||||
storeFlags := &imap.StoreFlags{
|
||||
Op: imap.StoreFlagsAdd,
|
||||
Silent: true,
|
||||
Flags: []imap.Flag{imap.FlagDeleted},
|
||||
}
|
||||
if _, err := cl.c.Store(seqSet, storeFlags, nil).Collect(); err != nil {
|
||||
return 0, fmt.Errorf("IMAP store flags: %w", err)
|
||||
}
|
||||
|
||||
if _, err := cl.c.Expunge().Collect(); err != nil {
|
||||
return 0, fmt.Errorf("IMAP expunge: %w", err)
|
||||
}
|
||||
|
||||
return len(seqNums), nil
|
||||
}
|
||||
|
||||
// FetchWithBody holt bis zu n Emails aus dem angegebenen Ordner mit Text-Body.
|
||||
// Emails werden in Batches von 50 gefetcht um den IMAP-Server nicht zu überlasten.
|
||||
func (cl *Client) FetchWithBody(folder string, n uint32) ([]MessageWithBody, error) {
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
selectData, err := cl.c.Select(folder, &imap.SelectOptions{ReadOnly: true}).Wait()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP select %s: %w", folder, err)
|
||||
}
|
||||
if selectData.NumMessages == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// Letzte n Nachrichten
|
||||
total := selectData.NumMessages
|
||||
start := uint32(1)
|
||||
if total > n {
|
||||
start = total - n + 1
|
||||
}
|
||||
|
||||
bodySec := &imap.FetchItemBodySection{Specifier: imap.PartSpecifierText}
|
||||
hdrSec := &imap.FetchItemBodySection{Specifier: imap.PartSpecifierHeader}
|
||||
|
||||
var result []MessageWithBody
|
||||
batchSize := uint32(50)
|
||||
|
||||
for i := start; i <= total; i += batchSize {
|
||||
end := i + batchSize - 1
|
||||
if end > total {
|
||||
end = total
|
||||
}
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddRange(i, end)
|
||||
|
||||
msgs, err := cl.c.Fetch(seqSet, &imap.FetchOptions{
|
||||
Envelope: true,
|
||||
BodySection: []*imap.FetchItemBodySection{bodySec, hdrSec},
|
||||
}).Collect()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP fetch batch %d-%d: %w", i, end, err)
|
||||
}
|
||||
|
||||
for _, msg := range msgs {
|
||||
if msg.Envelope == nil {
|
||||
continue
|
||||
}
|
||||
m := MessageWithBody{Message: parseMessage(msg)}
|
||||
|
||||
// Content-Transfer-Encoding aus Header lesen
|
||||
enc := ""
|
||||
if hdr := msg.FindBodySection(hdrSec); hdr != nil {
|
||||
for _, line := range strings.Split(string(hdr), "\n") {
|
||||
if strings.HasPrefix(strings.ToLower(line), "content-transfer-encoding:") {
|
||||
enc = strings.TrimSpace(strings.ToLower(strings.SplitN(line, ":", 2)[1]))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if body := msg.FindBodySection(bodySec); body != nil {
|
||||
m.Body = decodeBody(body, enc)
|
||||
}
|
||||
result = append(result, m)
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// decodeBody dekodiert einen Email-Body je nach Content-Transfer-Encoding.
|
||||
func decodeBody(raw []byte, enc string) string {
|
||||
var text string
|
||||
switch enc {
|
||||
case "base64":
|
||||
cleaned := strings.ReplaceAll(strings.TrimSpace(string(raw)), "\r\n", "")
|
||||
if decoded, err := base64.StdEncoding.DecodeString(cleaned); err == nil {
|
||||
text = string(decoded)
|
||||
} else if decoded, err := base64.RawStdEncoding.DecodeString(cleaned); err == nil {
|
||||
text = string(decoded)
|
||||
} else {
|
||||
text = string(raw) // Fallback: roh
|
||||
}
|
||||
case "quoted-printable":
|
||||
r := quotedprintable.NewReader(strings.NewReader(string(raw)))
|
||||
if buf := new(strings.Builder); true {
|
||||
buf.Grow(len(raw))
|
||||
tmp := make([]byte, 4096)
|
||||
for {
|
||||
n, err := r.Read(tmp)
|
||||
buf.Write(tmp[:n])
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
text = buf.String()
|
||||
}
|
||||
default:
|
||||
text = string(raw)
|
||||
}
|
||||
|
||||
// Kürzen auf max 2000 Zeichen
|
||||
text = strings.TrimSpace(text)
|
||||
if len(text) > 2000 {
|
||||
text = text[:2000]
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
// parseMessage extrahiert eine Message aus einem FetchMessageBuffer.
|
||||
func parseMessage(msg *imapclient.FetchMessageBuffer) Message {
|
||||
m := Message{
|
||||
Subject: msg.Envelope.Subject,
|
||||
Date: msg.Envelope.Date.Format("2006-01-02 15:04"),
|
||||
}
|
||||
if len(msg.Envelope.From) > 0 {
|
||||
addr := msg.Envelope.From[0]
|
||||
if addr.Name != "" {
|
||||
m.From = fmt.Sprintf("%s <%s@%s>", addr.Name, addr.Mailbox, addr.Host)
|
||||
} else {
|
||||
m.From = fmt.Sprintf("%s@%s", addr.Mailbox, addr.Host)
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// FetchWithBodyAndAttachments holt bis zu n Emails aus dem angegebenen Ordner mit Text-Body
|
||||
// und extrahiert Text aus PDF-Anhängen. Der kombinierte Text wird in MessageWithBody.Body gespeichert.
|
||||
// Nutzt stdlib mime/multipart — keine externen Abhängigkeiten außer dem bereits vorhandenen PDF-Parser.
|
||||
func (cl *Client) FetchWithBodyAndAttachments(folder string, n uint32) ([]MessageWithBody, error) {
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
|
||||
selectData, err := cl.c.Select(folder, &imap.SelectOptions{ReadOnly: true}).Wait()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP select %s: %w", folder, err)
|
||||
}
|
||||
if selectData.NumMessages == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
total := selectData.NumMessages
|
||||
start := uint32(1)
|
||||
if total > n {
|
||||
start = total - n + 1
|
||||
}
|
||||
|
||||
// Komplette RFC822-Nachricht fetchen (Header + Body + Attachments)
|
||||
fullMsgSec := &imap.FetchItemBodySection{}
|
||||
|
||||
var result []MessageWithBody
|
||||
batchSize := uint32(50)
|
||||
|
||||
for i := start; i <= total; i += batchSize {
|
||||
end := i + batchSize - 1
|
||||
if end > total {
|
||||
end = total
|
||||
}
|
||||
var seqSet imap.SeqSet
|
||||
seqSet.AddRange(i, end)
|
||||
|
||||
msgs, err := cl.c.Fetch(seqSet, &imap.FetchOptions{
|
||||
Envelope: true,
|
||||
BodySection: []*imap.FetchItemBodySection{fullMsgSec},
|
||||
}).Collect()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("IMAP fetch batch %d-%d: %w", i, end, err)
|
||||
}
|
||||
|
||||
for _, msg := range msgs {
|
||||
if msg.Envelope == nil {
|
||||
continue
|
||||
}
|
||||
m := MessageWithBody{Message: parseMessage(msg)}
|
||||
|
||||
rawMsg := msg.FindBodySection(fullMsgSec)
|
||||
if rawMsg != nil {
|
||||
body, err := extractBodyAndAttachments(rawMsg)
|
||||
if err != nil {
|
||||
slog.Warn("[Email] MIME-Parsing fehlgeschlagen", "betreff", m.Subject, "fehler", err)
|
||||
} else {
|
||||
m.Body = body
|
||||
}
|
||||
}
|
||||
result = append(result, m)
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// extractBodyAndAttachments parst eine rohe RFC822-Nachricht und gibt den kombinierten Text zurück.
|
||||
// Text/plain-Teile werden direkt übernommen, PDF-Anhänge werden in Text extrahiert.
|
||||
func extractBodyAndAttachments(rawMsg []byte) (string, error) {
|
||||
parsed, err := mail.ReadMessage(bytes.NewReader(rawMsg))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("mail parsen: %w", err)
|
||||
}
|
||||
|
||||
contentType := parsed.Header.Get("Content-Type")
|
||||
mediaType, params, err := mime.ParseMediaType(contentType)
|
||||
if err != nil {
|
||||
// Kein valider Content-Type — Body direkt lesen
|
||||
body, readErr := io.ReadAll(parsed.Body)
|
||||
if readErr != nil {
|
||||
return "", fmt.Errorf("body lesen: %w", readErr)
|
||||
}
|
||||
enc := strings.ToLower(parsed.Header.Get("Content-Transfer-Encoding"))
|
||||
return decodeBody(body, enc), nil
|
||||
}
|
||||
|
||||
var parts []string
|
||||
|
||||
if strings.HasPrefix(mediaType, "multipart/") {
|
||||
boundary := params["boundary"]
|
||||
mr := multipart.NewReader(parsed.Body, boundary)
|
||||
for {
|
||||
part, err := mr.NextPart()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
slog.Warn("[Email] Multipart-Teil fehlgeschlagen", "fehler", err)
|
||||
break
|
||||
}
|
||||
|
||||
partContentType := part.Header.Get("Content-Type")
|
||||
partMediaType, _, parseErr := mime.ParseMediaType(partContentType)
|
||||
if parseErr != nil {
|
||||
part.Close()
|
||||
continue
|
||||
}
|
||||
|
||||
enc := strings.ToLower(part.Header.Get("Content-Transfer-Encoding"))
|
||||
data, readErr := io.ReadAll(part)
|
||||
part.Close()
|
||||
if readErr != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
switch {
|
||||
case partMediaType == "text/plain":
|
||||
parts = append(parts, decodeBody(data, enc))
|
||||
case partMediaType == "application/pdf":
|
||||
pdfText, pdfErr := extractPDFTextFromBytes(data, enc)
|
||||
if pdfErr != nil {
|
||||
slog.Warn("[Email] PDF-Anhang konnte nicht gelesen werden", "fehler", pdfErr)
|
||||
} else if pdfText != "" {
|
||||
parts = append(parts, "[PDF-Anhang] "+pdfText)
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Einfache (nicht-multipart) Nachricht
|
||||
body, readErr := io.ReadAll(parsed.Body)
|
||||
if readErr != nil {
|
||||
return "", fmt.Errorf("body lesen: %w", readErr)
|
||||
}
|
||||
enc := strings.ToLower(parsed.Header.Get("Content-Transfer-Encoding"))
|
||||
parts = append(parts, decodeBody(body, enc))
|
||||
}
|
||||
|
||||
combined := strings.TrimSpace(strings.Join(parts, "\n"))
|
||||
if len(combined) > 2000 {
|
||||
combined = combined[:2000]
|
||||
}
|
||||
return combined, nil
|
||||
}
|
||||
|
||||
// extractPDFTextFromBytes dekodiert die rohen Anhang-Bytes (ggf. base64) und extrahiert PDF-Text.
|
||||
func extractPDFTextFromBytes(data []byte, enc string) (string, error) {
|
||||
// PDF-Anhänge sind fast immer base64-kodiert
|
||||
var pdfBytes []byte
|
||||
switch enc {
|
||||
case "base64":
|
||||
cleaned := strings.ReplaceAll(strings.TrimSpace(string(data)), "\r\n", "")
|
||||
cleaned = strings.ReplaceAll(cleaned, "\n", "")
|
||||
decoded, err := base64.StdEncoding.DecodeString(cleaned)
|
||||
if err != nil {
|
||||
decoded, err = base64.RawStdEncoding.DecodeString(cleaned)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("base64 dekodieren: %w", err)
|
||||
}
|
||||
}
|
||||
pdfBytes = decoded
|
||||
default:
|
||||
pdfBytes = data
|
||||
}
|
||||
|
||||
// PDF in temporäre Datei schreiben, da die pdf-Bibliothek einen Datei-Pfad erwartet
|
||||
tmp, err := os.CreateTemp("", "email-pdf-*.pdf")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("temp-Datei anlegen: %w", err)
|
||||
}
|
||||
tmpPath := tmp.Name()
|
||||
defer os.Remove(tmpPath)
|
||||
|
||||
if _, err := tmp.Write(pdfBytes); err != nil {
|
||||
tmp.Close()
|
||||
return "", fmt.Errorf("temp-Datei schreiben: %w", err)
|
||||
}
|
||||
tmp.Close()
|
||||
|
||||
return extractPDFTextFromFile(tmpPath)
|
||||
}
|
||||
|
||||
// extractPDFTextFromFile liest alle Seiten einer PDF-Datei und gibt den Plain-Text zurück.
|
||||
// Dupliziert die Logik aus brain/ingest_pdf.go um Import-Zyklen zu vermeiden.
|
||||
func extractPDFTextFromFile(filePath string) (string, error) {
|
||||
f, r, err := pdf.Open(filePath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
var sb strings.Builder
|
||||
totalPages := r.NumPage()
|
||||
for pageNum := 1; pageNum <= totalPages; pageNum++ {
|
||||
page := r.Page(pageNum)
|
||||
if page.V.IsNull() {
|
||||
continue
|
||||
}
|
||||
text, err := page.GetPlainText(nil)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
sb.WriteString(text)
|
||||
sb.WriteString("\n")
|
||||
}
|
||||
return strings.TrimSpace(sb.String()), nil
|
||||
}
|
||||
|
||||
func parseMessages(msgs []*imapclient.FetchMessageBuffer) []Message {
|
||||
result := make([]Message, 0, len(msgs))
|
||||
for _, msg := range msgs {
|
||||
if msg.Envelope == nil {
|
||||
continue
|
||||
}
|
||||
m := Message{
|
||||
Subject: msg.Envelope.Subject,
|
||||
Date: msg.Envelope.Date.Format("2006-01-02 15:04"),
|
||||
}
|
||||
if len(msg.Envelope.From) > 0 {
|
||||
addr := msg.Envelope.From[0]
|
||||
if addr.Name != "" {
|
||||
m.From = fmt.Sprintf("%s <%s@%s>", addr.Name, addr.Mailbox, addr.Host)
|
||||
} else {
|
||||
m.From = fmt.Sprintf("%s@%s", addr.Mailbox, addr.Host)
|
||||
}
|
||||
}
|
||||
result = append(result, m)
|
||||
}
|
||||
return result
|
||||
}
|
||||
153
internal/agents/tool/email/idle.go
Normal file
153
internal/agents/tool/email/idle.go
Normal file
@@ -0,0 +1,153 @@
|
||||
// email/idle.go – IMAP IDLE Watcher für Echtzeit-Email-Benachrichtigungen
|
||||
package email
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
imap "github.com/emersion/go-imap/v2"
|
||||
"github.com/emersion/go-imap/v2/imapclient"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
|
||||
// IdleWatcher überwacht einen IMAP-Account per IDLE auf neue Nachrichten.
|
||||
type IdleWatcher struct {
|
||||
acc config.EmailAccount
|
||||
onNew func(accountName, summary string)
|
||||
fetching atomic.Bool
|
||||
}
|
||||
|
||||
// NewIdleWatcher erstellt einen IdleWatcher für einen einzelnen Account.
|
||||
// onNew wird aufgerufen wenn neue Emails gefunden wurden (mit Account-Name und Zusammenfassung).
|
||||
func NewIdleWatcher(acc config.EmailAccount, onNew func(accountName, summary string)) *IdleWatcher {
|
||||
return &IdleWatcher{acc: acc, onNew: onNew}
|
||||
}
|
||||
|
||||
// Run startet die IDLE-Schleife. Blockiert bis ctx abgebrochen wird.
|
||||
func (w *IdleWatcher) Run(ctx context.Context) {
|
||||
for {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
slog.Info("IDLE: Verbinde", "account", accountLabel(w.acc), "host", w.acc.Host)
|
||||
if err := w.runOnce(ctx); err != nil {
|
||||
slog.Warn("IDLE: Fehler, Neuverbindung in 60s", "account", accountLabel(w.acc), "fehler", err)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(60 * time.Second):
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *IdleWatcher) runOnce(ctx context.Context) error {
|
||||
// numMsgs wird atomar geschrieben/gelesen: UnilateralDataHandler läuft in einem
|
||||
// separaten Goroutine (imapclient-intern), IDLE-Loop liest im Hauptgoroutine.
|
||||
var numMsgs atomic.Uint32
|
||||
hasNew := make(chan struct{}, 1)
|
||||
|
||||
addr := fmt.Sprintf("%s:%d", w.acc.Host, w.acc.Port)
|
||||
options := &imapclient.Options{
|
||||
UnilateralDataHandler: &imapclient.UnilateralDataHandler{
|
||||
Mailbox: func(data *imapclient.UnilateralDataMailbox) {
|
||||
if data.NumMessages != nil && *data.NumMessages > numMsgs.Load() {
|
||||
numMsgs.Store(*data.NumMessages)
|
||||
select {
|
||||
case hasNew <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
var (
|
||||
c *imapclient.Client
|
||||
err error
|
||||
)
|
||||
switch {
|
||||
case w.acc.TLS:
|
||||
tlsCfg := &tls.Config{ServerName: w.acc.Host}
|
||||
options.TLSConfig = tlsCfg
|
||||
c, err = imapclient.DialTLS(addr, options)
|
||||
case w.acc.StartTLS:
|
||||
tlsCfg := &tls.Config{ServerName: w.acc.Host}
|
||||
options.TLSConfig = tlsCfg
|
||||
c, err = imapclient.DialStartTLS(addr, options)
|
||||
default:
|
||||
c, err = imapclient.DialInsecure(addr, options)
|
||||
}
|
||||
if err != nil {
|
||||
return fmt.Errorf("verbinden: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
c.Logout().Wait()
|
||||
c.Close()
|
||||
}()
|
||||
|
||||
if err := c.Login(w.acc.User, w.acc.Password).Wait(); err != nil {
|
||||
return fmt.Errorf("login: %w", err)
|
||||
}
|
||||
|
||||
folder := w.acc.Folder
|
||||
if folder == "" {
|
||||
folder = "INBOX"
|
||||
}
|
||||
selectData, err := c.Select(folder, &imap.SelectOptions{ReadOnly: true}).Wait()
|
||||
if err != nil {
|
||||
return fmt.Errorf("select: %w", err)
|
||||
}
|
||||
numMsgs.Store(selectData.NumMessages)
|
||||
slog.Info("IDLE: Aktiv", "account", accountLabel(w.acc), "folder", folder, "numMsgs", selectData.NumMessages)
|
||||
|
||||
for {
|
||||
if ctx.Err() != nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
idleCmd, err := c.Idle()
|
||||
if err != nil {
|
||||
return fmt.Errorf("IDLE starten: %w", err)
|
||||
}
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
idleCmd.Close()
|
||||
idleCmd.Wait()
|
||||
return nil
|
||||
|
||||
case <-hasNew:
|
||||
idleCmd.Close()
|
||||
if err := idleCmd.Wait(); err != nil {
|
||||
slog.Warn("IDLE Wait Fehler", "account", accountLabel(w.acc), "fehler", err)
|
||||
}
|
||||
slog.Info("IDLE: Neue Email erkannt", "account", accountLabel(w.acc))
|
||||
// Nur einen gleichzeitigen Fetch erlauben
|
||||
if !w.fetching.Swap(true) {
|
||||
go func() {
|
||||
defer w.fetching.Store(false)
|
||||
w.notifyNewEmail()
|
||||
}()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *IdleWatcher) notifyNewEmail() {
|
||||
summary, err := SummarizeUnreadAccount(w.acc)
|
||||
if err != nil {
|
||||
slog.Error("IDLE: Email-Zusammenfassung fehlgeschlagen", "account", accountLabel(w.acc), "fehler", err)
|
||||
return
|
||||
}
|
||||
if summary == "📭 Keine ungelesenen Emails." {
|
||||
return
|
||||
}
|
||||
w.onNew(accountLabel(w.acc), summary)
|
||||
}
|
||||
710
internal/agents/tool/email/summary.go
Normal file
710
internal/agents/tool/email/summary.go
Normal file
@@ -0,0 +1,710 @@
|
||||
// email/summary.go – LLM-Zusammenfassung von Emails via LocalAI
|
||||
package email
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
openai "github.com/sashabaranov/go-openai"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
"my-brain-importer/internal/triage"
|
||||
)
|
||||
|
||||
// Summarize verbindet sich mit IMAP, holt die letzten 20 Emails und fasst sie per LLM zusammen.
|
||||
func Summarize() (string, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
if len(accounts) == 0 {
|
||||
return "", fmt.Errorf("Kein Email-Account konfiguriert")
|
||||
}
|
||||
if len(accounts) == 1 {
|
||||
return fetchAndSummarizeAccount(accounts[0], 20, "Fasse diese Emails kurz zusammen und hebe wichtige oder dringende hervor.")
|
||||
}
|
||||
var parts []string
|
||||
for _, acc := range accounts {
|
||||
result, err := fetchAndSummarizeAccount(acc, 20, "Fasse diese Emails kurz zusammen und hebe wichtige oder dringende hervor.")
|
||||
if err != nil {
|
||||
slog.Error("Email-Fehler", "account", acc.Name, "fehler", err)
|
||||
parts = append(parts, fmt.Sprintf("❌ **%s:** %v", accountLabel(acc), err))
|
||||
continue
|
||||
}
|
||||
parts = append(parts, fmt.Sprintf("**%s:**\n%s", accountLabel(acc), result))
|
||||
}
|
||||
return strings.Join(parts, "\n\n"), nil
|
||||
}
|
||||
|
||||
// SummarizeUnread fasst ungelesene Emails für alle konfigurierten Accounts zusammen.
|
||||
// Wenn email.processed_folder konfiguriert ist, werden Emails danach dorthin verschoben.
|
||||
func SummarizeUnread() (string, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
if len(accounts) == 0 {
|
||||
return "", fmt.Errorf("Kein Email-Account konfiguriert")
|
||||
}
|
||||
if len(accounts) == 1 {
|
||||
return SummarizeUnreadAccount(accounts[0])
|
||||
}
|
||||
|
||||
var parts []string
|
||||
allEmpty := true
|
||||
for _, acc := range accounts {
|
||||
result, err := SummarizeUnreadAccount(acc)
|
||||
if err != nil {
|
||||
slog.Error("Email-Fehler", "account", acc.Name, "fehler", err)
|
||||
parts = append(parts, fmt.Sprintf("❌ **%s:** %v", accountLabel(acc), err))
|
||||
continue
|
||||
}
|
||||
if result != "📭 Keine ungelesenen Emails." {
|
||||
allEmpty = false
|
||||
parts = append(parts, fmt.Sprintf("**%s:**\n%s", accountLabel(acc), result))
|
||||
}
|
||||
}
|
||||
if allEmpty {
|
||||
return "📭 Keine ungelesenen Emails.", nil
|
||||
}
|
||||
return strings.Join(parts, "\n\n"), nil
|
||||
}
|
||||
|
||||
// SummarizeUnreadAccount fasst ungelesene Emails für einen bestimmten Account zusammen.
|
||||
// Wenn triage_folder konfiguriert ist, werden unwichtige Emails vorher aussortiert.
|
||||
func SummarizeUnreadAccount(acc config.EmailAccount) (string, error) {
|
||||
// Phase 1: Triage – Emails sortieren (eigene Verbindung)
|
||||
if acc.TriageUnimportantFolder != "" || acc.TriageImportantFolder != "" {
|
||||
if err := triageUnread(acc); err != nil {
|
||||
slog.Warn("[Triage] fehlgeschlagen, übersprungen", "account", accountLabel(acc), "fehler", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 2: Zusammenfassung der verbleibenden wichtigen Emails (frische Verbindung)
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("Email-Verbindung (%s): %w", accountLabel(acc), err)
|
||||
}
|
||||
defer cl.Close()
|
||||
|
||||
var msgs []Message
|
||||
var seqNums []uint32
|
||||
|
||||
if acc.ProcessedFolder != "" {
|
||||
msgs, seqNums, err = cl.FetchUnreadSeqNums()
|
||||
} else {
|
||||
msgs, err = cl.FetchUnread()
|
||||
}
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("Emails abrufen (%s): %w", accountLabel(acc), err)
|
||||
}
|
||||
if len(msgs) == 0 {
|
||||
return "📭 Keine ungelesenen Emails.", nil
|
||||
}
|
||||
|
||||
slog.Info("Email-Zusammenfassung gestartet", "account", accountLabel(acc), "anzahl", len(msgs), "typ", "unread")
|
||||
result, err := summarizeWithLLMModel(msgs, "Fasse diese ungelesenen Emails zusammen. Hebe wichtige, dringende oder actionable Emails hervor.", accountModel(acc))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// Nach dem Zusammenfassen: Emails in Processed-Ordner verschieben
|
||||
if acc.ProcessedFolder != "" && len(seqNums) > 0 {
|
||||
if moveErr := cl.MoveMessages(seqNums, acc.ProcessedFolder); moveErr != nil {
|
||||
slog.Warn("Emails konnten nicht verschoben werden", "fehler", moveErr, "ordner", acc.ProcessedFolder)
|
||||
} else {
|
||||
slog.Info("Emails verschoben", "anzahl", len(seqNums), "ordner", acc.ProcessedFolder)
|
||||
}
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// TriageRecentAllAccounts klassifiziert die letzten n Emails aller Accounts manuell
|
||||
// und verschiebt sie in die konfigurierten Triage-Ordner.
|
||||
func TriageRecentAllAccounts(n uint32) (string, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
if len(accounts) == 0 {
|
||||
return "", fmt.Errorf("kein Email-Account konfiguriert")
|
||||
}
|
||||
|
||||
var lines []string
|
||||
for _, acc := range accounts {
|
||||
if acc.TriageImportantFolder == "" && acc.TriageUnimportantFolder == "" {
|
||||
lines = append(lines, fmt.Sprintf("⚠️ **%s:** kein triage_important_folder / triage_unimportant_folder konfiguriert", accountLabel(acc)))
|
||||
continue
|
||||
}
|
||||
wichtig, unwichtig, err := triageRecentAccount(acc, n)
|
||||
if err != nil {
|
||||
lines = append(lines, fmt.Sprintf("❌ **%s:** %v", accountLabel(acc), err))
|
||||
continue
|
||||
}
|
||||
lines = append(lines, fmt.Sprintf("✅ **%s:** %d wichtig → `%s`, %d unwichtig → `%s`",
|
||||
accountLabel(acc), wichtig, acc.TriageImportantFolder, unwichtig, acc.TriageUnimportantFolder))
|
||||
}
|
||||
return strings.Join(lines, "\n"), nil
|
||||
}
|
||||
|
||||
// triageRecentAccount klassifiziert die letzten n Emails eines Accounts.
|
||||
// Gibt Anzahl wichtiger und unwichtiger Emails zurück.
|
||||
func triageRecentAccount(acc config.EmailAccount, n uint32) (wichtig, unwichtig int, err error) {
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("verbinden: %w", err)
|
||||
}
|
||||
defer cl.Close()
|
||||
|
||||
// Ordner vorab anlegen, unabhängig davon ob Emails verschoben werden
|
||||
if acc.TriageImportantFolder != "" {
|
||||
if err := cl.EnsureFolder(acc.TriageImportantFolder); err != nil {
|
||||
slog.Warn("[Triage] Ordner anlegen fehlgeschlagen", "ordner", acc.TriageImportantFolder, "fehler", err)
|
||||
}
|
||||
}
|
||||
if acc.TriageUnimportantFolder != "" {
|
||||
if err := cl.EnsureFolder(acc.TriageUnimportantFolder); err != nil {
|
||||
slog.Warn("[Triage] Ordner anlegen fehlgeschlagen", "ordner", acc.TriageUnimportantFolder, "fehler", err)
|
||||
}
|
||||
}
|
||||
|
||||
msgs, err := cl.FetchRecentForSelect(n)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("fetch: %w", err)
|
||||
}
|
||||
if len(msgs) == 0 {
|
||||
return 0, 0, nil
|
||||
}
|
||||
|
||||
model := accountModel(acc)
|
||||
var wichtigSeqNums, unwichtigSeqNums []uint32
|
||||
|
||||
slog.Info("[Triage] Manuell gestartet", "account", accountLabel(acc), "anzahl", len(msgs))
|
||||
for _, msg := range msgs {
|
||||
if ClassifyImportance(msg.Message, model) {
|
||||
wichtigSeqNums = append(wichtigSeqNums, msg.SeqNum)
|
||||
} else {
|
||||
unwichtigSeqNums = append(unwichtigSeqNums, msg.SeqNum)
|
||||
}
|
||||
}
|
||||
|
||||
if acc.TriageUnimportantFolder != "" && len(unwichtigSeqNums) > 0 {
|
||||
if ensureErr := cl.EnsureFolder(acc.TriageUnimportantFolder); ensureErr != nil {
|
||||
slog.Warn("[Triage] Ordner anlegen fehlgeschlagen", "ordner", acc.TriageUnimportantFolder, "fehler", ensureErr)
|
||||
}
|
||||
if moveErr := cl.MoveMessages(unwichtigSeqNums, acc.TriageUnimportantFolder); moveErr != nil {
|
||||
slog.Warn("[Triage] Verschieben unwichtig fehlgeschlagen", "fehler", moveErr)
|
||||
}
|
||||
}
|
||||
|
||||
if acc.TriageImportantFolder != "" && len(wichtigSeqNums) > 0 {
|
||||
if ensureErr := cl.EnsureFolder(acc.TriageImportantFolder); ensureErr != nil {
|
||||
slog.Warn("[Triage] Ordner anlegen fehlgeschlagen", "ordner", acc.TriageImportantFolder, "fehler", ensureErr)
|
||||
}
|
||||
if moveErr := cl.MoveMessages(wichtigSeqNums, acc.TriageImportantFolder); moveErr != nil {
|
||||
slog.Warn("[Triage] Verschieben wichtig fehlgeschlagen", "fehler", moveErr)
|
||||
}
|
||||
}
|
||||
|
||||
return len(wichtigSeqNums), len(unwichtigSeqNums), nil
|
||||
}
|
||||
|
||||
// triageUnread klassifiziert alle ungelesenen Emails eines Accounts und verschiebt
|
||||
// wichtige in TriageImportantFolder und unwichtige in TriageUnimportantFolder.
|
||||
// Läuft sequentiell: eine Email nach der anderen.
|
||||
func triageUnread(acc config.EmailAccount) error {
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return fmt.Errorf("verbinden: %w", err)
|
||||
}
|
||||
defer cl.Close()
|
||||
|
||||
// Ordner vorab anlegen
|
||||
if acc.TriageImportantFolder != "" {
|
||||
cl.EnsureFolder(acc.TriageImportantFolder)
|
||||
}
|
||||
if acc.TriageUnimportantFolder != "" {
|
||||
cl.EnsureFolder(acc.TriageUnimportantFolder)
|
||||
}
|
||||
|
||||
msgs, seqNums, err := cl.FetchUnreadSeqNums()
|
||||
if err != nil {
|
||||
return fmt.Errorf("fetch: %w", err)
|
||||
}
|
||||
if len(msgs) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
model := accountModel(acc)
|
||||
var wichtigSeqNums, unwichtigSeqNums []uint32
|
||||
|
||||
slog.Info("[Triage] Klassifizierung gestartet", "account", accountLabel(acc), "anzahl", len(msgs))
|
||||
for i, msg := range msgs {
|
||||
if ClassifyImportance(msg, model) {
|
||||
wichtigSeqNums = append(wichtigSeqNums, seqNums[i])
|
||||
} else {
|
||||
unwichtigSeqNums = append(unwichtigSeqNums, seqNums[i])
|
||||
}
|
||||
}
|
||||
|
||||
if acc.TriageUnimportantFolder != "" && len(unwichtigSeqNums) > 0 {
|
||||
if err := cl.EnsureFolder(acc.TriageUnimportantFolder); err != nil {
|
||||
slog.Warn("[Triage] Ordner konnte nicht angelegt werden", "ordner", acc.TriageUnimportantFolder, "fehler", err)
|
||||
}
|
||||
if err := cl.MoveMessages(unwichtigSeqNums, acc.TriageUnimportantFolder); err != nil {
|
||||
slog.Warn("[Triage] Verschieben unwichtig fehlgeschlagen", "ordner", acc.TriageUnimportantFolder, "fehler", err)
|
||||
} else {
|
||||
slog.Info("[Triage] Unwichtige Emails verschoben", "anzahl", len(unwichtigSeqNums), "ordner", acc.TriageUnimportantFolder)
|
||||
}
|
||||
}
|
||||
|
||||
if acc.TriageImportantFolder != "" && len(wichtigSeqNums) > 0 {
|
||||
if err := cl.EnsureFolder(acc.TriageImportantFolder); err != nil {
|
||||
slog.Warn("[Triage] Ordner konnte nicht angelegt werden", "ordner", acc.TriageImportantFolder, "fehler", err)
|
||||
}
|
||||
if err := cl.MoveMessages(wichtigSeqNums, acc.TriageImportantFolder); err != nil {
|
||||
slog.Warn("[Triage] Verschieben wichtig fehlgeschlagen", "ordner", acc.TriageImportantFolder, "fehler", err)
|
||||
} else {
|
||||
slog.Info("[Triage] Wichtige Emails verschoben", "anzahl", len(wichtigSeqNums), "ordner", acc.TriageImportantFolder)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ClassifyImportance klassifiziert eine einzelne Email als wichtig (true) oder unwichtig (false).
|
||||
// Sucht zuerst ähnliche vergangene Entscheidungen in Qdrant (RAG) und gibt sie als Kontext mit.
|
||||
// Im Fehlerfall oder bei unklarer Antwort wird true (wichtig) zurückgegeben – sicherer Default.
|
||||
func ClassifyImportance(msg Message, model string) bool {
|
||||
// RAG: ähnliche vergangene Triage-Entscheidungen als Few-Shot-Beispiele
|
||||
ragQuery := fmt.Sprintf("Von: %s Betreff: %s", msg.From, msg.Subject)
|
||||
examples := triage.SearchSimilar(ragQuery)
|
||||
|
||||
var examplesText string
|
||||
if len(examples) > 0 {
|
||||
var sb strings.Builder
|
||||
sb.WriteString("Ähnliche Entscheidungen aus der Vergangenheit:\n")
|
||||
for _, ex := range examples {
|
||||
sb.WriteString("- ")
|
||||
sb.WriteString(ex.Text)
|
||||
sb.WriteString("\n")
|
||||
}
|
||||
sb.WriteString("\n")
|
||||
examplesText = sb.String()
|
||||
}
|
||||
|
||||
prompt := fmt.Sprintf("%sVon: %s\nBetreff: %s\n\nIst diese Email wichtig? Antworte NUR mit einem einzigen Wort: wichtig oder unwichtig.",
|
||||
examplesText, msg.From, msg.Subject)
|
||||
|
||||
chatClient := config.NewChatClient()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
resp, err := chatClient.CreateChatCompletion(ctx, openai.ChatCompletionRequest{
|
||||
Model: model,
|
||||
Messages: []openai.ChatCompletionMessage{
|
||||
{Role: openai.ChatMessageRoleSystem, Content: "Du bist ein Email-Filter. Antworte immer nur mit einem einzigen Wort: wichtig oder unwichtig."},
|
||||
{Role: openai.ChatMessageRoleUser, Content: prompt},
|
||||
},
|
||||
Temperature: 0.1,
|
||||
MaxTokens: 300,
|
||||
})
|
||||
if err != nil {
|
||||
slog.Warn("[Triage] LLM-Fehler, Email als wichtig eingestuft", "betreff", msg.Subject, "fehler", err)
|
||||
return true
|
||||
}
|
||||
if len(resp.Choices) == 0 {
|
||||
return true
|
||||
}
|
||||
|
||||
raw := resp.Choices[0].Message.Content
|
||||
// Reasoning-Modelle (z.B. Qwen3) geben Antwort nach </think>-Tag aus
|
||||
if idx := strings.LastIndex(raw, "</think>"); idx >= 0 {
|
||||
raw = raw[idx+len("</think>"):]
|
||||
}
|
||||
answer := strings.ToLower(strings.TrimSpace(raw))
|
||||
isImportant := !strings.Contains(answer, "unwichtig")
|
||||
slog.Info("[Triage] Email klassifiziert",
|
||||
"betreff", msg.Subject,
|
||||
"von", msg.From,
|
||||
"wichtig", isImportant,
|
||||
"rag_beispiele", len(examples),
|
||||
"antwort", answer,
|
||||
)
|
||||
|
||||
// Entscheidung für künftiges Lernen in Qdrant speichern
|
||||
// Body ist bei ClassifyImportance nicht verfügbar (nur Envelope), daher leer.
|
||||
if err := triage.StoreDecision(msg.Subject, msg.From, msg.Date, "", isImportant); err != nil {
|
||||
slog.Warn("[Triage] Entscheidung nicht gespeichert", "fehler", err)
|
||||
}
|
||||
|
||||
return isImportant
|
||||
}
|
||||
|
||||
// ExtractReminders sucht in den letzten Emails nach Terminen/Deadlines.
|
||||
func ExtractReminders() (string, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
if len(accounts) == 0 {
|
||||
return "", fmt.Errorf("Kein Email-Account konfiguriert")
|
||||
}
|
||||
if len(accounts) == 1 {
|
||||
return fetchAndSummarizeAccount(accounts[0], 30, "Extrahiere alle Termine, Deadlines, Erinnerungen und wichtigen Daten aus diesen Emails. Liste sie strukturiert auf.")
|
||||
}
|
||||
var parts []string
|
||||
for _, acc := range accounts {
|
||||
result, err := fetchAndSummarizeAccount(acc, 30, "Extrahiere alle Termine, Deadlines, Erinnerungen und wichtigen Daten aus diesen Emails. Liste sie strukturiert auf.")
|
||||
if err != nil {
|
||||
slog.Error("Email-Fehler", "account", acc.Name, "fehler", err)
|
||||
parts = append(parts, fmt.Sprintf("❌ **%s:** %v", accountLabel(acc), err))
|
||||
continue
|
||||
}
|
||||
parts = append(parts, fmt.Sprintf("**%s:**\n%s", accountLabel(acc), result))
|
||||
}
|
||||
return strings.Join(parts, "\n\n"), nil
|
||||
}
|
||||
|
||||
// MoveUnread verschiebt alle ungelesenen Emails eines Accounts in den Zielordner.
|
||||
// Gibt die Anzahl verschobener Emails zurück.
|
||||
func MoveUnread(acc config.EmailAccount, destFolder string) (int, error) {
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("Email-Verbindung (%s): %w", accountLabel(acc), err)
|
||||
}
|
||||
defer cl.Close()
|
||||
|
||||
_, seqNums, err := cl.FetchUnreadSeqNums()
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("Emails abrufen: %w", err)
|
||||
}
|
||||
if len(seqNums) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
if err := cl.MoveMessages(seqNums, destFolder); err != nil {
|
||||
return 0, fmt.Errorf("Verschieben nach %s: %w", destFolder, err)
|
||||
}
|
||||
return len(seqNums), nil
|
||||
}
|
||||
|
||||
// AccountSelectMessages enthält ungelesene Emails eines Accounts für die Discord-Auswahl.
|
||||
type AccountSelectMessages struct {
|
||||
Account config.EmailAccount
|
||||
AccIndex int
|
||||
Messages []SelectMessage
|
||||
}
|
||||
|
||||
// FetchUnreadForSelectAllAccounts holt ungelesene Emails aller Accounts für die Discord-Auswahl.
|
||||
func FetchUnreadForSelectAllAccounts() ([]AccountSelectMessages, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
var result []AccountSelectMessages
|
||||
for i, acc := range accounts {
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Verbindung %s: %w", accountLabel(acc), err)
|
||||
}
|
||||
msgs, err := cl.FetchUnreadForSelect()
|
||||
cl.Close()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Emails %s: %w", accountLabel(acc), err)
|
||||
}
|
||||
result = append(result, AccountSelectMessages{
|
||||
Account: acc,
|
||||
AccIndex: i,
|
||||
Messages: msgs,
|
||||
})
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// FetchRecentForSelectAllAccounts holt die letzten n Emails aller Accounts für die Discord-Auswahl.
|
||||
func FetchRecentForSelectAllAccounts(n uint32) ([]AccountSelectMessages, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
var result []AccountSelectMessages
|
||||
for i, acc := range accounts {
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Verbindung %s: %w", accountLabel(acc), err)
|
||||
}
|
||||
msgs, err := cl.FetchRecentForSelect(n)
|
||||
cl.Close()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Emails %s: %w", accountLabel(acc), err)
|
||||
}
|
||||
result = append(result, AccountSelectMessages{
|
||||
Account: acc,
|
||||
AccIndex: i,
|
||||
Messages: msgs,
|
||||
})
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// MoveOldEmailsAllAccounts verschiebt alle Emails aller Accounts, die älter als olderThanDays Tage sind, nach destFolder.
|
||||
// Gibt die Gesamtanzahl verschobener Emails zurück.
|
||||
func MoveOldEmailsAllAccounts(destFolder string, olderThanDays int) (int, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
total := 0
|
||||
for _, acc := range accounts {
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return total, fmt.Errorf("Verbindung %s: %w", accountLabel(acc), err)
|
||||
}
|
||||
n, err := cl.MoveOldMessages(acc.Folder, destFolder, olderThanDays)
|
||||
cl.Close()
|
||||
if err != nil {
|
||||
return total, fmt.Errorf("Verschieben %s: %w", accountLabel(acc), err)
|
||||
}
|
||||
total += n
|
||||
}
|
||||
return total, nil
|
||||
}
|
||||
|
||||
// MoveSpecificUnread verschiebt spezifische Emails (per Sequenznummer) eines Accounts in den Zielordner.
|
||||
func MoveSpecificUnread(accIndex int, seqNums []uint32, destFolder string) (int, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
if accIndex < 0 || accIndex >= len(accounts) {
|
||||
return 0, fmt.Errorf("ungültiger Account-Index %d", accIndex)
|
||||
}
|
||||
acc := accounts[accIndex]
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("Email-Verbindung (%s): %w", accountLabel(acc), err)
|
||||
}
|
||||
defer cl.Close()
|
||||
if err := cl.MoveSpecificMessages(seqNums, destFolder); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return len(seqNums), nil
|
||||
}
|
||||
|
||||
// CleanupArchiveFolders löscht abgelaufene Emails aus allen konfigurierten Archivordnern.
|
||||
// Gibt eine menschenlesbare Zusammenfassung zurück.
|
||||
func CleanupArchiveFolders() (string, error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
var lines []string
|
||||
var errs []string
|
||||
total := 0
|
||||
|
||||
for _, acc := range accounts {
|
||||
for _, af := range acc.ArchiveFolders {
|
||||
if af.RetentionDays <= 0 {
|
||||
continue
|
||||
}
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
errs = append(errs, fmt.Sprintf("%s/%s: %v", accountLabel(acc), af.Name, err))
|
||||
continue
|
||||
}
|
||||
n, err := cl.CleanupOldEmails(af.IMAPFolder, af.RetentionDays)
|
||||
cl.Close()
|
||||
if err != nil {
|
||||
errs = append(errs, fmt.Sprintf("%s/%s: %v", accountLabel(acc), af.Name, err))
|
||||
continue
|
||||
}
|
||||
if n > 0 {
|
||||
lines = append(lines, fmt.Sprintf("🗑️ %s/%s: %d Email(s) gelöscht (älter als %d Tage)", accountLabel(acc), af.Name, n, af.RetentionDays))
|
||||
total += n
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var result string
|
||||
if len(lines) == 0 && len(errs) == 0 {
|
||||
result = "Kein Aufräumen notwendig."
|
||||
} else {
|
||||
result = strings.Join(lines, "\n")
|
||||
}
|
||||
|
||||
var combinedErr error
|
||||
if len(errs) > 0 {
|
||||
combinedErr = fmt.Errorf("%s", strings.Join(errs, "; "))
|
||||
}
|
||||
slog.Info("Archiv-Aufräumen abgeschlossen", "gelöscht", total, "fehler", len(errs))
|
||||
return result, combinedErr
|
||||
}
|
||||
|
||||
// SummarizeMessages fasst eine übergebene Liste von Nachrichten zusammen (für Tests ohne IMAP).
|
||||
func SummarizeMessages(msgs []Message, instruction string) (string, error) {
|
||||
return summarizeWithLLM(msgs, instruction)
|
||||
}
|
||||
|
||||
func fetchAndSummarizeAccount(acc config.EmailAccount, n uint32, instruction string) (string, error) {
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("Email-Verbindung: %w", err)
|
||||
}
|
||||
defer cl.Close()
|
||||
|
||||
msgs, err := cl.FetchRecent(n)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("Emails abrufen: %w", err)
|
||||
}
|
||||
if len(msgs) == 0 {
|
||||
return "📭 Keine Emails gefunden.", nil
|
||||
}
|
||||
|
||||
slog.Info("Email-Zusammenfassung gestartet", "account", accountLabel(acc), "anzahl", len(msgs))
|
||||
return summarizeWithLLMModel(msgs, instruction, accountModel(acc))
|
||||
}
|
||||
|
||||
// accountLabel gibt einen lesbaren Namen für einen Account zurück.
|
||||
func accountLabel(acc config.EmailAccount) string {
|
||||
if acc.Name != "" {
|
||||
return acc.Name
|
||||
}
|
||||
return acc.User
|
||||
}
|
||||
|
||||
// accountModel gibt das konfigurierte LLM-Modell für einen Account zurück.
|
||||
func accountModel(acc config.EmailAccount) string {
|
||||
if acc.Model != "" {
|
||||
return acc.Model
|
||||
}
|
||||
return config.Cfg.Chat.Model
|
||||
}
|
||||
|
||||
// emailModel gibt das konfigurierte Modell für den Legacy-Account zurück.
|
||||
func emailModel() string {
|
||||
if config.Cfg.Email.Model != "" {
|
||||
return config.Cfg.Email.Model
|
||||
}
|
||||
return config.Cfg.Chat.Model
|
||||
}
|
||||
|
||||
// formatEmailList formatiert Emails als lesbaren Text (Fallback und Eingabe fürs LLM).
|
||||
func formatEmailList(msgs []Message) string {
|
||||
var sb strings.Builder
|
||||
for i, m := range msgs {
|
||||
fmt.Fprintf(&sb, "[%d] Von: %s | Datum: %s | Betreff: %s\n", i+1, m.From, m.Date, m.Subject)
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
func summarizeWithLLM(msgs []Message, instruction string) (string, error) {
|
||||
return summarizeWithLLMModel(msgs, instruction, emailModel())
|
||||
}
|
||||
|
||||
func summarizeWithLLMModel(msgs []Message, instruction, model string) (string, error) {
|
||||
emailList := formatEmailList(msgs)
|
||||
|
||||
chatClient := config.NewChatClient()
|
||||
ctx := context.Background()
|
||||
|
||||
systemPrompt := `Du bist ein hilfreicher persönlicher Assistent. Analysiere Email-Listen und antworte auf Deutsch, präzise und strukturiert.`
|
||||
userPrompt := fmt.Sprintf("%s\n\nEmail-Liste:\n%s", instruction, emailList)
|
||||
|
||||
slog.Debug("[LLM] Email Prompt",
|
||||
"model", model,
|
||||
"emails", len(msgs),
|
||||
"system", systemPrompt,
|
||||
"user", userPrompt,
|
||||
)
|
||||
|
||||
start := time.Now()
|
||||
stream, err := chatClient.CreateChatCompletionStream(ctx, openai.ChatCompletionRequest{
|
||||
Model: model,
|
||||
Messages: []openai.ChatCompletionMessage{
|
||||
{Role: openai.ChatMessageRoleSystem, Content: systemPrompt},
|
||||
{Role: openai.ChatMessageRoleUser, Content: userPrompt},
|
||||
},
|
||||
Temperature: 0.5,
|
||||
MaxTokens: 600,
|
||||
})
|
||||
if err != nil {
|
||||
slog.Warn("[LLM] nicht erreichbar, Fallback-Liste", "fehler", err)
|
||||
return fallbackList(msgs), nil
|
||||
}
|
||||
defer stream.Close()
|
||||
|
||||
var answer strings.Builder
|
||||
for {
|
||||
resp, err := stream.Recv()
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
if len(resp.Choices) > 0 {
|
||||
answer.WriteString(resp.Choices[0].Delta.Content)
|
||||
}
|
||||
}
|
||||
|
||||
result := answer.String()
|
||||
slog.Debug("[LLM] Email Antwort",
|
||||
"dauer", time.Since(start).Round(time.Millisecond),
|
||||
"zeichen", len(result),
|
||||
"antwort", result,
|
||||
)
|
||||
|
||||
if strings.TrimSpace(result) == "" {
|
||||
slog.Warn("[LLM] leere Antwort, Fallback-Liste")
|
||||
return fallbackList(msgs), nil
|
||||
}
|
||||
|
||||
slog.Info("[LLM] Email-Zusammenfassung abgeschlossen", "dauer", time.Since(start).Round(time.Millisecond), "zeichen", len(result))
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// fallbackList gibt eine einfache formatierte Liste zurück wenn das LLM nicht verfügbar ist.
|
||||
func fallbackList(msgs []Message) string {
|
||||
var sb strings.Builder
|
||||
sb.WriteString("⚠️ *LLM nicht verfügbar – ungefilterte Email-Liste:*\n\n")
|
||||
for i, m := range msgs {
|
||||
fmt.Fprintf(&sb, "**[%d]** %s\n📤 %s\n📌 %s\n\n", i+1, m.Date, m.From, m.Subject)
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
// LearnFromFolders scannt die Archiv-Ordner eines Accounts und speichert Triage-Entscheidungen in Qdrant.
|
||||
// Ordner mit retention_days == 0 (Archiv/dauerhaft) → wichtig, retention_days > 0 → unwichtig.
|
||||
// Pro Ordner werden maximal die letzten 50 Emails verarbeitet.
|
||||
func LearnFromFolders(acc config.EmailAccount) (wichtig, unwichtig int, err error) {
|
||||
if len(acc.ArchiveFolders) == 0 {
|
||||
return 0, 0, nil
|
||||
}
|
||||
|
||||
cl, err := ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("IMAP verbinden: %w", err)
|
||||
}
|
||||
defer cl.Close()
|
||||
|
||||
for _, af := range acc.ArchiveFolders {
|
||||
isImportant := af.RetentionDays == 0
|
||||
|
||||
msgs, err := cl.FetchWithBodyAndAttachments(af.IMAPFolder, 50)
|
||||
if err != nil {
|
||||
slog.Warn("[Triage-Learn] Ordner nicht lesbar", "ordner", af.IMAPFolder, "fehler", err)
|
||||
continue
|
||||
}
|
||||
|
||||
for _, m := range msgs {
|
||||
if err := triage.StoreDecision(m.Subject, m.From, m.Date, m.Body, isImportant); err != nil {
|
||||
slog.Warn("[Triage-Learn] Speichern fehlgeschlagen", "betreff", m.Subject, "fehler", err)
|
||||
continue
|
||||
}
|
||||
if isImportant {
|
||||
wichtig++
|
||||
} else {
|
||||
unwichtig++
|
||||
}
|
||||
}
|
||||
|
||||
slog.Info("[Triage-Learn] Ordner verarbeitet",
|
||||
"account", accountLabel(acc),
|
||||
"ordner", af.IMAPFolder,
|
||||
"emails", len(msgs),
|
||||
"wichtig", isImportant,
|
||||
)
|
||||
}
|
||||
|
||||
return wichtig, unwichtig, nil
|
||||
}
|
||||
|
||||
// LearnFromFoldersAllAccounts führt LearnFromFolders für alle konfigurierten Accounts aus.
|
||||
func LearnFromFoldersAllAccounts() (wichtig, unwichtig int, err error) {
|
||||
accounts := config.AllEmailAccounts()
|
||||
for _, acc := range accounts {
|
||||
w, u, accErr := LearnFromFolders(acc)
|
||||
if accErr != nil {
|
||||
slog.Error("[Triage-Learn] Account fehlgeschlagen", "account", accountLabel(acc), "fehler", accErr)
|
||||
continue
|
||||
}
|
||||
wichtig += w
|
||||
unwichtig += u
|
||||
}
|
||||
return wichtig, unwichtig, nil
|
||||
}
|
||||
117
internal/agents/tool/email/summary_test.go
Normal file
117
internal/agents/tool/email/summary_test.go
Normal file
@@ -0,0 +1,117 @@
|
||||
package email
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
var testMessages = []Message{
|
||||
{Subject: "Rechnung März", From: "buchhaltung@firma.de", Date: "2026-03-01 09:00"},
|
||||
{Subject: "Meeting Einladung", From: "chef@firma.de", Date: "2026-03-02 10:30"},
|
||||
{Subject: "Newsletter", From: "news@shop.de", Date: "2026-03-03 08:00"},
|
||||
}
|
||||
|
||||
func TestFormatEmailList_ContainsFields(t *testing.T) {
|
||||
result := formatEmailList(testMessages)
|
||||
|
||||
checks := []string{"Rechnung März", "buchhaltung@firma.de", "Meeting Einladung", "Newsletter"}
|
||||
for _, s := range checks {
|
||||
if !strings.Contains(result, s) {
|
||||
t.Errorf("formatEmailList: fehlt %q in:\n%s", s, result)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatEmailList_NumberedLines(t *testing.T) {
|
||||
result := formatEmailList(testMessages)
|
||||
if !strings.Contains(result, "[1]") {
|
||||
t.Error("fehlt [1] in Ausgabe")
|
||||
}
|
||||
if !strings.Contains(result, "[3]") {
|
||||
t.Error("fehlt [3] in Ausgabe")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatEmailList_Empty(t *testing.T) {
|
||||
result := formatEmailList([]Message{})
|
||||
if result != "" {
|
||||
t.Errorf("leere Liste sollte leeren String ergeben, got: %q", result)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFallbackList_ContainsWarning(t *testing.T) {
|
||||
result := fallbackList(testMessages)
|
||||
if !strings.Contains(result, "LLM nicht verfügbar") {
|
||||
t.Errorf("Fallback-Hinweis fehlt: %q", result)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFallbackList_ContainsAllMessages(t *testing.T) {
|
||||
result := fallbackList(testMessages)
|
||||
for _, m := range testMessages {
|
||||
if !strings.Contains(result, m.Subject) {
|
||||
t.Errorf("Betreff fehlt: %q", m.Subject)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseMessages_Empty(t *testing.T) {
|
||||
result := parseMessages(nil)
|
||||
if len(result) != 0 {
|
||||
t.Errorf("erwartet leer, got %d", len(result))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMessage_DateFormat(t *testing.T) {
|
||||
// Datum muss im Format "2006-01-02 15:04" formatiert werden
|
||||
m := testMessages[0]
|
||||
if _, err := time.Parse("2006-01-02 15:04", m.Date); err != nil {
|
||||
t.Errorf("Datumsformat ungültig: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAccountLabel_WithName(t *testing.T) {
|
||||
acc := config.EmailAccount{Name: "Privat", User: "user@example.de"}
|
||||
if got := accountLabel(acc); got != "Privat" {
|
||||
t.Errorf("accountLabel: erwartet %q, got %q", "Privat", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAccountLabel_FallsBackToUser(t *testing.T) {
|
||||
acc := config.EmailAccount{User: "user@example.de"}
|
||||
if got := accountLabel(acc); got != "user@example.de" {
|
||||
t.Errorf("accountLabel: erwartet %q, got %q", "user@example.de", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAccountModel_WithModel(t *testing.T) {
|
||||
acc := config.EmailAccount{Model: "custom-model"}
|
||||
if got := accountModel(acc); got != "custom-model" {
|
||||
t.Errorf("accountModel: erwartet %q, got %q", "custom-model", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAccountModel_FallsBackToChatModel(t *testing.T) {
|
||||
orig := config.Cfg
|
||||
defer func() { config.Cfg = orig }()
|
||||
config.Cfg.Chat.Model = "default-model"
|
||||
|
||||
acc := config.EmailAccount{} // kein Model gesetzt
|
||||
if got := accountModel(acc); got != "default-model" {
|
||||
t.Errorf("accountModel: erwartet chat.model %q, got %q", "default-model", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSummarizeUnread_NoAccountsConfigured(t *testing.T) {
|
||||
orig := config.Cfg
|
||||
defer func() { config.Cfg = orig }()
|
||||
config.Cfg = config.Config{} // leere Config, kein Email-Account
|
||||
|
||||
_, err := SummarizeUnread()
|
||||
if err == nil {
|
||||
t.Error("erwartet Fehler wenn kein Account konfiguriert")
|
||||
}
|
||||
}
|
||||
168
internal/agents/tool/rss/watcher.go
Normal file
168
internal/agents/tool/rss/watcher.go
Normal file
@@ -0,0 +1,168 @@
|
||||
// rss/watcher.go – Überwacht RSS-Feeds und importiert neue Artikel in Qdrant
|
||||
package rss
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/mmcdole/gofeed"
|
||||
|
||||
"my-brain-importer/internal/brain"
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// FeedResult fasst das Ergebnis eines Feed-Imports zusammen.
|
||||
type FeedResult struct {
|
||||
URL string
|
||||
Title string
|
||||
Imported int
|
||||
Err error
|
||||
}
|
||||
|
||||
// IngestFeed fetcht einen RSS-Feed und importiert neue Artikel in Qdrant.
|
||||
// Gibt Anzahl der importierten Artikel zurück.
|
||||
func IngestFeed(feedURL string) (int, string, error) {
|
||||
fp := gofeed.NewParser()
|
||||
fp.Client = gofeed.NewParser().Client // default HTTP client with timeout
|
||||
feed, err := fp.ParseURL(feedURL)
|
||||
if err != nil {
|
||||
return 0, "", fmt.Errorf("Feed-Parsing fehlgeschlagen: %w", err)
|
||||
}
|
||||
|
||||
feedTitle := feed.Title
|
||||
if feedTitle == "" {
|
||||
feedTitle = feedURL
|
||||
}
|
||||
|
||||
imported := 0
|
||||
for _, item := range feed.Items {
|
||||
text := buildArticleText(item)
|
||||
if len(strings.TrimSpace(text)) < 20 {
|
||||
continue
|
||||
}
|
||||
source := fmt.Sprintf("rss/%s", feedURL)
|
||||
if item.Link != "" {
|
||||
source = item.Link
|
||||
}
|
||||
if err := brain.IngestText(text, source, "rss"); err != nil {
|
||||
slog.Warn("RSS: Artikel konnte nicht importiert werden", "url", item.Link, "fehler", err)
|
||||
continue
|
||||
}
|
||||
imported++
|
||||
}
|
||||
return imported, feedTitle, nil
|
||||
}
|
||||
|
||||
// buildArticleText formatiert einen RSS-Artikel als importierbaren Text.
|
||||
func buildArticleText(item *gofeed.Item) string {
|
||||
var sb strings.Builder
|
||||
if item.Title != "" {
|
||||
fmt.Fprintf(&sb, "# %s\n\n", item.Title)
|
||||
}
|
||||
if item.Published != "" {
|
||||
fmt.Fprintf(&sb, "Veröffentlicht: %s\n", item.Published)
|
||||
}
|
||||
if item.Link != "" {
|
||||
fmt.Fprintf(&sb, "URL: %s\n\n", item.Link)
|
||||
}
|
||||
if item.Description != "" {
|
||||
sb.WriteString(strings.TrimSpace(item.Description))
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
// IngestAllFeeds importiert alle konfigurierten RSS-Feeds.
|
||||
// Gibt eine Zusammenfassung der Ergebnisse zurück.
|
||||
func IngestAllFeeds() []FeedResult {
|
||||
feeds := config.Cfg.RSSFeeds
|
||||
if len(feeds) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
results := make([]FeedResult, 0, len(feeds))
|
||||
for _, f := range feeds {
|
||||
n, title, err := IngestFeed(f.URL)
|
||||
results = append(results, FeedResult{
|
||||
URL: f.URL,
|
||||
Title: title,
|
||||
Imported: n,
|
||||
Err: err,
|
||||
})
|
||||
}
|
||||
return results
|
||||
}
|
||||
|
||||
// FormatResults gibt eine Discord-formatierte Zusammenfassung zurück.
|
||||
func FormatResults(results []FeedResult) string {
|
||||
if len(results) == 0 {
|
||||
return "📭 Keine RSS-Feeds konfiguriert."
|
||||
}
|
||||
var sb strings.Builder
|
||||
for _, r := range results {
|
||||
if r.Err != nil {
|
||||
fmt.Fprintf(&sb, "❌ **%s**: %v\n", r.URL, r.Err)
|
||||
} else {
|
||||
name := r.Title
|
||||
if name == "" {
|
||||
name = r.URL
|
||||
}
|
||||
fmt.Fprintf(&sb, "✅ **%s**: %d Artikel importiert\n", name, r.Imported)
|
||||
}
|
||||
}
|
||||
return strings.TrimSpace(sb.String())
|
||||
}
|
||||
|
||||
// Watcher überwacht alle konfigurierten RSS-Feeds in regelmäßigen Abständen.
|
||||
type Watcher struct {
|
||||
OnResults func(summary string)
|
||||
}
|
||||
|
||||
// Run startet die RSS-Überwachungsschleife. Blockiert bis ctx abgebrochen wird.
|
||||
func (w *Watcher) Run(ctx context.Context) {
|
||||
feeds := config.Cfg.RSSFeeds
|
||||
if len(feeds) == 0 {
|
||||
slog.Info("RSS-Watcher: Keine Feeds konfiguriert, beende")
|
||||
return
|
||||
}
|
||||
|
||||
// Ersten Durchlauf sofort starten
|
||||
w.runOnce()
|
||||
|
||||
// Dann Timer basierend auf minimalem Intervall
|
||||
minInterval := 24 * time.Hour
|
||||
for _, f := range feeds {
|
||||
h := f.IntervalHours
|
||||
if h <= 0 {
|
||||
h = 24
|
||||
}
|
||||
d := time.Duration(h) * time.Hour
|
||||
if d < minInterval {
|
||||
minInterval = d
|
||||
}
|
||||
}
|
||||
|
||||
ticker := time.NewTicker(minInterval)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
w.runOnce()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Watcher) runOnce() {
|
||||
results := IngestAllFeeds()
|
||||
if w.OnResults != nil && len(results) > 0 {
|
||||
summary := FormatResults(results)
|
||||
if summary != "" {
|
||||
w.OnResults(summary)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,12 +5,15 @@ import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
pb "github.com/qdrant/go-client/qdrant"
|
||||
openai "github.com/sashabaranov/go-openai"
|
||||
"google.golang.org/grpc/metadata"
|
||||
|
||||
"my-brain-importer/internal/agents"
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
@@ -23,7 +26,8 @@ type KnowledgeChunk struct {
|
||||
|
||||
// AskQuery sucht relevante Chunks und generiert eine LLM-Antwort.
|
||||
// Gibt die Antwort als String und die verwendeten Quellen zurück.
|
||||
func AskQuery(question string) (string, []KnowledgeChunk, error) {
|
||||
// history enthält vorherige Gesprächsnachrichten (optional, nil für stateless).
|
||||
func AskQuery(question string, history []agents.HistoryMessage) (string, []KnowledgeChunk, error) {
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
@@ -37,15 +41,20 @@ func AskQuery(question string) (string, []KnowledgeChunk, error) {
|
||||
|
||||
contextText := buildContext(chunks)
|
||||
|
||||
systemPrompt := `Du bist ein hilfreicher persönlicher Assistent.
|
||||
Deine Aufgabe ist es, Fragen basierend auf den bereitgestellten Informationen zu beantworten.
|
||||
coreMemory := LoadCoreMemory()
|
||||
systemPromptBase := `Du bist ein hilfreicher persönlicher Assistent.
|
||||
Beantworte Fragen primär anhand der bereitgestellten Informationen aus der Wissensdatenbank.
|
||||
Ergänze fehlende Details mit deinem eigenen Wissen, kennzeichne dies aber klar mit "Aus meinem Wissen:".
|
||||
|
||||
WICHTIGE REGELN:
|
||||
- Antworte nur basierend auf den bereitgestellten Informationen
|
||||
- Wenn die Informationen die Frage nicht beantworten, sage das ehrlich
|
||||
- Nutze die bereitgestellten Informationen als Hauptquelle
|
||||
- Ergänze mit eigenem Wissen wenn sinnvoll, kennzeichne es deutlich
|
||||
- Antworte auf Deutsch
|
||||
- Sei präzise und direkt
|
||||
- Erfinde keine Informationen hinzu`
|
||||
- Sei präzise und direkt`
|
||||
systemPrompt := systemPromptBase
|
||||
if coreMemory != "" {
|
||||
systemPrompt = systemPromptBase + "\n\n## Fakten über den Nutzer:\n" + coreMemory
|
||||
}
|
||||
|
||||
userPrompt := fmt.Sprintf(`Hier sind die relevanten Informationen aus meiner Wissensdatenbank:
|
||||
|
||||
@@ -54,12 +63,31 @@ WICHTIGE REGELN:
|
||||
Basierend auf diesen Informationen, beantworte bitte folgende Frage:
|
||||
%s`, contextText, question)
|
||||
|
||||
slog.Debug("[LLM] AskQuery Prompt",
|
||||
"model", config.Cfg.Chat.Model,
|
||||
"history_len", len(history),
|
||||
"system", systemPrompt,
|
||||
"user", userPrompt,
|
||||
)
|
||||
|
||||
msgs := []openai.ChatCompletionMessage{
|
||||
{Role: openai.ChatMessageRoleSystem, Content: systemPrompt},
|
||||
}
|
||||
for _, h := range history {
|
||||
msgs = append(msgs, openai.ChatCompletionMessage{
|
||||
Role: h.Role,
|
||||
Content: h.Content,
|
||||
})
|
||||
}
|
||||
msgs = append(msgs, openai.ChatCompletionMessage{
|
||||
Role: openai.ChatMessageRoleUser,
|
||||
Content: userPrompt,
|
||||
})
|
||||
|
||||
start := time.Now()
|
||||
stream, err := chatClient.CreateChatCompletionStream(ctx, openai.ChatCompletionRequest{
|
||||
Model: config.Cfg.Chat.Model,
|
||||
Messages: []openai.ChatCompletionMessage{
|
||||
{Role: openai.ChatMessageRoleSystem, Content: systemPrompt},
|
||||
{Role: openai.ChatMessageRoleUser, Content: userPrompt},
|
||||
},
|
||||
Model: config.Cfg.Chat.Model,
|
||||
Messages: msgs,
|
||||
Temperature: 0.7,
|
||||
MaxTokens: 500,
|
||||
})
|
||||
@@ -79,6 +107,12 @@ Basierend auf diesen Informationen, beantworte bitte folgende Frage:
|
||||
}
|
||||
}
|
||||
|
||||
slog.Debug("[LLM] AskQuery Antwort",
|
||||
"dauer", time.Since(start).Round(time.Millisecond),
|
||||
"zeichen", answer.Len(),
|
||||
"antwort", answer.String(),
|
||||
)
|
||||
|
||||
return answer.String(), chunks, nil
|
||||
}
|
||||
|
||||
@@ -87,7 +121,7 @@ func Ask(question string) {
|
||||
fmt.Printf("🤔 Frage: \"%s\"\n\n", question)
|
||||
fmt.Println("🔍 Durchsuche lokale Wissensdatenbank...")
|
||||
|
||||
answer, chunks, err := AskQuery(question)
|
||||
answer, chunks, err := AskQuery(question, nil)
|
||||
if err != nil {
|
||||
log.Fatalf("❌ %v", err)
|
||||
}
|
||||
@@ -146,7 +180,7 @@ func searchKnowledge(ctx context.Context, embClient *openai.Client, query string
|
||||
seen := make(map[string]bool)
|
||||
for _, hit := range searchResult.Result {
|
||||
text := hit.Payload["text"].GetStringValue()
|
||||
if seen[text] {
|
||||
if text == "" || seen[text] {
|
||||
continue
|
||||
}
|
||||
seen[text] = true
|
||||
@@ -176,6 +210,12 @@ func ChatDirect(question string) (string, error) {
|
||||
|
||||
systemPrompt := `Du bist ein hilfreicher persönlicher Assistent. Antworte auf Deutsch, präzise und direkt.`
|
||||
|
||||
slog.Debug("[LLM] ChatDirect Prompt",
|
||||
"model", config.Cfg.Chat.Model,
|
||||
"user", question,
|
||||
)
|
||||
|
||||
start := time.Now()
|
||||
stream, err := chatClient.CreateChatCompletionStream(ctx, openai.ChatCompletionRequest{
|
||||
Model: config.Cfg.Chat.Model,
|
||||
Messages: []openai.ChatCompletionMessage{
|
||||
@@ -200,6 +240,13 @@ func ChatDirect(question string) (string, error) {
|
||||
answer.WriteString(response.Choices[0].Delta.Content)
|
||||
}
|
||||
}
|
||||
|
||||
slog.Debug("[LLM] ChatDirect Antwort",
|
||||
"dauer", time.Since(start).Round(time.Millisecond),
|
||||
"zeichen", answer.Len(),
|
||||
"antwort", answer.String(),
|
||||
)
|
||||
|
||||
return answer.String(), nil
|
||||
}
|
||||
|
||||
|
||||
54
internal/brain/core_memory.go
Normal file
54
internal/brain/core_memory.go
Normal file
@@ -0,0 +1,54 @@
|
||||
// core_memory.go – Persistente Kernfakten über den Nutzer (core_memory.md)
|
||||
package brain
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// CoreMemoryPath gibt den Pfad zur core_memory.md-Datei zurück.
|
||||
func CoreMemoryPath() string {
|
||||
return filepath.Join(config.Cfg.BrainRoot, "core_memory.md")
|
||||
}
|
||||
|
||||
// LoadCoreMemory liest den Inhalt der core_memory.md-Datei.
|
||||
// Gibt leeren String zurück wenn die Datei nicht existiert.
|
||||
func LoadCoreMemory() string {
|
||||
data, err := os.ReadFile(CoreMemoryPath())
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimSpace(string(data))
|
||||
}
|
||||
|
||||
// AppendCoreMemory fügt einen Fakt zur core_memory.md-Datei hinzu.
|
||||
func AppendCoreMemory(text string) error {
|
||||
path := CoreMemoryPath()
|
||||
// Datei erstellen falls nicht vorhanden, sonst anhängen
|
||||
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if err != nil {
|
||||
return fmt.Errorf("core_memory.md öffnen: %w", err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
// Führenden Bindestrich ergänzen wenn nicht vorhanden
|
||||
line := strings.TrimSpace(text)
|
||||
if !strings.HasPrefix(line, "-") {
|
||||
line = "- " + line
|
||||
}
|
||||
_, err = fmt.Fprintf(f, "%s\n", line)
|
||||
return err
|
||||
}
|
||||
|
||||
// ShowCoreMemory gibt den Inhalt der core_memory.md als formatierte Nachricht zurück.
|
||||
func ShowCoreMemory() string {
|
||||
content := LoadCoreMemory()
|
||||
if content == "" {
|
||||
return "📭 Keine Kernfakten gespeichert. Nutze `/memory profile <text>` um Fakten hinzuzufügen."
|
||||
}
|
||||
return fmt.Sprintf("🧠 **Kerngedächtnis:**\n```\n%s\n```", content)
|
||||
}
|
||||
224
internal/brain/deepask.go
Normal file
224
internal/brain/deepask.go
Normal file
@@ -0,0 +1,224 @@
|
||||
// deepask.go – Multi-Step Reasoning: iterative RAG-Suche mit Folgefragen
|
||||
package brain
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
openai "github.com/sashabaranov/go-openai"
|
||||
"google.golang.org/grpc/metadata"
|
||||
|
||||
"my-brain-importer/internal/agents"
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
const maxDeepSteps = 3
|
||||
|
||||
// DeepAskQuery führt eine iterative RAG-Suche durch:
|
||||
// 1. Initiale Suche → LLM generiert Folgefragen
|
||||
// 2. Vertiefungssuchen mit Folgefragen (max 2 Iterationen)
|
||||
// 3. Synthese: alle gesammelten Chunks → umfassende Antwort
|
||||
func DeepAskQuery(question string, history []agents.HistoryMessage) (string, []KnowledgeChunk, error) {
|
||||
start := time.Now()
|
||||
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
chatClient := config.NewChatClient()
|
||||
|
||||
// Deduplizierung über alle Schritte
|
||||
seen := make(map[string]bool)
|
||||
var allChunks []KnowledgeChunk
|
||||
|
||||
addChunks := func(chunks []KnowledgeChunk) int {
|
||||
added := 0
|
||||
for _, c := range chunks {
|
||||
if !seen[c.Text] {
|
||||
seen[c.Text] = true
|
||||
allChunks = append(allChunks, c)
|
||||
added++
|
||||
}
|
||||
}
|
||||
return added
|
||||
}
|
||||
|
||||
// Phase 1: Initiale Suche
|
||||
slog.Info("[DeepAsk] Phase 1: Initiale Suche", "frage", question)
|
||||
initialChunks := searchKnowledge(ctx, embClient, question)
|
||||
addChunks(initialChunks)
|
||||
|
||||
if len(allChunks) == 0 {
|
||||
return "", nil, nil
|
||||
}
|
||||
|
||||
// Phase 2: Folgefragen generieren und suchen (max 2 Iterationen)
|
||||
queries := []string{question}
|
||||
for step := 1; step < maxDeepSteps; step++ {
|
||||
followUps := generateFollowUpQueries(ctx, chatClient, question, allChunks)
|
||||
if len(followUps) == 0 {
|
||||
slog.Info("[DeepAsk] Keine Folgefragen generiert, überspringe", "schritt", step)
|
||||
break
|
||||
}
|
||||
|
||||
slog.Info("[DeepAsk] Phase 2: Vertiefung",
|
||||
"schritt", step,
|
||||
"folgefragen", len(followUps),
|
||||
"fragen", followUps,
|
||||
)
|
||||
|
||||
newFound := 0
|
||||
for _, fq := range followUps {
|
||||
chunks := searchKnowledge(ctx, embClient, fq)
|
||||
newFound += addChunks(chunks)
|
||||
queries = append(queries, fq)
|
||||
}
|
||||
|
||||
if newFound == 0 {
|
||||
slog.Info("[DeepAsk] Keine neuen Chunks gefunden, beende Vertiefung", "schritt", step)
|
||||
break
|
||||
}
|
||||
|
||||
slog.Info("[DeepAsk] Neue Chunks gefunden", "schritt", step, "neu", newFound, "gesamt", len(allChunks))
|
||||
}
|
||||
|
||||
// Phase 3: Synthese — alle Chunks + Frage → umfassende Antwort
|
||||
slog.Info("[DeepAsk] Phase 3: Synthese", "chunks", len(allChunks), "schritte", len(queries))
|
||||
|
||||
contextText := buildContext(allChunks)
|
||||
coreMemory := LoadCoreMemory()
|
||||
|
||||
systemPrompt := `Du bist ein hilfreicher persönlicher Assistent mit Zugang zu einer umfangreichen Wissensdatenbank.
|
||||
Dir wurden Informationen aus mehreren Suchdurchläufen bereitgestellt.
|
||||
|
||||
WICHTIGE REGELN:
|
||||
- Nutze ALLE bereitgestellten Informationen für eine umfassende Antwort
|
||||
- Verbinde Informationen aus verschiedenen Quellen zu einer kohärenten Antwort
|
||||
- Ergänze mit eigenem Wissen wenn sinnvoll, kennzeichne es mit "Aus meinem Wissen:"
|
||||
- Antworte auf Deutsch
|
||||
- Sei gründlich aber strukturiert`
|
||||
|
||||
if coreMemory != "" {
|
||||
systemPrompt += "\n\n## Fakten über den Nutzer:\n" + coreMemory
|
||||
}
|
||||
|
||||
userPrompt := fmt.Sprintf(`Hier sind relevante Informationen aus mehreren Suchdurchläufen in meiner Wissensdatenbank:
|
||||
|
||||
%s
|
||||
|
||||
Basierend auf ALLEN diesen Informationen, beantworte bitte umfassend folgende Frage:
|
||||
%s`, contextText, question)
|
||||
|
||||
msgs := []openai.ChatCompletionMessage{
|
||||
{Role: openai.ChatMessageRoleSystem, Content: systemPrompt},
|
||||
}
|
||||
for _, h := range history {
|
||||
msgs = append(msgs, openai.ChatCompletionMessage{
|
||||
Role: h.Role,
|
||||
Content: h.Content,
|
||||
})
|
||||
}
|
||||
msgs = append(msgs, openai.ChatCompletionMessage{
|
||||
Role: openai.ChatMessageRoleUser,
|
||||
Content: userPrompt,
|
||||
})
|
||||
|
||||
stream, err := chatClient.CreateChatCompletionStream(ctx, openai.ChatCompletionRequest{
|
||||
Model: config.Cfg.Chat.Model,
|
||||
Messages: msgs,
|
||||
Temperature: 0.7,
|
||||
MaxTokens: 800,
|
||||
})
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("LLM Fehler: %w", err)
|
||||
}
|
||||
defer stream.Close()
|
||||
|
||||
var answer strings.Builder
|
||||
for {
|
||||
response, err := stream.Recv()
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
if len(response.Choices) > 0 {
|
||||
answer.WriteString(response.Choices[0].Delta.Content)
|
||||
}
|
||||
}
|
||||
|
||||
slog.Info("[DeepAsk] Abgeschlossen",
|
||||
"dauer", time.Since(start).Round(time.Millisecond),
|
||||
"chunks_gesamt", len(allChunks),
|
||||
"suchanfragen", len(queries),
|
||||
"antwort_zeichen", answer.Len(),
|
||||
)
|
||||
|
||||
return answer.String(), allChunks, nil
|
||||
}
|
||||
|
||||
// generateFollowUpQueries lässt das LLM basierend auf bisherigen Ergebnissen Folgefragen generieren.
|
||||
// Gibt 0-3 Folgefragen zurück.
|
||||
func generateFollowUpQueries(ctx context.Context, chatClient *openai.Client, question string, chunks []KnowledgeChunk) []string {
|
||||
contextText := buildContext(chunks)
|
||||
|
||||
prompt := fmt.Sprintf(`Originalfrage: %s
|
||||
|
||||
Bisherige Suchergebnisse:
|
||||
%s
|
||||
|
||||
Generiere 1-3 Folgefragen, die helfen würden, die Originalfrage besser zu beantworten.
|
||||
Jede Folgefrage muss auf einer eigenen Zeile stehen und mit "FOLGEFRAGE:" beginnen.
|
||||
Wenn die bisherigen Ergebnisse die Frage bereits vollständig beantworten, schreibe: KEINE FOLGEFRAGEN`, question, contextText)
|
||||
|
||||
resp, err := chatClient.CreateChatCompletion(ctx, openai.ChatCompletionRequest{
|
||||
Model: config.Cfg.Chat.Model,
|
||||
Messages: []openai.ChatCompletionMessage{
|
||||
{Role: openai.ChatMessageRoleSystem, Content: "Du generierst präzise Suchfragen für eine Wissensdatenbank. Antworte NUR im vorgegebenen Format."},
|
||||
{Role: openai.ChatMessageRoleUser, Content: prompt},
|
||||
},
|
||||
Temperature: 0.3,
|
||||
MaxTokens: 300,
|
||||
})
|
||||
if err != nil {
|
||||
slog.Warn("[DeepAsk] Folgefragen-Generierung fehlgeschlagen", "fehler", err)
|
||||
return nil
|
||||
}
|
||||
if len(resp.Choices) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
return parseFollowUpQueries(resp.Choices[0].Message.Content)
|
||||
}
|
||||
|
||||
// parseFollowUpQueries extrahiert Folgefragen aus der LLM-Antwort.
|
||||
// Erwartet Zeilen im Format "FOLGEFRAGE: <frage>".
|
||||
func parseFollowUpQueries(response string) []string {
|
||||
// Reasoning-Modelle: Antwort nach </think>-Tag
|
||||
if idx := strings.LastIndex(response, "</think>"); idx >= 0 {
|
||||
response = response[idx+len("</think>"):]
|
||||
}
|
||||
|
||||
if strings.Contains(strings.ToUpper(response), "KEINE FOLGEFRAGEN") {
|
||||
return nil
|
||||
}
|
||||
|
||||
var queries []string
|
||||
for _, line := range strings.Split(response, "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
upper := strings.ToUpper(line)
|
||||
if strings.HasPrefix(upper, "FOLGEFRAGE:") {
|
||||
q := strings.TrimSpace(line[len("FOLGEFRAGE:"):])
|
||||
if q != "" {
|
||||
queries = append(queries, q)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Max 3 Folgefragen
|
||||
if len(queries) > 3 {
|
||||
queries = queries[:3]
|
||||
}
|
||||
return queries
|
||||
}
|
||||
@@ -256,3 +256,30 @@ func IngestChatMessage(text, author, source string) error {
|
||||
}
|
||||
|
||||
func boolPtr(b bool) *bool { return &b }
|
||||
|
||||
// IngestText speichert einen beliebigen Text mit Quelle und Typ in Qdrant.
|
||||
// Verwendet die gleiche Chunking-Logik wie der Markdown-Ingest.
|
||||
func IngestText(text, source, docType string) error {
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
ensureCollection(ctx, pb.NewCollectionsClient(conn))
|
||||
pointsClient := pb.NewPointsClient(conn)
|
||||
|
||||
var chunks []chunk
|
||||
for _, part := range splitLongSection(text) {
|
||||
part = strings.TrimSpace(part)
|
||||
if len(part) < 20 {
|
||||
continue
|
||||
}
|
||||
chunks = append(chunks, chunk{Text: part, Source: source, Type: docType})
|
||||
}
|
||||
if len(chunks) == 0 {
|
||||
return nil
|
||||
}
|
||||
return ingestChunks(ctx, embClient, pointsClient, chunks)
|
||||
}
|
||||
|
||||
98
internal/brain/ingest_email.go
Normal file
98
internal/brain/ingest_email.go
Normal file
@@ -0,0 +1,98 @@
|
||||
// ingest_email.go – Importiert Emails aus einem IMAP-Ordner in Qdrant
|
||||
package brain
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
pb "github.com/qdrant/go-client/qdrant"
|
||||
"google.golang.org/grpc/metadata"
|
||||
|
||||
"my-brain-importer/internal/agents/tool/email"
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// IngestEmailFolder importiert alle Emails aus einem IMAP-Ordner in Qdrant.
|
||||
// Gibt Anzahl der importierten Emails zurück.
|
||||
// maxEmails = 0 bedeutet: alle (bis max. 500).
|
||||
func IngestEmailFolder(acc config.EmailAccount, folder string, maxEmails uint32) (int, error) {
|
||||
if maxEmails == 0 {
|
||||
maxEmails = 500
|
||||
}
|
||||
|
||||
cl, err := email.ConnectAccount(acc)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("IMAP-Verbindung: %w", err)
|
||||
}
|
||||
defer cl.Close()
|
||||
|
||||
slog.Info("Email-Ingest: Lade Emails", "account", acc.Name, "folder", folder, "max", maxEmails)
|
||||
msgs, err := cl.FetchWithBody(folder, maxEmails)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("Emails laden: %w", err)
|
||||
}
|
||||
if len(msgs) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
ensureCollection(ctx, pb.NewCollectionsClient(conn))
|
||||
pointsClient := pb.NewPointsClient(conn)
|
||||
|
||||
var chunks []chunk
|
||||
for _, m := range msgs {
|
||||
text := formatEmailForIngest(m)
|
||||
if len(strings.TrimSpace(text)) < 20 {
|
||||
continue
|
||||
}
|
||||
source := fmt.Sprintf("email/%s/%s", folder, m.Date)
|
||||
chunks = append(chunks, chunk{Text: text, Source: source, Type: "email"})
|
||||
}
|
||||
|
||||
if len(chunks) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
slog.Info("Email-Ingest: Starte Embedding", "emails", len(msgs), "chunks", len(chunks))
|
||||
|
||||
// In Batches von 20 ingesten (Embeddings können langsam sein)
|
||||
ingested := 0
|
||||
for i := 0; i < len(chunks); i += 20 {
|
||||
end := i + 20
|
||||
if end > len(chunks) {
|
||||
end = len(chunks)
|
||||
}
|
||||
batch := chunks[i:end]
|
||||
if err := ingestChunks(ctx, embClient, pointsClient, batch); err != nil {
|
||||
slog.Warn("Email-Ingest Batch-Fehler", "batch_start", i, "fehler", err)
|
||||
continue
|
||||
}
|
||||
ingested += len(batch)
|
||||
slog.Info("Email-Ingest Fortschritt", "ingested", ingested, "total", len(chunks))
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
}
|
||||
|
||||
return ingested, nil
|
||||
}
|
||||
|
||||
// formatEmailForIngest formatiert eine Email als durchsuchbaren Text.
|
||||
func formatEmailForIngest(m email.MessageWithBody) string {
|
||||
var sb strings.Builder
|
||||
fmt.Fprintf(&sb, "Betreff: %s\n", m.Subject)
|
||||
fmt.Fprintf(&sb, "Von: %s\n", m.From)
|
||||
fmt.Fprintf(&sb, "Datum: %s\n", m.Date)
|
||||
if m.Body != "" {
|
||||
sb.WriteString("\n")
|
||||
sb.WriteString(m.Body)
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
82
internal/brain/ingest_pdf.go
Normal file
82
internal/brain/ingest_pdf.go
Normal file
@@ -0,0 +1,82 @@
|
||||
// ingest_pdf.go – Extrahiert Text aus einer PDF-Datei und importiert ihn in Qdrant
|
||||
package brain
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/ledongthuc/pdf"
|
||||
pb "github.com/qdrant/go-client/qdrant"
|
||||
"google.golang.org/grpc/metadata"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// IngestPDF extrahiert Text aus einer PDF-Datei und importiert ihn in Qdrant.
|
||||
// source ist der Anzeigename der Quelle (z.B. Dateiname).
|
||||
// Gibt Anzahl der importierten Chunks zurück.
|
||||
func IngestPDF(filePath, source string) (int, error) {
|
||||
text, err := extractPDFText(filePath)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("PDF-Parsing fehlgeschlagen: %w", err)
|
||||
}
|
||||
|
||||
text = strings.TrimSpace(text)
|
||||
if len(text) < 20 {
|
||||
return 0, fmt.Errorf("kein verwertbarer Text in PDF gefunden")
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
ensureCollection(ctx, pb.NewCollectionsClient(conn))
|
||||
pointsClient := pb.NewPointsClient(conn)
|
||||
|
||||
var chunks []chunk
|
||||
for _, part := range splitLongSection(text) {
|
||||
part = strings.TrimSpace(part)
|
||||
if len(part) < 20 {
|
||||
continue
|
||||
}
|
||||
chunks = append(chunks, chunk{Text: part, Source: source, Type: "pdf"})
|
||||
}
|
||||
|
||||
if len(chunks) == 0 {
|
||||
return 0, fmt.Errorf("kein verwertbarer Inhalt nach Aufteilung")
|
||||
}
|
||||
|
||||
if err := ingestChunks(ctx, embClient, pointsClient, chunks); err != nil {
|
||||
return 0, fmt.Errorf("Ingest fehlgeschlagen: %w", err)
|
||||
}
|
||||
return len(chunks), nil
|
||||
}
|
||||
|
||||
// extractPDFText liest alle Seiten einer PDF-Datei und gibt den Text zurück.
|
||||
func extractPDFText(filePath string) (string, error) {
|
||||
f, r, err := pdf.Open(filePath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
var sb strings.Builder
|
||||
totalPages := r.NumPage()
|
||||
for pageNum := 1; pageNum <= totalPages; pageNum++ {
|
||||
page := r.Page(pageNum)
|
||||
if page.V.IsNull() {
|
||||
continue
|
||||
}
|
||||
text, err := page.GetPlainText(nil)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
sb.WriteString(text)
|
||||
sb.WriteString("\n")
|
||||
}
|
||||
return sb.String(), nil
|
||||
}
|
||||
124
internal/brain/ingest_url.go
Normal file
124
internal/brain/ingest_url.go
Normal file
@@ -0,0 +1,124 @@
|
||||
// ingest_url.go – Fetcht eine URL und importiert den Textinhalt in Qdrant
|
||||
package brain
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
pb "github.com/qdrant/go-client/qdrant"
|
||||
"golang.org/x/net/html"
|
||||
"google.golang.org/grpc/metadata"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// IngestURL fetcht eine URL, extrahiert den Textinhalt und importiert ihn in Qdrant.
|
||||
// Gibt Anzahl der importierten Chunks zurück.
|
||||
func IngestURL(rawURL string) (int, error) {
|
||||
client := &http.Client{Timeout: 30 * time.Second}
|
||||
resp, err := client.Get(rawURL)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("HTTP-Fehler: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return 0, fmt.Errorf("HTTP %d: %s", resp.StatusCode, resp.Status)
|
||||
}
|
||||
|
||||
contentType := resp.Header.Get("Content-Type")
|
||||
var text string
|
||||
if strings.Contains(contentType, "text/html") {
|
||||
text, err = extractHTMLText(resp.Body)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("HTML-Parsing fehlgeschlagen: %w", err)
|
||||
}
|
||||
} else {
|
||||
raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) // max 1MB
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("Lesen fehlgeschlagen: %w", err)
|
||||
}
|
||||
text = string(raw)
|
||||
}
|
||||
|
||||
text = strings.TrimSpace(text)
|
||||
if len(text) < 20 {
|
||||
return 0, fmt.Errorf("kein verwertbarer Inhalt gefunden")
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
ensureCollection(ctx, pb.NewCollectionsClient(conn))
|
||||
pointsClient := pb.NewPointsClient(conn)
|
||||
|
||||
var chunks []chunk
|
||||
for _, part := range splitLongSection(text) {
|
||||
part = strings.TrimSpace(part)
|
||||
if len(part) < 20 {
|
||||
continue
|
||||
}
|
||||
chunks = append(chunks, chunk{Text: part, Source: rawURL, Type: "url"})
|
||||
}
|
||||
|
||||
if len(chunks) == 0 {
|
||||
return 0, fmt.Errorf("kein verwertbarer Inhalt nach Aufteilung")
|
||||
}
|
||||
|
||||
if err := ingestChunks(ctx, embClient, pointsClient, chunks); err != nil {
|
||||
return 0, fmt.Errorf("Ingest fehlgeschlagen: %w", err)
|
||||
}
|
||||
return len(chunks), nil
|
||||
}
|
||||
|
||||
// extractHTMLText extrahiert sichtbaren Text aus einem HTML-Dokument.
|
||||
func extractHTMLText(r io.Reader) (string, error) {
|
||||
doc, err := html.Parse(r)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
var sb strings.Builder
|
||||
extractTextNode(doc, &sb)
|
||||
// Mehrfach-Leerzeilen reduzieren
|
||||
lines := strings.Split(sb.String(), "\n")
|
||||
var cleaned []string
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line != "" {
|
||||
cleaned = append(cleaned, line)
|
||||
}
|
||||
}
|
||||
return strings.Join(cleaned, "\n"), nil
|
||||
}
|
||||
|
||||
// skipTags sind HTML-Elemente deren Inhalt nicht extrahiert wird.
|
||||
var skipTags = map[string]bool{
|
||||
"script": true, "style": true, "noscript": true,
|
||||
"head": true, "meta": true, "link": true,
|
||||
"nav": true, "footer": true, "header": true,
|
||||
}
|
||||
|
||||
func extractTextNode(n *html.Node, sb *strings.Builder) {
|
||||
if n.Type == html.TextNode {
|
||||
text := strings.TrimSpace(n.Data)
|
||||
if text != "" {
|
||||
sb.WriteString(text)
|
||||
sb.WriteString("\n")
|
||||
}
|
||||
return
|
||||
}
|
||||
if n.Type == html.ElementNode && skipTags[n.Data] {
|
||||
return
|
||||
}
|
||||
for c := n.FirstChild; c != nil; c = c.NextSibling {
|
||||
extractTextNode(c, sb)
|
||||
}
|
||||
}
|
||||
108
internal/brain/knowledge.go
Normal file
108
internal/brain/knowledge.go
Normal file
@@ -0,0 +1,108 @@
|
||||
// knowledge.go – Listet und löscht Einträge in der Qdrant-Wissensdatenbank
|
||||
package brain
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sort"
|
||||
|
||||
pb "github.com/qdrant/go-client/qdrant"
|
||||
"google.golang.org/grpc/metadata"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// ListSources gibt alle eindeutigen Quellen in der Wissensdatenbank zurück.
|
||||
// Limit begrenzt die Anzahl der zu scrollenden Punkte (0 = Standard 1000).
|
||||
func ListSources(limit uint32) ([]string, error) {
|
||||
if limit == 0 {
|
||||
limit = 1000
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
pointsClient := pb.NewPointsClient(conn)
|
||||
|
||||
seen := map[string]bool{}
|
||||
var offset *pb.PointId
|
||||
|
||||
for {
|
||||
req := &pb.ScrollPoints{
|
||||
CollectionName: config.Cfg.Qdrant.Collection,
|
||||
WithPayload: &pb.WithPayloadSelector{
|
||||
SelectorOptions: &pb.WithPayloadSelector_Include{
|
||||
Include: &pb.PayloadIncludeSelector{Fields: []string{"source"}},
|
||||
},
|
||||
},
|
||||
Limit: uint32Ptr(250),
|
||||
}
|
||||
if offset != nil {
|
||||
req.Offset = offset
|
||||
}
|
||||
|
||||
result, err := pointsClient.Scroll(ctx, req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Scroll fehlgeschlagen: %w", err)
|
||||
}
|
||||
|
||||
for _, pt := range result.Result {
|
||||
if src := pt.Payload["source"].GetStringValue(); src != "" {
|
||||
seen[src] = true
|
||||
}
|
||||
}
|
||||
|
||||
if result.NextPageOffset == nil || uint32(len(seen)) >= limit {
|
||||
break
|
||||
}
|
||||
offset = result.NextPageOffset
|
||||
}
|
||||
|
||||
sources := make([]string, 0, len(seen))
|
||||
for s := range seen {
|
||||
sources = append(sources, s)
|
||||
}
|
||||
sort.Strings(sources)
|
||||
return sources, nil
|
||||
}
|
||||
|
||||
// DeleteBySource löscht alle Punkte mit dem gegebenen Quellennamen aus Qdrant.
|
||||
// Gibt Anzahl gelöschter Punkte zurück (Qdrant liefert keine genaue Zahl — gibt 0 zurück wenn erfolgreich).
|
||||
func DeleteBySource(source string) error {
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
pointsClient := pb.NewPointsClient(conn)
|
||||
|
||||
_, err := pointsClient.Delete(ctx, &pb.DeletePoints{
|
||||
CollectionName: config.Cfg.Qdrant.Collection,
|
||||
Points: &pb.PointsSelector{
|
||||
PointsSelectorOneOf: &pb.PointsSelector_Filter{
|
||||
Filter: &pb.Filter{
|
||||
Must: []*pb.Condition{
|
||||
{
|
||||
ConditionOneOf: &pb.Condition_Field{
|
||||
Field: &pb.FieldCondition{
|
||||
Key: "source",
|
||||
Match: &pb.Match{
|
||||
MatchValue: &pb.Match_Keyword{Keyword: source},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
Wait: boolPtr(true),
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func uint32Ptr(v uint32) *uint32 { return &v }
|
||||
@@ -12,6 +12,30 @@ import (
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// ArchiveFolder beschreibt einen IMAP-Archivordner mit optionaler Aufbewahrungsdauer.
|
||||
type ArchiveFolder struct {
|
||||
Name string `yaml:"name"` // Anzeigename (z.B. "5Jahre")
|
||||
IMAPFolder string `yaml:"imap_folder"` // Echter IMAP-Ordnername (z.B. "5Jahre")
|
||||
RetentionDays int `yaml:"retention_days"` // 0 = dauerhaft behalten
|
||||
}
|
||||
|
||||
// EmailAccount beschreibt einen einzelnen IMAP-Account.
|
||||
type EmailAccount struct {
|
||||
Name string `yaml:"name"`
|
||||
Host string `yaml:"host"`
|
||||
Port int `yaml:"port"`
|
||||
User string `yaml:"user"`
|
||||
Password string `yaml:"password"`
|
||||
TLS bool `yaml:"tls"`
|
||||
StartTLS bool `yaml:"starttls"`
|
||||
Folder string `yaml:"folder"`
|
||||
ProcessedFolder string `yaml:"processed_folder"`
|
||||
Model string `yaml:"model"`
|
||||
ArchiveFolders []ArchiveFolder `yaml:"archive_folders"`
|
||||
TriageImportantFolder string `yaml:"triage_important_folder"` // Ordner für wichtige Emails (leer = in INBOX lassen)
|
||||
TriageUnimportantFolder string `yaml:"triage_unimportant_folder"` // Ordner für unwichtige Emails (leer = kein Triage)
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
Qdrant struct {
|
||||
Host string `yaml:"host"`
|
||||
@@ -32,17 +56,84 @@ type Config struct {
|
||||
} `yaml:"chat"`
|
||||
|
||||
Discord struct {
|
||||
Token string `yaml:"token"`
|
||||
GuildID string `yaml:"guild_id"`
|
||||
Token string `yaml:"token"`
|
||||
GuildID string `yaml:"guild_id"`
|
||||
AllowedUsers []string `yaml:"allowed_users"` // Wenn gesetzt, dürfen nur diese User-IDs den Bot nutzen
|
||||
} `yaml:"discord"`
|
||||
|
||||
// Email ist der Legacy-Block für einen einzelnen Account.
|
||||
Email struct {
|
||||
Host string `yaml:"host"`
|
||||
Port int `yaml:"port"`
|
||||
User string `yaml:"user"`
|
||||
Password string `yaml:"password"`
|
||||
TLS bool `yaml:"tls"`
|
||||
StartTLS bool `yaml:"starttls"`
|
||||
Folder string `yaml:"folder"`
|
||||
ProcessedFolder string `yaml:"processed_folder"`
|
||||
Model string `yaml:"model"`
|
||||
ArchiveFolders []ArchiveFolder `yaml:"archive_folders"`
|
||||
TriageImportantFolder string `yaml:"triage_important_folder"`
|
||||
TriageUnimportantFolder string `yaml:"triage_unimportant_folder"`
|
||||
} `yaml:"email"`
|
||||
|
||||
// EmailAccounts ermöglicht mehrere IMAP-Accounts. Hat Vorrang vor email:.
|
||||
EmailAccounts []EmailAccount `yaml:"email_accounts"`
|
||||
|
||||
Tasks struct {
|
||||
StorePath string `yaml:"store_path"`
|
||||
} `yaml:"tasks"`
|
||||
|
||||
Daemon struct {
|
||||
ChannelID string `yaml:"channel_id"`
|
||||
EmailIntervalMin int `yaml:"email_interval_min"`
|
||||
TaskReminderHour int `yaml:"task_reminder_hour"`
|
||||
CleanupHour int `yaml:"cleanup_hour"` // Uhrzeit für tägliches Archiv-Aufräumen (Standard: 2)
|
||||
IngestHour int `yaml:"ingest_hour"` // Uhrzeit für nächtlichen Email-Ingest (Standard: 23, 0 = deaktiviert)
|
||||
} `yaml:"daemon"`
|
||||
|
||||
BrainRoot string `yaml:"brain_root"`
|
||||
TopK uint64 `yaml:"top_k"`
|
||||
ScoreThreshold float32 `yaml:"score_threshold"`
|
||||
|
||||
// RSSFeeds definiert RSS-Feeds die automatisch überwacht werden.
|
||||
RSSFeeds []RSSFeed `yaml:"rss_feeds"`
|
||||
}
|
||||
|
||||
// RSSFeed beschreibt einen RSS-Feed mit Polling-Intervall.
|
||||
type RSSFeed struct {
|
||||
URL string `yaml:"url"`
|
||||
IntervalHours int `yaml:"interval_hours"` // 0 = Standard 24h
|
||||
}
|
||||
|
||||
var Cfg Config
|
||||
|
||||
// AllEmailAccounts gibt alle konfigurierten Email-Accounts zurück.
|
||||
// Wenn email_accounts konfiguriert ist, hat das Vorrang vor dem Legacy-email:-Block.
|
||||
func AllEmailAccounts() []EmailAccount {
|
||||
if len(Cfg.EmailAccounts) > 0 {
|
||||
return Cfg.EmailAccounts
|
||||
}
|
||||
if Cfg.Email.Host == "" {
|
||||
return nil
|
||||
}
|
||||
return []EmailAccount{{
|
||||
Name: "Email",
|
||||
Host: Cfg.Email.Host,
|
||||
Port: Cfg.Email.Port,
|
||||
User: Cfg.Email.User,
|
||||
Password: Cfg.Email.Password,
|
||||
TLS: Cfg.Email.TLS,
|
||||
StartTLS: Cfg.Email.StartTLS,
|
||||
Folder: Cfg.Email.Folder,
|
||||
ProcessedFolder: Cfg.Email.ProcessedFolder,
|
||||
Model: Cfg.Email.Model,
|
||||
ArchiveFolders: Cfg.Email.ArchiveFolders,
|
||||
TriageImportantFolder: Cfg.Email.TriageImportantFolder,
|
||||
TriageUnimportantFolder: Cfg.Email.TriageUnimportantFolder,
|
||||
}}
|
||||
}
|
||||
|
||||
// NewQdrantConn öffnet eine gRPC-Verbindung zur Qdrant-Instanz.
|
||||
// Der Aufrufer ist verantwortlich für conn.Close().
|
||||
func NewQdrantConn() *grpc.ClientConn {
|
||||
@@ -70,7 +161,7 @@ func NewChatClient() *openai.Client {
|
||||
return openai.NewClientWithConfig(c)
|
||||
}
|
||||
|
||||
// LoadConfig liest config.yml aus dem aktuellen Verzeichnis.
|
||||
// LoadConfig liest config.yml aus dem aktuellen Verzeichnis und validiert Pflichtfelder.
|
||||
func LoadConfig() {
|
||||
data, err := os.ReadFile("config.yml")
|
||||
if err != nil {
|
||||
@@ -79,4 +170,34 @@ func LoadConfig() {
|
||||
if err := yaml.Unmarshal(data, &Cfg); err != nil {
|
||||
log.Fatalf("❌ config.yml ungültig: %v", err)
|
||||
}
|
||||
validateConfig()
|
||||
}
|
||||
|
||||
// validateConfig prüft Pflichtfelder und gibt früh eine klare Fehlermeldung.
|
||||
func validateConfig() {
|
||||
var errs []string
|
||||
if Cfg.Qdrant.Host == "" {
|
||||
errs = append(errs, "qdrant.host fehlt")
|
||||
}
|
||||
if Cfg.Qdrant.Port == "" {
|
||||
errs = append(errs, "qdrant.port fehlt")
|
||||
}
|
||||
if Cfg.Embedding.URL == "" {
|
||||
errs = append(errs, "embedding.url fehlt")
|
||||
}
|
||||
if Cfg.Embedding.Model == "" {
|
||||
errs = append(errs, "embedding.model fehlt")
|
||||
}
|
||||
if Cfg.Chat.URL == "" {
|
||||
errs = append(errs, "chat.url fehlt")
|
||||
}
|
||||
if Cfg.Chat.Model == "" {
|
||||
errs = append(errs, "chat.model fehlt")
|
||||
}
|
||||
if len(errs) > 0 {
|
||||
for _, e := range errs {
|
||||
log.Printf("❌ config.yml: %s", e)
|
||||
}
|
||||
log.Fatal("❌ Konfiguration unvollständig – Bot wird nicht gestartet.")
|
||||
}
|
||||
}
|
||||
|
||||
87
internal/config/config_test.go
Normal file
87
internal/config/config_test.go
Normal file
@@ -0,0 +1,87 @@
|
||||
package config
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestAllEmailAccounts_Empty(t *testing.T) {
|
||||
orig := Cfg
|
||||
defer func() { Cfg = orig }()
|
||||
Cfg = Config{}
|
||||
|
||||
accounts := AllEmailAccounts()
|
||||
if len(accounts) != 0 {
|
||||
t.Errorf("erwartet 0 Accounts, got %d", len(accounts))
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllEmailAccounts_LegacyFallback(t *testing.T) {
|
||||
orig := Cfg
|
||||
defer func() { Cfg = orig }()
|
||||
Cfg = Config{}
|
||||
Cfg.Email.Host = "imap.example.de"
|
||||
Cfg.Email.Port = 143
|
||||
Cfg.Email.User = "user@example.de"
|
||||
Cfg.Email.Password = "geheim"
|
||||
Cfg.Email.Folder = "INBOX"
|
||||
Cfg.Email.ProcessedFolder = "Processed"
|
||||
Cfg.Email.Model = "testmodel"
|
||||
|
||||
accounts := AllEmailAccounts()
|
||||
if len(accounts) != 1 {
|
||||
t.Fatalf("erwartet 1 Account, got %d", len(accounts))
|
||||
}
|
||||
a := accounts[0]
|
||||
if a.Host != "imap.example.de" {
|
||||
t.Errorf("Host: got %q", a.Host)
|
||||
}
|
||||
if a.Port != 143 {
|
||||
t.Errorf("Port: got %d", a.Port)
|
||||
}
|
||||
if a.User != "user@example.de" {
|
||||
t.Errorf("User: got %q", a.User)
|
||||
}
|
||||
if a.ProcessedFolder != "Processed" {
|
||||
t.Errorf("ProcessedFolder: got %q", a.ProcessedFolder)
|
||||
}
|
||||
if a.Model != "testmodel" {
|
||||
t.Errorf("Model: got %q", a.Model)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllEmailAccounts_MultipleAccounts(t *testing.T) {
|
||||
orig := Cfg
|
||||
defer func() { Cfg = orig }()
|
||||
Cfg = Config{}
|
||||
Cfg.EmailAccounts = []EmailAccount{
|
||||
{Name: "Privat", Host: "imap1.de", Port: 143},
|
||||
{Name: "Arbeit", Host: "imap2.de", Port: 993, TLS: true},
|
||||
}
|
||||
|
||||
accounts := AllEmailAccounts()
|
||||
if len(accounts) != 2 {
|
||||
t.Fatalf("erwartet 2 Accounts, got %d", len(accounts))
|
||||
}
|
||||
if accounts[0].Host != "imap1.de" {
|
||||
t.Errorf("Account 0 Host: got %q", accounts[0].Host)
|
||||
}
|
||||
if accounts[1].Host != "imap2.de" {
|
||||
t.Errorf("Account 1 Host: got %q", accounts[1].Host)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllEmailAccounts_NewTakesPrecedence(t *testing.T) {
|
||||
orig := Cfg
|
||||
defer func() { Cfg = orig }()
|
||||
Cfg = Config{}
|
||||
Cfg.Email.Host = "legacy.de"
|
||||
Cfg.EmailAccounts = []EmailAccount{
|
||||
{Name: "Neu", Host: "new.de"},
|
||||
}
|
||||
|
||||
accounts := AllEmailAccounts()
|
||||
if len(accounts) != 1 {
|
||||
t.Fatalf("erwartet 1 Account, got %d", len(accounts))
|
||||
}
|
||||
if accounts[0].Host != "new.de" {
|
||||
t.Errorf("email_accounts sollte Vorrang haben, got host %q", accounts[0].Host)
|
||||
}
|
||||
}
|
||||
127
internal/diag/diag.go
Normal file
127
internal/diag/diag.go
Normal file
@@ -0,0 +1,127 @@
|
||||
// diag – Start-Diagnose: prüft Erreichbarkeit aller externen Dienste
|
||||
package diag
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// Result ist das Ergebnis einer einzelnen Prüfung.
|
||||
type Result struct {
|
||||
Name string
|
||||
OK bool
|
||||
Message string
|
||||
}
|
||||
|
||||
// RunAll führt alle Verbindungs-Checks durch und gibt eine Zusammenfassung zurück.
|
||||
// warnings = Dienste die konfiguriert aber nicht erreichbar sind.
|
||||
func RunAll() (results []Result, allOK bool) {
|
||||
allOK = true
|
||||
cfg := config.Cfg
|
||||
|
||||
check := func(name string, ok bool, msg string) {
|
||||
results = append(results, Result{Name: name, OK: ok, Message: msg})
|
||||
if !ok {
|
||||
allOK = false
|
||||
}
|
||||
}
|
||||
|
||||
// Qdrant
|
||||
qdrantAddr := fmt.Sprintf("%s:%s", cfg.Qdrant.Host, cfg.Qdrant.Port)
|
||||
ok, msg := tcpCheck(qdrantAddr)
|
||||
check("Qdrant ("+qdrantAddr+")", ok, msg)
|
||||
|
||||
// LocalAI Chat
|
||||
if cfg.Chat.URL != "" {
|
||||
ok, msg = httpCheck(cfg.Chat.URL)
|
||||
check("LocalAI Chat ("+cfg.Chat.URL+")", ok, msg)
|
||||
}
|
||||
|
||||
// LocalAI Embedding (nur prüfen wenn andere URL als Chat)
|
||||
if cfg.Embedding.URL != "" && cfg.Embedding.URL != cfg.Chat.URL {
|
||||
ok, msg = httpCheck(cfg.Embedding.URL)
|
||||
check("LocalAI Embedding ("+cfg.Embedding.URL+")", ok, msg)
|
||||
}
|
||||
|
||||
// IMAP – alle konfigurierten Accounts prüfen
|
||||
for _, acc := range config.AllEmailAccounts() {
|
||||
if acc.Host == "" {
|
||||
continue
|
||||
}
|
||||
imapAddr := fmt.Sprintf("%s:%d", acc.Host, acc.Port)
|
||||
label := acc.Name
|
||||
if label == "" {
|
||||
label = acc.User
|
||||
}
|
||||
ok, msg = tcpCheck(imapAddr)
|
||||
check("IMAP "+label+" ("+imapAddr+")", ok, msg)
|
||||
}
|
||||
|
||||
return results, allOK
|
||||
}
|
||||
|
||||
// Log gibt die Ergebnisse über slog aus.
|
||||
func Log(results []Result) {
|
||||
for _, r := range results {
|
||||
if r.OK {
|
||||
slog.Info("Dienst-Check", "dienst", r.Name, "status", "OK", "info", r.Message)
|
||||
} else {
|
||||
slog.Warn("Dienst-Check", "dienst", r.Name, "status", "FEHLER", "info", r.Message)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Format gibt eine menschenlesbare Zusammenfassung zurück (für Discord/stdout).
|
||||
func Format(results []Result, allOK bool) string {
|
||||
var sb strings.Builder
|
||||
sb.WriteString("🔍 **Start-Diagnose:**\n")
|
||||
for _, r := range results {
|
||||
icon := "✅"
|
||||
if !r.OK {
|
||||
icon = "❌"
|
||||
}
|
||||
fmt.Fprintf(&sb, "%s %s – %s\n", icon, r.Name, r.Message)
|
||||
}
|
||||
if allOK {
|
||||
sb.WriteString("\n✅ Alle Dienste erreichbar.")
|
||||
} else {
|
||||
sb.WriteString("\n⚠️ Einige Dienste sind nicht erreichbar — Bot läuft, aber Funktionen könnten fehlen.")
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
func tcpCheck(addr string) (bool, string) {
|
||||
conn, err := net.DialTimeout("tcp", addr, 3*time.Second)
|
||||
if err != nil {
|
||||
return false, "nicht erreichbar: " + err.Error()
|
||||
}
|
||||
conn.Close()
|
||||
return true, "TCP OK"
|
||||
}
|
||||
|
||||
func httpCheck(baseURL string) (bool, string) {
|
||||
// Normalisiere URL: entferne trailing /v1 etc., hänge /v1/models an
|
||||
url := strings.TrimRight(baseURL, "/")
|
||||
if !strings.HasSuffix(url, "/models") {
|
||||
// Gehe zur Basis-URL zurück und frage /v1/models
|
||||
if idx := strings.LastIndex(url, "/v1"); idx >= 0 {
|
||||
url = url[:idx]
|
||||
}
|
||||
url += "/v1/models"
|
||||
}
|
||||
|
||||
client := &http.Client{Timeout: 3 * time.Second}
|
||||
resp, err := client.Get(url)
|
||||
if err != nil {
|
||||
return false, "nicht erreichbar: " + err.Error()
|
||||
}
|
||||
resp.Body.Close()
|
||||
return resp.StatusCode == http.StatusOK,
|
||||
fmt.Sprintf("HTTP %d", resp.StatusCode)
|
||||
}
|
||||
388
internal/triage/triage.go
Normal file
388
internal/triage/triage.go
Normal file
@@ -0,0 +1,388 @@
|
||||
// triage/triage.go – Speichert und sucht Email-Triage-Entscheidungen in Qdrant (RAG-Lernen)
|
||||
// Eigenes Package um Import-Zyklen zwischen brain und email zu vermeiden.
|
||||
package triage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
|
||||
pb "github.com/qdrant/go-client/qdrant"
|
||||
openai "github.com/sashabaranov/go-openai"
|
||||
"google.golang.org/grpc/metadata"
|
||||
|
||||
"my-brain-importer/internal/config"
|
||||
)
|
||||
|
||||
// TriageResult repräsentiert ein Suchergebnis aus vergangenen Triage-Entscheidungen.
|
||||
type TriageResult struct {
|
||||
Text string
|
||||
Score float32
|
||||
}
|
||||
|
||||
// StoreDecision speichert eine Triage-Entscheidung in Qdrant.
|
||||
// Bei gleicher Email (Von + Betreff als deterministischer Schlüssel) wird die Entscheidung überschrieben.
|
||||
// date und bodySummary sind optional (leerer String = weglassen).
|
||||
// bodySummary wird auf max. 200 Zeichen gekürzt.
|
||||
func StoreDecision(subject, from, date, bodySummary string, isImportant bool) error {
|
||||
label := "wichtig"
|
||||
if !isImportant {
|
||||
label = "unwichtig"
|
||||
}
|
||||
|
||||
// Kopfzeile immer vorhanden
|
||||
header := fmt.Sprintf("Email-Triage | Von: %s | Betreff: %s", from, subject)
|
||||
if date != "" {
|
||||
header += fmt.Sprintf(" | Datum: %s", date)
|
||||
}
|
||||
header += fmt.Sprintf(" | Entscheidung: %s", label)
|
||||
|
||||
// Body-Zusammenfassung anhängen wenn vorhanden
|
||||
text := header
|
||||
if bodySummary != "" {
|
||||
summary := bodySummary
|
||||
if len(summary) > 200 {
|
||||
summary = summary[:200]
|
||||
}
|
||||
text = header + "\nBody: " + summary
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
embResp, err := embClient.CreateEmbeddings(ctx, openai.EmbeddingRequest{
|
||||
Input: []string{text},
|
||||
Model: openai.EmbeddingModel(config.Cfg.Embedding.Model),
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("embedding: %w", err)
|
||||
}
|
||||
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
// ID basiert auf Von+Betreff — nicht auf dem vollständigen Text, damit
|
||||
// Re-Processing derselben Email den bestehenden Eintrag überschreibt.
|
||||
id := triageID(from + "|" + subject)
|
||||
wait := true
|
||||
_, err = pb.NewPointsClient(conn).Upsert(ctx, &pb.UpsertPoints{
|
||||
CollectionName: config.Cfg.Qdrant.Collection,
|
||||
Points: []*pb.PointStruct{{
|
||||
Id: &pb.PointId{
|
||||
PointIdOptions: &pb.PointId_Uuid{Uuid: id},
|
||||
},
|
||||
Vectors: &pb.Vectors{
|
||||
VectorsOptions: &pb.Vectors_Vector{
|
||||
Vector: &pb.Vector{Data: embResp.Data[0].Embedding},
|
||||
},
|
||||
},
|
||||
Payload: map[string]*pb.Value{
|
||||
"text": {Kind: &pb.Value_StringValue{StringValue: text}},
|
||||
"source": {Kind: &pb.Value_StringValue{StringValue: "email_triage"}},
|
||||
"type": {Kind: &pb.Value_StringValue{StringValue: "email_triage"}},
|
||||
},
|
||||
}},
|
||||
Wait: &wait,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("qdrant upsert: %w", err)
|
||||
}
|
||||
|
||||
slog.Debug("[Triage] Entscheidung gespeichert", "betreff", subject, "wichtig", isImportant)
|
||||
return nil
|
||||
}
|
||||
|
||||
// SearchSimilar sucht ähnliche vergangene Triage-Entscheidungen in Qdrant.
|
||||
// Gibt bis zu 3 Ergebnisse zurück (nur type=email_triage, Score ≥ 0.7).
|
||||
func SearchSimilar(query string) []TriageResult {
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
embResp, err := embClient.CreateEmbeddings(ctx, openai.EmbeddingRequest{
|
||||
Input: []string{query},
|
||||
Model: openai.EmbeddingModel(config.Cfg.Embedding.Model),
|
||||
})
|
||||
if err != nil {
|
||||
slog.Warn("[Triage] Embedding für RAG fehlgeschlagen", "fehler", err)
|
||||
return nil
|
||||
}
|
||||
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
threshold := float32(0.7)
|
||||
result, err := pb.NewPointsClient(conn).Search(ctx, &pb.SearchPoints{
|
||||
CollectionName: config.Cfg.Qdrant.Collection,
|
||||
Vector: embResp.Data[0].Embedding,
|
||||
Limit: 3,
|
||||
WithPayload: &pb.WithPayloadSelector{
|
||||
SelectorOptions: &pb.WithPayloadSelector_Enable{Enable: true},
|
||||
},
|
||||
ScoreThreshold: &threshold,
|
||||
Filter: &pb.Filter{
|
||||
Must: []*pb.Condition{{
|
||||
ConditionOneOf: &pb.Condition_Field{
|
||||
Field: &pb.FieldCondition{
|
||||
Key: "type",
|
||||
Match: &pb.Match{
|
||||
MatchValue: &pb.Match_Keyword{Keyword: "email_triage"},
|
||||
},
|
||||
},
|
||||
},
|
||||
}},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
slog.Warn("[Triage] RAG-Suche fehlgeschlagen", "fehler", err)
|
||||
return nil
|
||||
}
|
||||
|
||||
var results []TriageResult
|
||||
for _, hit := range result.Result {
|
||||
text := hit.Payload["text"].GetStringValue()
|
||||
if text == "" {
|
||||
continue
|
||||
}
|
||||
results = append(results, TriageResult{Text: text, Score: hit.Score})
|
||||
}
|
||||
return results
|
||||
}
|
||||
|
||||
// ListRecent gibt die letzten N Triage-Entscheidungen aus Qdrant zurück.
|
||||
func ListRecent(limit uint32) ([]TriageResult, error) {
|
||||
if limit == 0 {
|
||||
limit = 10
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
pointsClient := pb.NewPointsClient(conn)
|
||||
|
||||
var results []TriageResult
|
||||
var offset *pb.PointId
|
||||
|
||||
for {
|
||||
req := &pb.ScrollPoints{
|
||||
CollectionName: config.Cfg.Qdrant.Collection,
|
||||
WithPayload: &pb.WithPayloadSelector{
|
||||
SelectorOptions: &pb.WithPayloadSelector_Enable{Enable: true},
|
||||
},
|
||||
Filter: triageFilter(),
|
||||
Limit: &limit,
|
||||
}
|
||||
if offset != nil {
|
||||
req.Offset = offset
|
||||
}
|
||||
|
||||
result, err := pointsClient.Scroll(ctx, req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("scroll fehlgeschlagen: %w", err)
|
||||
}
|
||||
|
||||
for _, pt := range result.Result {
|
||||
text := pt.Payload["text"].GetStringValue()
|
||||
if text != "" {
|
||||
results = append(results, TriageResult{Text: text})
|
||||
}
|
||||
}
|
||||
|
||||
if result.NextPageOffset == nil || uint32(len(results)) >= limit {
|
||||
break
|
||||
}
|
||||
offset = result.NextPageOffset
|
||||
}
|
||||
|
||||
if uint32(len(results)) > limit {
|
||||
results = results[:limit]
|
||||
}
|
||||
return results, nil
|
||||
}
|
||||
|
||||
// CorrectDecision sucht eine Triage-Entscheidung per Embedding-Suche und flippt sie (wichtig↔unwichtig).
|
||||
// Gibt eine Bestätigungsmeldung zurück.
|
||||
func CorrectDecision(query string) (string, error) {
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
embResp, err := embClient.CreateEmbeddings(ctx, openai.EmbeddingRequest{
|
||||
Input: []string{query},
|
||||
Model: openai.EmbeddingModel(config.Cfg.Embedding.Model),
|
||||
})
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("embedding: %w", err)
|
||||
}
|
||||
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
threshold := float32(0.8)
|
||||
result, err := pb.NewPointsClient(conn).Search(ctx, &pb.SearchPoints{
|
||||
CollectionName: config.Cfg.Qdrant.Collection,
|
||||
Vector: embResp.Data[0].Embedding,
|
||||
Limit: 1,
|
||||
WithPayload: &pb.WithPayloadSelector{
|
||||
SelectorOptions: &pb.WithPayloadSelector_Enable{Enable: true},
|
||||
},
|
||||
ScoreThreshold: &threshold,
|
||||
Filter: triageFilter(),
|
||||
})
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("suche fehlgeschlagen: %w", err)
|
||||
}
|
||||
if len(result.Result) == 0 {
|
||||
return "", fmt.Errorf("keine passende Triage-Entscheidung gefunden (Score < 0.8)")
|
||||
}
|
||||
|
||||
hit := result.Result[0]
|
||||
text := hit.Payload["text"].GetStringValue()
|
||||
|
||||
// Text parsen: "Email-Triage | Von: X | Betreff: Y | Entscheidung: Z"
|
||||
from, subject, wasImportant, err := parseTriageText(text)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// Alten Eintrag löschen — ID basiert jetzt auf from|subject
|
||||
oldID := triageID(from + "|" + subject)
|
||||
pointsClient := pb.NewPointsClient(conn)
|
||||
if err := deleteByUUID(ctx, pointsClient, oldID); err != nil {
|
||||
return "", fmt.Errorf("löschen fehlgeschlagen: %w", err)
|
||||
}
|
||||
|
||||
// Neuen Eintrag mit geflipptem Label speichern (Datum/Body aus altem Text nicht übertragen)
|
||||
newImportant := !wasImportant
|
||||
if err := StoreDecision(subject, from, "", "", newImportant); err != nil {
|
||||
return "", fmt.Errorf("speichern fehlgeschlagen: %w", err)
|
||||
}
|
||||
|
||||
oldLabel := "wichtig"
|
||||
newLabel := "unwichtig"
|
||||
if newImportant {
|
||||
oldLabel = "unwichtig"
|
||||
newLabel = "wichtig"
|
||||
}
|
||||
|
||||
msg := fmt.Sprintf("Korrigiert: '%s' von %s → %s (vorher: %s)", subject, from, newLabel, oldLabel)
|
||||
slog.Info("[Triage] Entscheidung korrigiert", "betreff", subject, "von", from, "neu", newLabel)
|
||||
return msg, nil
|
||||
}
|
||||
|
||||
// SearchExtended sucht ähnliche Triage-Entscheidungen mit niedrigerem Threshold und konfigurierbarem Limit.
|
||||
func SearchExtended(query string, limit uint64) ([]TriageResult, error) {
|
||||
ctx := context.Background()
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "api-key", config.Cfg.Qdrant.APIKey)
|
||||
|
||||
embClient := config.NewEmbeddingClient()
|
||||
embResp, err := embClient.CreateEmbeddings(ctx, openai.EmbeddingRequest{
|
||||
Input: []string{query},
|
||||
Model: openai.EmbeddingModel(config.Cfg.Embedding.Model),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("embedding: %w", err)
|
||||
}
|
||||
|
||||
conn := config.NewQdrantConn()
|
||||
defer conn.Close()
|
||||
|
||||
threshold := float32(0.5)
|
||||
result, err := pb.NewPointsClient(conn).Search(ctx, &pb.SearchPoints{
|
||||
CollectionName: config.Cfg.Qdrant.Collection,
|
||||
Vector: embResp.Data[0].Embedding,
|
||||
Limit: limit,
|
||||
WithPayload: &pb.WithPayloadSelector{
|
||||
SelectorOptions: &pb.WithPayloadSelector_Enable{Enable: true},
|
||||
},
|
||||
ScoreThreshold: &threshold,
|
||||
Filter: triageFilter(),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("suche fehlgeschlagen: %w", err)
|
||||
}
|
||||
|
||||
var results []TriageResult
|
||||
for _, hit := range result.Result {
|
||||
text := hit.Payload["text"].GetStringValue()
|
||||
if text != "" {
|
||||
results = append(results, TriageResult{Text: text, Score: hit.Score})
|
||||
}
|
||||
}
|
||||
return results, nil
|
||||
}
|
||||
|
||||
// triageFilter gibt den Standard-Filter für Triage-Einträge zurück.
|
||||
func triageFilter() *pb.Filter {
|
||||
return &pb.Filter{
|
||||
Must: []*pb.Condition{{
|
||||
ConditionOneOf: &pb.Condition_Field{
|
||||
Field: &pb.FieldCondition{
|
||||
Key: "type",
|
||||
Match: &pb.Match{
|
||||
MatchValue: &pb.Match_Keyword{Keyword: "email_triage"},
|
||||
},
|
||||
},
|
||||
},
|
||||
}},
|
||||
}
|
||||
}
|
||||
|
||||
// parseTriageText extrahiert Von, Betreff und Entscheidung aus dem gespeicherten Text.
|
||||
// Unterstützt sowohl das alte Format (4 Felder) als auch das neue Format mit optionalem Datum und Body.
|
||||
// Altes Format: "Email-Triage | Von: X | Betreff: Y | Entscheidung: Z"
|
||||
// Neues Format: "Email-Triage | Von: X | Betreff: Y | Datum: D | Entscheidung: Z\nBody: ..."
|
||||
func parseTriageText(text string) (from, subject string, isImportant bool, err error) {
|
||||
// Body-Zeile abtrennen – nur die Header-Zeile parsen
|
||||
headerLine := text
|
||||
if idx := strings.Index(text, "\nBody:"); idx >= 0 {
|
||||
headerLine = text[:idx]
|
||||
}
|
||||
|
||||
parts := strings.Split(headerLine, " | ")
|
||||
if len(parts) < 4 {
|
||||
return "", "", false, fmt.Errorf("ungültiges Triage-Format: %s", text)
|
||||
}
|
||||
from = strings.TrimPrefix(parts[1], "Von: ")
|
||||
subject = strings.TrimPrefix(parts[2], "Betreff: ")
|
||||
|
||||
// Letztes Feld enthält immer "Entscheidung: X", egal ob Datum dazwischen steht
|
||||
lastPart := parts[len(parts)-1]
|
||||
decision := strings.TrimPrefix(lastPart, "Entscheidung: ")
|
||||
isImportant = !strings.Contains(decision, "unwichtig")
|
||||
return from, subject, isImportant, nil
|
||||
}
|
||||
|
||||
// deleteByUUID löscht einen einzelnen Punkt aus Qdrant anhand seiner UUID.
|
||||
func deleteByUUID(ctx context.Context, client pb.PointsClient, uuid string) error {
|
||||
wait := true
|
||||
_, err := client.Delete(ctx, &pb.DeletePoints{
|
||||
CollectionName: config.Cfg.Qdrant.Collection,
|
||||
Points: &pb.PointsSelector{
|
||||
PointsSelectorOneOf: &pb.PointsSelector_Points{
|
||||
Points: &pb.PointsIdsList{
|
||||
Ids: []*pb.PointId{{
|
||||
PointIdOptions: &pb.PointId_Uuid{Uuid: uuid},
|
||||
}},
|
||||
},
|
||||
},
|
||||
},
|
||||
Wait: &wait,
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
// triageID erzeugt eine deterministische UUID aus einem Schlüssel.
|
||||
// Für Email-Triage sollte der Schlüssel "from|subject" sein, damit
|
||||
// Re-Processing derselben Email (mit ggf. anderem Body/Datum) upsert statt insert auslöst.
|
||||
func triageID(key string) string {
|
||||
hash := sha256.Sum256([]byte("email_triage:" + key))
|
||||
return hex.EncodeToString(hash[:16])
|
||||
}
|
||||
16
tasks.json
Normal file
16
tasks.json
Normal file
@@ -0,0 +1,16 @@
|
||||
[
|
||||
{
|
||||
"id": "1773950110942000154",
|
||||
"text": "Synology DSM Update durchfuehren",
|
||||
"done": true,
|
||||
"created_at": "2026-03-19T20:55:10.942001964+01:00",
|
||||
"done_at": "2026-03-20T20:56:51.054077698+01:00"
|
||||
},
|
||||
{
|
||||
"id": "1773950110942351936",
|
||||
"text": "Zahnarzt Termin bestaetigen",
|
||||
"done": true,
|
||||
"created_at": "2026-03-19T20:55:10.942353012+01:00",
|
||||
"done_at": "2026-03-20T20:58:27.785644744+01:00"
|
||||
}
|
||||
]
|
||||
134
test-integration.sh
Executable file
134
test-integration.sh
Executable file
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-integration.sh – Prüft Erreichbarkeit aller externen Dienste
|
||||
# Kann manuell oder automatisch beim Bot-Start aufgerufen werden.
|
||||
#
|
||||
# Exit-Code: 0 = alle Dienste OK, 1 = mindestens ein Dienst nicht erreichbar
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
CONFIG_FILE="${1:-./config.yml}"
|
||||
TIMEOUT=5
|
||||
ERRORS=0
|
||||
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
ok() { echo -e " ${GREEN}✓${NC} $*"; }
|
||||
fail() { echo -e " ${RED}✗${NC} $*"; ((ERRORS++)); }
|
||||
warn() { echo -e " ${YELLOW}⚠${NC} $*"; }
|
||||
|
||||
# ── Konfiguration lesen ──────────────────────────────────────────────────────
|
||||
if [[ ! -f "$CONFIG_FILE" ]]; then
|
||||
echo "❌ config.yml nicht gefunden: $CONFIG_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
read_yaml() {
|
||||
python3 -c "
|
||||
import yaml, sys
|
||||
with open('$CONFIG_FILE') as f:
|
||||
cfg = yaml.safe_load(f)
|
||||
keys = '$1'.split('.')
|
||||
val = cfg
|
||||
for k in keys:
|
||||
val = val.get(k, '') if isinstance(val, dict) else ''
|
||||
print(val or '')
|
||||
" 2>/dev/null || echo ""
|
||||
}
|
||||
|
||||
QDRANT_HOST=$(read_yaml qdrant.host)
|
||||
QDRANT_PORT=$(read_yaml qdrant.port)
|
||||
CHAT_URL=$(read_yaml chat.url)
|
||||
EMB_URL=$(read_yaml embedding.url)
|
||||
IMAP_HOST=$(read_yaml email.host)
|
||||
IMAP_PORT=$(read_yaml email.port)
|
||||
|
||||
echo ""
|
||||
echo "🔍 Integrations-Diagnose"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
# ── Qdrant ───────────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "📦 Qdrant (${QDRANT_HOST}:${QDRANT_PORT})"
|
||||
if nc -z -w "$TIMEOUT" "$QDRANT_HOST" "$QDRANT_PORT" 2>/dev/null; then
|
||||
ok "TCP-Verbindung erfolgreich"
|
||||
else
|
||||
fail "TCP-Verbindung fehlgeschlagen (${QDRANT_HOST}:${QDRANT_PORT})"
|
||||
fi
|
||||
|
||||
# ── LocalAI (Chat) ───────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "🤖 LocalAI Chat (${CHAT_URL})"
|
||||
if [[ -n "$CHAT_URL" ]]; then
|
||||
MODELS_URL="${CHAT_URL%/v1*}/v1/models"
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time "$TIMEOUT" "$MODELS_URL" 2>/dev/null || echo "000")
|
||||
if [[ "$HTTP_CODE" == "200" ]]; then
|
||||
ok "HTTP ${HTTP_CODE} – erreichbar"
|
||||
elif [[ "$HTTP_CODE" == "000" ]]; then
|
||||
fail "Keine Verbindung (Timeout/Refused)"
|
||||
else
|
||||
warn "HTTP ${HTTP_CODE} – unerwartet aber erreichbar"
|
||||
fi
|
||||
else
|
||||
warn "chat.url nicht konfiguriert"
|
||||
fi
|
||||
|
||||
# ── LocalAI (Embedding) ──────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "🔢 LocalAI Embedding (${EMB_URL})"
|
||||
if [[ -n "$EMB_URL" && "$EMB_URL" != "$CHAT_URL" ]]; then
|
||||
MODELS_URL="${EMB_URL%/v1*}/v1/models"
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time "$TIMEOUT" "$MODELS_URL" 2>/dev/null || echo "000")
|
||||
if [[ "$HTTP_CODE" == "200" ]]; then
|
||||
ok "HTTP ${HTTP_CODE} – erreichbar"
|
||||
elif [[ "$HTTP_CODE" == "000" ]]; then
|
||||
fail "Keine Verbindung (Timeout/Refused)"
|
||||
else
|
||||
warn "HTTP ${HTTP_CODE}"
|
||||
fi
|
||||
else
|
||||
ok "Gleicher Endpunkt wie Chat – übersprungen"
|
||||
fi
|
||||
|
||||
# ── IMAP ────────────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "📧 IMAP (${IMAP_HOST}:${IMAP_PORT})"
|
||||
if [[ -n "$IMAP_HOST" && -n "$IMAP_PORT" ]]; then
|
||||
if nc -z -w "$TIMEOUT" "$IMAP_HOST" "$IMAP_PORT" 2>/dev/null; then
|
||||
ok "TCP-Verbindung erfolgreich"
|
||||
else
|
||||
fail "TCP-Verbindung fehlgeschlagen (${IMAP_HOST}:${IMAP_PORT})"
|
||||
fi
|
||||
else
|
||||
warn "IMAP nicht konfiguriert – übersprungen"
|
||||
fi
|
||||
|
||||
# ── Unit-Tests ───────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "🧪 Go Unit-Tests"
|
||||
if go test ./... -count=1 -timeout 30s 2>&1 | grep -E "^(ok|FAIL|---)" | while read -r line; do
|
||||
if echo "$line" | grep -q "^ok"; then
|
||||
ok "$line"
|
||||
elif echo "$line" | grep -q "^FAIL\|^--- FAIL"; then
|
||||
fail "$line"
|
||||
fi
|
||||
done; then
|
||||
:
|
||||
fi
|
||||
# Prüfe Exit-Code der Tests separat
|
||||
if ! go test ./... -count=1 -timeout 30s > /dev/null 2>&1; then
|
||||
((ERRORS++))
|
||||
fi
|
||||
|
||||
# ── Ergebnis ────────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
if [[ "$ERRORS" -eq 0 ]]; then
|
||||
echo -e "${GREEN}✅ Alle Checks bestanden${NC}"
|
||||
exit 0
|
||||
else
|
||||
echo -e "${RED}❌ ${ERRORS} Check(s) fehlgeschlagen${NC}"
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user