llm mail integration
This commit is contained in:
Christoph K.
166
CLAUDE.md
166
CLAUDE.md
@@ -1,50 +1,116 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
**my-brain-importer** is a personal RAG (Retrieval-Augmented Generation) system written in Go. It ingests Markdown notes and image descriptions into a Qdrant vector database and answers questions using a local LLM via LocalAI.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Build all binaries (Linux + Windows cross-compile)
|
||||
bash build.sh
|
||||
|
||||
# Run directly without building
|
||||
go run ./cmd/ingest/
|
||||
go run ./cmd/ask/ "your question here"
|
||||
|
||||
# Build individual binaries
|
||||
go build ./cmd/ingest/
|
||||
go build ./cmd/ask/
|
||||
|
||||
# Run tests
|
||||
go test ./...
|
||||
|
||||
# Tidy dependencies
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
Binaries are output to `./bin/`. The `config.yml` file must exist in the working directory at runtime.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two CLI tools share a common internal library:
|
||||
|
||||
**`cmd/ingest/`** → `internal/brain/ingest.go` + `internal/brain/ingest_json.go`
|
||||
- Markdown mode: recursively finds `.md` files, splits by `# `/`## ` headings, chunks long sections (max 800 chars) by paragraphs, embeds in batches of 10, upserts to Qdrant
|
||||
- JSON mode (when arg ends in `.json`): imports image description records with `file_path`, `file_name`, `description` fields
|
||||
|
||||
**`cmd/ask/`** → `internal/brain/ask.go`
|
||||
- Embeds the question, searches Qdrant (top-k, score threshold 0.5), deduplicates by text content, streams LLM response constrained to retrieved context
|
||||
|
||||
**`internal/config/config.go`** initializes all clients: gRPC connection to Qdrant and OpenAI-compatible HTTP clients for embeddings and chat (both point to LocalAI).
|
||||
|
||||
## Key Patterns
|
||||
|
||||
- **Deterministic IDs**: SHA256 of `source:text` — upserting the same content is always idempotent
|
||||
- **Excluded directories**: `05_Agents` and `.git` are skipped during markdown ingest
|
||||
- **config.yml** must be present in the working directory; defines Qdrant host/port/api_key, embedding model + dimensions, chat model, `brain_root` path, and `top_k`
|
||||
- External services: Qdrant (gRPC port 6334) and LocalAI (HTTP, OpenAI-compatible API)
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
**my-brain-importer** is a personal AI assistant and RAG (Retrieval-Augmented Generation) system written in Go. It ingests Markdown notes into a Qdrant vector database, answers questions using a local LLM (LocalAI), and is primarily controlled via Discord. A background daemon sends proactive email summaries and task reminders.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Build all binaries (Linux + Windows cross-compile)
|
||||
bash build.sh
|
||||
|
||||
# Primary entry point: Discord Bot (includes daemon)
|
||||
go run ./cmd/discord/
|
||||
|
||||
# CLI tools
|
||||
go run ./cmd/ingest/ # Markdown importieren
|
||||
go run ./cmd/ingest/ bild.json # JSON importieren
|
||||
go run ./cmd/ask/ "your question here" # Frage stellen
|
||||
|
||||
# Test: IMAP-Verbindung
|
||||
go run ./cmd/mailtest/
|
||||
|
||||
# Test: LLM-Zusammenfassung ohne IMAP
|
||||
go run ./cmd/mailtest/ -llm-only
|
||||
|
||||
# Run tests
|
||||
go test ./...
|
||||
|
||||
# Tidy dependencies
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
Binaries are output to `./bin/`. The `config.yml` file must exist in the working directory at runtime.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Discord (primäres Interface)
|
||||
↓ Slash-Commands + @Mention
|
||||
cmd/discord/main.go
|
||||
├── internal/agents/research/ → brain.AskQuery()
|
||||
├── internal/agents/memory/ → brain.RunIngest(), brain.IngestChatMessage()
|
||||
├── internal/agents/task/ → tasks.json (atomisches JSON)
|
||||
└── internal/agents/tool/email/ → IMAP + LLM-Zusammenfassung
|
||||
↓
|
||||
[Daemon-Goroutine] startDaemon()
|
||||
├── Email-Check (alle N min) → #localagent Discord-Channel
|
||||
└── Task-Reminder (täglich) → #localagent Discord-Channel
|
||||
|
||||
cmd/ingest/ + cmd/ask/ (CLI-Tools, direkt nutzbar)
|
||||
↓
|
||||
internal/brain/ (Core RAG-Logik, unverändert)
|
||||
↓
|
||||
Qdrant (gRPC) + LocalAI (HTTP, OpenAI-kompatibel)
|
||||
```
|
||||
|
||||
### Packages
|
||||
|
||||
| Package | Zweck |
|
||||
|---------|-------|
|
||||
| `cmd/discord/` | Discord Bot + Daemon (primärer Einstiegspunkt) |
|
||||
| `cmd/ask/` | CLI-Tool: Fragen stellen |
|
||||
| `cmd/ingest/` | CLI-Tool: Markdown/JSON importieren |
|
||||
| `cmd/mailtest/` | Testprogramm: IMAP + LLM-Test |
|
||||
| `internal/brain/` | Core RAG: Embeddings, Qdrant-Suche, LLM-Streaming |
|
||||
| `internal/config/` | Konfiguration + Client-Initialisierung (globale `Cfg`) |
|
||||
| `internal/agents/` | Agent-Interface (`Request`/`Response`) |
|
||||
| `internal/agents/research/` | Research-Agent: Wissensdatenbank-Abfragen |
|
||||
| `internal/agents/memory/` | Memory-Agent: Ingest + Chat-Speicherung |
|
||||
| `internal/agents/task/` | Task-Agent: Aufgabenverwaltung (tasks.json) |
|
||||
| `internal/agents/tool/` | Tool-Dispatcher |
|
||||
| `internal/agents/tool/email/` | IMAP-Client + LLM-Email-Analyse |
|
||||
|
||||
### Discord Commands
|
||||
|
||||
| Slash-Command | @Mention | Funktion |
|
||||
|---------------|----------|---------|
|
||||
| `/ask`, `/research` | `@bot <frage>` | Wissensdatenbank abfragen |
|
||||
| `/asknobrain` | – | Direkt an LLM (kein RAG) |
|
||||
| `/memory store` | `@bot remember <text>` | Text speichern |
|
||||
| `/memory ingest` | `@bot ingest` | Markdown neu einlesen |
|
||||
| `/task add/list/done/delete` | `@bot task <aktion>` | Aufgaben verwalten |
|
||||
| `/email summary/unread/remind` | `@bot email <aktion>` | Email-Analyse |
|
||||
| `/remember` | – | Alias für `/memory store` |
|
||||
| `/ingest` | – | Alias für `/memory ingest` |
|
||||
|
||||
## Key Patterns
|
||||
|
||||
- **Deterministic IDs**: SHA256 of `source:text` — upserting the same content is always idempotent
|
||||
- **Excluded directories**: `05_Agents` and `.git` are skipped during markdown ingest
|
||||
- **config.yml** must be present in the working directory at runtime
|
||||
- **Agent Interface**: alle Agenten implementieren `Handle(Request) Response`
|
||||
- **Defer-first Pattern**: Discord-Handlers senden sofort Defer, dann berechnen — nie >3s warten
|
||||
- **LLM-Fallback**: Email-Zusammenfassung zeigt Rohliste wenn LLM nicht erreichbar
|
||||
- **Daemon**: läuft als Goroutine im Discord-Bot-Prozess (`startDaemon()`)
|
||||
- **config.Cfg**: globale Variable — bei Tests muss `config.LoadConfig()` aufgerufen oder Cfg direkt gesetzt werden
|
||||
|
||||
## Model Limitations
|
||||
|
||||
Das konfigurierte Modell (`Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-GGUF`) hat folgende Grenzen:
|
||||
|
||||
- **Kontextfenster**: Begrenzt — bei sehr langen Email-Listen oder vielen Chunks kann die Antwort abgeschnitten werden (`MaxTokens: 600`)
|
||||
- **Latenz**: Lokales Modell auf NAS — Antwortzeiten variieren (5–60s je nach Last)
|
||||
- **Encoding**: Betreffzeilen in `windows-1252` (Strato) werden nicht dekodiert — das LLM interpretiert sie trotzdem meist korrekt
|
||||
- **Halluzinationen**: Das Modell kann bei unklarem Kontext eigenes Wissen einmischen — ist im System-Prompt mit "Aus meinem Wissen:" markiert
|
||||
- **Streaming-Timeout**: Kein expliziter Timeout auf LLM-Calls — bei Hänger wird Discord-Interaktion erst nach 15min ungültig
|
||||
|
||||
## External Services
|
||||
|
||||
- **Qdrant** (`192.168.1.4:6334`) — Vektordatenbank, gRPC
|
||||
- **LocalAI** (`192.168.1.118:8080`) — lokales LLM, OpenAI-kompatibles API
|
||||
- **Strato IMAP** (`imap.strato.de:143`, STARTTLS) — Email-Abruf
|
||||
- **Discord** — primäres Interface (Bot-Token in `config.yml`)
|
||||
|
||||
Reference in New Issue
Block a user