Part 4 of 6

Memory Systems & Knowledge Management

Configure persistent memory, build custom knowledge bases, set up private web search with SearXNG, and optimize memory for long-running agents.

14 minutes

Intermediate

What separates Agent Zero from stateless chatbots is its ability to remember. It learns from interactions, stores solutions to problems it has solved, and builds a persistent understanding of your preferences and context.

Memory Architecture Overview

Agent Zero uses a hybrid memory system that combines several storage mechanisms:

┌─────────────────────────────────────────────────────────────┐
│                    Agent Zero Memory                         │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐  │
│  │   Short-Term    │  │   Long-Term     │  │  Knowledge  │  │
│  │    Memory       │  │    Memory       │  │    Base     │  │
│  │                 │  │                 │  │             │  │
│  │ • Current       │  │ • Facts         │  │ • Documents │  │
│  │   conversation  │  │ • Solutions     │  │ • Custom    │  │
│  │ • Active        │  │ • Fragments     │  │   data      │  │
│  │   context       │  │ • Preferences   │  │ • Imported  │  │
│  │                 │  │                 │  │   files     │  │
│  └────────┬────────┘  └────────┬────────┘  └──────┬──────┘  │
│           │                    │                   │         │
│           └────────────────────┼───────────────────┘         │
│                                ▼                             │
│                    ┌─────────────────────┐                   │
│                    │  Vector Database    │                   │
│                    │   (Embeddings)      │                   │
│                    └─────────────────────┘                   │
└─────────────────────────────────────────────────────────────┘

Short-Term Memory

The current conversation context. Managed automatically through context window handling. Cleared between sessions unless explicitly saved.

Long-Term Memory

Persistent storage for information the agent learns over time:

Facts — Declarative knowledge ("User prefers Python over JavaScript")
Solutions — Successful approaches to problems ("To fix X, do Y")
Fragments — Reusable code snippets, templates, and patterns

Knowledge Base

Your custom document library that agents can search and reference.

All long-term storage uses vector embeddings for semantic search—agents find relevant memories by meaning, not just keywords.

Memory Storage Location

Agent Zero stores memory data in the memory/ directory:

Terminal

ls -la ~/agent-zero/memory/

Structure:

memory/
├── facts/           # Learned facts and preferences
├── solutions/       # Problem-solution pairs
├── fragments/       # Code snippets and templates
└── metadata.json    # Memory index and statistics

This directory grows over time. For production deployments, monitor its size:

Terminal

du -sh ~/agent-zero/memory/

Configuring Memory Settings

Memory behavior is controlled through environment variables. Edit your .env file:

Terminal

nano ~/agent-zero/.env

Key Memory Settings

.env

# Memory Configuration
MEMORY_ENABLED=true
MEMORY_AUTO_SAVE=true
MEMORY_SAVE_INTERVAL=300  # Save every 5 minutes

# Vector Database Settings
MEMORY_VECTOR_DB=chroma      # Options: chroma, faiss, qdrant
MEMORY_EMBEDDING_MODEL=nomic-embed-text

# Memory Retrieval
MEMORY_MAX_RESULTS=10        # Max memories returned per query
MEMORY_SIMILARITY_THRESHOLD=0.7  # Minimum relevance score (0-1)

Context Window Management

Agent Zero automatically manages context to stay within model limits:

.env

# Context Management
CONTEXT_WINDOW_SIZE=32000    # Match your model's context limit
CONTEXT_COMPRESSION_ENABLED=true
CONTEXT_COMPRESSION_THRESHOLD=0.8  # Compress when 80% full

When context approaches the limit, Agent Zero:

Summarizes older conversation segments
Moves important information to long-term memory
Keeps recent exchanges and critical context intact

This allows extended conversations without losing important information.

Auto Memory Feature

Agent Zero's auto memory system automatically identifies and stores useful information from conversations. Enable it:

.env

# Auto Memory
AUTO_MEMORY_ENABLED=true
AUTO_MEMORY_FACTS=true       # Store learned facts
AUTO_MEMORY_SOLUTIONS=true   # Store problem solutions
AUTO_MEMORY_FRAGMENTS=true   # Store code snippets

When enabled, the agent automatically:

Extracts facts from statements like "I work at Acme Corp" or "My server runs Ubuntu 24.04"
Saves successful solutions when you confirm something worked
Stores useful code snippets for future reuse

You can also manually trigger memory storage:

"Remember that our production database is PostgreSQL 16 running on port 5432."

Or ask the agent to recall:

"What do you remember about my database setup?"

Building a Knowledge Base

The knowledge base lets you give Agent Zero access to custom documents—internal documentation, code repositories, reference materials, or any text-based information.

Knowledge Directory Structure

Terminal

ls -la ~/agent-zero/knowledge/

knowledge/
├── documents/       # Your imported documents
├── embeddings/      # Generated vector embeddings
└── index.json       # Document metadata

Importing Documents

Place documents in the knowledge directory:

Terminal

# Create subdirectories for organization
mkdir -p ~/agent-zero/knowledge/documents/{docs,code,references}

# Copy your files
cp /path/to/your/documentation/*.md ~/agent-zero/knowledge/documents/docs/
cp /path/to/your/codebase/*.py ~/agent-zero/knowledge/documents/code/

Supported formats:

Markdown (.md)
Plain text (.txt)
Python (.py)
JavaScript (.js)
JSON (.json)
YAML (.yaml, .yml)
PDF (.pdf) — requires additional dependencies

Processing Documents

After adding documents, process them to generate embeddings:

Terminal

cd ~/agent-zero
source venv/bin/activate
python -c "from python.helpers.knowledge import process_knowledge; process_knowledge()"

Or trigger processing through the web UI by asking: "Reindex your knowledge base."

Processing time depends on document volume and your embedding model. Local models like nomic-embed-text process roughly 100 pages per minute on a standard VPS.

Querying the Knowledge Base

Once indexed, agents automatically search the knowledge base for relevant information:

"What does our documentation say about API authentication?"

"Find the function that handles user registration in our codebase."

"Summarize our deployment procedures."

The agent searches by semantic similarity, finding relevant content even when your query uses different terminology than the source documents.

Keeping Knowledge Updated

For dynamic content, set up periodic reindexing:

~/agent-zero/update-knowledge.sh

#!/bin/bash
cd ~/agent-zero
source venv/bin/activate

# Pull latest docs from git (example)
cd knowledge/documents/docs
git pull origin main
cd ~/agent-zero

# Reprocess knowledge base
python -c "from python.helpers.knowledge import process_knowledge; process_knowledge()"

echo "Knowledge base updated: $(date)"

Make it executable and schedule:

Terminal

chmod +x ~/agent-zero/update-knowledge.sh

# Add to crontab - runs daily at 3 AM
crontab -e

Add:

0 3 * * * /home/agentzero/agent-zero/update-knowledge.sh >> /var/log/knowledge-update.log 2>&1

SearXNG Integration for Private Web Search

Agent Zero can search the web for current information. By default, it uses public search engines, but you can deploy SearXNG for fully private searches that don't leak queries to third parties.

Deploy SearXNG with Docker

Terminal

mkdir -p ~/searxng
nano ~/searxng/docker-compose.yml

docker-compose.yml

version: '3.8'

services:
  searxng:
    image: searxng/searxng:latest
    container_name: searxng
    restart: unless-stopped
    ports:
      - "8080:8080"
    volumes:
      - ./searxng:/etc/searxng:rw
    environment:
      - SEARXNG_BASE_URL=http://localhost:8080/
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID

Create the settings directory and configuration:

Terminal

mkdir -p ~/searxng/searxng
nano ~/searxng/searxng/settings.yml

settings.yml

use_default_settings: true

server:
  secret_key: "generate-a-random-string-here"
  bind_address: "0.0.0.0"
  
search:
  safe_search: 0
  autocomplete: ""
  
engines:
  - name: google
    engine: google
    disabled: false
  - name: duckduckgo
    engine: duckduckgo
    disabled: false
  - name: bing
    engine: bing
    disabled: false
  - name: wikipedia
    engine: wikipedia
    disabled: false

outgoing:
  request_timeout: 5.0
  max_request_timeout: 15.0

Generate a secret key:

Terminal

sed -i "s/generate-a-random-string-here/$(openssl rand -hex 32)/" ~/searxng/searxng/settings.yml

Start SearXNG:

Terminal

cd ~/searxng
docker compose up -d

Verify it's running:

Terminal

curl http://localhost:8080/search?q=test&format=json | head -50

Configure Agent Zero to Use SearXNG

.env

# Web Search Configuration
WEB_SEARCH_ENABLED=true
WEB_SEARCH_ENGINE=searxng
SEARXNG_URL=http://localhost:8080

Restart Agent Zero:

Terminal

sudo systemctl restart agent-zero

Test web search: "Search the web for the latest Python release notes."

All searches now route through your private SearXNG instance—no queries sent to Google, Bing, or other third-party services.

SearXNG Engine Selection

For technical searches, enable specialized engines in settings.yml:

settings.yml

engines:
  # General
  - name: google
    disabled: false
  - name: duckduckgo
    disabled: false
    
  # Technical
  - name: github
    engine: github
    disabled: false
  - name: stackoverflow
    engine: stackoverflow
    disabled: false
  - name: dockerhub
    engine: docker_hub
    disabled: false
    
  # Documentation
  - name: archwiki
    engine: archlinux
    disabled: false
  - name: mdn
    engine: mdn
    disabled: false

Restart SearXNG after changes:

Terminal

cd ~/searxng
docker compose restart

Memory Optimization

As your Agent Zero instance accumulates memories, optimize for performance and relevance.

Pruning Stale Memories

Old or rarely-accessed memories can be pruned:

Terminal

cd ~/agent-zero
source venv/bin/activate
python -c "
from python.helpers.memory import prune_memories
# Remove memories older than 90 days with less than 3 accesses
prune_memories(max_age_days=90, min_access_count=3)
"

Or schedule automatic pruning:

# Add to crontab - runs weekly on Sunday at 4 AM
0 4 * * 0 cd ~/agent-zero && source venv/bin/activate && python -c "from python.helpers.memory import prune_memories; prune_memories(max_age_days=90, min_access_count=3)"

Memory Consolidation

Consolidate similar memories to reduce redundancy:

"Review your memories and consolidate any duplicate or redundant information."

The agent will identify overlapping memories and merge them.

Backing Up Memory

Memory represents learned knowledge—back it up regularly:

~/agent-zero/backup-memory.sh

#!/bin/bash
BACKUP_DIR="/home/agentzero/backups/memory"
DATE=$(date +%Y%m%d_%H%M%S)

mkdir -p $BACKUP_DIR

# Create compressed backup
tar -czf $BACKUP_DIR/memory_$DATE.tar.gz -C ~/agent-zero memory/

# Keep only last 30 days of backups
find $BACKUP_DIR -name "memory_*.tar.gz" -mtime +30 -delete

echo "Memory backup completed: memory_$DATE.tar.gz"

Terminal

chmod +x ~/agent-zero/backup-memory.sh

# Add to crontab - daily at 2 AM
crontab -e

0 2 * * * /home/agentzero/agent-zero/backup-memory.sh >> /var/log/memory-backup.log 2>&1

Restoring Memory

To restore from backup:

Terminal

# Stop Agent Zero
sudo systemctl stop agent-zero

# Remove current memory
rm -rf ~/agent-zero/memory/*

# Extract backup
tar -xzf ~/backups/memory/memory_20250115_020000.tar.gz -C ~/agent-zero/

# Restart
sudo systemctl start agent-zero

Memory Privacy Considerations

Memory persists sensitive information. Consider these practices:

Sensitive Data Handling

Ask the agent to forget sensitive data:

"Forget any API keys or passwords I may have shared."

Memory Isolation

For multi-user scenarios (covered in Part 5), each agent can have isolated memory.

Encryption at Rest

Encrypt the memory directory for additional security:

Terminal

# Install encfs
sudo apt install encfs -y

# Create encrypted directory
encfs ~/agent-zero/.memory_encrypted ~/agent-zero/memory

# On system startup, mount before starting Agent Zero

Audit Memory Contents

"List all facts you've stored about me."

"Show me what solutions you've saved."

Monitoring Memory Usage

Track memory growth and performance:

Terminal

# Memory directory size
du -sh ~/agent-zero/memory/

# Count memories by type
find ~/agent-zero/memory/facts -type f | wc -l
find ~/agent-zero/memory/solutions -type f | wc -l
find ~/agent-zero/memory/fragments -type f | wc -l

# Knowledge base size
du -sh ~/agent-zero/knowledge/

For detailed statistics, ask the agent: "Show me statistics about your memory usage."

What's Next

Your Agent Zero instance now has a robust memory system that learns and retains knowledge across sessions, backed by private web search and a custom knowledge base. In Part 5: Multi-Agent Systems & Custom Tools, we'll explore:

Configuring agent hierarchies (superior/subordinate relationships)
Creating specialized agents for different tasks
Building custom Instruments (callable functions)
Modifying system prompts for custom behavior
MCP (Model Context Protocol) integration

Multi-agent architectures let you build sophisticated AI workflows where specialized agents collaborate on complex tasks.

Part 3: LLM Configuration Part 5: Multi-Agent Systems