Agent Zero Mastery Series
    Part 5 of 6

    Multi-Agent Systems & Custom Tools

    Build agent hierarchies, create custom instruments, modify system prompts for specialized behavior, and integrate MCP for extended capabilities.

    16 minutes
    Intermediate to Advanced

    Agent Zero's real power emerges when you move beyond single-agent interactions. The framework supports hierarchical multi-agent systems where specialized agents collaborate on complex tasks, and a fully customizable tool system that lets you extend agent capabilities without touching core code.

    Understanding the Agent Hierarchy

    Agent Zero uses a superior-subordinate model for task delegation:

    ┌─────────────────────────────────────────────────────────┐
    │                         User                             │
    │                           │                              │
    │                           ▼                              │
    │                    ┌─────────────┐                       │
    │                    │   Agent 0   │ ◄── Primary Agent     │
    │                    │  (Superior) │                       │
    │                    └──────┬──────┘                       │
    │                           │                              │
    │            ┌──────────────┼──────────────┐               │
    │            ▼              ▼              ▼               │
    │     ┌─────────────┐ ┌─────────────┐ ┌─────────────┐     │
    │     │   Agent 1   │ │   Agent 2   │ │   Agent 3   │     │
    │     │  (Research) │ │   (Code)    │ │  (Review)   │     │
    │     └─────────────┘ └──────┬──────┘ └─────────────┘     │
    │                            │                             │
    │                            ▼                             │
    │                     ┌─────────────┐                      │
    │                     │   Agent 4   │                      │
    │                     │   (Test)    │                      │
    │                     └─────────────┘                      │
    └─────────────────────────────────────────────────────────┘

    Key Concepts

    • Agent 0 is always the primary agent that communicates with you
    • Any agent can spawn subordinate agents to handle subtasks
    • Subordinates report results back to their superior
    • Each agent maintains its own context, keeping focus clean
    • The hierarchy can go multiple levels deep

    This architecture enables complex workflows: Agent 0 receives your request, breaks it into subtasks, delegates to specialists, and synthesizes results.

    Configuring Multi-Agent Behavior

    Multi-agent settings are controlled through environment variables. Edit your .env file:

    Terminal
    nano ~/agent-zero/.env
    .env
    # Multi-Agent Configuration
    AGENT_MAX_DEPTH=5              # Maximum hierarchy depth
    AGENT_MAX_SUBORDINATES=10      # Max subordinates per agent
    AGENT_TIMEOUT=300              # Subordinate task timeout (seconds)
    AGENT_MEMORY_SHARED=false      # Share memory between agents

    AGENT_MAX_DEPTH — Limits how deep the hierarchy can go. Deeper hierarchies handle more complex decomposition but increase latency and token usage.

    AGENT_MAX_SUBORDINATES — Prevents runaway agent spawning. Adjust based on your task complexity and resource availability.

    AGENT_MEMORY_SHARED — When true, all agents access the same memory pool. When false, each agent has isolated memory.

    Prompt-Level Control

    Agent delegation behavior is defined in the system prompt. View the default:

    Terminal
    cat ~/agent-zero/prompts/default/agent.system.md

    Spawning Subordinate Agents

    Agents spawn subordinates naturally when tasks warrant delegation. You can also explicitly request it:

    "Break this task into subtasks and use subordinate agents: Build a Python web scraper that extracts product prices from three e-commerce sites, stores them in a SQLite database, and generates a comparison report."

    Agent 0 might respond by:

    1. Creating Agent 1 to research the target site structures
    2. Creating Agent 2 to write the scraping code
    3. Creating Agent 3 to design the database schema
    4. Creating Agent 4 to build the reporting module
    5. Synthesizing all components into a working solution

    Controlling Delegation Verbosity

    By default, you see summarized subordinate activity. For debugging or learning, request detailed output or configure globally:

    .env
    AGENT_VERBOSE_SUBORDINATES=true

    Creating Custom Instruments

    Instruments are custom functions that agents can call. Unlike ad-hoc code execution, instruments are predefined, tested, and reusable.

    Instrument Structure

    Instruments live in the python/instruments/ directory:

    python/instruments/example_instrument.py
    from python.helpers.instrument import Instrument, Response
    
    class ExampleInstrument(Instrument):
        """
        Brief description of what this instrument does.
        The agent sees this docstring when deciding whether to use the instrument.
        """
        
        # Define parameters the agent must provide
        async def execute(self, parameter1: str, parameter2: int = 10) -> Response:
            """
            Execute the instrument.
            
            Args:
                parameter1: Description of first parameter
                parameter2: Description of second parameter (default: 10)
                
            Returns:
                Response object with result or error
            """
            try:
                # Your implementation here
                result = f"Processed {parameter1} with value {parameter2}"
                return Response(success=True, message=result)
            except Exception as e:
                return Response(success=False, message=str(e))

    Example: Server Health Instrument

    Terminal
    nano ~/agent-zero/python/instruments/server_health.py
    server_health.py
    from python.helpers.instrument import Instrument, Response
    import psutil
    import platform
    from datetime import datetime
    
    class ServerHealthInstrument(Instrument):
        """
        Check server health metrics including CPU, memory, disk usage, and uptime.
        Use this instrument when asked about server status, resource usage, or system health.
        """
        
        async def execute(self, detailed: bool = False) -> Response:
            """
            Get server health metrics.
            
            Args:
                detailed: If True, include per-CPU and per-disk breakdown
                
            Returns:
                Response with health metrics
            """
            try:
                # Basic metrics
                cpu_percent = psutil.cpu_percent(interval=1)
                memory = psutil.virtual_memory()
                disk = psutil.disk_usage('/')
                boot_time = datetime.fromtimestamp(psutil.boot_time())
                uptime = datetime.now() - boot_time
                
                health = {
                    "timestamp": datetime.now().isoformat(),
                    "hostname": platform.node(),
                    "cpu_percent": cpu_percent,
                    "memory_percent": memory.percent,
                    "memory_available_gb": round(memory.available / (1024**3), 2),
                    "disk_percent": disk.percent,
                    "disk_free_gb": round(disk.free / (1024**3), 2),
                    "uptime_days": uptime.days,
                    "uptime_hours": uptime.seconds // 3600
                }
                
                # Generate summary
                status = "healthy"
                warnings = []
                
                if cpu_percent > 80:
                    warnings.append(f"High CPU usage: {cpu_percent}%")
                    status = "warning"
                if memory.percent > 85:
                    warnings.append(f"High memory usage: {memory.percent}%")
                    status = "warning"
                if disk.percent > 90:
                    warnings.append(f"Low disk space: {disk.percent}% used")
                    status = "critical"
                
                health["status"] = status
                health["warnings"] = warnings
                
                return Response(
                    success=True, 
                    message=f"Server status: {status}",
                    data=health
                )
                
            except Exception as e:
                return Response(success=False, message=f"Health check failed: {str(e)}")

    Install the required dependency:

    Terminal
    cd ~/agent-zero
    source venv/bin/activate
    pip install psutil

    Restart and test:

    Terminal
    sudo systemctl restart agent-zero

    "Check the server health with detailed metrics."

    Registering Instruments

    Instruments are auto-discovered from the python/instruments/ directory. Ensure your file:

    1. Is a .py file in the correct directory
    2. Contains a class that inherits from Instrument
    3. Has a descriptive docstring (agents use this to decide when to use the instrument)
    4. Implements the execute method with typed parameters

    Customizing System Prompts

    The system prompt defines agent personality, capabilities, and behavior. Customizing it transforms a generic assistant into a specialized tool.

    Prompt File Structure

    Terminal
    ls ~/agent-zero/prompts/default/
    prompts/default/
    ├── agent.system.md      # Main system prompt
    ├── agent.tools.md       # Tool usage instructions
    ├── agent.formats.md     # Response formatting rules
    ├── memory.save.md       # Memory storage instructions
    ├── memory.recall.md     # Memory retrieval instructions
    └── ...

    Creating a Custom Prompt Profile

    Don't edit the defaults directly. Create a custom profile:

    Terminal
    cp -r ~/agent-zero/prompts/default ~/agent-zero/prompts/devops

    Edit your custom system prompt:

    Terminal
    nano ~/agent-zero/prompts/devops/agent.system.md

    Example: DevOps Specialist Profile

    agent.system.md
    # Agent Identity
    
    You are a DevOps specialist AI assistant focused on infrastructure automation, 
    system administration, and deployment pipelines. You have deep expertise in:
    
    - Linux system administration (Ubuntu, RHEL, Alpine)
    - Container technologies (Docker, Podman, Kubernetes)
    - CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins)
    - Infrastructure as Code (Terraform, Ansible, Pulumi)
    - Cloud platforms (AWS, GCP, Azure)
    - Monitoring and observability (Prometheus, Grafana, ELK)
    
    ## Communication Style
    
    - Be direct and technical; avoid unnecessary explanations of basic concepts
    - Provide working commands and configurations, not just descriptions
    - Include error handling and edge cases in scripts
    - Explain security implications of recommended actions
    - Default to infrastructure-as-code approaches over manual configuration
    
    ## Safety Rules
    
    - Never store credentials in code or logs
    - Always use the principle of least privilege
    - Verify backup existence before destructive operations
    - Test in staging-equivalent environments when possible
    - Alert the user to potential security issues proactively

    Activating Custom Prompts

    Set your custom profile in .env:

    .env
    PROMPT_PROFILE=devops

    Restart Agent Zero:

    Terminal
    sudo systemctl restart agent-zero

    Task-Specific Prompt Injection

    For one-off customization without changing profiles, inject context at the start of conversations:

    "For this conversation, act as a security auditor. Focus on identifying vulnerabilities, misconfigurations, and compliance issues. Be thorough and assume nothing is secure until verified. Now, audit the Nginx configuration I'm about to share."

    Building Specialized Agent Teams

    Combine custom prompts with multi-agent architecture to create specialized teams.

    Example: Code Review Pipeline

    Create agents that work together on code review:

    1. Static Analysis Agent

    prompts/code-analysis/agent.system.md

    Analyzes code for syntax errors, type issues, code style violations, complexity metrics, and potential bugs. Outputs structured findings with severity levels and line numbers.

    2. Security Review Agent

    prompts/security-review/agent.system.md

    Analyzes code for security vulnerabilities: injection risks, authentication flaws, sensitive data exposure, insecure dependencies, and OWASP Top 10 issues.

    3. Review Coordinator

    prompts/review-coordinator/agent.system.md

    Orchestrates code reviews by delegating to analysis and security specialist agents, collecting and deduplicating findings, and generating a unified review report.

    Use the coordinator as your primary agent, and it will delegate to specialists automatically.

    MCP Integration

    Model Context Protocol (MCP) allows Agent Zero to connect with external services and data sources through a standardized interface.

    Understanding MCP

    MCP servers expose capabilities that agents can discover and use:

    • Resources — Data sources the agent can read
    • Tools — Actions the agent can perform
    • Prompts — Templates for common interactions

    Configuring MCP Servers

    .env
    # MCP Configuration
    MCP_ENABLED=true
    MCP_SERVERS=filesystem,github,slack

    Example: Filesystem MCP Server

    The filesystem server gives agents controlled access to directories:

    Terminal
    # Install MCP filesystem server
    npm install -g @anthropic/mcp-server-filesystem
    .env
    # MCP Filesystem Server
    MCP_FILESYSTEM_ENABLED=true
    MCP_FILESYSTEM_PATHS=/home/agentzero/projects,/var/log
    MCP_FILESYSTEM_READ_ONLY=true

    Now agents can read files from allowed paths:

    "Read the contents of /home/agentzero/projects/myapp/config.yaml"

    Example: GitHub MCP Server

    Terminal
    npm install -g @anthropic/mcp-server-github
    .env
    # MCP GitHub Server
    MCP_GITHUB_ENABLED=true
    MCP_GITHUB_TOKEN=ghp_your_personal_access_token
    MCP_GITHUB_REPOS=myorg/myrepo,myorg/another-repo

    Agents can now:

    "Show me open pull requests in myorg/myrepo"

    "Create an issue in myorg/myrepo titled 'Bug: Login fails on mobile'"

    Building Custom MCP Servers

    For internal systems, build custom MCP servers:

    ~/agent-zero/mcp_servers/internal_api.py
    from mcp import Server, Tool, Resource
    import httpx
    import os
    
    server = Server("internal-api")
    
    @server.tool("get_user")
    async def get_user(user_id: str) -> dict:
        """Fetch user details from internal API."""
        async with httpx.AsyncClient() as client:
            response = await client.get(
                f"https://internal.example.com/api/users/{user_id}",
                headers={"Authorization": f"Bearer {os.environ['INTERNAL_API_KEY']}"}
            )
            return response.json()
    
    @server.tool("create_ticket")
    async def create_ticket(title: str, description: str, priority: str = "medium") -> dict:
        """Create a support ticket in the internal system."""
        async with httpx.AsyncClient() as client:
            response = await client.post(
                "https://internal.example.com/api/tickets",
                json={"title": title, "description": description, "priority": priority},
                headers={"Authorization": f"Bearer {os.environ['INTERNAL_API_KEY']}"}
            )
            return response.json()
    
    if __name__ == "__main__":
        server.run()

    Register in .env:

    .env
    MCP_CUSTOM_SERVERS=/home/agentzero/agent-zero/mcp_servers/internal_api.py

    Testing Multi-Agent Workflows

    Verify your multi-agent setup with increasingly complex tasks:

    Simple Delegation

    "Research the pros and cons of PostgreSQL vs MySQL for a high-traffic web application. Use a subordinate agent for each database."

    Multi-Step Workflow

    "1. Check current server health
    2. Review the nginx access logs for the past hour
    3. Identify any unusual traffic patterns
    4. Generate a summary report
    Use subordinate agents where appropriate."

    Full Pipeline

    "I need to deploy a new version of our application:
    1. Run the test suite
    2. Build the Docker image
    3. Push to registry
    4. Update the staging deployment
    5. Run smoke tests
    6. Report results
    Coordinate this with subordinate agents and report progress."

    What's Next

    Your Agent Zero deployment now supports sophisticated multi-agent workflows with custom tools and integrations. In Part 6: Production Hardening & Monitoring, we'll cover:

    • Reverse proxy configuration with Caddy
    • SSL/TLS and authentication
    • Docker security hardening
    • Resource limits and monitoring
    • Backup strategies and update procedures

    These final steps prepare your deployment for reliable, secure production operation.