Building a local-first multi-agent coding assistant where specialized agents (Analyst, Architect, Planner, Critic, Consensus) collaborate on software tasks—running entirely on your machine with Ollama. Focus on role specialization, structured intermediates, and local execution.

Most coding assistants are single-model chat interfaces. AgentForge asks: what if we decompose software engineering into specialized roles, each with its own agent?

The Insight

Human software teams don't have one engineer do everything. We have:

Analysts who clarify requirements
Architects who design systems
Planners who break down implementation
Critics who review designs
Consensus that integrates feedback

AgentForge replicates this with local LLMs via Ollama.

Architecture

User Request
    ↓
Analyst Agent → Requirements Spec
    ↓
Architect Agent → System Design
    ↓
Planner Agent → Implementation Plan
    ↓
Critic Agent → Review & Issues
    ↓
Consensus Agent → Final Output

Each agent:

Receives structured input from previous stage
Has specialized system prompt for its role
Produces structured output (JSON/Markdown)
Can be swapped/upgraded independently

Agent Roles

Agent	Responsibility	Key Prompt Focus
Analyst	Requirement decomposition	Ambiguity detection, edge cases, acceptance criteria
Architect	High-level design	Modularity, scalability, tech selection, trade-offs
Planner	Task breakdown	Dependencies, ordering, estimates, risk mitigation
Critic	Design review	Security, maintainability, performance, completeness
Consensus	Integration	Conflict resolution, final synthesis, actionable output

Local-First with Ollama

Why local?

Privacy: Code never leaves machine
Offline: Works on planes, secure environments
Cost: No per-token API fees
Control: Model choice, quantization, context window

Models tested:

qwen2.5-coder:7b - Strong coding, good instruction following
qwen2.5:14b - Better reasoning, slower
codellama:13b - Specialized for code
gpt-oss:20b - Open weights, strong general reasoning

Implementation

Agent Base Class

class Agent:
    def __init__(self, name: str, system_prompt: str, model: str):
        self.name = name
        self.system_prompt = system_prompt
        self.model = model
    
    async def run(self, input_data: dict) -> dict:
        prompt = self.format_prompt(input_data)
        response = await ollama.chat(self.model, prompt)
        return self.parse_output(response)

Pipeline Orchestrator

class AgentForge:
    def __init__(self, agents: List[Agent]):
        self.agents = agents
        self.cache = {}
    
    async def execute(self, request: str) -> dict:
        context = {"request": request}
        for agent in self.agents:
            if agent.name in self.cache and self.same_input(context):
                context[agent.name] = self.cache[agent.name]
            else:
                context[agent.name] = await agent.run(context)
                self.cache[agent.name] = context[agent.name]
        return self.consensus(context)

Caching Strategy

Input hashing: Cache key = hash(agent_name + relevant_context)
Invalidation: Clear on request change or model switch
Observed: Substantial latency reduction on iterative refinement

Key Engineering Challenges

1. Context Preservation Across Agents

Problem: Later agents lose nuance from earlier stages Solution: Structured intermediate representations (not free text)

{
  "requirements": [...],
  "constraints": {...},
  "acceptance_criteria": [...]
}

2. Prompt Consistency

Problem: Slight prompt changes cascade unpredictably Solution:

Version-controlled prompts (separate files, Git-tracked)
Automated prompt testing with golden examples
A/B comparison framework

3. Local Model Limitations

Problem: 7B models hallucinate APIs, miss edge cases Solution:

Retrieval-augmented context (local docs, past decisions)
Critic agent catches hallucinations before consensus
Human-in-the-loop for critical decisions

4. Latency Management

Problem: 5 sequential agents × 10s = 50s response Solutions:

Parallel where independent (Analyst + Architect can run together)
Streaming responses for perceived speed
Caching as above
Smaller models for simpler agents (Analyst: 7B, Architect: 14B)

Development Findings

Note: The following are qualitative observations from development and testing, not controlled experimental results with defined baselines, multiple runs, or statistical significance.

Role Specialization

Separating analysis, architecture, planning, and critique made individual reasoning stages easier to inspect and debug. Each agent's output could be independently validated.

Structured Intermediate State

Passing structured representations (JSON) between agents reduced loss of requirements between stages and enabled programmatic validation at each boundary.

Critic Agent Effectiveness

The Critic agent caught issues (hallucinated APIs, missing auth checks, incomplete error handling) that single-model generation missed — separation of concerns works for review tasks.

Local Execution

Ollama enabled the workflow to operate without sending source code to an external inference API. Privacy-sensitive codebases can be processed entirely locally.

Caching Impact

Caching repeated agent computations substantially reduced unnecessary inference during iterative refinement. Second and subsequent iterations felt near-instant for unchanged pipeline stages.

Lessons Learned

Role specialization > prompt engineering: Explicit roles produce more consistent outputs than "you are a senior engineer"
Structured intermediates enable debugging: Can inspect each stage's output independently
Local models are viable for structured tasks: Coding, review, planning—less so for open-ended creativity
Consensus is the force multiplier: Integrating critique before output > single-pass generation
Caching transforms UX: Second iteration feels instant

Future Work

Parallel execution: Analyst || Architect, then sequential
Persistent memory: Vector DB of past decisions, patterns, preferences
RAG integration: Local codebase indexing for context-aware agents
Code execution sandbox: Validate generated code before consensus
Automated testing agents: Generate tests, run, iterate
Multi-model routing: Task → best local model (coding vs reasoning)
Cloud deployment option: Same architecture, swap Ollama for vLLM/TGI
IDE integration: VS Code extension, JetBrains plugin
Team workspace: Shared memory, collaborative agent runs

Related Project

This blog post accompanies the AgentForge project case study.

View Project Case Study →

AgentForge: Multi-Agent Software Engineering with Local LLMs