How to Build an AI Agent That Understands Your Entire Codebase

An AI code agent is a system that combines a large language model with retrieval tools, execution capabilities, and persistent memory to answer questions about and operate on a codebase. Unlike simple code completion, a code agent understands cross-file dependencies, architectural patterns, and business logic spread across thousands of files. At Skopx, our agent architecture processes codebases up to 500,000 files and answers architectural questions with 91% accuracy as rated by senior engineers.

What Makes a Code Agent Different From Code Completion?

Code completion predicts the next few tokens in a single file. A code agent reasons across your entire codebase. When you ask "How does our payment processing handle refunds?", a code agent must identify the relevant services (possibly spread across 15+ files), understand the data flow between them, trace error handling paths, and synthesize a coherent explanation with source citations.

This requires four capabilities that code completion lacks: multi-file retrieval, dependency graph traversal, context prioritization, and source-backed answer generation. Each of these is a distinct engineering challenge.

How Does Multi-File Retrieval Work?

The retrieval system must find the 5-15 most relevant code sections from a codebase that may contain 100,000+ files. We use a three-stage retrieval pipeline.

Stage 1: Vector search. We embed the user's question and find the top 50 semantically similar code chunks from our ChromaDB index. This casts a wide net and captures conceptually related code even when terminology differs.

Stage 2: Graph expansion. For each of the top 50 results, we traverse the import/dependency graph to find closely related files. If vector search finds PaymentService.processRefund(), graph expansion adds RefundValidator, PaymentGateway.reverse(), and the relevant database migration. This typically expands the candidate set to 80-120 files.

Stage 3: Re-ranking. We use Claude to re-rank the expanded candidate set against the original question, selecting the 8-12 most relevant code sections. This re-ranking step is critical, it improved answer accuracy from 74% to 91% in our evaluations.

// Three-stage retrieval pipeline
async function retrieveContext(
  question: string,
  codebase: CodebaseIndex
): Promise<CodeContext[]> {
  // Stage 1: Cast wide net with vector search
  const vectorHits = await codebase.vectorSearch(question, { topK: 50 });

  // Stage 2: Expand via dependency graph
  const expanded = await codebase.expandDependencies(vectorHits, {
    maxDepth: 2,
    maxFiles: 120
  });

  // Stage 3: LLM re-ranking for precision
  const reranked = await rerankWithClaude(question, expanded, {
    maxResults: 12,
    maxTokens: 8000  // fit within context budget
  });

  return reranked;
}

How Do You Build the Dependency Graph?

The dependency graph is a directed graph where nodes are files/modules and edges represent import relationships. We build it by parsing import statements across all supported languages using tree-sitter parsers. For a TypeScript/JavaScript codebase, this means resolving import and require statements, following tsconfig.json path aliases, and handling re-exports.

Building the graph for a 100,000-file codebase takes approximately 45 seconds on first index and 2-3 seconds for incremental updates. The graph is stored in memory and persisted to disk as a serialized adjacency list. Typical graph density is 4.2 edges per node for TypeScript projects and 3.1 for Python projects.

How Do You Fit a Large Codebase Into a Context Window?

Claude's context window is large but finite. A 100,000-file codebase might contain 50 million tokens of source code, far more than any context window can hold. Our context packing strategy uses a budget-based approach inspired by token packing techniques from ML training.

We allocate a context budget (typically 12,000 tokens for code context out of a 32,000 token prompt) and pack the most relevant code sections using a best-fit decreasing algorithm. Each code section is annotated with its file path, line numbers, and a relevance score. Sections are packed in relevance order until the budget is exhausted.

Critically, we include structural metadata even for code we cannot fit: file names, function signatures, and class hierarchies for the top 100 relevant files. This gives the model awareness of the broader architecture even when it can only see the implementation details of 8-12 files.

How Does the Agent Use Tools?

Our agent is not just a question-answering system, it can execute tools to gather information that is not in the static codebase index. The tool set includes:

• Database query: Execute read-only SQL against connected databases to answer questions about production data patterns
• API inspection: Fetch OpenAPI/Swagger specs from running services to understand API contracts
• Git history: Search commit history for when and why code changed
• Test runner: Execute specific test files to verify behavior claims
• Documentation search: Query connected Notion, Confluence, or markdown docs

The agent decides which tools to use based on the question. "What is our API rate limit?" might trigger both code search (to find the rate limiting middleware) and API inspection (to check the deployed configuration). Tool selection accuracy is 88%, meaning the agent picks the right tools for the question 88% of the time.

How Do You Ensure Answer Accuracy?

Every claim in the agent's response must be backed by a source, a specific file, line number, database record, or document. We enforce this through structured output: the model generates both the answer text and a list of citations, and we validate that every citation references a real, accessible source.

When the model cannot find sufficient evidence to answer a question, it says so explicitly rather than guessing. Our hallucination rate (claims not supported by cited sources) is 3.2%, measured by weekly manual audits of 200 randomly sampled responses.

What Are the Performance Characteristics?

End-to-end response time varies by question complexity. Simple questions ("Where is the User model defined?") resolve in 1.2 seconds. Moderate questions ("How does authentication work?") take 3-5 seconds. Complex architectural questions ("What happens when a payment fails?") take 8-15 seconds due to multi-stage retrieval and tool use.

We optimize perceived latency through streaming, the answer begins appearing within 800ms even for complex questions, as the retrieval and generation stages are pipelined. The model starts generating the answer while tool calls are still completing, updating its response as new information arrives.

Key Takeaways

Building a code agent requires solving four interconnected problems: multi-file retrieval with graph expansion, context packing within token budgets, tool orchestration for dynamic information gathering, and source-backed answer generation. The retrieval pipeline is the most impactful component, our three-stage approach (vector search, graph expansion, LLM re-ranking) accounts for most of the accuracy improvement over naive retrieval. The hardest unsolved problem is reasoning about runtime behavior from static code analysis, which we partially address through database queries and test execution.