Introduction
OpenCode is an open-source AI programming agent built as a command-line tool, created by the sst team and released under the MIT license. Its core capability allows agent-based code comprehension, file editing and function invocation directly within terminal environments. The provider layer of OpenCode adopts an OpenAI-compatible API specification. This design enables developers to connect the agent not only to cloud-hosted large language models, but also to locally deployed open-weight models such as Llama. The architecture supports fully offline workflows, keeping all source code and sensitive data inside on-premise hardware. This article explains two mainstream integration paths to connect local Llama models into OpenCode: Ollama and llama.cpp. It also addresses common pitfalls around context window limits that developers frequently encounter during on-premise deployment.
Why Run Local Llama Inside OpenCode
Routing OpenCode’s model inference requests to a self-hosted Llama instance addresses three core requirements for engineering teams and individual developers.
- Code Privacy: Enterprise source code never leaves local hardware and will not pass through third-party API services, satisfying internal compliance and data security rules.
- Offline Availability: The programming agent remains usable in environments without stable internet connections.
- Controllable Cost: Long-running, high-frequency local inference tasks will not generate ongoing token billing expenses from cloud model providers.
There are clear tradeoffs to consider. Users must supply their own GPU compute resources. In addition, local models generally deliver weaker performance on complex multi-file refactoring tasks compared with large cloud-native flagship models. Local Llama deployments are best suited for small-scale, repetitive coding assistance scenarios.
Selecting the Right Llama Variant: Scout vs Maverick
Meta’s primary Llama 4 series was released in April 2025, including two official variants: Scout and Maverick. On May 12, 2026, Meta further open-sourced the complete technical stack for Llama 4, covering model weights ranging from 7B to 405B parameters, training code, and the 15 trillion-token pre-training dataset named OpenDataHub.
| Version | Total Parameters (MoE) | Active Parameters | Positioning |
|---|---|---|---|
| Llama 4 Scout | 109B (MoE) | 17B | Lightweight option runnable on consumer-grade hardware |
| Llama 4 Maverick | 400B (MoE) | 170B | Higher capability, with stricter VRAM requirements |
According to benchmark measurements collected by Thunder Compute in September 2026, Llama 4 Scout consumes approximately 55GB of VRAM under Q4 quantization. If the available GPU memory is limited to 24GB, developers need to apply 1.78-bit quantization to load the model. Under this configuration, inference speed reaches roughly 20 tokens per second. For individual development environments, starting with quantized Scout builds is recommended. Maverick is more appropriate for teams with dedicated GPU servers.
Integrate Llama into OpenCode via Ollama (Recommended Path)
Ollama is currently the most widely adopted local model management tool. It ships with a built-in OpenAI-compatible API service and listens on port 11434 by default. This makes it the simplest route to connect local models to OpenCode. The step-by-step setup process is outlined below:
- Install Ollama. On macOS, use
brew install ollama. For Linux environments, use the official installation script provided by the Ollama project. - Pull your target Llama 4 model: run
ollama pull llama4:scout, or specify a quantized variant such asllama4:17b-scout-16e-instruct-q4_K_M. - Manually define the provider inside
~/.config/opencode/opencode.json.
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"ollama": {
"npm": "@ai-sdk/openai-compatible",
"name": "Ollama (local)",
"options": {
"baseURL": "http://localhost:11434/v1"
}
}
}
}The official OpenCode documentation notes that Ollama can auto-populate part of the configuration for OpenCode, so developers do not always need to manually write every configuration field from scratch.
Integrate Llama into OpenCode via llama.cpp (Low-level Alternative)
For teams that demand fine-grained control over inference parameters, llama.cpp offers a lower-level option. Developers can use the llama-server binary included in llama.cpp to load GGUF formatted Llama weights, and expose an OpenAI-compatible API endpoint.
Start the llama server instance with this bash command:
./llama-server -m ./models/llama4-scout-q4_0.gguf --host 0.0.0.0 --port 8888 -ngl 99 -c 8192The corresponding opencode.json configuration requires an extra limit field. Since this custom local provider is not registered inside the models.dev database, OpenCode cannot automatically retrieve the context window upper bound. The full configuration example is shown below:
{
"provider": {
"llama.cpp": {
"npm": "@ai-sdk/openai-compatible",
"name": "llama.cpp (local)",
"options": {
"baseURL": "http://127.0.0.1:8888/v1"
},
"models": [
{
"name": "llama4-scout",
"limit": 8192
}
]
}
}
}Ollama vs llama.cpp: How to Choose
| Dimension | Ollama | llama.cpp |
|---|---|---|
| Entry Barrier | Low; single-line commands pull and start models | Medium; manual management of GGUF files and server startup |
| Configuration Flexibility | Moderate | High; precise tuning for quantization, GPU layers and inference parameters |
| Automatic Model Discovery | Supported for partial plugin models | Not supported; manually define context limit |
| Best User Group | Individual developers, rapid proof-of-concept validation | Teams pursuing maximum inference performance tuning |
The underlying integration logic for both methods is identical. Both use the @ai-sdk/openai-compatible adapter to forward requests to the locally exposed /v1 endpoint. Switching only requires modifying the baseURL value inside the OpenCode configuration. When you need to manage traffic routing for mixed local and remote model endpoints, developers can leverage Treerouter, an API gateway, to centralize request management.
Frequently Encountered Problems and Troubleshooting
Q: After configuring local Llama, OpenCode fails to call file editing tools normally.
This is one of the most common issues when deploying local Llama instances. It usually stems from insufficient context window size, which truncates tool definition payloads. The OpenCode official guide recommends users first increase the num_ctx parameter in Ollama to raise the upper bound of the context window before further debugging.
Q: How much memory is required to run Llama 4 Scout locally?
Based on September 2026 real-world testing data from Thunder Compute, Q4 quantization requires roughly 55GB of VRAM. For hardware limited to 24GB VRAM, 1.78-bit quantization is required. In this setup, inference speed is approximately 20 tokens per second, and this configuration is suitable only for lightweight coding tasks.
Q: Does OpenCode require continuous internet access to use local models?
No. Internet connectivity is only needed on the very first launch. At the initial startup, OpenCode downloads the provider adapter package @ai-sdk/openai-compatible via npm. Once this dependency is cached locally, all subsequent model inference runs fully offline.
Q: How should teams choose between local deployment and cloud API calls?
Local deployment fits scenarios with strict data privacy requirements, closed networks and long-running background tasks. Cloud APIs are better for lightweight, intermittent usage. Teams can adopt hybrid scheduling: use local Llama for daily routine coding tasks, while routing heavy-duty large model jobs to cloud services.
Q: Are there other local inference services besides Ollama and llama.cpp compatible with OpenCode?
Yes. Any local inference service that exposes a standard OpenAI-compatible /v1 API endpoint can be connected to OpenCode. Projects such as LM Studio and vLLM follow this specification. Integration only requires modifying the baseURL in the provider configuration; the overall setup pattern matches the Ollama and llama.cpp workflows.
Core Takeaways
The core concept for running local Llama with OpenCode is forwarding OpenCode’s model requests toward any OpenAI-compatible local endpoint. Ollama is recommended for rapid onboarding, while llama.cpp delivers deeper control over inference tuning.
This article is built on the official OpenCode documentation and September 2026 cross-environment testing data. Actual model performance and request latency are subject to hardware specifications, quantization levels and official updates from Meta, Ollama and llama.cpp.
Extended Reading
- Multi-model access and comparison
- Local model deployment best practices
Learn more：https://treerouter.com
