---
question: How do I switch the default models to Ollama?
kind: task
via_web: na
via_cli: na
via_python: available
via_api: na
via_config: available
ref_python: provider="ollama"
ref_config: naas_abi_marketplace.ai.ollama
prereq: '[Ollama](https://ollama.com/download) is installed and running on the machine, or on a host the engine can reach.'
next:
  - /capabilities/build/models/choose-the-default-model
  - /capabilities/build/agents/chat-with-an-agent
---

# Run models locally

In local mode, the models run on your machine through Ollama: no API key, and no data leaves your network. The `naas_abi_marketplace.ai.ollama` module registers Qwen2.5 3B for chat (`qwen-2.5-3b`) and nomic-embed-text for embeddings (`nomic-embed-text`). A new project has the module in `config.yaml`, commented out; cloud mode, through OpenRouter, is the default.

## Web UI

Not applicable. Nexus does not switch the engine's models: the Ollama module is enabled in `config.yaml`, on the Config tab.

## CLI

Not applicable. No `abi` command switches to local models: the switch is in `config.yaml`, on the Config tab.

## Python

### Enable the Ollama module

Do the Config steps first: an agent can only ask for models the enabled modules provide.

### Ask for a local model

In an agent's `New()`, ask the registry for the local model by its canonical id:

```python title="agents/ReviewerAgent.py"
chat_model = registry.get_chat_model("qwen-2.5-3b")
```

Any other Ollama tag works through the module's factory, once it is pulled:

```python title="agents/ReviewerAgent.py"
chat_model = registry.get_chat_model("qwen2.5:1.5b", provider="ollama")  # ollama pull qwen2.5:1.5b
embedding = registry.get_embedding_model("embeddinggemma", provider="ollama")
```

## REST API

Not applicable. No endpoint switches the engine to local models; agents then answer at [the same routes](/user-guide/build/agents/chat-with-an-agent?via=api).

## Config

### Pull the models

```bash
ollama pull qwen2.5:3b
ollama pull nomic-embed-text
```

### Install the module's extra

```bash
uv add "naas-abi-marketplace[ai-ollama]"
```

### Enable the module and the local defaults

Uncomment the `naas_abi_marketplace.ai.ollama` block in `config.yaml` (the generated file shows each step in its comments), then set:

```yaml title="config.yaml"
global_config:
  ai_mode: "local"

modules:
  - module: naas_abi
    enabled: true
    config:
      abi_agent_model: "qwen-2.5-3b"
      ontology_engineer_model: "qwen-2.5-3b"
      # nexus_config: unchanged
  - module: naas_abi_marketplace.ai.ollama
    enabled: true

services:
  model_registry:
    default_chat_model: "qwen-2.5-3b"
    default_embedding_model: "nomic-embed-text"
```

The template pins `naas_abi`'s agents to `gemma-4-26b-a4b-it`, an OpenRouter model: move them too, or keep the OpenRouter module enabled. `ai_mode` is `cloud`, `local` or `airgap`; modules may read it to pick their models.

### Check and restart

```bash
abi config validate --modules
abi dev up -d
```

At load, the module looks for Ollama and logs a warning, not an error, when no server answers. Settings › Models then marks the Ollama models **Configured**.

## Where Ollama runs

The module finds the server itself: `localhost` on macOS and Linux, and under WSL the distro's own server or the Windows app on the host. To point it elsewhere, set `base_url` in the module's `config:` (`base_url: "http://localhost:11434"`), or the `ABI_OLLAMA_BASE_URL` or `OLLAMA_HOST` environment variable. `probe_on_load: false` skips the search and trusts `base_url`.

`abi dev up` runs natively and reaches it on `localhost`. The containerised deployment (`abi deploy local`) reaches the host through `host.docker.internal`: on Linux, Ollama then has to listen beyond `127.0.0.1`, for example with `OLLAMA_HOST=0.0.0.0:11434` in its systemd service, on a trusted network only.

:::warning Context sizes
Ollama allocates a 4096-token context by default. The chat model asks for its full 32k. `nomic-embed-text` reads 2048 tokens and silently drops the rest: chunk a document before you embed it.
:::