--- sidebar_position: 4 question: What pieces do I run on my own cluster? kind: guide --- > **You are reading the v2.x docs (engine modules). New projects use v3.** [Read this page in v3](/capabilities/deploy/production) · [Migrate from 2.x to 3.0](/capabilities/upgrade/migrate-to-3) # Production How to run a project on servers that are not your laptop, and which pieces to run yourself or take as a service. [Docker](/v2/capabilities/deploy/local-docker) and, on ABI 3.x, [Dokploy](/v2/capabilities/deploy/dokploy) are ready-made ways to do it on one host; this page is for everything else: your own Compose or Kubernetes, a container platform, Naas Spaces, a managed NATS (ABI 3.x). Your modules run inside the engine: the engine image is the project, with every module listed under `modules:` in the config. --- ## The pieces of a production project ```text title="A production project" Users ──> Nexus web ──> API (the engine, with your modules) │ ├──> message broker (NATS on ABI 3.x; RabbitMQ without NATS) ├──> PostgreSQL ├──> triple store (Fuseki, Oxigraph) ├──> vector store (Qdrant) ├──> object storage (S3, MinIO) └──> Redis ``` | Piece | Notes | |---|---| | **The engine** (`python -m naas_abi_core.apps.api.api`) | Loads the modules of `modules:` and serves the API and Nexus's backend. In a NATS project (ABI 3.x) it also serves the kernel services over NATS and hosts the module jobs: **one serving engine per NATS account.** | | **Nexus web** | The web app; it calls the API. | | **The message broker** | ABI 3.x, NATS project: NATS 2.14 or later, JetStream on. See [the broker](#the-broker). ABI 2.x, and 3.x without NATS: RabbitMQ, from `services.bus`, with Dagster running the orchestrations. | | **Backing services** | Whatever `services:` points at: PostgreSQL (documents, Nexus), a triple store, Qdrant, S3-compatible storage, Redis. | | **A model provider** | A cloud key (OpenRouter by default) or your own inference endpoint. | All of them must be restarted when they stop, by a supervisor (Docker `restart: unless-stopped`, Kubernetes). In a NATS project (ABI 3.x) the engine exits when the broker is unreachable for about two minutes, on purpose, so the supervisor starts a fresh one. See [When the broker is gone](/architecture/real-time-fabric#when-the-broker-is-gone). --- ## `config.remote.yaml` A project made by `abi new project` has one config per environment. The engine reads `config..yaml` when `ENV` names a file that exists, so a deployment sets `ENV=remote` and gets `config.remote.yaml`: | What it sets | How | |---|---| | Secrets | A dotenv adapter on `.env.remote` (fix its `config:` first, see below). `{{ secret.X }}` in the config takes a platform environment variable before the file; code that calls `secret.get` reads the file first. Neither is committed or baked into an image: see [Keep secrets out of git and out of the image](#keep-secrets-out-of-git-and-out-of-the-image). | | The broker | ABI 3.x, NATS project: `nats.nats_url: "{{ secret.NATS_URL }}"`, logged in as the broker user `abi`, and `nats.jwt_secret: "{{ secret.NATS_JWT_SECRET }}"`. ABI 2.x, and 3.x without NATS: `services.bus` with the `rabbitmq` adapter, from the `RABBITMQ_*` secrets. | | Backends | PostgreSQL (`services.document`), S3-compatible object storage, Fuseki, Qdrant and Redis, each from `{{ secret.* }}` host, port and credentials. | | Models | `ai_mode: "cloud"` with the OpenRouter module, so the host needs no inference hardware. Ollama is opt-in and needs a server the app can reach (`ABI_OLLAMA_BASE_URL`). | | Public hosts | `PUBLIC_WEB_HOST` and `PUBLIC_API_HOST`, which feed `cors_origins` and `public_api_host`. | ```yaml title="config.remote.yaml (ABI 3.x, NATS project)" nats: nats_url: "{{ secret.NATS_URL }}" # nats://abi:@broker.internal:4222 jwt_secret: "{{ secret.NATS_JWT_SECRET }}" # 32 bytes or more discovery: project: "my_ai" lease_seconds: 180 monitoring_url: "http://broker.internal:8222" # optional: the System app's NATS tab ``` ```yaml title="config.remote.yaml (ABI 2.x, and 3.x without NATS)" services: bus: bus_adapter: adapter: "rabbitmq" config: rabbitmq_url: "amqp://{{ secret.RABBITMQ_USER }}:{{ secret.RABBITMQ_PASSWORD }}@{{ secret.RABBITMQ_HOST }}:{{ secret.RABBITMQ_PORT }}" ``` ### Fix the secret adapter first The `config.remote.yaml` that `abi new project` writes, on ABI 2.x and 3.x, gives the `dotenv` adapter a list (`config:` then `- path: .env.remote`), which the engine refuses with "Input should be a valid dictionary". Make it a mapping: ```yaml title="config.remote.yaml" services: secret: secret_adapters: - adapter: "dotenv" config: path: .env.remote ``` :::warning The file must exist The dotenv adapter stops the engine when its file is missing. A `.dockerignore` that keeps `.env.*` out of the image (see below) means a container needs `.env.remote` mounted, or a config whose secrets all come from the environment. ::: Check it before you ship it: ```bash title="Project root" ENV=remote abi config validate --modules ``` `abi config validate --modules` also imports each enabled module and validates its config. Every `{{ secret.X }}` of the active YAML must resolve, or the engine refuses to load: add each key to the environment or to `.env.remote`. See [Configuration](/user-guide/get-started/configuration#environment-files). In a NATS project (ABI 3.x), give any second engine of the project (a worker, a notebook) a config **without** the `discovery:` block: exactly one engine per project hosts the module registry. --- ## The broker This section is for a NATS project on ABI 3.x. On ABI 2.x, and on 3.x without NATS, the broker is RabbitMQ (`services.bus`): run it next to the engine, on a private network. A NATS project needs one broker. Run it next to the project, or take a managed NATS cluster, as long as it provides: - **JetStream**, on persistent storage. It holds the bus, the module jobs, the engine's lease and discovery. - **nats-server 2.14 or later**: module jobs use JetStream message schedules, and time zones on cron triggers need 2.14. - **A `max_payload` of 8 MB**: the largest value NATS recommends. Bigger values stream in chunks. - **A login**: the engine logs in as the user `abi`, with every right. The `nats.conf` that `abi deploy local` writes under `.deploy/docker/nats/` is the reference: copy its `authorization` block to your broker's configuration, or translate it to your provider's permission model. - **A private network.** Traffic to NATS is not encrypted and the monitoring port has no authentication. Put the broker where only the engine reaches it. :::note Managed NATS ABI is built and tested against the `nats-server` 2.14 it runs itself. A managed service works when it offers the list above; run your project against it before you depend on it. See [Security model](/developer-guide/architecture/security). ::: --- ## Deploy without downtime This is for a NATS project on ABI 3.x. Set `ABI_ROLLOUT_ID` (for example the image tag) on each deploy. The new engine stands by, takes over when the old one stops, and a serving engine with a rollout id needs shared backends (PostgreSQL, S3, Fuseki, Qdrant), not SQLite or local files. On Kubernetes use one rollout id per deploy, `maxSurge: 1` and `maxUnavailable: 0`. The details are in [one serving engine](/architecture/real-time-fabric#one-serving-engine). Your modules are part of the engine, so they roll out with it. --- ## Your own Docker Compose or Kubernetes The image is the project's `Dockerfile`: it copies the project folder (`COPY . .`), installs it with `uv sync --no-dev` and runs `python -m naas_abi_core.apps.api.api` on port 9879. Check that [`.dockerignore`](#keep-secrets-out-of-git-and-out-of-the-image) excludes `.env*` before you build, then build for your platform: ```bash docker build -t my_ai:latest . docker run --rm -p 9879:9879 -e ENV=remote --env-file .env.remote my_ai:latest ``` Secrets reach the container as environment variables (`--env-file`, your platform's secret store), not as files in the image. To start from a working stack, run `abi deploy local` and take `docker-compose.yml` and `.deploy/` as the base: they hold every service with its health checks and volumes. Put TLS in front with your reverse proxy (Caddy, Traefik, Nginx, an ingress). Keep the broker, the databases and the object store on a private network, and back up their volumes. See [High availability](/v2/capabilities/deploy/high-availability) and [Security and compliance](/v2/capabilities/deploy/security-compliance). --- ## Naas Spaces `abi deploy naas` builds the project's image, pushes it to a Naas container registry and creates or updates a Naas Space that runs it (one `api` container on port 9879, 1 CPU, 1 GiB). It reads a `deploy:` section from `config..yaml` (`-e prod` by default, so `config.prod.yaml`, else `config.yaml`): ```yaml title="config.prod.yaml" deploy: workspace_id: "" space_name: "my-ai-api" naas_api_key: "{{ secret.NAAS_API_KEY }}" env: # environment variables of the container ENV: "remote" NATS_URL: "{{ secret.NATS_URL }}" # ABI 3.x, NATS project NATS_JWT_SECRET: "{{ secret.NATS_JWT_SECRET }}" # ABI 3.x, NATS project OPENROUTER_API_KEY: "{{ secret.OPENROUTER_API_KEY }}" ``` ```bash abi deploy naas # needs Docker and a Naas subscription with Spaces ``` It prints the Space's URL (`https://.default.space.naas.ai`). A Space runs the engine and its modules only: give it a broker it can reach (NATS on ABI 3.x, a managed one or one you host; RabbitMQ without NATS) and the backing services in `env`. A project made by `abi new project` includes a GitHub Actions workflow (`.github/workflows/release.yaml`) meant to run `abi deploy naas` on every push to `main`, with `ENV=prod` and `NAAS_API_KEY` as a repository secret. As generated it fails: it runs `python -m naas_abi_cli.cli`, which is not runnable, and no generated config has a `deploy:` block. Make the step run the `abi` CLI (`abi deploy naas`) and add the block before you rely on it. --- ## Keep secrets out of git and out of the image `.env`, `.env.remote`, `.env.dokploy` and the volume archives of `abi stack snapshot` (`.snapshots/`) hold secrets: `NATS_JWT_SECRET` (ABI 3.x), passwords, API keys. A project made by `abi new project` on ABI 3.x ignores them in `.gitignore` and `.dockerignore`: ```text title=".gitignore and .dockerignore" # .gitignore # .dockerignore .env .env .env.* .env.* .snapshots/ .snapshots/ .git ``` A project made on ABI 2.x, or by an earlier 3.x, has a `.gitignore` that ignores only `.env` and no `.dockerignore`: add these lines before you build an image or commit `.env.remote`. On ABI 3.x, `abi deploy dokploy` adds `.env.dokploy` to `.gitignore` and `.env`, `.env.*` to `.dockerignore` itself. An image that contains `.env*` leaks every secret to whoever can pull it. ## Secrets Production secrets reach the engine as environment variables or through a secret adapter chain (`services.secret.secret_adapters`: `dotenv`, `naas`, `base64`, `custom`, and on ABI 3.x `nats_rpc`). See [Add secrets](/v2/capabilities/deploy/add_secrets_to_production). ## Next - [Deployment models](/v2/capabilities/deploy/deployment-models): cloud, on premises, air gapped, managed. - [Publish remote agents](/v2/capabilities/deploy/publish_remote_agents). - [Upgrade to 3.0](/user-guide/upgrade/migrate-to-3), when the project is a 2.x one.