Ansible Warp Drive: Faster Playbooks with Mitogen

Ansible is great at expressing intent.

Ansible is also very good at… spending most of its runtime doing everything except the thing you asked it to do:

• spawning processes
• shuffling small payloads
• and paying roundtrip tax

If your playbooks are “many small tasks” (copy, lineinfile, template, loops), the overhead dominates.

Mitogen is a rare optimization that changes the game. It keeps Python interpreters alive on the other side and drives Ansible modules over fast RPC, instead of redoing setup work for every task.

Why classic Ansible is slow

#

For each task on each host, classic Ansible typically does:

• build module payload
• transfer/stage it
• start Python, import, execute
• parse output and clean up temp files

Even with SSH multiplexing and pipelining, the per-task overhead is real.

Baseline execution model (conceptual)

#

sequenceDiagram
  participant C as Controller (ansible-playbook)
  participant H as Target

  loop each task
    C->>H: stage module payload
    H->>H: python startup + imports
    H-->>C: module result
  end

What Mitogen changes

#

Mitogen keeps a persistent interpreter on the target. Modules stay hot in memory. Most per-task setup cost disappears.

Mitogen execution model (conceptual)

#

sequenceDiagram
  participant C as Controller
  participant M as Mitogen multiplexer
  participant H as Target (persistent Python)

  C->>M: connect once per target
  M->>H: start interpreter once

  loop each task
    C->>M: RPC call
    M->>H: execute in persistent interpreter
    H-->>M: result
    M-->>C: result
  end

Mitogen also changes local execution. When you run against localhost, classic Ansible still spawns a lot of transient processes. Mitogen prefers a persistent model, which is why local benchmarks can look dramatic.

Benchmark repo

#

• Python environment is managed via uv (pyproject.toml + uv.lock)
• Orchestration and run entry points are in Taskfile.yml
• Ansible configuration is pinned in ansible.cfg
• Collections are pinned in requirements.yml
• The benchmark playbook is playbooks/bench.yml
• Default inventory runs the benchmark on localhost (so anyone can reproduce)

All code lives in the ansible-mitogen-benchmarks

Taskfile as the single entry point

#

Everything starts from Taskfile.yml.

flowchart LR
  os["task deps-os
(Arch helper)"] --> py["task deps-python
uv sync --frozen"]
  py --> galaxy["task deps-galaxy
ansible-galaxy install"]
  galaxy --> ssh["task deps-ssh
prepare ControlPath dir"]
  ssh --> bench["task bench
hyperfine A/B run"]
  ssh --> mitogen["task mitogen-only
Mitogen run only"]
  ssh --> strategy["task get-strategy
list strategies"]
  py --> lint["task lint
ansible-lint"]

Key points:

• task deps-python creates a reproducible environment via uv sync --frozen --quiet
• task deps-galaxy installs pinned collections from requirements.yml
• task deps-ssh prepares the SSH ControlPath directory used by ansible.cfg
• task bench runs the same playbook twice via hyperfine and reports statistics
• task lint runs ansible-lint (also wired into pre-commit)

The playbook entry is a single variable:

• RUN_PLAYBOOK="uv run ansible-playbook playbooks/bench.yml {{.CLI_ARGS}}"

That makes command strings consistent across tasks.

Ansible config: baseline is already tuned

#

My baseline is not default Ansible. I intentionally tuned it, because I want the comparison to be honest.

ansible.cfg highlights:

• SSH multiplexing via ControlMaster, ControlPersist, and a dedicated ControlPath
• pipelining = True
• use_persistent_connections = True
• ansible.posix.profile_tasks callback enabled for timing visibility

That means the speedup is not coming from disabling obvious bottlenecks. It is Mitogen doing real work.

The playbook: overhead-heavy on purpose

#

The benchmark playbook is playbooks/bench.yml.

It does “boring ops” that stress overhead:

• create a temp directory
• write 200 small files (copy)
• apply 200 config-like edits (lineinfile)
• clean up files and directory
• set gather_facts: false to avoid measuring fact collection instead of task execution

Default inventory: runs everywhere

#

The default inventory is YAML and runs the playbook on localhost:

benchmark_targets:
  hosts:
    localhost:
      ansible_connection: local

This is fine to reproduce the result without setting up a lab. If you want to run against real machines, add hosts under benchmark_targets. Keep the group name the same, because the playbook targets it.

Running it

#

One command runs the full chain (Python deps, Galaxy deps, SSH ControlPath prep, and the A/B benchmark):

task bench

Under the hood it runs:

hyperfine --warmup 1 --runs 3 \
  'uv run ansible-playbook playbooks/bench.yml' \
  'ANSIBLE_STRATEGY=serverscom.mitogen.mitogen_linear uv run ansible-playbook playbooks/bench.yml'

If you only want the Mitogen run:

task mitogen-only

Results (localhost)

#

Here is the hyperfine output from my run on the default inventory (localhost, local connection):

Benchmark 1: uv run ansible-playbook playbooks/bench.yml
  Time (mean ± σ):     129.019 s ±  3.871 s
  Range (min ... max):   124.715 s ... 132.216 s    3 runs

Benchmark 2: ANSIBLE_STRATEGY=serverscom.mitogen.mitogen_linear uv run ansible-playbook playbooks/bench.yml
  Time (mean ± σ):       8.695 s ±  0.102 s
  Range (min ... max):     8.577 s ...  8.758 s    3 runs

Summary
  Mitogen ran 14.84 ± 0.48 times faster

Translated into human time:

• Baseline: ~2m 09s
• Mitogen: ~8.7s
• Saved per run: ~2m 00s

Table view

#

Results on remote hosts (SSH)

#

Same playbook and same harness, but executed against real SSH targets.

Benchmark 1: uv run ansible-playbook playbooks/bench.yml
  Time (mean ± σ):     506.752 s ±  8.057 s
  Range (min ... max):   501.885 s ... 516.052 s    3 runs

Benchmark 2: ANSIBLE_STRATEGY=serverscom.mitogen.mitogen_linear uv run ansible-playbook playbooks/bench.yml
  Time (mean ± σ):      71.390 s ±  0.719 s
  Range (min ... max):    70.806 s ... 72.193 s    3 runs

Summary
  Mitogen ran 7.10 ± 0.13 times faster

Translated into human time:

• Baseline: ~8m 27s
• Mitogen: ~1m 11s
• Saved per run: ~7m 16s

Table view (SSH)

#

Why the speedup is so large

#

Two reasons:

1. The benchmark is overhead-heavy by design. There are hundreds of tiny operations.
2. Local runs get an extra boost because the control-plane overhead dominates even without SSH.

On remote hosts over SSH, Mitogen still cuts the per-task overhead, which is why the speedup remains large.

That is the signature of “overhead-bound” runs turning into “work-bound” runs.

Caveats

#

Mitogen is fast because it reuses interpreter processes. That can expose modules that are not clean about global state. If a task starts misbehaving, Mitogen supports task isolation via fork.

Also, Ansible is evolving. Newer Ansible versions warn that third-party strategy plugins may be deprecated in the future. Mitogen works today, but treat it as an optimization you can toggle per-run.

Takeaways

#

• Mitogen is not a micro-optimization. On overhead-heavy playbooks it can be a different tier of speed.
• A benchmark is only convincing when it is reproducible. This repo is designed to be cloned and run.
• Tuning matters. My baseline is already optimized, which makes the Mitogen result more meaningful.
• If you do infra work, fast feedback loops are a feature. This is how you buy them.

Compatibility note: external strategy plugins

#

Mitogen integrates with Ansible via a custom strategy plugin (mitogen_linear).

Recent Ansible releases deprecate strategy plugins not shipped in ansible.builtin. A future release will remove the ability to use external strategy plugins, and no alternative is currently planned.

References:

• Ansible 12 Porting Guide: Strategy Plugins
• Mitogen issue tracker discussion about Ansible 12 deprecation

Appendix

#

Repo files worth reading first

#

• Taskfile.yml
• ansible.cfg
• playbooks/bench.yml
• requirements.yml

Disk and system load disclaimer

#

This benchmark is intentionally I/O-heavy.

It creates, edits, and deletes hundreds of small files on the target(s). On remote hosts this can generate sustained disk activity, metadata churn, and noticeable load.

Do not run it on production systems, shared storage, or anything you cannot afford to stress. You are responsible for where you run it and for any impact on performance, wear, or data.