---
title: "SWE-bench infrastructure, verified, reproducible environments per instance"
description: "Promigence runs SWE-bench-class benchmarks on verified environments, reproducible by hash, with a report separating environment failures from agent failures."
url: "https://www.promigence.ai/for/swe-bench"
site: "Promigence"
---

# SWE-bench and friends, without the broken-environment tax

Promigence runs SWE-bench-class benchmarks on environments verified before the instance starts and reproducible from a hash afterwards. Published work has found roughly a third of the hardest instances broken at the environment level, a tax paid out of every score computed on them.

**Who this is for:** Teams publishing or consuming SWE-bench, SWE-rebench, Terminal-Bench, SetupBench and similar results.

## What breaks today
### A third of the hardest instances do not set up
Breakage in public task sets is concentrated in exactly the instances that separate good agents from great ones, so scores computed across them are not wrong by a random amount.

### Nobody can reproduce your run
A benchmark defined by a repo and a script measures the repo as much as the agent. Rebuild it six months later and you are comparing two different things.

### The harness is fine; the fleet is the bottleneck
Most harnesses are fine. What fails at 500 instances is image pulls, concurrency limits and cold setup repeated per instance.

## What changes with Promigence
### One verified snapshot per instance family
Built once, verified once, forked per instance. Instances that cannot build show up as a list at build time, not as mysterious agent failures.

### A hash you can put in the paper
`snap_…` identifies a bit-identical environment. Anyone re-running against that hash gets what you had.

### Drop-in behind your existing harness
Adapters for common eval harnesses (pre-release), plus a compatibility SDK class so existing sandbox call sites keep working.

## Point an existing harness at Promigence
```typescript
// Most existing sandbox call sites keep working through the compatibility class.
import { Sandbox } from "promigence";

const sandbox = await Sandbox.create("swebench-django");   // a snapshot you built
await sandbox.commands.run("git apply /tmp/model_patch.diff");
const result = await sandbox.commands.run("pytest -q tests/");
await sandbox.kill();               // ← the episode ends here; billing stops
```