Run AI security-testing agents against a fresh copy of your own application

Authorization#

Test only systems you own or have written permission to test. Here that system is a copy of your own application, inside a sandbox you started. Do not point the agent at production, at a third party, or at anything else.

Promigence's Acceptable Use Policy draws the same line. It permits security testing inside your sandboxes when the targets are systems you own, systems you have written authorization to test, or deliberately vulnerable test environments, and when network access for that workload is restricted to those targets. It prohibits scanning, probing or attacking any system you do not own or are not explicitly authorized to test. What an agent does counts as done by you, even when it acted without your instruction. Testing the Promigence platform itself falls under the Vulnerability Disclosure Policy, which this guide does not cover.

Keep the permission with the results. The run script below records its reference as a run label, authorization=AUTH-142, which comes back with the run, and promigence sandbox list --label authorization=AUTH-142 finds every sandbox whose run carries it.

What runs where#

One sandbox holds one test run. Inside it are the copy of your application, already running on test data, and your agent, which reaches the app at http://127.0.0.1:8080 (use your app's own port). The sandbox's outbound allow-list names a single host, your model API. Other hosts it asks for are refused, and promigence sandbox network lists each one. No sandbox can see, reach or address another, so twenty runs at once are twenty separate copies.

Give the agent nothing outside the sandbox worth reaching. The copy holds test data and no production credentials, and the one key the agent uses is bound to the model API.

Before you start#

You need a Promigence account and the CLI. Sign up once: the CLI stores your key, and the SDKs read it too. Claim the free credit before anything else, because until it is claimed a first run stops with credit_unclaimed.

npm i -g promigence
promigence signup          # or: promigence login, for an account you already have
promigence credits claim

The run script in step 4 also uses jq.

1. Build the copy once#

From a Dockerfile#

Most applications already have a Dockerfile. Use it to build the snapshot, start the app at build time, and make the verify command wait until the app answers:

promigence snapshot create --dockerfile ./Dockerfile --tier medium \
  --start "cd /app && ./start.sh > /tmp/app.log 2>&1" \
  --verify "timeout 120 sh -c 'until curl -fsS http://127.0.0.1:8080/health; do sleep 2; done'" \
  --alias app-under-test --wait --wait-timeout 15m

The build context is the Dockerfile's directory unless --context names another directory, or a git URL pinned to a full commit SHA. --start takes your app's start command, the one your Dockerfile's CMD runs, here with its output sent to /tmp/app.log. It is started at build time and is already running in every sandbox made from the snapshot.

--verify has to exit 0 or nothing is stored. Either way, the build is billed per second for the time it runs. This check waits up to two minutes for the app's health route, so a copy that cannot start never becomes a snapshot. It needs curl in the image; without it, call your own check script. --tier medium gives every sandbox started from the snapshot 4 vCPU and 16 GiB, room for the app and the agent side by side. A build that outlasts --wait-timeout keeps going, and promigence snapshot status BUILD_ID --wait picks it up again.

Put the agent in the same image, under /opt/tester for example, with everything it needs already installed. During a run the allow-list names only the model API, so a download from anywhere else is refused. The app and the agent are then pinned together, and every run tests the same app with the same agent.

Seed the image with test data only: test accounts, fixtures, a local database. Leave production credentials and customer data out of the image and out of the build context, since the agent will read and change whatever the copy holds.

From a snapshot of a build#

Without a Dockerfile, build the snapshot from your repository at a pinned commit with setup steps, as the AI pentest agents page does:

promigence snapshot create --image debian:12 --tier medium \
  --setup "apt-get update && apt-get install -y git curl" \
  --setup "git clone https://github.com/your-org/your-app /work/app && git -C /work/app checkout <40-char sha>" \
  --setup "cd /work/app && ./provision.sh" \
  --verify "cd /work/app && ./healthcheck.sh" \
  --alias app-under-test --wait --wait-timeout 15m

provision.sh is your own script. It installs and builds the app, loads test data, installs your agent, and starts the server in the background with its output in /tmp/app.log, so the step can finish while the app keeps running. healthcheck.sh exits 0 once the app answers. Pin the commit by its full SHA: a snapshot built from a moving ref is not reproducible. A private repository builds through the GitHub connection instead (promigence github connect, then promigence snapshot create --repo your-org/your-app); the CLI reference has the details.

Either way, app-under-test now names a verified copy of your application at one version, and building again from identical inputs reuses that snapshot.

2. Store the model key, bound to its host#

Your agent needs a model API key, and nothing running in the sandbox should be able to read it. Store it once and bind it to the model API's host:

export ANTHROPIC_API_KEY=...   # already set in most setups
promigence secrets set ANTHROPIC_API_KEY --from-env ANTHROPIC_API_KEY \
  --host api.anthropic.com --no-raw

The value comes from the environment, never from the command line. With --host, requests a sandbox sends to the address this command prints are authenticated for you, and the value stays out of the sandbox. --no-raw makes that permanent: no run can ask for the value itself. A sandbox given the secret also has ANTHROPIC_BASE_URL pointed at that address, so an agent built on Anthropic's SDK works as it is, and a key bound to api.openai.com gets OPENAI_BASE_URL the same way. A request sent straight to https://api.anthropic.com is not authenticated, so an agent with a base URL of its own should use the printed address.

That key is the only secret a run needs. A snapshot of a sandbox that was given a secret is refused unless you force it. Leave it refused.

3. Price it and cap it#

promigence run quote --snapshot app-under-test --count 1 --timeout 1h

A quote is the most a run can cost: every sandbox running for its whole timeout. At medium, an hour lists at $0.46, billed per second for what actually runs. The --cap in the script below is a hard limit. A run quoted above it is refused before anything starts, and a sandbox that reaches it while running is paused with its files kept until you raise the cap or end it. An owner can also give the project a monthly ceiling with promigence project cap PROJECT USD. Tokens are billed by your model provider, separately.

4. Run the agent against a fresh copy#

This script is one complete test run. Save it as security-test.sh:

#!/bin/bash
# One security-test run against a fresh copy of your app; needs promigence and jq
set -euo pipefail

SNAPSHOT=app-under-test
OUT="runs/$(date -u +%Y%m%dT%H%M%SZ)"
mkdir -p "$OUT"

# The exact snapshot this run starts from, kept with the results
promigence snapshot inspect "$SNAPSHOT" --json > "$OUT/snapshot.json"
SNAP_ID=$(jq -r '.snapshot.snapshot_id' "$OUT/snapshot.json")

# A fresh copy behind an allow-list that names only the model API, with a $1 cap and one hour
SID=$(promigence fork --snapshot "$SNAP_ID" --count 1 --cap 1 --timeout 1h \
  --allow api.anthropic.com --secret ANTHROPIC_API_KEY \
  --label authorization=AUTH-142 --json | jq -r '.sandboxes[0].sandbox_id')

# However the script ends, the sandbox is deleted with everything the run changed in it
trap 'promigence sandbox kill "$SID" > /dev/null' EXIT

# Your agent, pointed at the copy; a non-zero exit is recorded and the run carries on
STATUS=0
promigence exec --sandbox "$SID" --stream --timeout 50m \
  -- /opt/tester/run --target http://127.0.0.1:8080 --report-dir /work/report \
  > "$OUT/agent.log" 2>&1 || STATUS=$?
echo "agent exited $STATUS" | tee "$OUT/status.txt"

# Every host the sandbox was refused, then the report and the app's log
promigence sandbox network "$SID" --json > "$OUT/network.json"
promigence files get /work/report "$OUT/report" --sandbox "$SID" --recursive
promigence files get /tmp/app.log "$OUT/app.log" --sandbox "$SID"

fork waits until the copy is ready and prints its sandbox id. That copy is the app as the snapshot holds it: running, seeded with test data, and untouched by any earlier run. Forking by the snapshot id rather than the alias ties the results to the exact copy they came from. --allow api.anthropic.com is the whole allow-list, and --secret ANTHROPIC_API_KEY gives the sandbox the bound key.

/opt/tester/run stands for however your agent starts: give it the copy's address and a directory for its report. --timeout 50m stops the agent after fifty minutes, which leaves the last ten of the sandbox's hour for collecting results, and --stream writes its output to the log as it is produced. If a run looks wrong while it is going, promigence sandbox network SID --cut cuts the sandbox's outbound access at once. It cannot be turned back on.

To run the script from CI or on a schedule, give it a key that can start work and nothing more: promigence keys create --name security-tests --access run makes one that can run sandboxes and builds but cannot change secrets, keys or billing.

5. Read the results#

Each run leaves a folder under runs/. Start with network.json: its refused hosts are what the agent or the app asked for and did not get. A copy that tries to call a payment or email provider shows up there, and so does an agent that wandered. report/ and agent.log are your agent's own output, app.log is the application's side of the same run, and status.txt holds the agent's exit code.

snapshot.json names the snapshot the run started from. Keep it with the report, because it is how a finding gets re-run.

6. Throw it away, start the next one#

The trap line ends the sandbox when the script exits, whether the agent finished, failed or ran out of time. A killed sandbox stops billing, and whatever the run changed is discarded with it. The snapshot is untouched, so the next ./security-test.sh starts from the same copy with nothing left over from the last attempt. None of your code, your agent's outputs or your test results is used for training.

To run several attempts side by side, fork with --count N. Each sandbox is its own fresh copy, promigence exec --run RUN_ID --all runs the agent in every one of them, and promigence run kill RUN_ID ends them all. A new account runs 20 sandboxes at once.

From TypeScript#

The same run with @promigence/sdk (npm i @promigence/sdk, Node.js 20 or newer). It type-checks against 0.1.0-beta.7:

import { mkdir, writeFile } from "node:fs/promises";
import { CommandExitError, PromigenceClient, Sandbox } from "@promigence/sdk";

const SNAPSHOT = "app-under-test";
const AGENT = "/opt/tester/run --target http://127.0.0.1:8080 --report-dir /work/report";

const out = `runs/${new Date().toISOString().replace(/[:.]/g, "-")}`;
await mkdir(out, { recursive: true });
const client = new PromigenceClient(); // reads PROMIGENCE_API_KEY, or the key signup stored

// The exact snapshot this run starts from, kept with the results
const snapshot = await client.snapshots.get(SNAPSHOT);
await writeFile(`${out}/snapshot.json`, JSON.stringify(snapshot, null, 2));

// A fresh copy behind an allow-list that names only the model API, with a $1 cap and one hour
const [sandbox] = await Sandbox.fork(snapshot.snapshot_id, {
  client,
  count: 1,
  spendCap: 1,
  timeoutMs: 60 * 60_000,
  allowHosts: ["api.anthropic.com"],
  secrets: ["ANTHROPIC_API_KEY"],
  metadata: { authorization: "AUTH-142" },
});

try {
  // Your agent, pointed at the copy; a non-zero exit is recorded and the run carries on
  const agent = await sandbox.commands.run(AGENT, { background: true, timeoutMs: 50 * 60_000 });
  let exitCode = 0;
  try {
    await agent.wait();
  } catch (e) {
    if (!(e instanceof CommandExitError)) throw e;
    exitCode = e.exitCode;
  }
  await writeFile(`${out}/agent.log`, `${agent.stdout}${agent.stderr}\nexit ${exitCode}\n`);

  // Every host the sandbox was refused, then the report and the app's log
  const network = await client.sandboxes.network(sandbox.sandboxId);
  await writeFile(`${out}/network.json`, JSON.stringify(network, null, 2));
  await writeFile(`${out}/report.tar`, await sandbox.files.downloadDir("/work/report"));
  await writeFile(`${out}/app.log`, await sandbox.files.read("/tmp/app.log"));
} finally {
  await sandbox.kill(); // the copy and everything the run changed in it are gone
}

Sandbox.fork takes the allow-list as allowHosts, the secrets by name and the cap in US dollars. The agent runs as a background command, and wait() returns when it ends, however long it takes. files.downloadDir hands back the report directory as one tar archive.

From Python#

The same run with promigence (pip install promigence, Python 3.9 or newer). It type-checks against 0.1.0b7:

import json
from datetime import datetime, timezone
from pathlib import Path

from promigence import CommandExitError, Promigence, Sandbox

SNAPSHOT = "app-under-test"
AGENT = "/opt/tester/run --target http://127.0.0.1:8080 --report-dir /work/report"

out = Path("runs") / datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
out.mkdir(parents=True)
client = Promigence()  # reads PROMIGENCE_API_KEY, or the key signup stored

# The exact snapshot this run starts from, kept with the results
snapshot = client.snapshots.get(SNAPSHOT)
(out / "snapshot.json").write_text(json.dumps(snapshot, indent=2))

# A fresh copy behind an allow-list that names only the model API, with a $1 cap and one hour
sandbox = Sandbox.create(
    snapshot["snapshot_id"],
    client=client,
    spend_cap=1,
    timeout_ms=60 * 60_000,
    allow_hosts=["api.anthropic.com"],
    secrets=["ANTHROPIC_API_KEY"],
    metadata={"authorization": "AUTH-142"},
)
try:
    # Your agent, pointed at the copy; a non-zero exit is recorded and the run carries on
    agent = sandbox.commands.run(AGENT, background=True, timeout_ms=50 * 60_000)
    try:
        exit_code = agent.wait().exit_code
    except CommandExitError as e:
        exit_code = e.exit_code
    (out / "agent.log").write_text(f"{agent.stdout}{agent.stderr}\nexit {exit_code}\n")

    # Every host the sandbox was refused, then the report and the app's log
    network = client.sandboxes.network(sandbox.sandbox_id)
    (out / "network.json").write_text(json.dumps(network, indent=2))
    (out / "report.tar").write_bytes(sandbox.files.download_dir("/work/report"))
    (out / "app.log").write_text(sandbox.files.read("/tmp/app.log"))
finally:
    sandbox.kill()  # the copy and everything the run changed in it are gone

In Python the allow-list is allow_hosts on Sandbox.create, and files.download_dir returns the report directory as tar bytes.

Sources#

Every page below was read on 17 Oct 2026.

Updated 2026-10-17