How to Automate AI Agent Readiness Audits with a Free API

1. Pick the automation pattern that matches the question

“Automate the audit” usually hides one of three different questions. Answer that question first, because it determines how often you scan, what you store, and what counts as a failure.

The API is the same in all three cases: one POST /api/scan per URL with a bearer key. The differences are pacing, storage, and the rule that turns a JSON report into a decision. Sections 4 to 6 give one worked example for each pattern.

2. Make one call and read the headers

Create a free account, then create a key in your API keys panel. Keep the key in a secret store or an environment variable on the machine that runs the automation. Never put it in a workflow file, an n8n prompt, or a shared spreadsheet. The API page has the quickstart and the OpenAPI 3.1 document.

Make your first call from a terminal and keep the response headers. curl’s --fail-with-body returns a non-zero exit code on an error status while still saving the JSON body, which is what an automation needs.

curl -sS --max-time 60 --fail-with-body \
  -D headers.txt -o report.json \
  https://adaptmypage.com/api/scan \
  -H "Authorization: Bearer $AGENT_READY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://www.example.com","profile":"website"}'

grep -i '^ratelimit' headers.txt
jq '{score, summary, fetched: .metadata.fetchedUrls}' report.json

Three headers matter for pacing. RateLimit-Limit is the number of scans allowed per account in the current hourly window, 60 for free accounts. RateLimit-Remaining counts down as you scan. RateLimit-Reset is the Unix timestamp, in seconds, at which the fixed window resets. The window is per account, not per key, so two keys from the same account share the same 60 scans.

These field names follow an earlier draft of the IETF’s RateLimit header fields work. The current draft defines combined RateLimit and RateLimit-Policy fields instead. Read the three separate headers the API actually sends rather than assuming a client library’s default parsing.

A scan is bounded. The scanner fetches at most 12 HTML pages and 50 requests, allows 4 seconds per request at up to 3 requests per second, and respects the target’s robots.txt for secondary probes. The API stops the whole scan after 45 seconds with 504 scan_timeout. Set your client timeout to 60 seconds so you receive that error instead of producing your own.

3. Read the report so the automation makes good decisions

The response is the open-source AgentReady report with its original statuses. Below is an abbreviated real result: a scanner run against our own homepage on 14 September 2026 with the API’s limits. We shortened the lists; the field names and values are unchanged. Your target will produce different numbers.

{
  "agentreadyVersion": "0.1.0-alpha.0",
  "target": "https://www.adaptmypage.com",
  "normalizedTarget": "https://www.adaptmypage.com/",
  "profile": "website",
  "score": 54,
  "scores": [
    { "key": "discoverability", "label": "Discoverability",
      "weight": 35, "score": 23, "applicableFindings": 6 },
    { "key": "access-policy", "label": "Agent access policy",
      "weight": 20, "score": 8, "applicableFindings": 2 }
  ],
  "summary": { "critical": 0, "major": 4, "minor": 7, "info": 1,
               "pass": 7, "fail": 0, "warn": 7, "unknown": 3 },
  "findings": [
    {
      "id": "discoverability.llms_txt",
      "title": "llms.txt is available",
      "status": "warn",
      "severity": "minor",
      "category": "discoverability",
      "evidence": [{ "message": "/llms.txt not available (HTTP 404)." }],
      "recommendation": "Add /llms.txt with a concise Markdown map of agent-relevant pages."
    }
  ],
  "metadata": {
    "fetchedUrls": 42,
    "skippedByRobots": ["https://www.adaptmypage.com/api/openapi.json"],
    "elapsedMs": 13753,
    "active": false,
    "browser": false
  }
}

Yes, our own homepage has open findings. That is the reason to automate: the checks that are easy to forget are the ones a scheduled scan keeps asking about.

Four reading rules keep an automation honest:

• Category scores are points, not percentages. Each entry in scores has a weight; its score is at most that weight. Discoverability at 23 of 35 is a different statement from 23 percent. The website profile weights discoverability 35, agent access policy 20, content structure 20, security 15, and support 10.
• Compare finding IDs, not the total. A total score can move because a page was unreachable for a second, not because you changed anything. A finding such as discoverability.sitemap_xml switching from pass to warn between two runs is a concrete event you can investigate.
• unknown means the scanner could not observe the signal. It is not a failure. not_applicable means the check does not apply to the chosen profile. Preserve both in your storage instead of collapsing them into “missing.”
• Read metadata.skippedByRobots. If your robots.txt disallows the scanner’s user agent from a path, the related findings turn unknown. That is the scanner respecting your policy, and it is worth knowing before you conclude that a file is absent.

The profile field changes which checks and weights apply. Use auto when you scan a mixed list; set merchant, api, marketplace, mcp-server, or agent-service when you know what the site is, so commerce or API findings are not reported as not applicable by mistake. The documented checks list what each profile covers.

4. Batch-scan a portfolio from a shell script

For a one-off review of many sites, a shell loop is enough. The script below reads one public URL per line, saves each report, and sleeps 65 seconds between calls so a long list stays under 60 scans per hour without any bookkeeping. Sixty sites take about an hour; two hundred take roughly three and a half. If you need faster runs, the API page explains how to ask for a higher limit. Do not parallelize calls to get around the window; the count is per account.

#!/usr/bin/env bash
# sites.txt: one public https:// URL per line, no query strings.
set -u
mkdir -p reports

while read -r url; do
  [ -z "$url" ] && continue
  name=$(printf '%s' "$url" | sed 's#https\?://##; s#[^A-Za-z0-9.-]#_#g')
  status=$(curl -sS --max-time 60 -o "reports/$name.json" -w '%{http_code}' \
    https://adaptmypage.com/api/scan \
    -H "Authorization: Bearer $AGENT_READY_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"url\":\"$url\",\"profile\":\"auto\"}")
  echo "$status $url"
  if [ "$status" = "429" ]; then
    echo "Hourly limit reached. Re-run after RateLimit-Reset." >&2
    break
  fi
  sleep 65
done < sites.txt

Error responses are also JSON, with an error.code and error.message, so they land in the same folder. When you summarize, select only the files that contain a report. This jq command produces a CSV with the target, score, counts, and the IDs of every failed or warned finding:

jq -r 'select(has("score")) |
  [ .normalizedTarget, .score, .summary.fail, .summary.warn, .summary.unknown,
    ([ .findings[] | select(.status == "fail" or .status == "warn") | .id ]
      | join(" ")) ] | @csv' reports/*.json > summary.csv

Sort the CSV by the failed and warned IDs, not by score. Two sites with the same score can need entirely different work. Then send each site’s owner the evidence messages from their report. Evidence is verifiable in a browser; a number is not. Our llms.txt versus robots.txt guide and the structured data guide cover the fixes behind the two most common discoverability findings.

Only scan sites you are authorized to assess. For a prospecting list, scanning a public homepage is a passive read of public files, but the report belongs in a conversation with the owner, not in a public ranking.

5. Schedule weekly monitoring in n8n

For ongoing monitoring of the sites you maintain, a low-code workflow is easier to hand over than a script on someone’s laptop. In n8n, the shape is: a Schedule Trigger, a node that reads your list of sites, an HTTP Request per site, a comparison against the last stored result, and a notification only when a finding changed.

The HTTP Request node documentation describes the JSON body options and Batching; the credential documentation notes that Bearer Auth is header authentication with Authorization: Bearer <token> filled in for you. Enable the response option that includes headers and status so the branch that follows can read RateLimit-Remaining and act on a 429.

For the comparison, store the previous run’s findings per site in a sheet or database keyed by normalizedTarget and finding id. Notify when a status changes, when a new fail or warn appears, or when unknown counts jump, which often means the site started blocking the scanner. Do not notify on score changes alone. A weekly message that says “discoverability.robots_txt moved from pass to warn on client-b.example; evidence: /robots.txt not available (HTTP 404)” is something a developer can act on in a minute.

6. Check production after every deploy with GitHub Actions

The scanner only reaches public URLs. A staging site behind authentication returns 422 target_unavailable. So the useful CI pattern is a check that runs against production after the deploy job finishes, or on a schedule, and fails the workflow if a small set of critical findings regressed. Treat it as a smoke test for agent-facing signals, not as a pre-merge gate.

name: Agent readiness check
on:
  workflow_dispatch:
  schedule:
    - cron: '30 6 * * 1'   # Mondays, 06:30 UTC

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - name: Scan the production homepage
        env:
          AGENT_READY_API_KEY: ${{ secrets.AGENT_READY_API_KEY }}
        run: |
          curl -sS --max-time 60 --fail-with-body -o report.json \
            https://adaptmypage.com/api/scan \
            -H "Authorization: Bearer $AGENT_READY_API_KEY" \
            -H "Content-Type: application/json" \
            -d '{"url":"https://www.example.com","profile":"website"}'

      - name: Fail if a signal agents depend on regressed
        run: |
          jq -e '
            [ .findings[]
              | select(.id == "discoverability.robots_txt"
                    or .id == "discoverability.sitemap_xml"
                    or .id == "discoverability.structured_data"
                    or .id == "security.https_required")
              | select(.status == "fail" or .status == "warn") ]
            | length == 0' report.json

      - name: Keep the report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: agent-readiness-report
          path: report.json

Store the key as a repository secret and reference it through the secrets context, as shown in GitHub’s secrets documentation. Pass it through env, not as a command-line argument that could appear in a process list or log. GitHub redacts registered secrets from logs, but only the values you registered.

jq -e exits non-zero when the final expression is false, so the step fails when any listed finding is not passing. Choose that list deliberately. Ask “would an agent or crawler be blocked or misled if this regressed?” rather than “is this finding open today?”. If llms.txt is a project decision you have not made, do not fail builds over it. If a deploy could accidentally remove robots.txt or serve the site over HTTP, those belong on the list.

To run after a deploy instead of on a schedule, put this job in your deploy workflow with needs: deploy, or trigger it from a deployment event described in GitHub’s workflow trigger reference. Either way, keep the report as an artifact. When the check fails, the evidence message in the report is the first thing you will want to read.

7. Handle rate limits and errors without losing data

Every error is JSON with error.code and error.message. The table maps the documented responses to the action an automation should take. The general rule: fix the input for a 4xx caused by your request, wait for a 429, retry later with backoff for 503 and 504, and never retry on 401.

Retry-After is defined in RFC 9110 and the 429 status in RFC 6585; both allow the server to tell you how long to wait, and this API does. A job that pauses on 429 and continues from the next URL is far more useful than one that fails half a list and reports nothing.

Two hygiene rules complete the picture. Rotate the key from your API keys panel if it ever appears in a log or a shared export; the old key stops working at once. And remember that reports are processed in memory for the response and are not kept in a report archive on our side, as the API FAQ explains. If you want history, you store it.

8. Know the alternatives and the limits

This API is one way to get agent readiness signals as JSON. Two others deserve a mention so you choose deliberately.

• Run the open-source scanner yourself. The API wraps AgentReady, an Apache-2.0 scanner with a CLI. Its scan command has a --fail-on severity option for CI and no account or hourly limit. You take on the hosting, updates, and outbound network policy. The API is the choice when you want a hosted endpoint with a key you can revoke and no Node.js toolchain in your workflow.
• Cloudflare’s URL Scanner. In 2026 Cloudflare added an Agent Readiness report to its URL Scanner, available in the dashboard and through its URL Scanner API. It uses its own categories, so a Cloudflare score and an AgentReady score are not interchangeable. If your stack already lives in Cloudflare, compare the finding lists rather than the totals.

Whatever you use, the passive scan has the same boundary. It reads public HTTP responses. It does not execute JavaScript, so a fact that only appears after rendering is invisible to it; use the crawler rendering guide for that question. It does not submit forms or exercise browser controls; the demo-form test plan and the accessibility-tree guide cover those. MCP metadata discovery in the report is not a test of browser WebMCP tools; use the WebMCP testing workflow. And a score is an alpha diagnostic, not a certification, a ranking factor, or evidence that any agent completed a task on the site.

• The key lives in a secret store or credential manager, never in a workflow file or prompt.
• Calls are paced to stay under 60 per account per hour, and the job pauses on 429 instead of failing.
• Every report is stored with a date and the target, including error responses.
• Alerts and CI failures are based on named finding IDs, not on total score movement.
• unknown and not_applicable are kept as their own states.
• Browser, form, and WebMCP behavior is tested separately and reported as such.