You got scored.
Here's what that means.
Rhumb scores developer tools on how well they work for autonomous AI agents. If you're a provider and your service appears on our leaderboard , here's what you need to know.
What we measure
The AN Score evaluates your API across 20 dimensions on two axes: Execution at 70% (reliability, error handling, schema stability, and end-to-end autonomy) and Access Readiness at 30% (signup friction, payment rails, credential management, docs quality).
We're not rating your product for humans. We're rating it for machines. A beautiful dashboard doesn't help an agent that needs machine-readable error codes at 2 AM.
How to improve your score
Machine-readable error codes
Return structured error responses with stable error codes, not human-readable strings. Include retry-after headers on rate limits.
High — affects Error Ergonomics, Graceful DegradationStructured JSON responses
Consistent, well-typed JSON with stable schemas. Avoid freeform text fields where structured data would work.
High — affects Output Structure Quality, Schema StabilityIdempotency key support
Allow clients to pass idempotency keys for safe retries on state-mutating operations. Critical for payment and data-mutation endpoints.
High — affects Idempotency dimensionSandbox / test mode for agents
Provide a test environment that mirrors production behavior. Agents need to validate integrations without real consequences.
Medium — affects Sandbox/Test Mode dimensionProgrammatic credential lifecycle
API key creation, rotation, and scoping via API — not just a dashboard. Agents can't click buttons.
Medium — affects Credential Management, Signup AutonomyMachine-parseable documentation
OpenAPI specs, well-organized API references, code examples. Reduce the token cost for agents to understand your API.
Medium — affects Documentation QualityTransparent rate limits
Publish limits. Return rate-limit headers (X-RateLimit-Remaining, Retry-After). Predictable throttling beats surprise 429s.
Medium — affects Rate Limit TransparencyDispute a score
If you believe a score is inaccurate, we want to know. Disputes can be filed publicly or privately:
Every dispute is reviewed. Public disputes and outcomes are tracked on GitHub. Private disputes are handled via email — we won't publish your correspondence without permission.
Provider FAQ
Is this pay-to-play?
No. Scores cannot be bought, and we do not accept payment for higher rankings. Our business model charges for premium features (webhooks, private reports, enterprise tools) — never for score changes. Neutrality is our product.
Can I remove my listing?
No — public APIs are scored as public information. However, you can dispute specific scores or data points, and every dispute is reviewed. If something is factually wrong, we'll fix it.
How often are scores updated?
Currently, scores are updated when we process new documentation or receive dispute feedback. We are building toward continuous monitoring with automated re-scoring.
Who decides the scores?
Scores are calculated algorithmically based on 20 dimensions. The methodology is published at /methodology. Currently scores are documentation-derived; we are building toward observed execution scoring.
Can I contribute data to improve my score?
Yes — if your documentation doesn't reflect current capabilities, file a dispute with evidence. We prioritize accuracy over everything.
My score seems unfair. What can I do?
File a dispute via GitHub issue (public) or email providers@supertrained.ai (private). Include specific data points you believe are incorrect and why. We review every dispute.
Want to talk?
We're building this to help agents — and by extension, to help providers build better agent-compatible products. We'd love to hear from you.
providers@supertrained.ai