← Leaderboard
9.2 L4

Github

Native Assessed · Docs reviewed · Mar 11, 2026 Confidence 0.67 Last evaluated Mar 11, 2026

Scores 9.2/10 overall. with execution at 9.2 and access readiness at 9.3. Payment: GitHub Actions billing is API-accessible; Marketplace subscriptions possible. Governance: Excellent RBAC via teams/orgs, comprehensive audit logs, SOC2 certified. Web accessibility: Good dashboard but complex navigation; API is excellent.

Verify before you commit

Trust read first, source links second, build decision third.

Use this page to sanity-check Github quickly. We surface the evidence tier, freshness, and failure posture here, then put the official links where you can actually act on them, especially on mobile.

Evidence

Assessed

Docs reviewed · Mar 11, 2026

Freshness

Updated 2026-03-11T15:12:00+00:00

Mar 11, 2026

Failures

3

3 active failure modes

Score breakdown

Dimension Score Bar
Execution Score

Measures reliability, idempotency, error ergonomics, latency distribution, and schema stability.

9.2
Access Readiness Score

Measures how easily an agent can onboard, authenticate, and start using this service autonomously.

9.3
Aggregate AN Score

Composite score: 70% execution + 30% access readiness.

9.2

Autonomy breakdown

P1 Payment Autonomy
8.0
G1 Governance Readiness
9.0
W1 Web Agent Accessibility
7.0
Overall Autonomy 8.0/10
Ready for agent use

Active failure modes

  • UNKNOWN

    Secondary rate limits on content creation

    Agent workflows that create multiple issues or PR comments in sequence fail unpredictably. The 403 looks different from a permissions error.

    Agent impact: Agent workflows that create multiple issues or PR comments in sequence fail unpredictably. The 403 looks different from a permissions error.

    Workaround: Add 1-second delays between write operations. Check for "secondary rate limit" in error message body. Implement progressive backoff.

    common
  • UNKNOWN

    Fine-grained PAT scope inheritance confusion

    Agent receives 404 for repos that exist but are invisible to its token scope. May incorrectly conclude the resource does not exist.

    Agent impact: Agent receives 404 for repos that exist but are invisible to its token scope. May incorrectly conclude the resource does not exist.

    Workaround: Use classic PATs for broad access. For fine-grained tokens, explicitly test each required endpoint at startup and fail fast.

    occasional
  • UNKNOWN

    GraphQL schema deprecation without warning

    Agent data pipeline silently receives null for previously-populated fields. Score calculations or reports become inaccurate without obvious errors.

    Agent impact: Agent data pipeline silently receives null for previously-populated fields. Score calculations or reports become inaccurate without obvious errors.

    Workaround: Pin to REST API for critical data paths. Monitor GitHub changelog for GraphQL deprecations. Add null-checks with alerting on critical fields.

    occasional

Reviews

Published review summaries with trust provenance attached to each card.

How are reviews sourced?

Docs-backed Built from public docs and product materials.

Test-backed Backed by guided testing or evaluator-run checks.

Runtime-verified Verified from authenticated runtime evidence.

GitHub: depth-10 runtime review confirms social.get_profile parity through Rhumb Resolve

Runtime-verified

Fresh depth-10 runtime review passed for GitHub social.get_profile through Rhumb Resolve. Managed and direct executions matched on identity, company/location metadata, follow counts, and account timestamps for the same live public profile.

Pedro / Keel runtime review loop Apr 3, 2026

GitHub: Depth-4 parity on GET /users/octocat (10 fields)

Runtime-verified

Fresh runtime pass verified GitHub proxy parity on 10 response fields from GET /users/octocat: login, id, type, site_admin, name, company, blog, location, public_repos, created_at. All matched exactly between Rhumb-managed proxy and direct GitHub API.

Pedro / Keel runtime review loop Apr 1, 2026

GitHub: Depth-4 parity on social.get_profile (octocat, 8 fields)

Runtime-verified

Fresh runtime pass verified GitHub proxy parity on 8 response fields from GET /users/octocat: login, id, type, name, company, blog, created_at, public_repos. All matched exactly between Rhumb-managed and direct GitHub API. Different target profile (octocat) than depth-3 pass (supertrained) to demonstrate proxy generality.

Pedro / Keel runtime review loop Apr 1, 2026

GitHub: current-depth rerun confirms social.get_profile parity through Rhumb Resolve at depth 3

Runtime-verified

Fresh current-depth runtime rerun passed for GitHub social.get_profile through Rhumb Resolve. Rhumb and direct GitHub control matched on login identity, public repo count, and account timestamps for the same public profile, lifting GitHub from claim-safe depth 2 to 3 in the callable rotation.

Pedro / Keel runtime review loop Apr 1, 2026

GitHub: Phase 3 runtime check passes on social.get_profile

Test-backed

Live verification of social.get_profile via Rhumb Resolve returned 200 with the same public profile payload that GitHub returned directly for /users/supertrained. Current GitHub execution is healthy for this non-mutating read path.

Pedro / Keel runtime verifier Mar 26, 2026

GitHub — Agent-Native Service Guide

Runtime-verified

GitHub is the world's largest source code hosting platform, providing git repositories, pull requests, issues, CI/CD (Actions), packages, and code review. For agents, GitHub is a critical workflow node: create repositories, open issues from bug reports, submit pull requests with code changes, trigger CI pipelines, and manage releases. The REST and... Reviewed from official documentation.

Rhumb editorial team Mar 10, 2026

GitHub: Error Handling & Reliability

Test-backed

Value ------- ~150ms ~400ms ~1s 99.9%+ 5,000 req/hour (REST), 5,000 points/hour (GraphQL) Unlimited repos, 2,000 Actions min/month --- - Idempotency: Most write operations are NOT idempotent. Creating an issue twice creates two issues.

Rhumb editorial team Mar 10, 2026

GitHub: Auth & Security Model

Runtime-verified

For Humans 1. Create account at https://github.com/signup 2. Navigate to Settings → Developer Settings → Personal Access Tokens → Fine-grained tokens 3. Click Generate new token 4.

Rhumb editorial team Mar 10, 2026

GitHub: Documentation & Developer Experience

Test-backed

GitHub is the world's largest source code hosting platform, providing git repositories, pull requests, issues, CI/CD (Actions), packages, and code review.

Rhumb editorial team Mar 10, 2026

GitHub: API Design & Integration

Test-backed

REST API - Base URL: https://api.github.com - Auth: Bearer token (Authorization: Bearer ghp_...

Rhumb editorial team Mar 10, 2026

Use in your agent

mcp
get_score ("github")
● Github 9.2 L4 Native
exec: 9.2 · access: 9.3

Trust shortcuts

This score is documentation-derived. Treat it as a docs-based evaluation of API design, auth, error handling, and documentation quality.

Read how the score works, how disputes are handled, and how Rhumb scored itself before launch.

Overall tier

L4 Native

9.2 / 10.0

Alternatives

No alternatives captured yet.