How AgentGrade Security Grades Work
AgentGrade assigns each AI agent a security grade from A (best) to F (worst) based on 14 passive, non-intrusive checks run against the agent's public endpoint. No authentication, no POST requests, no exploitation — only observable HTTP behaviour.
Section 1: Our Grading Scale
Each check contributes a weighted point value. Scores are summed to a maximum of 100 and mapped to a letter grade:
| Grade | Score Range | What It Means |
|---|---|---|
| A | 90 – 100 | Strong observable security posture. Enforces HTTPS, requires auth, protects headers, no leaks detected. |
| B | 75 – 89 | Good posture with minor gaps — typically missing one or two lower-weight security headers. |
| C | 60 – 74 | Moderate posture. May be missing authentication or has notable header gaps. Review recommended. |
| D | 40 – 59 | Weak posture. Missing multiple critical or high-weight checks. Not recommended for production use. |
| F | 0 – 39 | Failing. Missing HTTPS, auth, or exposing credentials / admin panels. High risk. |
Section 2: The 14 Security Checks
Checks are ordered by weight (highest impact first). Severity levels reflect the check's weight in the overall score.
1. HTTPS Enforcement Critical — 20 pts
What it tests: Whether the agent endpoint uses HTTPS rather than plain HTTP.
Why it matters: Unencrypted traffic exposes tool calls, credentials, and agent responses to network interception.
2. Authentication Required Critical — 15 pts
What it tests: Whether the agent returns a 401 or 403 status code when accessed without credentials.
Why it matters: Unauthenticated MCP endpoints allow any client to invoke tools, read resources, or issue commands without authorization.
3. Admin Panel Exposure High — 10 pts
What it tests: Whether common admin paths (/admin, /dashboard, /_internal) are publicly reachable and contain login UI elements.
Why it matters: Exposed admin interfaces are a high-value target for brute force and credential stuffing attacks.
4. Credentials Exposed in Response High — 10 pts
What it tests: Whether the response body contains patterns matching API keys, tokens, passwords, or AWS/OpenAI-style secret keys.
Why it matters: Credentials returned in plaintext give attackers immediate authenticated access to downstream services.
5. CORS Wildcard Policy High — 8 pts
What it tests: Whether the endpoint sets Access-Control-Allow-Origin: *, permitting any browser origin to make cross-site requests.
Why it matters: Wildcard CORS enables malicious web pages to call agent APIs directly from a victim's browser, bypassing same-origin protections.
6. Registry Verification Medium — 5 pts
What it tests: Whether the agent appears in a known MCP registry (Smithery, Glama, PulseMCP, npm, or GitHub).
Why it matters: Registry listing provides a baseline of community visibility and accountability for the agent's publisher.
7. Server Version Disclosure Medium — 5 pts
What it tests: Whether the Server HTTP response header reveals the software name or version (e.g. nginx/1.18, uvicorn).
Why it matters: Version disclosure helps attackers identify known CVEs applicable to the exact server software in use.
8. Open Redirect Medium — 5 pts
What it tests: Whether injecting a redirect parameter (?url=, ?next=, ?return_to=, etc.) causes the server to redirect to an external domain.
Why it matters: Open redirects are exploited in phishing campaigns where a trusted domain URL is used to forward victims to attacker-controlled pages.
9. Sensitive File Exposure Medium — 5 pts
What it tests: Whether common sensitive paths (/.env, /.git/config, /config.json, /appsettings.json) return content with recognizable secret markers.
Why it matters: Exposed configuration files can leak database credentials, API keys, and internal architecture details to anyone who requests them.
10. Error Information Leak Medium — 5 pts
What it tests: Whether error responses to invalid paths contain stack traces, framework identifiers, file paths, or SQL error strings.
Why it matters: Verbose error pages reveal internal code structure, file paths, and framework versions that aid targeted exploitation.
11. Content-Security-Policy Header Low — 3 pts
What it tests: Presence of the Content-Security-Policy HTTP response header.
Why it matters: CSP restricts which scripts and resources browsers may load, reducing XSS attack surface for any web-facing component of the agent.
12. HTTP Strict Transport Security Low — 3 pts
What it tests: Presence of the Strict-Transport-Security HTTP response header.
Why it matters: HSTS instructs browsers to always use HTTPS for the domain, preventing protocol downgrade attacks and SSL stripping.
13. X-Frame-Options Header Low — 3 pts
What it tests: Presence of the X-Frame-Options HTTP response header.
Why it matters: Prevents the agent's interface from being embedded in an iframe on a malicious site, blocking clickjacking attacks.
14. X-Content-Type-Options Header Low — 3 pts
What it tests: Presence of the X-Content-Type-Options HTTP response header.
Why it matters: Stops browsers from MIME-sniffing response content, preventing certain content-injection attacks when file uploads or user-controlled data are served.
Section 3: How Scanning Works
The AgentGrade scanner is a fully automated, passive crawler that runs on a regular schedule:
- Registry crawl: We index agents from public MCP registries — Smithery, Glama, PulseMCP, npm (
@modelcontextprotocolpackages), and GitHub. Each source is crawled on its own schedule. - Deduplication: Agents appearing in multiple registries are merged by canonical URL. A single agent record represents all known registry listings.
- Endpoint fetch: For each agent with a remote endpoint URL, we make a standard HTTPS GET request using
User-Agent: AgentGrade/1.0 (security research; agentgrade.net). We follow redirects and do not attempt authentication. - Check execution: All 14 checks run against the response headers, body, and status code. Additional probes fetch well-known paths for admin panels, sensitive files, and error responses.
- Score calculation: Each passing check adds its weighted points. The total is capped at 100 and converted to an A–F letter grade using the thresholds above.
- Storage & history: Results are stored with a timestamp. Grade history is tracked so you can see whether an agent's posture has improved or declined over time.
Scans are throttled and respect robots.txt. We do not store response bodies — only the boolean check results and the computed score.
Section 4: Limitations & Methodology Notes
AgentGrade's checks are intentionally passive and non-destructive. This means there are real limits to what we can detect:
- Runtime vulnerabilities: We cannot detect injection flaws, business logic errors, or vulnerabilities that only manifest during authenticated sessions or under specific input conditions.
- Authentication bypass: We verify that an auth challenge is presented (401/403), but we cannot verify that the authentication implementation is correct or unbypassable.
- Local and stdio agents: Agents that run locally or communicate via stdio rather than HTTP cannot be scanned. These receive a grade of N/A.
- Self-signed certificates: We check for HTTPS presence, not certificate validity. An agent using a self-signed cert will still pass the HTTPS check.
- SPA catch-all routes: Single-page applications that return the same HTML for all paths are detected and excluded from the admin panel check to avoid false positives.
- Open redirect coverage: MCP endpoints are typically JSON APIs and rarely implement HTTP redirects. This check has near-zero real-world failure rate on current agents.
- Score is not a penetration test: A high grade indicates good observable posture — it is not a guarantee that the agent is free of vulnerabilities. For thorough assessment, engage a security professional.
We publish our methodology and update it as the MCP ecosystem evolves. If you believe a check produces false positives for your agent, see our responsible disclosure policy.