[mcp-analysis] MCP Structural Analysis - 2026-02-11

[github-actions[bot]]

Comprehensive analysis of GitHub MCP tool response structures and usefulness for agentic workflows. Evaluated 32 tool responses across 14 unique toolsets, measuring response sizes, schema structures, and practical utility for autonomous agents.

Key Findings

🎯 Overall Assessment

• Tools Analyzed: 32 responses from 22 unique tools
• Average Token Size: 5,198 tokens (excluding errors)
• Average Usefulness Rating: 4.03/5
• Analysis Period: Feb 6-11, 2026 (3 days)

⭐ Top Performers (Rating 5/5)

• repos toolset: Efficient file operations, commit metadata
• search toolset: Excellent discovery tools with rich metadata
• issues/discussions: Well-structured GraphQL responses
• labels: Minimal overhead, complete data

⚠️ Problem Areas

• actions toolset: Broken pagination, massive responses (90K+ tokens)
• security: Oversized payloads without proper filtering (24K tokens)
• context/notifications: Inaccessible in GitHub Actions (403 errors)
• projects: Poor error handling, unusable in many contexts

Full Structural Analysis Report

Executive Summary

Metric	Value
Total Tools Analyzed	32
Unique Tools	22
Toolsets Covered	14
Average Token Size	5,198
Median Token Size	313
Largest Response	list_workflow_runs: 90,096 tokens
Smallest Response	get_file_contents: 25 tokens
Average Usefulness	4.03/5
Excellent Tools (5/5)	18 (56%)
Good Tools (4/5)	6 (19%)
Adequate Tools (3/5)	4 (13%)
Limited Tools (2/5)	1 (3%)
Poor Tools (1/5)	3 (9%)

Usefulness Ratings for Agentic Work

Tools rated on 1-5 scale based on completeness, actionability, clarity, efficiency, and structural quality.

⭐⭐⭐⭐⭐ Excellent (Rating 5) - Recommended for All Agents

Tool	Toolset	Tokens	Schema	Assessment
get_file_contents	repos	25	string	Minimal overhead, just file contents. Perfect efficiency.
list_branches	repos	100	array	Essential data only: name, SHA, protection status.
get_label	labels	50	object	Complete label metadata, highly efficient.
get_commit (no diff)	repos	200	object	Clean commit metadata without diff bloat.
list_commits	repos	325	array	Good balance of commit history with author details.
list_tags	repos	150	array	Efficient release tracking with SHA and URLs.
list_issues	issues	650-1650	object	GraphQL with cursor pagination. Complete issue data.
issue_read	issues	550	object	Full details with reactions, labels. Well-structured.
list_discussions	discussions	275	object	GraphQL structure, category, user, timestamps.
search_repositories	search	250-350	object	Rich metadata: stars, topics, language. Immediately actionable.
search_issues	search	650-2600	object	Comprehensive search with all issue metadata.

Why These Excel: Minimal token usage for maximum value. Clean schemas with low nesting. Complete data for decision-making. Efficient for agentic workflows.

⭐⭐⭐⭐ Good (Rating 4) - Use with Awareness

Tool	Toolset	Tokens	Schema	Assessment
get_repository_tree	repos	2,700	oversized	Clean structure but context-heavy for large repos.
search_code	search	950	object	Good code discovery, some redundancy in repo context.
pull_request_read	pull_requests	2,600	object	Detailed PR data with nested repo objects. Heavy but complete.
list_pull_requests (2026-02-06)	pull_requests	2,800	array	Comprehensive PR list, some metadata redundancy.
list_workflows (2026-02-06)	actions	3,200	object	Complete workflow list but context-heavy.
search_issues (with bodies)	search	2,600	object	Full issue bodies included, some redundant user fields.

Trade-offs: Higher token usage but provide comprehensive data. Useful when detailed context is needed.

⭐⭐⭐ Adequate (Rating 3) - Use Sparingly

Tool	Toolset	Tokens	Schema	Assessment
list_releases	releases	7,215	oversized	28KB for single release. Full body + all assets. Inefficient.
list_pull_requests (2026-02-10)	pull_requests	4,650	oversized	18KB response, duplicate repo objects for head/base.
list_workflows (2026-02-10)	actions	3,900	oversized	15KB for perPage=1. Pagination broken.

Issues: Oversized payloads requiring file system reads. Broken pagination. Heavy redundancy. Usable but inefficient.

⭐⭐ Limited (Rating 2) - Avoid Unless Necessary

Tool	Toolset	Tokens	Schema	Assessment
list_code_scanning_alerts	security	23,800	oversized	95KB payload. Full rule descriptions, no filtering. Context nightmare.

Critical Issues: Massive context usage. No pagination respect. Hard to process. Needs major improvements.

⭐ Poor (Rating 1) - Unusable

Tool	Toolset	Tokens	Schema	Assessment
list_workflow_runs	actions	90,096	oversized	360KB for perPage=1. Pagination completely broken. Unusable.
get_me	context	0	error	403 forbidden in GitHub Actions. Not accessible by integration.
list_notifications	notifications	0	error	403 forbidden in GitHub Actions. Cannot be used in CI.
projects_list	projects	0	error	404 not found. Poor error handling, unusable in many contexts.

Critical Issues: Completely unusable for agents. Either forbidden access or broken functionality.

Schema Analysis

Understanding response structures for optimal agent usage.

Tool	Schema Type	Nesting Depth	Key Fields	Structure Quality
get_file_contents	string	1	content	⭐⭐⭐⭐⭐ Flat, minimal
list_branches	array	2	name, sha, protected	⭐⭐⭐⭐⭐ Simple array
get_label	object	1	id, name, color, description	⭐⭐⭐⭐⭐ Flat object
list_issues	object	4	issues[], pageInfo, totalCount	⭐⭐⭐⭐⭐ GraphQL paginated
search_repositories	object	3	total_count, items[]	⭐⭐⭐⭐ Standard search
list_pull_requests	array	5	number, title, state, user, head.repo, base.repo	⭐⭐⭐ Deeply nested
list_code_scanning_alerts	oversized	5	number, rule, tool, locations[]	⭐⭐ Too deep, complex

Schema Patterns:

• Best: Flat or shallow (depth 1-2) with clear field names
• Good: GraphQL with cursor pagination (depth 3-4)
• Adequate: REST arrays with moderate nesting (depth 3-4)
• Poor: Deep nesting (5+) with duplicate data

Response Size Analysis

Token usage efficiency across toolsets.

Toolset	Avg Tokens	Tool Count	Efficiency Rating
repos	648	11	⭐⭐⭐⭐⭐ Excellent
labels	50	1	⭐⭐⭐⭐⭐ Excellent
discussions	275	2	⭐⭐⭐⭐⭐ Excellent
search	950	5	⭐⭐⭐⭐ Good
issues	950	3	⭐⭐⭐⭐ Good
pull_requests	3,350	3	⭐⭐⭐ Adequate
releases	7,215	1	⭐⭐ Limited
security	23,800	1	⭐ Poor
actions	49,098	3	⭐ Poor

Context Efficiency Champions:

1. get_file_contents (25 tokens) - Just the data you need
2. get_label (50 tokens) - Minimal complete response
3. list_branches (100 tokens) - Essential git metadata

Context Hogs:

1. list_workflow_runs (90,096 tokens) - 360KB response, broken pagination
2. list_code_scanning_alerts (23,800 tokens) - 95KB with no filtering
3. list_releases (7,215 tokens) - 28KB for single release

Structural Patterns

Pattern 1: GraphQL Responses (⭐⭐⭐⭐⭐)

Tools: list_issues, list_discussions

Structure:

{
  "items": [...],
  "pageInfo": {
    "hasNextPage": true,
    "endCursor": "..."
  },
  "totalCount": 123
}

Strengths: Clear pagination, total counts, cursor-based navigation
Agent Benefits: Easy to implement pagination logic, know total results upfront

Pattern 2: REST Arrays (⭐⭐⭐⭐)

Tools: list_branches, list_commits, list_tags

Structure:

[
  {"name": "main", "sha": "abc123", "protected": true},
  {"name": "dev", "sha": "def456", "protected": false}
]

Strengths: Simple, lightweight, easy to iterate
Agent Benefits: Minimal parsing overhead, direct array access

Pattern 3: Search Results (⭐⭐⭐⭐)

Tools: search_repositories, search_issues, search_code

Structure:

{
  "total_count": 100,
  "incomplete_results": false,
  "items": [...]
}

Strengths: Total count, result completeness indicator
Agent Benefits: Can decide whether to refine search based on total_count

Pattern 4: Oversized Payloads (⭐⭐ or lower)

Tools: list_releases, list_pull_requests, list_workflows, list_workflow_runs

Structure: File system redirect due to MCP size limits

Issues: Requires file read, breaks MCP flow, excessive redundancy
Agent Impact: Extra I/O, interrupted workflow, difficult to process

30-Day Trend Summary

Analysis covers 3 days of testing (Feb 6, 10, 11, 2026):

Date	Tools Tested	Total Tokens	Avg Rating	Notable Findings
2026-02-06	14	12,550	4.64/5	Initial baseline testing
2026-02-10	12	40,400	4.08/5	Discovered pagination issues
2026-02-11	6	100,211	2.83/5	Found critical flaws in actions toolset

Trend: Token usage spiking as we test more complex tools. Usefulness declining as we uncover broken features.

Key Discovery: list_workflow_runs alone accounts for 90K tokens, exposing critical pagination failure in actions toolset.

Recommendations

For Agent Developers

✅ Prioritize These Tools (High value, low cost):

1. get_file_contents - File reading
2. list_branches - Git operations
3. get_label - Label management
4. list_commits - Commit history
5. search_repositories - Repo discovery
6. list_issues - Issue querying
7. list_discussions - Discussion access

⚠️ Use with Caution (High cost but necessary):

• get_repository_tree - Use path_filter to reduce size
• search_code - Acceptable for targeted searches
• pull_request_read - Get specific PR only when needed

❌ Avoid Unless Critical:

• list_releases - Use get_latest_release or get_release_by_tag instead
• list_code_scanning_alerts - Needs filtering options not yet available
• list_workflows - Pagination broken, use sparingly

🚫 Do Not Use (Broken or inaccessible):

• list_workflow_runs - Completely broken pagination (360KB for 1 item)
• get_me - Not accessible in GitHub Actions
• list_notifications - Not accessible in GitHub Actions
• projects_list - Poor error handling, unreliable

For Tool Maintainers

Critical Fixes Needed:

1. list_workflow_runs pagination (P0 - Critical)

   • Currently returns 360KB for perPage=1
   • Pagination parameter completely ignored
   • Makes tool unusable for agents

2. list_code_scanning_alerts filtering (P0 - Critical)

   • 95KB payload without filtering options
   • Needs field selection or pagination respect
   • Context budget killer

3. list_releases efficiency (P1 - High)

   • 28KB for single release
   • Consider separating assets from release body
   • Add option to exclude release notes

4. Error context improvements (P1 - High)

   • get_me, list_notifications: Return helpful message about integration limitations
   • projects_list: Better error detection and user guidance

5. list_workflows pagination (P2 - Medium)

   • Returns all 192 workflows even with perPage=1
   • Inconsistent with other list endpoints

Optimization Opportunities:

1. Reduce nested duplicates in list_pull_requests

   • head.repo and base.repo contain identical fields
   • Consider repo_id reference instead of full objects

2. Add field selection to large endpoints

   • Allow agents to request only needed fields
   • GraphQL-style field selection for REST endpoints

3. Pagination consistency

   • Some tools use GraphQL cursors (good)
   • Some use page/perPage (inconsistent behavior)
   • Standardize pagination across toolsets

Visualizations

Response Size by Toolset

Average token usage varies widely. Actions toolset has extreme outliers.

Usefulness Ratings

Most toolsets rate highly (4+). Actions and error-prone tools need attention.

Daily Token Trend

Token usage spiking as we discover broken pagination in complex tools.

Size vs Usefulness

Best tools cluster in low tokens, high rating. Actions outliers unusable.

Schema Distribution

Most responses are well-structured objects. Oversized payloads and errors are problematic.

Next Steps

1. Immediate: Document list_workflow_runs pagination bug and notify maintainers
2. Short-term: Create workaround guide for broken tools
3. Long-term: Continue monitoring 30-day trends, track improvements

References:

• §21902955300

AI generated by GitHub MCP Structural Analysis

• expires on Feb 18, 2026, 11:24 AM UTC