Find agent skills by outcome

After creating your evaluation file

you can use the provided evaluation harness to test your MCP server.

Productivity

2. Load each task instruction and in parallel using the MCP server and tools

identify the correct answer by attempting to solve the task YOURSELF

Productivity

Remember to parallelize solving tasks to avoid running out of context

then accumulate all answers and make changes to the file at the end.

- LLM might format differently (JSON array

comma-separated

<answer>repo1

repo2

- The answer will change as issues are created

closed

- Requires filtering bugs by date

priority

- Answer is a simple

verifiable value

<question>Among all bugs reported in January 2024 that were marked as critical priority

which assignee resolved the highest percentage of their assigned bugs within 48 hours? Provide the assignee's username.</question>

Coding & Debugging

<question>Look for pull requests that modified files in the /api directory and were merged between January 1 and January 31

2024. How many different contributors worked on these PRs?</question>

- In all tool call requests

use the `limit` parameter to limit results (<10)

Writing & Content

After inspecting the content

create 10 human-readable questions:

- Make INCREMENTAL

SMALL

- Ensure each subagent is only performing READ-ONLY

NON-DESTRUCTIVE

Writing & Content

- Goal: identify specific content (e.g.

users

- Use your intuition and understanding to create reasonable

realistic

- Understand input/output schemas

docstrings

Coding & Debugging

After understanding the API and tools

USE the MCP server tools:

- Message concept: message ID

message string

- If ambiguity exists

fetch additional information from the web

- Channel concept: channel ID

channel name

Research & Learning

- Example: if finding a paper name

be SPECIFIC enough so answer is not confused with papers published later

- User concept: user ID

user name

Creative

- Questions must be designed so there is a single

clear answer