Find agent skills by outcome

2. Load each task instruction and in parallel using the MCP server and tools

identify the correct answer by attempting to solve the task YOURSELF

After creating your evaluation file

you can use the provided evaluation harness to test your MCP server.

Productivity

Remember to parallelize solving tasks to avoid running out of context

then accumulate all answers and make changes to the file at the end.

- The answer will change as issues are created

closed

- LLM might format differently (JSON array

comma-separated

<answer>repo1

repo2

- Requires filtering bugs by date

priority

- Answer is a simple

verifiable value

<question>Among all bugs reported in January 2024 that were marked as critical priority

which assignee resolved the highest percentage of their assigned bugs within 48 hours? Provide the assignee's username.</question>

- In all tool call requests

use the `limit` parameter to limit results (<10)

Coding & Debugging

<question>Look for pull requests that modified files in the /api directory and were merged between January 1 and January 31

2024. How many different contributors worked on these PRs?</question>

- Make INCREMENTAL

SMALL

- Ensure each subagent is only performing READ-ONLY

NON-DESTRUCTIVE

- Use your intuition and understanding to create reasonable

realistic

- Understand input/output schemas

docstrings

Coding & Debugging

After understanding the API and tools

USE the MCP server tools:

- If ambiguity exists

fetch additional information from the web

- Channel concept: channel ID

channel name

- Message concept: message ID

message string

- User concept: user ID

user name

- Examples: names

first name

- Answers must not require special formatting or complex

structured output

- File ID

file name

- Timestamp

datetime