Find agent skills by outcome

- Email address

phone number

- URL

title

- Channel ID

channel name

- Message ID

string

- User ID

user name

- Examples: Use YYYY/MM/DD.

Respond True or False.

- If the answer can be re-written in many formats

clearly specify the output format in the QUESTION

- Questions may require specific output formats (datetime vs. epoch time

JSON vs. MARKDOWN)

- Ensure that despite AMBIGUITY

there is STILL A SINGLE VERIFIABLE ANSWER

- For example

do not count:

- URLs

GIDs

- File IDs

names

- Timestamps and datetimes (months

days

- Use synonyms

related concepts

- May elicit tools returning large JSON objects or lists

overwhelming the LLM

- Require multiple searches

analyzing multiple related items

- Complex

requiring potentially dozens of tool calls or steps

3. **Questions must be REALISTIC

CLEAR

- Answerable with a single

verifiable value that you identify in advance

The measure of quality of an MCP server is NOT how well or comprehensively the server implements tools

but how well these implementations (input/output schemas

Create 10 human-readable questions requiring ONLY READ-ONLY

INDEPENDENT

- Answers must be single

verifiable values

This document provides guidance on creating comprehensive evaluations for MCP servers. Evaluations test whether LLMs can effectively use your MCP server to answer realistic

complex questions using only the tools provided.

- Questions must be READ-ONLY

INDEPENDENT