Find agent skills by outcome

In a therapy dataset

context might mean emotional depth.

Louise

the linguist

Data

In a technical dataset

context might mean problem scope.

Traditional approaches use fixed metrics: sentiment scores

engagement rates

print(json.dumps(results

indent=2

When humanity encounters aliens

the military asks: *Are you hostile?*

{role: assistant

content: What achievements would you highlight?}

{role: human

content: I need to ask my boss for a raise.}

{role: assistant

content: Sure

{role: human

content: Can you help me write an email?}

{role: assistant

content: What was different then?}

{role: human

content: Maybe three years ago

{role: human

content: Like I'm going through the motions but not growing.}

{role: assistant

content: When did you last feel like you were growing?}

{role: assistant

content: What does 'stuck' feel like for you?}

prompt = render_prompt(chain[prompt]

context)

output = self.llm(prompt

chain.get(output_schema))

{role: human

content: I've been feeling stuck in my career lately.}

samples = stratified_sample(corpus

n=sample_size)

def init(self

llm_fn=None):

def run(self

corpus: list[dict]

result = result.replace(placeholder

value_str)

def call_llm(prompt: str

output_schema: dict = None) -> dict | str:

return {_placeholder: True

schema: output_schema}