Optimizing Metrics with BootstrapFewShot

[Haknt]

I want to evaluate the grammatical accuracy of outputs.

Directly requesting scores gave poor results, so I switched to calculating scores based on the distance after grammar correction.

However, the grammar correction using Signature and Predict still isn’t performing well. To improve the Signature, I’ve prepared 10 examples and plan to use BootstrapFewShot. What inputs should I use when optimizing an existing metric with BootstrapFewShot? If so, what inputs would be suitable for this purpose?

[arnavsinghvi11]

Hi @Haknt , this BootstrapFewShot doc should help clarifying what inputs to use to optimize for your metric.