You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, we can specify an output_regex in the task config to extract the model answer from its generated content. This config option is used in apply_generative_metrics.
We could replace output_regex with an answer extractor object that performs arbitary operations to extract the answer. For example, that of MMLU-Pro is as follows:
The text was updated successfully, but these errors were encountered:
Hi there!
Currently, we can specify an
output_regex
in the task config to extract the model answer from its generated content. This config option is used inapply_generative_metrics
.We could replace
output_regex
with an answer extractor object that performs arbitary operations to extract the answer. For example, that of MMLU-Pro is as follows:The text was updated successfully, but these errors were encountered: