Review TextClassification task

[sdiazlor]

No description provided.

[github-actions]

Documentation for this PR has been built. You can view it at: https://distilabel.argilla.io/pr-1073/

[codspeed-hq]

CodSpeed Performance Report

Merging #1073 will improve performances by ×7.2

_{Comparing feat/review-text-classification (1e65f6a) with develop (63c75c5)}

Summary

⚡ 1 improvements

Benchmarks breakdown

	Benchmark	develop	feat/review-text-classification	Change
⚡	test_cache_time	3,985.3 ms	550.3 ms	×7.2

[plaguss]

Hi @sdiazlor! The thing with the n attribute is that it's needed for a task like TextClustering, where you want to enforce the same number of labels. I see that it can be useful to have it as single/multi label, but both this option and a predefined set of n options should be available, so an extra argument could work

[davidberenstein1957]

Hi @sdiazlor, I think you should also update the prompt template to make this work smoothly.

[davidberenstein1957]

why a randomint 1,3?

[sdiazlor]

Yes, I was doubting about the best approach as I didn't want to implicit a fixed number of labels. So, that's why I randomized it.

[davidberenstein1957]

I think we need a more fundamental change to this implementation on the level of the prompt template. We ideally want to have an arbitrary number of labels based on a potential set, where we should allow for 0 to n labels without it forcefully setting a fixed number.

[davidberenstein1957]

@plaguss perhaps we don't know enough about the context of the paper/implementation but when would it be useful to set a fixed number? Normally in a mulit-label textcat setting, you would go for a random number of labels without enforcing the exact required number because it leads to mis-labelling.

[sdiazlor]

@plaguss Thanks! I read the paper and it was more focused on structured generation rather than texcat, I guess we can more or less modify the task, right? So, would it be possible to optionally select between n or multi_label on an exclusive basis (updating the code/prompt conditionally), so TexClustering won't be broken, but for "standard textcat" we don't need to use n.

[plaguss]

@plaguss perhaps we don't know enough about the context of the paper/implementation but when would it be useful to set a fixed number? Normally in a mulit-label textcat setting, you would go for a random number of labels without enforcing the exact required number because it leads to mis-labelling.

This is the task implementing the TextClustering: https://github.com/argilla-io/distilabel/blob/main/src/distilabel/steps/clustering/text_clustering.py, that's the reason to specify a given set of labels, the approach is a bit different to a given text classification problem, but instead it reads a bunch of texts and try to obtain a set of representative labels.

[plaguss]

@plaguss Thanks! I read the paper and it was more focused on structured generation rather than texcat, I guess we can more or less modify the task, right? So, would it be possible to optionally select between n or multi_label on an exclusive basis (updating the code/prompt conditionally), so TexClustering won't be broken, but for "standard textcat" we don't need to use n.

Totally. As long as both options are available work it's perfect