TypeError: an integer is required

[PiXianDouBanJiang]

I failed on the given example with a typeerror.

TypeError Traceback (most recent call last)
Cell In[1], line 6
3 prediction = "def add ( a , b ) :\n return a + b"
4 reference = "def sum ( first , second ) :\n return second + first"
----> 6 result = calc_codebleu([reference], [prediction], lang="python", weights=(0.25, 0.25, 0.25, 0.25), tokenizer=None)
7 print(result)

File d:\cc\LLM_DynaEval_Coding\mbpp+\codebleu\codebleu.py:41, in calc_codebleu(references, predictions, lang, weights, tokenizer, keywords_dir)
38 assert keywords_dir.exists(), f"keywords_dir {keywords_dir} does not exist"
40 # get the tree-sitter language for a given language
---> 41 tree_sitter_language = get_tree_sitter_language(lang)
43 # preprocess inputs
44 references = [[x.strip() for x in ref] if isinstance(ref, list) else [ref.strip()] for ref in references]

File d:\cc\LLM_DynaEval_Coding\mbpp+\codebleu\utils.py:163, in get_tree_sitter_language(lang)
160 elif lang == "python":
161 import tree_sitter_python
--> 163 return Language(tree_sitter_python.language())
164 elif lang == "go":
165 import tree_sitter_go

TypeError: an integer is required

[chao-peng]

pip install tree-sitter-python==0.21

will solve the problem.

[ranimkhojah]

pip install tree-sitter-python==0.21 did not work for me. @PiXianDouBanJiang did you find a way to make it work?

[aur3l14no]

I got it working with tree-sitter==0.23.1 and codebleu==0.7.1.
The latter is not built and uploaded to PyPI yet, so you'd have to pip install git+https://github.com/k4black/codebleu#egg=codebleu.

[raptech-jp]

I’m also encountering the TypeError: an integer is required issue with tree-sitter==0.23.1 and codebleu==0.7.1. I tried pip install tree-sitter-python==0.21 as suggested by @chao-peng and installing CodeBLEU directly from GitHub as suggested by @aur3l14no, but neither resolved the issue. Has anyone found a reliable solution?

[raptech-jp]

I have an update. I wasn’t able to resolve the issue on my x86 Ubuntu environment, but I managed to get it working on an M3 MacBook with the following setup:

% pip freeze
codebleu==0.7.0
tree-sitter==0.23.2
tree-sitter-python==0.23.2

[Weigang-Wu]

pip install tree-sitter-python==0.21

will solve the problem.

Thanks，this method has solved my question.

[nielstron]

Since this is resolved by installing tree-sitter-0.23.1, the commit in bedf9e8 should resolve the issue. It is just not yet released.

[RyanLoil]

Confirmed from @nielstron.

Since this is resolved by installing tree-sitter-0.23.1, the commit in bedf9e8 should resolve the issue. It is just not yet released.

The problem is caused by capabilities of tree-sitter.

Somehow the metadata of codebleu write the following incompatible dependency in ver. 0.7.0:

Requires-Dist: tree-sitter <0.23.0,>=0.22.0
Requires-Dist: setuptools >=61.0.0
Provides-Extra: all
Requires-Dist: tree-sitter-python~=0.21 ; extra == 'all'
Requires-Dist: tree-sitter-go ~=0.21 ; extra == 'all'
Requires-Dist: tree-sitter-javascript ~=0.21 ; extra == 'all'
Requires-Dist: tree-sitter-ruby ~=0.21 ; extra == 'all'
Requires-Dist: tree-sitter-php ~=0.22 ; extra == 'all'
Requires-Dist: tree-sitter-java ~=0.21 ; extra == 'all'
Requires-Dist: tree-sitter-c-sharp ~=0.21 ; extra == 'all'
Requires-Dist: tree-sitter-c ~=0.21 ; extra == 'all'
Requires-Dist: tree-sitter-cpp ~=0.22 ; extra == 'all'
Requires-Dist: tree-sitter-rust ~=0.21 ; extra == 'all'

this "tree-sitter-language~=0.21" means that the pip will automatically install any version of tree-sitter below 1.0.
Right now the tree-sitter-language main version is 0.23.* that will be installed automatically by the line.
This version is not compatible with tree-sitter==0.22.* tree-sitter/tree-sitter-python#280 which will also be installed automatically by line Requires-Dist: tree-sitter <0.23.0,>=0.22.0 and then will cause exception as followed:

return Language(tree_sitter_python.language())
TypeError: an integer is required

The correct writting is ~=0.21.0, which means install any version below 0.22.0, which you can't dirrectly fixed in pypi or pip.
At this moment, you will have both tree-sitter-language==0.23.* and tree-sitter==0.22.* in your environment.

So the manual fixing is to:

Update your tree-sitter to 0.23.* (this will cause your IDE capabilities warning as the metadata requires tree-sitter <0.23.0,>=0.22.0, but it works).

OR

Uninstall codebleu by pip-autoremove and reinstall ver. 0.7.1 through git+https://github.com/k4black/codebleu.git

OR

Downgrade ALL tree-sitter-language packages to ver. 0.22.0

Anyway, this bug will finally be fixed as the team release 0.7.1 on PyPi.
I wish they can notice the problems of metadata and fix them before the next major update of tree-sitter.