Newer transformers is incompatible with polyjuice_nlp

[qiaw99]

1.

While testing cfe code, which requires both transformers and polyjuice_nlp packages, figures out that after installing everything needed according to the offical website from polyjuice_nlp, there is always an error:

ValueError: The following `model_kwargs` are not used by the model: ['n'] (note: typos in the generate arguments will also show up in this list)

This issue is also mentioned in repo of transformers.

The easiest way to solve it, is just to use lower version.

2.

If running code and encounter this problem:

AttributeError: Can't get attribute 'Trie' on <module 'transformers.tokenization_utils'. 

One similar issue can be found here.

Make sure that your transformers version is 4.18.0

3.

If you want to install polyjuice_nlp, it's better to clone the git and install then directly(instead of pip/conda).

[qiaw99]

If you encounter this error using polyjuice:

KeyError: 'generated_text'

Then just install the package from source code

git clone [email protected]:tongshuangwu/polyjuice.git
cd polyjuice
pip install -e .

Meanwhile, make sure you have install nltk and spacy.

pip install -U pip setuptools wheel
pip install -U spacy
python -m spacy download en_core_web_sm

And you have to put these code before you execute code from polyjuice

import nltk
nltk.download('omw-1.4')

import spacy
spacy.load('en_core_web_sm')

[qiaw99]

Please just run the script under utils for convenience