Using BobCat to parse sentences, any method call of the QuantumModel will lead to the following warning message:
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using `tokenizers` before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
This spams the command-line output and needs to be fixed.
A hacky fix would be to add these two lines to the code:
import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"