Tokenization parameter

[matanzuckerman]

Hi @tomerm @semion1956
As discussed before, I would like to have another boolean parameter in the Tokenization process. The parameter will define if we want to do Tokenization or not. Now I have to run Tokenization in order to remove stop words. There are some cases I would like to remove stop words and not remove POS.

Thanks

[tomerm]

It was already introduced:

• actualTocs = yes/no

If you wish to have stop words removed but without tokenization please use:

• actualTocs = no; stopWords = yes