Not compatible with HF transformers >=5.0.0

[moritzharmel-doordash]

In our environment we use mosaic streaming and hugging face transformers libraries.

In order to run latest qwen models I need transformers >= 5.2.0
https://github.com/huggingface/transformers/releases/tag/v5.2.0

However, mosaic requires transformers<5 here
https://github.com/mosaicml/streaming/blame/6a4e12f66ee64d28bd4c5b308cec7fd825222205/setup.py#L55

This was set 4 years ago but probably is not a strict requirement.

Could you do some testing with newer transformers version and if it works remove the upper limit on the transformers version?

Thank you!