Fine-tuning a new language: Tokenizer compatibility and dataset requirements

[izyspania]

I am interested in fine-tuning the model for the Romanian language and i have a few technical questions regarding the adaptation process:

Fine-tuning Feasibility: Is it possible to fine-tune the current model specifically for Romanian phonetics? Does the model already have latent support for it?

Dataset Size: What is the recommended size of the dataset (audio-text pairs) required for a high-quality Romanian fine-tune?

Tokenizer Support: Does the underlying text tokenizer natively support Romanian diacritics? Specifically, does it correctly handle the following characters: Ă, ă, Â, â, Î, î, Ș, ș, Ț, ț?

Audio Tokenizer: Will the current audio tokenizer handle the specific vowel sounds and "closed" phonemes of Romanian without additional training?

Do you have any plans for adding Romanian yourself?

Thank you for your work.

[alpacaking]

Thanks for your interest.

With the currently released model, based on my own quick tests, Romanian inference seems to be feasible. I would be very interested in hearing your opinion on the current Romanian quality if you try it, and I would appreciate your feedback.

1. Fine-tuning Feasibility
   Fine-tuning for Romanian should be feasible. Our model seems to already have a basic ability to speak Romanian.

2. Dataset Size
   This depends on your specific task.
   In principle, a dataset about 100-1000 hours may be sufficient for the model to learn the language, but achieving stronger results will naturally require more data, better data quality, and greater data diversity.

3. Tokenizer Support
   Yes, the current text tokenizer natively supports Romanian diacritics.

4. Audio Tokenizer
   Our MOSS-Audio-Tokenizer has already been trained on large-scale data, so it is unlikely to be the main bottleneck here.

Looking forward to your feedback.