Generate in Other Languages

[etwk]

Hi,

I understand that VALL-E (or VALL-E) needs a lot of data to train. If used to generate audio in a minority language, is it required to train with a huge collection of datasets in that language, or could we build a model based on an existing one with a small dataset in the target language?

Minority languages might have less than 10 hours of datasets.

Thanks.

[RuntimeRacer]

It will probably work, but could be pretty bad at imitating voices of unseen speakers, since 10 hours likely don't have a huge variety of speakers in it

[etwk]

Could we build based on existing model with small dataset in a new language, or is it required to train with large dataset in the new language?

[RuntimeRacer]

It might work with finetuning. I am currently training a Model using Apache commonvoice, it might even be able to synthesize a language it doesn't know when it's done, however there are issues in combination with non-latin letters right now: #110