txt+ult2wav

Implementation of Tamás Gábor Csapó, László Tóth, Gábor Gosztolya, Alexandra Markó, ,,Speech Synthesis from Text and Ultrasound Tongue Image-based Articulatory Input'', ISCA 11th Speech Synthesis Workshop (SSW11), 2021, accepted, arXiv:2107.02003

txt+ult2wav is extending the original Merlin toolkit, for articulatory-to-acoustic mapping (ultrasound-to-speech) purposes.

For data, the UltraSuite-TaL corpus is used.

additional requirement: ultrasuite-tools

txt+ult2wav recipes:

• ultrasound-to-speech
• text-to-speech
• text&ultrasound-to-speech

txt+ult2wav pre-trained models :

• ultrasound-to-speech pre-trained model (Apr 21, 2021; 3 GB)
• text-to-speech pre-trained model (Apr 21, 2021; 3 GB)
• text&ultrasound-to-speech pre-trained model (Apr 22, 2021; 4 GB)

Merlin: The Neural Network (NN) based Speech Synthesis System

This repository contains the Neural Network (NN) based Speech Synthesis System
developed at the Centre for Speech Technology Research (CSTR), University of Edinburgh.

Merlin is a toolkit for building Deep Neural Network models for statistical parametric speech synthesis. It must be used in combination with a front-end text processor (e.g., Festival) and a vocoder (e.g., STRAIGHT or WORLD).

The system is written in Python and relies on the Theano numerical computation library.

Merlin comes with recipes (in the spirit of the Kaldi automatic speech recognition toolkit) to show you how to build state-of-the art systems.

Merlin is free software, distributed under an Apache License Version 2.0, allowing unrestricted commercial and non-commercial use alike.

Read the documentation at cstr-edinburgh.github.io/merlin.

Merlin is compatible with: Python 2.7-3.6.

Installation

Merlin uses the following dependencies:

• numpy, scipy
• matplotlib
• bandmat
• theano
• tensorflow (optional, required if you use tensorflow models)
• sklearn, keras, h5py (optional, required if you use keras models)

To install Merlin, cd merlin and run the below steps:

• Install some basic tools in Merlin

bash tools/compile_tools.sh

• Install python dependencies

pip install -r requirements.txt

For detailed instructions, to build the toolkit: see INSTALL and CSTR blog post.
These instructions are valid for UNIX systems including various flavors of Linux;

Getting started with Merlin

To run the example system builds, see egs/README.txt

As a first demo, please follow the scripts in egs/slt_arctic

Now, you can also follow Josh Meyer's blog post for detailed instructions
on how to install Merlin and build SLT demo voice.

For a more in-depth tutorial about building voices with Merlin, you can check out:

• Deep Learning for Text-to-Speech Synthesis, using the Merlin toolkit (Interspeech 2017 tutorial)
• Arctic voices
• Build your own voice

Synthetic speech samples

Listen to synthetic speech samples from our SLT arctic voice.

Development pattern for contributors

1. Create a personal fork of the main Merlin repository in GitHub.
2. Make your changes in a named branch different from master, e.g. you create a branch my-new-feature.
3. Generate a pull request through the Web interface of GitHub.

Contact Us

Post your questions, suggestions, and discussions to GitHub Issues.

Citation

If you publish work based on Merlin, please cite:

Zhizheng Wu, Oliver Watts, Simon King, "Merlin: An Open Source Neural Network Speech Synthesis System" in Proc. 9th ISCA Speech Synthesis Workshop (SSW9), September 2016, Sunnyvale, CA, USA.