22
33Speaker recognition toolkit
44
5- ## Cloning the repo
5+ ## Installation Instructions
66
7- - To clone the repo execute
7+ ### Prerequisites
8+
9+ We use anaconda or miniconda, though you should be able to make it work in other python distributions
10+ To start, you should create a new enviroment and install pytorch>=1.6, e.g.:
11+ ```
12+ conda create --name ${your_env} python=3.8
13+ conda activate ${your_env}
14+ conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch
15+ ```
16+
17+ ### Installing Hyperion
18+
19+ - First, clone the repo:
820``` bash
921git clone https://github.com/hyperion-ml/hyperion.git
1022```
1123
12- ## Dependencies:
13- - Anaconda3.5:
14- - Make a link to your anaconda installation in the tools directory:
15- ``` bash
16- cd hyperion/tools/anaconda
17- ln -s < your-anaconda-3.5> anaconda3.5
18- ```
19- - or follow instructions in hyperion/tools/anaconda/full_install.sh to install anaconda from scratch
20- - Kaldi speech recognition toolkit
21- - Make link to an existing kaldi installation
22- ` ` ` bash
23- cd hyperion/tools/kaldi
24- ln -s < your-kaldi> kaldi
25- ` ` `
26- - or follow instructions in hyperion/tools/anaconda/install_kaldi.sh to install kaldi from scratch
27-
28- - CuDNN: tensorflow and pytorch will need some version of cudnn
29- - Make a link to some existing cudnn version that matches the requirements of your tf or pytorch, e.g.:
30- ` ` ` bash
31- cd hyperion/tools/cudnn
32- # cudnn v7.4 for cuda 9.0 needed by pytorch 1.0
33- ln -s /home/janto/usr/local/cudnn-9.0-v7.4 cudnn-9.0-v7.4
34- ` ` `
35- - In the CLSP grid: you can use some preinstalled versions of anaconda and kaldi in the grid to avoid each person having its own.
36- - To create links to preinstalled kaldi, anaconda and cudnn, run:
37- ` ` ` bash
38- cd hyperion/
39- ./make_clsp_links.sh
40- ` ` `
41- - The anaconda that you will link with this has several environments:
42- - base: numpy, h5py, pandas, etc.
43- - tensorflow1.8g_cpu: tensorflow 1.8 for cpu
44- - tensorflow1.8g_gpu: tensorflow 1.8 for gpu
45- - pytorch1.0_cuda9.0: pytorch 1.0 with cuda 9.0
24+ - You can choolse to install hyperion in the environment
25+ ``` bash
26+ cd hyperion
27+ pip install -e .
28+ ```
29+
30+ - Or add the hyperion toolkit to the PYTHONPATH envirnoment variable
31+ This option will allow you to share the same environment if you are working with several hyperion branches
32+ at the same time, while installing it requires to have an enviroment per branch.
33+ For this, you need to install the requirements
34+ ``` bash
35+ cd hyperion
36+ pip install -r requirements.txt
37+ ```
38+ Then add these lines to your ` ~/.bashrc ` or to each script that uses hyperion
39+ ``` bash
40+ HYP_ROOT= # substitute this by your hyperion location
41+ export PYTHONPATH=${HYP_ROOT} :$PYTHONPATH
42+ export PATH=${HYP_ROOT} /bin:$PATH
43+ ```
44+
45+ ## Recipes
46+
47+ There are recipes for several tasks in the ` ./egs ` directory.
48+
49+ ### Prerequistes to run the recipes
50+
51+ These recipes require some extra tools (e.g. sph2pipe), which need to be installed first:
52+ ``` bash
53+ ./install_egs_requirements.sh
54+ ```
55+
56+ Most recipes do not require Kaldi, only the older ones using Kaldi x-vectors,
57+ so we do not install it by default. If you are going to need it install it
58+ yourself. Then make a link in ` ./tools ` to your kaldi installation
59+ ``` bash
60+ cd tools
61+ ln -s ${your_kaldi_path} kaldi
62+ cd -
63+ ```
64+
65+ Finally configure the python and environment name that you intend to use to run the recipes.
66+ For that run
67+ ``` bash
68+ ./prepare_egs_paths.sh
69+ ```
70+ This script will ask for the path to your anaconda installation and enviromentment name.
71+ It will also detect if hyperion is already installed in the environment,
72+ otherwise it will add hyperion to your python path.
73+ This will create the file
74+ ```
75+ tools/path.sh
76+ ```
77+ which sets all the enviroment variables required to run the recipes.
78+ This has been tested only on JHU computer grids, so you may need to
79+ modify this file manually to adapt it to your grid.
80+
81+ ## Recipes structure
82+
83+ The structure of the recipes is very similar to Kaldi, so if should be
84+ familiar for most people.
85+ Data preparation is also similar to Kaldi. Each dataset has
86+ a directory with files like
87+ ```
88+ wav.scp
89+ utt2spk
90+ spk2utt
91+ ...
92+ ```
93+
94+ ### Running the recipes
95+
96+ Contrary to other toolkits, the recipes do not contain a single ` run.sh ` script
97+ to run all the steps of the recipe.
98+ Since some recipes have many steps and most times you don't want to run all of then
99+ from the beginning, we have split the recipe in several run scripts.
100+ The scripts have a number indicating the order in the sequence.
101+ For example,
102+ ``` bash
103+ run_001_prepare_data.sh
104+ run_002_compute_vad.sh
105+ run_010_prepare_audios_to_train_xvector.sh
106+ run_011_train_xvector.sh
107+ run_030_extract_xvectors.sh
108+ run_040_evaluate_plda_backend.sh
109+ ```
110+ will evaluate the recipe with the default configuration.
111+ The default configuration is in the file ` default_config.sh `
112+
113+ We also include extra configurations, which may change
114+ the hyperparamters of the recipe. For example:
115+ - Acoustic features
116+ - Type of the x-vector neural netwok
117+ - Hyper-parameters of the models
118+ - etc.
119+
120+ Extra configs are in the ` global_conf ` directory of the recipe.
121+ Then you can run the recipe with the alternate config as:
122+ ``` bash
123+ run_001_prepare_data.sh --config-file global_conf/alternative_conf.sh
124+ run_002_compute_vad.sh --config-file global_conf/alternative_conf.sh
125+ run_010_prepare_audios_to_train_xvector.sh --config-file global_conf/alternative_conf.sh
126+ run_011_train_xvector.sh --config-file global_conf/alternative_conf.sh
127+ run_030_extract_xvectors.sh --config-file global_conf/alternative_conf.sh
128+ run_040_evaluate_plda_backend.sh --config-file global_conf/alternative_conf.sh
129+ ```
130+ Note that many alternative configus share hyperparameters with the default configs.
131+ That means that you may not need to rerun all the steps to evaluate a new configuration.
132+ It mast cases you just need to re-run the steps from the neural network training to the end.
133+
134+
135+ ## Citing
136+
137+ Each recipe README.md file contains the bibtex to the works that should be cited if you
138+ use that recipe in your research
46139
47140## Directory structure:
48141 - The directory structure of the repo looks like this:
@@ -53,14 +146,10 @@ hyperion/hyperion
53146hyperion/resources
54147hyperion/tests
55148hyperion/tools
56- hyperion/tools/anaconda
57- hyperion/tools/cudnn
58- hyperion/tools/kaldi
59- hyperion/tools/keras
60149```
61150 - Directories:
62151 - hyperion: python classes with utilities for speaker and language recognition
63- - egs: recipes for sevareal tasks: SRE18, voices, ...
152+ - egs: recipes for sevaral tasks: VoxCeleb, SRE18/19/20 , voices, ...
64153 - tools: contains external repos and tools like kaldi, python, cudnn, etc.
65154 - tests: unit tests for the classes in hyperion
66155 - resources: data files required by unittest or recipes
0 commit comments