Because our research interests tend towards creating statistical models trained from real speech data, Festival offers various support for extracting information from speech databases, in a way suitable for building models.
Models for accent prediction, F0 generation, duration, vowel reduction, homograph disambiguation, phrase break assignment and unit selection have been built using Festival to extract and process various databases.
In order for Festival to use a database it is most useful to build utterance structures for each utterance in the database. As discussed earlier, utterance structures contain relations of items. Given such a structure for each utterance in a database we can easily read in the utterance representation and access it, dumping information in a normalised way allowing for easy building and testing of models.
Of course the level of labelling that exists, or that you are willing to do by hand or using some automatic tool, for a particular database will vary. For many purposes you will at least need phonetic labelling. Hand labelled data is still better than auto-labelled data, but that could change. The size and consistency of the data is important too.
For this discussion we will assume labels for: segments, syllables, words, phrases, intonation events, pitch targets. Some of these can be derived, some need to be labelled. This would not fail with less labelling but of course you wouldn't be able to extract as much information from the result.
In our databases these labels are in Entropic's Xlabel format, though it is fairly easy to convert any reasonable format.
Once these files are created an utterance file can be automatically created from the above data. Note it is pretty easy to get the streams right but getting the relations between the streams is much harder. Firstly labelling is rarely accurate and small windows of error must be allowed to ensure things line up properly. The second problem is that some label files identify point type information (IntEvent and Target) while others identify segments (e.g. Segment, Words etc.). Relations have to know this in order to get it right. For example is not right for all syllables between two IntEvents to be linked to the IntEvent, only to the Syllable the IntEvent is within.
The script `festival/examples/make_utts' is an example Festival script which automatically builds the utterance files from the above labelled files.
The script, by default assumes, a hierarchy in an database directory of the following form. Under a directory `festival/' where all festival specific database ifnromation can be kept, a directory `relations/' contains a subdirectory for each basic relation (e.g. `Segment/', `Syllable/', etc.) Each of which contains the basic label files for that relation.
The following command will build a set of utterance structures (including building hte relations that link between these basic relations).
make_utts -phoneset radio festival/relation/Segment/*.Segment
This will create utterances in `festival/utts/'. There are
a number of options to `make_utts' use `-h' to find
them. The `-eval' option allows extra scheme code to be
loaded which may be called by the utterance building process.
The function make_utts_user_function will be called on all
utterance created. Redefining that in database specific loaded
code will allow database specific fixed to the utterance.
The easiest way to extract features from a labelled database of the form described in the previous section is by loading in each of the utterance structures and dumping the desired features.
Using the same mechanism to extract the features as will eventually be
used by models built from the features has the important advantage of
avoiding spurious errors easily introduced when collecting data. For
example a feature such as n.accent in a Festival utterance will
be defined as 0 when there is no next accent. Extracting all the
accents and using an external program to calculate the next accent may
make a different decision so that when the generated model is used a
different value for this feature will be produced. Such mismatches
in training models and actual use are unfortunately common, so using
the same mechanism to extract data for training, and for actual
use is worthwhile.
The recommedn method for extracting features is using the festival script `dumpfeats'. It basically takes a list of feature names and a list of utterance files and dumps the desired features.
Features may be dumped into a single file or into separate files one for each utterance. Feature names may be specified on the command line or in a separate file. Extar code to define new ture nam has the following syntax
CART ::= QUESTION-NODE || ANSWER-NODE
QUESTION-NODE ::= ( QUESTION YES-NODE NO-NODE )
YES-NODE ::= CART
NO-NODE ::= CART
QUESTION ::= ( FEATURE in LIST )
QUESTION ::= ( FEATURE is STRVALUE )
QUESTION ::= ( FEATURE = NUMVALUE )
QUESTION ::= ( FEATURE > NUMVALUE )
QUESTION ::= ( FEATURE < NUMVALUE )
QUESTION ::= ( FEATURE matches REGEX )
ANSWER-NODE ::= CLASS-ANSWER || REGRESS-ANSWER
CLASS-ANSWER ::= ( (VALUE0 PROB) (VALUE1 PROB) ... MOST-PROB-VALUE )
REGRESS-ANSWER ::= ( ( STANDARD-DEVIATION MEAN ) )
Note that answer nodes are distinguished by their car not being atomic.
The interpretation of a tree is with respect to a Stream_Item The FEATURE in a tree is a standard feature (see section 14.6 Features).
The following example tree is used in one of the Spanish voices to predict variations from average durations.
(set! spanish_dur_tree
'
(set! spanish_dur_tree
'
((R:SylStructure.parent.R:Syllable.p.syl_break > 1 ) ;; clause initial
((R:SylStructure.parent.stress is 1)
((1.5))
((1.2)))
((R:SylStructure.parent.syl_break > 1) ;; clause final
((R:SylStructure.parent.stress is 1)
((2.0))
((1.5)))
((R:SylStructure.parent.stress is 1)
((1.2))
((1.0))))))
It is applied to the segment stream to give a factor to multiply the average by.
wagon is constantly improving and with version 1.2 of the speech
tools may now be considered fairly stable for its basic operations.
Experimental features are described in help it gives. See the
Speech Tools manual for a more comprehensive discussion of using
`wagon'.
However the above format of trees is similar to those produced by many other systems and hence it is reasonable to translate their formats into one which Festival can use.
Bigram, trigrams, and general ngrams are used in the part of speech tagger and the phrase break predicter. An Ngram C++ Class is defined in the speech tools library and some simple facilities are added within Festival itself.
Ngrams may be built from files of tokens using the program
ngram_build which is part of the speech tools. See
the speech tools documentation for details.
Within Festival ngrams may be named and loaded from files
and used when required. The LISP function load_ngram
takes a name and a filename as argument and loads the Ngram
from that file. For an example of its use once loaded see
`src/modules/base/pos.cc' or
`src/modules/base/phrasify.cc'.
Another common tool is a Viterbi decoder. This C++ Class is defined in the speech tools library `speech_tooks/include/EST_viterbi.h' and `speech_tools/stats/EST_viterbi.cc'. A Viterbi decoder requires two functions at declaration time. The first constructs candidates at each stage, while the second combines paths. A number of options are available (which may change).
The prototypical example of use is in the part of speech tagger which using standard Ngram models to predict probabilities of tags. See `src/modules/base/pos.cc' for an example.
The Viterbi decoder can also be used through the Scheme function
Gen_Viterbi. This function respects the parameters defined
in the variable get_vit_params. Like other modules this
parameter list is an assoc list of feature name and value. The
parameters supported are:
Relation
cand_function
return_feat
p_word
pp_word
ngramname
ngram.load) to be used
as a "language model".
wfstmname
wfst.load) to be used
as a "language model", this is ignored if an ngramname is also
specified.
debug
gscale_p
Here is a short example to help make the use of this facility clearer.
There are two parts required for the Viterbi decode a set of candidate observations and some "language model". For the math to work properly the candidate observat