This section is of little use for anyone except for programmers willing to contribute to the development of Epos or going to modify its source code. It is also not trying to become a beginner's guide to Epos. Anyway, if you are personally missing something here or elsewhere, tell me and I may add it; that will become almost the only source of progress in this section of documentation. The section may also slowly become outdated due to lack of interest, except for the subsection on portability.
Overall coding priorities, approximately in order of decreasing precedence:
class parser, unit, rules, text and maybe a few others are isolated classes
that take no advantage from inheritance. The reason for the class-oriented design
is just a matter of code readability and decomposition in this case.
simpleparserThis class takes some input (such as a plain ASCII or STML text) and then can be
used in conjunction with the class unit constructor to build the
text structure representation. Its purpose is to
identify the Latin text tokens (usually ASCII characters, but some traditional
tokens like "..." would be difficult to identify later, as well as numerous other
context dependent uses of "."). The parser also identifies the level of description
which corresponds to the token and this is the information needed by the class
unit constructor to correctly build the
TSR. In this process,
the parser skips over any empty units, that is, units that contain no phones
(simple letters) at all.
Note that it is unnecessary and counterproductive to distinguish between homographic tokens used at the same level of description here; such intelligence can be handled more flexibly by the language dependent rules. In fact, they tend to be usually language dependent. The parser only avoids losing information (through empty unit deletion) by the minimum necessary tokenization.
The STML parser is still unimplemented.
unitThis class is the fundamental element of the
text structure representation. Its methods are listed in
unit.h. Every object of this type represents a single
text unit. Every unit includes pointers to its immediate
container, and to its contents. The contents are organized in a
bidirectional linked list; pointers to the head and tail units
of this lists are stored in the unit. These links, i.e. prev
and next, also serve to locate the neighboring units; they may be
NULL, indicating that this is the first/last unit in the
immediate container. For most uses, these pointers are not suitable
to be used directly; the Prev and Next methods find the
neighbor, even if a higher level boundary lies in between. It is also
possible to mark a unit as a scope one. In this case, the Next
and Prev methods will be unable to cross its boundary from inside
out (they will return NULL if this is attempted). If you need to
modify the TSR directly, you will benefit from calling unit::sanity
occasionally. This method checks the TSR structure near the unit which has
called it and will report a severe error, if an invariant is violated,
thus saving you from misguided error messages or crashes later.
To extract the prosodic information from a TSR, call the effective
method. It will combine the prosodic adjustments present at all the levels
of description above the current unit.
textThis class represents a logical line-oriented text file. It handles things like the @include directive, backslash-escaped special characters, initial whitespace and comment stripping. It is used for rule files, configuration files, and also for the dictionaries.
fileThis class represents a physical data file. Its main purpose is to cache and
share files repeatedly needed by Epos. The claim function (to be found in
interf.cc) should be used for opening the file (or only sharing an existing
copy if the file is already open) and reading the data out of the file. The
unclaim function is called separately for every claim call whenever
the file is no more needed.
Any code which uses this class should never extract the data member out of it and
use it independently, even if the class itself remains claimed. This is because
if the content of the file has changed, the data in memory will be reallocated and
re-read upon the next call to claim or possibly even sooner. This may cause
invalidation of the original data member at any point of a control switch
to another Epos agent. It is possible to call reclaim at any time to force
re-reading any file if its time stamp has changed.
hashclass hash is derived from class hash_table<char,char>.
The hash_table template is a generic hash table, keys and associated
data items being its class parameters. This implementation uses balanced (AVL)
trees to resolve collisions and is able to adjust (rehash) itself
when it gets too full or too sparse. It is a very robust and fast
implementation and it is independent of the rest of Epos, so you
may use it in other projects if you want to (subject to GPL).
If you want to have the hash table keep a copy of its contents,
the key and/or data may only be of a fixed size type, or a C-style
string. Alternatively, the hash table will only store pointers
to these items. These approaches can be mixed in any reasonable
sense of "mixing".
The hash tables are used frequently in Epos in various type combinations
(see hash.cc for a list. They're also used for parsing the
dictionary files.
rulesNote the difference between class rules and class rule.
Every set of rules in Epos (there is one per language) is a class
rules, which contains a single r_block, which in turn
contains the individual rules.
The class rules serves as the only communication interface
between the rule hierarchy and the rest of Epos, but there
is no inheritance relation between them.
ruleEach rule object represents a rule to be applied to
a structure of units. The class hierarchy:
rule
Classes not beginning in r_ can be considered abstract.
agentEpos can be configured to support multiple simultaneous TTSCP connections
and except for bugs, no single unauthorized connection should be able to
create a Denial of Service situation, such as long delays in processing
other connections. To achieve this, Epos uses a simple cooperative multitasking
facility called agents. An agent (process) is an entity, which is responsible
for carrying out some task, such as reading a few bytes from a file descriptor.
At any moment (except for the startup and the very moments of a transfer of
control), exactly one agent is active (Epos doesn't support SMP to avoid
the unnecessary overhead and complexity in the typical case). If an agent has to wait
for some event before its job is finished, for example, when the sound card reaches
full buffers or not enough data has arrived through a network connection, the agent
calls the block method (reading) or push method (writing) with the offending
file descriptor.
It is also possible for an agent to wait until some other agent
executes; see the block and push methods' implementation for an example.
If an agents wants
to have another agent running, it can call the schedule method to add it
to the queue of runnable processes. The scheduled agents always acquire control
through the run method; when this method returns, another agent is chosen.
If there are no more runnable agents, Epos will wait until an agent becomes runnable
through a status change of the file descriptor the agent is blocking for.
Most agents get their data input through the inb data member and place their output
into the outb data member. Whenever the agent has completed a stand-alone chunk
of output, the agent calls the pass method to pass it to its successor and
to schedule it for processing. The output agent never calls pass (it has actually
no successor and it is responsible for writing the data somewhere outside Epos),
but it calls finis when the data has been successfully written.
Most agents are organized into streams of interconnected agents. See the strm command for the semantics of that. Other agents are responsible for individual TTSCP connections, for accepting new connections and other tasks. A special agent is used for deleting other agents when they need to delete themselves.
The chunk agent may perform utterance chunking, that is, splitting the
text being processed at convenient points, such as after a period or at end
of paragraph. Such chunks travel through the rest of the stream independently
and they are queuedL>
bug_password option.
In TTSCP, a control connection can issue commands that affect any
other connection (e.g. interrupt a command in progress). To prevent
malicious use, the handle necessary to refer to a connection is
only available to the connection originator, and is generated at random.
Consequently, privacy in TTSCP relies on the properties of the underlying
network connection and on the length of the data connection handle.
Each byte of handle length carries a perplexity of six bits; reasonable
handle lengths (as controlled with the handle_size option therefore
fall within the range 10 to 200. Be sure to adjust
max_line_len as well if using extremely long connection handles.
restr.ini FileMany TTSCP options are rarely set over TTSCP, and some of them could be
used for malicious ends. In these cases, the administrator may restrict
their use in the restr.ini as described in
the respective section. Note especially that unlisted options can
be set freely, and the default restr.ini file as distributed with Epos
may occasionally omit a slightly dangerous option. Therefore,
you may want to grep out a list of all options and to leave out
only options you're going to treat as necessary for users and harmless
for the server. On the other hand, almost no options affect the security
directly, because they're only set within the context of the current
TTSCP connection, and the few exceptions are unlikely to have been overlooked.
At the moment, any TTSCP client is capable of writing arbitrary waveform data
to the local soundcard. If you consider this a security risk, disable
the #localsound output module by turning the localsound option off.
TTSCP in theory offers file system based input and output modules. For example,
that allows the client to write some intermittent product to a server-side file
and then the client may decide to use it repeatedly as an input to multiple streams.
However, the name space for the files doesn't have to correspond to the file system
root of the TTSCP server. With Epos, you can configure the location of the
TTSCP name space using the
--pseudo_root_dir option.
Furthermore, writing to it is disabled by default; this restriction can be overridden
by setting the --writefs option.
It is not recommended to run Epos as a setuid or setgid binary, not because of a known security weakness, but because of the possibility of yet unknown bugs. If necessary, Epos should run setgid to a group with minimal privileges.
Note also that the base_dir parameter can be given to Epos on the
command-line, thus by-passing the restrictions specified in the restr.ini
file. This issue may be solved in later versions of Epos; at the moment
you have to adjust the source manually.
Instead of a privileged executable, run Epos with the desired privileges at system start-up and use a TTSCP client to control it from then on.
local_only Option
By default, the TTSCP server only binds to the loopback interface.
That way, only local users can see and use Epos. Turn off the
local_only option to allow remote users to communicate
with Epos on your machine.
Theoretically, this security measure should not be needed, but it is very simple and thus likely to reduce the risk caused by bugs hidden in more complex corners of the Epos TTSCP implementation.