Recoll indexing can be performed along two main modes:

Recoll supports defining multiple indexes, each defined by its own configuration directory. A configuration directory contains several files which describe what should be indexed and how.

When recoll or recollindex is first executed, it creates a default configuration directory. This configuration is the one used for indexing and querying when no specific configuration is specified. It is located in $HOME/.recoll/ for Unix-like systems and %LOCALAPPDATA%/Recoll on Windows (typically C:/Users/[me]/Appdata/Local/Recoll).

All configuration parameters have defaults, defined in system-wide files. Without further customisation, the default configuration will process your complete home directory, with a reasonable set of defaults. It can be adjusted to process a different area of the file system, select files in different ways, and many other things.

In some cases, it may be useful to create additional configuration directories, for example, to separate personal and shared indexes, or to take advantage of the organization of your data to improve search precision.

In order to do this, you would create an empty directory in a location of your choice, and then instruct recoll or recollindex to use it by setting either a command line option (-c /some/directory), or an environment variable (RECOLL_CONFDIR=/some/directory). Any modification performed by the commands (e.g. configuration customisation or searches by recoll or index creation by recollindex) would then apply to the new directory and not to the default one.

Once multiple indexes are created, you can use each of them separately by setting the -c option or the RECOLL_CONFDIR environment variable when starting a command, to select the desired index.

It is also possible to instruct one configuration to query one or several other indexes in addition to its own, by using the External index function in the recoll GUI, or some equivalent in the command line and programming tools.

A plausible usage scenario for the multiple index feature would be for a system administrator to set up a central index for shared data, that you choose to search or not in addition to your personal data. Of course, there are other possibilities. for example, there are many cases where you know the subset of files that should be searched, and where narrowing the search can improve the results. You can achieve approximately the same effect by using a directory filter clause in a search, but multiple indexes may have better performance and may be worth the trouble in some cases.

A more advanced use case would be to use multiple indexes to improve indexing performance, by updating several indexes in parallel (using multiple CPU cores and disks, or possibly several machines), and then merging them, or querying them in parallel.

See the section about configuring multiple indexes for more detail

Recoll knows about quite a few different document types. The parameters for document types recognition and processing are set in configuration files.

Most file types, like HTML or word processing files, only hold one document. Some file types, like email folders or zip archives, can hold many individually indexed documents, which may themselves be compound ones. Such hierarchies can go quite deep, and Recoll can process, for example, a LibreOffice document stored as an attachment to an email message inside an email folder archived in a zip file...

recollindex processes plain text, HTML, OpenDocument (Open/LibreOffice), email formats, and a few others internally.

Other file types (e.g.: postscript, pdf, ms-word, rtf ...) need external applications for preprocessing. The list is in the installation section. After every indexing operation, Recoll updates a list of commands that would be needed for indexing existing files types. This list can be displayed by selecting the menu option File → Show Missing Helpers in the recoll GUI. It is stored in the missing text file inside the configuration directory.

After installing a missing handler, you may need to tell recollindex to retry the failed files, by adding option -k to the command line, or by using the GUI File → Special indexing menu. This is because recollindex, in its default operation mode, will not retry files which caused an error during an earlier pass. In special cases, it may be useful to reset the data for a category of files before indexing. See the recollindex manual page. If your index is not too big, it may be simpler to just reset it.

By default, Recoll will try to index any file type that it has a way to read. This is sometimes not desirable, and there are ways to either exclude some types, or on the contrary define a positive list of types to be indexed. In the latter case, any type not in the list will be ignored.

Excluding files by name can be done by adding wildcard name patterns to the skippedNames list, which can be done from the GUI Index configuration menu. Excluding by type can be done by setting the excludedmimetypes list in the configuration file. This can be redefined for subdirectories.

You can also define an exclusive list of MIME types to be indexed (no others will be indexed), by setting the indexedmimetypes configuration variable. Example:

indexedmimetypes = text/html application/pdf

It is possible to redefine this parameter for subdirectories. Example:

[/path/to/my/dir]
indexedmimetypes = application/pdf

(When using sections like this, don't forget that they remain in effect until the end of the file or another section indicator).

excludedmimetypes or indexedmimetypes, can be set either by editing the configuration file (recoll.conf) for the index, or by using the GUI index configuration tool.

Indexing may fail for some documents, for a number of reasons: a helper program may be missing, the document may be corrupt, we may fail to uncompress a file because no file system space is available, etc.

The Recoll indexer in versions 1.21 and later does not retry failed files by default, because some indexing failures can be quite costly (for example failing to uncompress a big file because of insufficient disk space). Retrying will only occur if an explicit option (-k) is set on the recollindex command line, or if a script executed when recollindex starts up says so. The script is defined by a configuration variable (checkneedretryindexscript), and makes a rather lame attempt at deciding if a helper command may have been installed, by checking if any of the common bin directories have changed.

In the rare case where the index becomes corrupted (which can signal itself by weird search results or crashes), the index files need to be erased before restarting a clean indexing pass. Just delete the xapiandb directory (see next section), or, alternatively, start the next recollindex with the -z option, which will reset the database before indexing. The difference between the two methods is that the second will not change the current index format, which may be undesirable if a newer format is supported by the Xapian version.

Xapian versions usually support several formats for index storage. A given major Xapian version will have a current format, used to create new indexes, and will also support the format from the previous major version.

Xapian will not convert automatically an existing index from the older format to the newer one. If you want to upgrade to the new format, or if a very old index needs to be converted because its format is not supported any more, you will have to explicitly delete the old index (typically ~/.recoll/xapiandb), then run a normal indexing command. Using recollindex option -z would not work in this situation.

The Recoll index does not hold complete copies of the indexed documents (it almost does after version 1.24). But it does hold enough data to allow for an almost complete reconstruction. If confidential data is indexed, access to the database directory should be restricted.

Recoll will create the configuration directory with a mode of 0700 (access by owner only). As the index data directory is by default a sub-directory of the configuration directory, this should result in appropriate protection.

If you use another setup, you should think of the kind of protection you need for your index, set the directory and files access modes appropriately, and also maybe adjust the umask used during index updates.

This only needs concern you if your index is going to be bigger than around 5 GBytes. Beyond 10 GBytes, it becomes a serious issue. Most people have much smaller indexes. For reference, 5 GBytes would be around 2000 bibles, a lot of text. If you have a huge text dataset (remember: images don't count, the text content of PDFs is typically less than 5% of the file size), read on.

The amount of writing performed by Xapian during index creation is not linear with the index size (it is somewhere between linear and quadratic). For big indexes this becomes a performance issue, and may even be an SSD disk wear issue.

The problem can be mitigated by using the following approaches:

Recoll versions 1.38 and newer have an option to use multiple temporary indexes and a final merge internally. This was designed as a CPU performance optimization (increasing parallelism), but it may also provide a simple solution for the index size issue, though it may not give enough control over the temporary indexes physical placement for really huge datasets.

Most index configuration parameters can be set from the recoll GUI (set RECOLL_CONFDIR or use the -c option to affect a non-default index.)

The interface is started from the Preferences → Index Configuration menu entry. It is divided in four tabs, Global parameters, Local parameters, Web history (details) and Search parameters.

The Global parameters tab allows setting global variables, like the lists of top/start directories, skipped paths, or stemming languages.

The Local parameters tab allows setting variables that can be redefined for subdirectories. This second tab has an initially empty list of customisation directories, to which you can add. The variables are then set for the currently selected directory (or at the top level if the empty line is selected).

The Search parameters section defines parameters which are used at query time, but are global to an index and affect all search tools, not only the GUI.

The meaning for most entries in the interface is self-evident and documented by a ToolTip popup on the text label. For more detail, you may need to refer to the configuration section of this guide.

The configuration tool normally respects the comments and most of the formatting inside the configuration file, so that it is quite possible to use it on hand-edited files, which you might nevertheless want to backup first...

Multiple Recoll indexes can be created by using several configuration directories which would typically be set to index different areas of the file system.

A specific configuration can be selected by setting the RECOLL_CONFDIR environment variable or giving the -c option to recoll and recollindex.

The recollindex program, used for creating or updating indexes, always works on a single index. The different configurations are entirely independent (no parameters are ever shared between configurations when indexing).

All the search interfaces (recoll, recollq, the Python API, etc.) operate with a main configuration, from which both configuration and index data are used, and can also query data from multiple additional indexes. Only the index data from additional indexes is used, their configuration parameters are ignored. This implies that some parameters should be consistent among index configurations which are to be used together.

When searching, the current main index (defined by RECOLL_CONFDIR or -c) is always active. If this is undesirable, you can set up your base configuration to index an empty directory.

Index configuration parameters can be set either by using a text editor on the files, or, for most parameters, by using the recoll index configuration GUI. In the latter case, the configuration directory for which parameters are modified is the one which was selected by RECOLL_CONFDIR or the -c parameter, and there is no way to switch configurations within the GUI.

See the configuration section for a detailed description of the parameters

Some configuration parameters must be consistent among a set of multiple indexes used together for searches. Most importantly, all indexes to be queried concurrently must have the same option concerning character case and diacritics stripping, but there are other constraints. Most of the relevant parameters affect the term generation.

Using multiple configurations implies a small level of command line or file manager usage. The user must explicitly create additional configuration directories, the GUI will not do it. This is to avoid mistakenly creating additional directories when an argument is mistyped. Also, the GUI or the indexer must be launched with a specific option or environment to work on the right configuration.

mkdir /path/to/my/new/config

recoll -c /path/to/my/new/config

someEditor /path/to/my/new/config/recoll.conf

recollindex -c /path/to/my/new/config

recoll -c /path/to/my/new/config

As of Recoll version 1.18 you have a choice of building an index with terms stripped of character case and diacritics, or one with raw terms. For a source term of Résumé, the former will store resume, the latter Résumé.

Each type of index allows performing searches insensitive to case and diacritics: with a raw index, the user entry will be expanded to match all case and diacritics variations present in the index. With a stripped index, the search term will be stripped before searching.

A raw index allows using case and diacritics to discriminate between terms, e.g., returning different results when searching for US and us or resume and résumé. Read the section about search case and diacritics sensitivity for more details.

The type of index to be created is controlled by the indexStripChars configuration variable which can only be changed by editing the configuration file. Any change implies an index reset (not automated by Recoll), and all indexes in a search must be set in the same way (again, not checked by Recoll).

Recoll creates a stripped index by default if indexStripChars is not set.

As a cost for added capability, a raw index will be slightly bigger than a stripped one (around 10%). Also, searches will be more complex, so probably slightly slower, and the feature is relatively little used, so that a certain amount of weirdness cannot be excluded.

One of the most adverse consequence of using a raw index is that some phrase and proximity searches may become impossible: because each term needs to be expanded, and all combinations searched for, the multiplicative expansion may become unmanageable.

Note: you don't probably don't need to read this. The default automatic configuration is fine is most cases. Only the part about disabling multithreading may be more commonly useful, so I'll prepend it here. In recoll.conf:

            thrQSizes = -1 -1 -1
          

The Recoll indexing process recollindex can use multithreading to speed up indexing on multiprocessor systems. This is currently enabled on MacOS systems and Unix-like systems systems, but not under Windows.

The data processing used to index files is divided in several stages and some of the stages can be executed by multiple threads. The stages are:

You can also read a longer document about the transformation of Recoll indexing to multithreading.

The threads configuration is controlled by two configuration file parameters.

The following example would use three queues (of depth 2), and 4 threads for converting source documents, 2 for processing their text, and one to update the index. This was tested to be the best configuration on the test system (quadri-processor with multiple disks).

            thrQSizes = 2 2 2
            thrTCounts =  4 2 1
          

The following example would use a single queue, and the complete processing for each document would be performed by a single thread (several documents will still be processed in parallel in most cases). The threads will use mutual exclusion when entering the index update stage. In practise the performance would be close to the precedent case in general, but worse in certain cases (e.g. a Zip archive would be performed purely sequentially), so the previous approach is preferred. YMMV... The 2 last values for thrTCounts are ignored.

            thrQSizes = 2 -1 -1
            thrTCounts =  6 1 1
          

The following example would disable multithreading. Indexing will be performed by a single thread.

thrQSizes = -1 -1 -1

In some cases, either when the input documents are simple and require little processing (e.g. HTML files), or possibly with a high number of available cores, the single-threaded Xapian index updates can become the performance bottleneck for indexing.

In this case, it is possible to configure the indexer to use multiple temporary indexes which are merged at the end of the operation. This can provide a huge gain in performance, but, as opposed to multithreading for document preparation, it can also have a (slight) negative impact in some cases, so that it is not enabled by default.

In most cases, this should also be turned off after the initial index creation is done, because it is extremely detrimental to the speed of small incremental updates.

The parameter which controls the number of temporary indexes in recoll.conf is named thrTmpDbCnt. The default value is 0, meaning that no temporary indexes are used.

If your document set is big, and you are using a processor with many cores for indexing, especially if the input documents are simple, it may be worth it to experiment with the value. For example, with a partial Wikipedia dump (many HTML small files), indexing times could be divided almost by three, by using four temporary indexes on a quad-core machine. More detail in this article on the Recoll WEB site.

All the tests were performed on SSDs, it is quite probable that this approach would not work well on spinning disks, at least not in its current form.

The Recoll indexing process, recollindex is usually configured to have very low priority and not disturb other activity on the machine. Still, on an idle system, even with multithreading disabled, it will use 100% of one core if needed and available. This may be enough to get a laptop fan to spin up in some cases. To prevent this, we want to limit the CPU utilisation for every short time quanta (e.g. not more than 20 mS for every 100 mS).

This would be extremely difficult to do from inside the indexing process, because of the many places where intensive CPU usage takes place, some not under our control (Xapian or external helpers). Happily enough, on Linux systems, you can use the cgroup facility to throttle the process CPU usage. This is further documented on the Recoll WEB site

Real time monitoring/indexing is performed by starting the recollindex -m command. With this option, recollindex will permanently monitor file changes and update the index.

On Windows systems, the monitoring process is started from the recoll GUI File menu. On Unix-like systems, there are other possibilities, see the following sections.

When this is in use, the recoll GUI File menu makes two operations available: Stop and Trigger incremental pass.

Trigger incremental pass has the same effect as restarting the indexer, and will cause a complete walk of the indexed area, processing the changed files, then switch to monitoring. This is only marginally useful, maybe in cases where the indexer is configured to delay updates, or to force an immediate rebuild of the stemming and phonetic data, which are only processed at intervals by the real time indexer.

While it is convenient that data is indexed in real time, repeated indexing can generate a significant load on the system when files such as email folders change. Also, monitoring large file trees by itself significantly taxes system resources. You probably do not want to enable it if your system is short on resources. Periodic indexing is adequate in most cases.

As of Recoll 1.24, you can set the monitordirs configuration variable to specify that only a subset of your indexed files will be monitored for instant indexing. In this situation, an incremental pass on the full tree can be triggered by either restarting the indexer, or just running recollindex, which will notify the running process. The recoll GUI also has a menu entry for this.

systemctl --user daemon-reload
systemctl --user enable --now recollindex.service

systemctl daemon-reload
systemctl enable --now recollindex@username.service

recollconf=$HOME/.recoll-home
recolldata=/usr/local/share/recoll
RECOLL_CONFDIR=$recollconf $recolldata/examples/rclmon.sh start

fvwm

### inotify
#
# cat  /proc/sys/fs/inotify/max_queued_events   - 16384
# cat  /proc/sys/fs/inotify/max_user_instances  - 128
# cat  /proc/sys/fs/inotify/max_user_watches    - 16384
#
# -- Change to:
#
fs.inotify.max_queued_events=32768
fs.inotify.max_user_instances=256
fs.inotify.max_user_watches=32768

By default, the Recoll indexer only uses most non-alphanumeric characters as separators, treating them as white space, so that inputs like all words, and all,words produce the same terms.

It may sometimes be useful to index some of these characters so that they can be used as discriminants for searches. This can be done by setting the indexedpunctuation configuration parameter. The value is an UTF-8 string, for example, setting:

indexedpunctuation = %€

would allow searching separately 100% or 100€.

The affected characters are indexed as terms with their own term positions, and they are their own separators, so that 100% and 100 % would be equivalent inputs.

The PDF format is very important for scientific and technical documentation, and document archival. It has extensive facilities for storing metadata along with the document, and these facilities are actually used in the real world.

In consequence, the rclpdf.py PDF input handler has more complex capabilities than most others, and it is also more configurable, because some extra features need executing external commands, so that they are not enabled by default. Specifically, rclpdf.py has the following optional features:

import sys
import re

class MetaFixer(object):
def __init__(self):
pass

def metafix(self, nm, txt):
if nm == 'bibtex:pages':
txt = re.sub(r'--', '-', txt)
elif nm == 'someothername':
# do something else
pass
elif nm == 'stillanother':
# etc.
pass

return txt
def wrapup(self, metaheaders):
pass

The Recoll PDF handler has the ability to call an external OCR program if the processed file has no text content. The OCR data is stored in a cache of separate files, avoiding any modification of the originals.

It must be noted that, if modifying the files (or a copy) is acceptable, then running something like OCRmyPDF to add a text layer to the PDF itself is a better solution (e.g. allowing Recoll to position the PDF viewer on the search target when opening the document, and permitting secondary search in the native tool).

To enable the Recoll OCR feature, you need to install one of the supported OCR applications (tesseract or ABBYY), enable OCR in the PDF handler (by setting pdfocr to 1 in the index configuration file), and tell Recoll how to run the OCR by setting configuration variables. All parameters can be localized in subdirectories through the usual main configuration mechanism (path sections).

Example configuration fragment in recoll.conf:

pdfocr = 1
ocrprogs = tesseract
pdfocrlang = eng

This facility got a major update in Recoll 1.26.5. Older versions had a more limited, non-caching capability to execute an external OCR program in the PDF handler. The new function has the following features:

If the OpenAI Whisper program is available and the appropriate parameters set in the configuration files, the Recoll audio file handler will run speech to text recognition on audio files and the resulting text will be indexed. See the the FAQ entry for more details.

The results of the speech recognition will be cached in the same manner as the results of image OCR.

Recoll used to have no support for indexing removable volumes (portable disks, USB keys, etc.). Recent versions have improved the situation and support indexing removable volumes in two different ways:

#!/bin/sh

fatal()
{
echo $*;exit 1
}
usage()
{
fatal "Usage: init-recoll-volume.sh <top-directory>"
}

test $# = 1 || usage
topdir=$1
test -d "$topdir" || fatal $topdir should be a directory

confdir="$topdir/recoll-config"
test ! -d "$confdir" || fatal $confdir should not exist

mkdir "$confdir"
cd "$topdir"
topdir=`pwd`
cd "$confdir"
confdir=`pwd`

(echo topdirs = '"'$topdir'"'; \
echo orgidxconfdir = $topdir/recoll-config) > "$confdir/recoll.conf"

recollindex -c "$confdir"

With the help of a Firefox extension, Recoll can index the Internet pages that you visit. The extension has a long history: it was initially designed for the Beagle indexer, then adapted to Recoll and the Firefox XUL API. The current version of the extension is located in the Mozilla add-ons repository uses the WebExtensions API, and works with current Firefox versions.

The extension works by copying visited Web pages to an indexing queue directory, which Recoll then processes, storing the data into a local cache, then indexing it, then removing the file from the queue.

The visited Web pages indexing feature can be enabled on the Recoll side from the GUI Index configuration panel, or by editing the configuration file (set processwebqueue to 1).

The Recoll GUI has a tool to list and edit the contents of the Web cache. (Tools → Webcache editor)

The recollindex command has two options to help manage the Web cache:

You can find more details on Web indexing, its usage and configuration in a Recoll 'Howto' entry.

The initial default search mode is Query language. Without special directives, this will look for documents containing all of the search terms (the ones with more terms will get better scores), just like the All Terms mode.

Any term will search for documents where at least one of the terms appear.

File name will exclusively look for file names, not contents

All search modes allow terms to be expanded with wildcards characters (*, ?, []). See the section about wildcards for more details.

In all modes except File name, you can search for exact phrases (adjacent words in a given order) by enclosing the input inside double quotes. Ex: "virtual reality".

The Query Language features are described in a separate section.

When using a stripped index (the default), character case has no influence on search, except that you can disable stem expansion for any term by capitalizing it. E.g.: a search for floor will also normally look for flooring, floored, etc., but a search for Floor will only look for floor, in any character case. Stemming can also be disabled globally in the preferences. When using a raw index, the rules are a bit more complicated.

Recoll remembers the last few searches that you performed. You can directly access the search history by clicking the clock button on the right of the search entry, while the latter is empty. Otherwise, the history is used for entry completion (see next). Only the search texts are remembered, not the mode (all/any/file name).

While text is entered in the search area, recoll will display possible completions, filtered from the history and the index search terms. This can be disabled with a GUI Preferences option.

Double-clicking on a word in the result list or a preview window will insert it into the simple search entry field.

You can cut and paste any text into an All terms or Any term search field, punctuation, newlines and all - except for wildcard characters (single ? characters are ok). Recoll will process it and produce a meaningful search. This is what most differentiates this mode from the Query Language mode, where you have to care about the syntax.

The File name search mode will specifically look for file names. The point of having a separate file name search is that wildcard expansion can be performed more efficiently on a small subset of the index (allowing wildcards on the left of terms without excessive cost). Things to know:

After starting a search, a list of results will instantly be displayed in the main window.

By default, the document list is presented in order of relevance (how well the application estimates that the document matches the query). You can sort the results by ascending or descending date by using the vertical arrows in the toolbar.

Each result is displayed as a structured text paragraph. The standard format is typically adequate, but the content and presentation are entirely customisable.

Most results will contain Preview and Open clickable links.

Clicking the Preview link will open an internal preview window for the document. Further Preview clicks for the same search will open tabs in the existing preview window. You can use Shift+Click to force the creation of another preview window, which may be useful to view the documents side by side. (You can also browse successive results in a single preview window by typing Shift+ArrowUp/Down in the window).

Clicking the Open link will start an external viewer for the document. By default, Recoll lets the desktop choose the appropriate application for most document types. See further for customising the applications.

The Preview and Open links may not be present for all entries. They are only available, respectively, for documents with MIME types that Recoll can extract text from, and for documents that have a configured viewer. However, you can modify the configuration to adjust this behavior. In more detail:

You can click on the Query details link at the top of the results page to see the actual Xapian query, after stem expansion and other processing.

Double-clicking on any word inside the result list or a preview window will insert it into the simple search text.

The result list is divided into pages. You can change the page size in the preferences. Use the arrow buttons in the toolbar or the links at the bottom of the page to browse the results.

As an alternative to the result list, the results can also be displayed in spreadsheet-like fashion. You can switch to this presentation by clicking the table-like icon in the toolbar (this is a toggle, click again to restore the list).

Clicking on the column headers will allow sorting by the values in the column. You can click again to invert the order, and use the header right-click menu to reset sorting to the default relevance order (you can also use the sort-by-date arrows to do this).

Both the list and the table display the same underlying results. The sort order set from the table is still active if you switch back to the list mode. You can click twice on a date sort arrow to reset it from there.

The header right-click menu allows adding or deleting columns. The columns can be resized, and their order can be changed (by dragging). All the changes are recorded when you quit recoll

Hovering over a table row will update the detail area at the bottom of the window with the corresponding values. You can click the row to freeze the display. The bottom area is equivalent to a result list paragraph, with links for starting a preview or a native application, and an equivalent right-click menu. Typing Esc (the Escape key) will unfreeze the display.

Using Shift-click on a row will display the document extracted text (somewhat like a preview) instead of the document details. The functions of Click and Shift-Click can be reversed in the GUI preferences.

By default, the GUI displays the filters panel on the left of the results area. This is new in version 1.32. You can adjust the width of the panel, and hide it by squeezing it completely. The width will be memorized for the next session.

The panel currently has two areas, for filtering the results by dates, or by filesystem location.

The panel is only active in Query Language search mode, and its effect is to add date: and dir: clauses to the actual search.

The dates filter can be activated by clicking the checkbox. It has two assisted date entry widgets, for the minimum and maximum dates of the search period.

The directory filter displays a subset of the filesystem directories, reduced to the indexed area, as defined by the topdirs list and the name exclusion parameters. Some directories may not be shown at all, depending on their (lack of) indexable content and other indexing parameters. The depth of the displayed tree is limited at 2 levels under the start directories by default. You can change this in the GUI Preferences, User interface panel.

You can independantly select and deselect directories by clicking them. Note that selecting a directory will activate the whole subtree for searching, there is no need to select the subdirectories, and no way to exclude some of them (use Query language dir: clauses if this is needed).

Apart from the Open and Open With operations, which allow starting an application on a result document (or a temporary copy), based on its MIME type, it is also possible to run arbitrary commands on results which are top-level files, using the Run Script entry in the results pop-up menu.

The commands which will appear in the Run Script submenu must be defined by .desktop files inside the scripts subdirectory of the current configuration directory.

Here follows an example of a .desktop file, which could be named for example, ~/.recoll/scripts/myscript.desktop (the exact file name inside the directory is irrelevant):

          [Desktop Entry]
          Type=Application
          Name=MyFirstScript
          Exec=/home/me/bin/tryscript %F
          MimeType=*/*
        

The Name attribute defines the label which will appear inside the Run Script menu. The Exec attribute defines the program to be run, which does not need to actually be a script, of course. The MimeType attribute is not used, but needs to exist.

The commands defined this way can also be used from links inside the result paragraph.

As an example, it might make sense to write a script which would move the document to the trash and purge it from the Recoll index.

The default format for the result list entries and the detail area of the result table display an icon for each result document. The icon is either a generic one determined from the MIME type, or a thumbnail of the document appearance. Thumbnails are only displayed if found in the standard freedesktop location, where they would typically have been created by a file manager.

Recoll has no capability to create thumbnails. A relatively simple trick is to use the Open parent document/folder entry in the result list popup menu. This should open a file manager window on the containing directory, which should in turn create the thumbnails (depending on your settings). Restarting the search should then display the thumbnails.

There are also some pointers about thumbnail generation in the Recoll FAQ.

The preview window opens when you first click a Preview link inside the result list.

Subsequent preview requests for a given search open new tabs in the existing window (except if you hold the Shift key while clicking which will open a new window for side by side viewing).

Starting another search and requesting a preview will create a new preview window. The old one stays open until you close it.

You can close a preview tab by typing Ctrl-W (Ctrl + W) in the window. Closing the last tab, or using the window manager button in the top of the frame will also close the window.

You can display successive or previous documents from the result list inside a preview tab by typing Shift+Down or Shift+Up (Down and Up are the arrow keys).

A right-click menu in the text area allows switching between displaying the main text or the contents of fields associated to the document (e.g.: author, abtract, etc.). This is especially useful in cases where the term match did not occur in the main text but in one of the fields. In the case of images, you can switch between three displays: the image itself, the image metadata as extracted by exiftool and the fields, which is the metadata stored in the index.

You can print the current preview window contents by typing Ctrl-P (Ctrl + P) in the window text.

The Query Fragments window can be used to control filtering query language elements modifying the current query, simply by clicking a button. This can be useful to save typing, or avoid memorizing, simple clauses of common usage (e.g. selecting only standalone documents or attachments, or filtering out WEB results, selecting a file system subtree, a file type, etc.).

Selecting the Tools → Query Fragments menu entry will open the dialog.

The contents of the window are entirely customisable, and defined by the contents of a XML text file, named fragment-buttons.xml and which will be looked for in the current index configuration directory. The sample file distributed with Recoll contains a number of example filters. This will be automatically copied to the configuration directory if the file does not exist in there (e.g. ~/.recoll/fragment-buttons.xml under Linux and MacOS, $HOME/AppData/Local/Recoll/fragment-buttons.xml for Windows). Editing the copy will allow you to configure the tool for your needs .

Here follows an example window:

And the corresponding configuration file:

<?xml version="1.0" encoding="UTF-8"?>
<fragbuttons version="1.0">

  <radiobuttons>
    <!-- Toggle WEB queue results inclusion -->
    <fragbutton>
      <label>Include Web Results</label>
      <frag></frag>
    </fragbutton>
    <fragbutton>
      <label>Exclude Web Results</label>
      <frag>-rclbes:BGL</frag>
    </fragbutton>
    <fragbutton>
      <label>Only Web Results</label>
      <frag>rclbes:BGL</frag>
    </fragbutton>
  </radiobuttons>

  <radiobuttons>
    <!-- Standalone vs embedded switch -->
    <fragbutton>
      <label>Include embedded documents</label>
      <frag></frag>
    </fragbutton>
    <fragbutton>
      <label>Only standalone documents</label>
      <frag>issub:0</frag>
    </fragbutton>
    <fragbutton>
      <label>Only embedded documents</label>
      <frag>issub:1</frag>
    </fragbutton>
  </radiobuttons>

  <buttons>
    <fragbutton>
      <label>Example: Year 2010</label>
      <frag>date:2010-01-01/2010-12-31</frag>
    </fragbutton>
    <fragbutton>
      <label>Example: c++ files</label>
      <frag>ext:cpp OR ext:cxx</frag>
    </fragbutton>
    <fragbutton>
      <label>Example: My Great Directory</label>
      <frag>dir:/my/great/directory</frag>
    </fragbutton>
  </buttons>
</fragbuttons>
        

There are two types of groupings radiobuttons and buttons, each defining a line of checkbuttons or radiobuttons inside the window. Any number of buttons can be selected, but the radiobuttons in a line are exclusive.

Buttons are defined by a fragbutton section, which provides the label for a button, and the Query Language fragment which will be added (as an AND filter) before performing the query if the button is active.

    <fragbutton>
      <label>Example: My Great Directory</label>
      <frag>dir:/my/great/directory</frag>
    </fragbutton>
  

It is also possible to add message elements inside the groups, for documenting the behaviour. message elements have a label but no frag element. Example:

  <buttons>
    <message>
      <label>This is a message</label> 
    </message>
  </buttons>
  

The label contents are interpreted as HTML. Take care to replace opening < characters with the < entity if you use tags.

The only thing that you need to know about XML for editing this file is that any opening tag like <label> needs to be matched by a closing tag after the value: </label>.

You will normally edit the file with a regular text editor, like, e.g. vi or notepad. Double-clicking the file in a file manager may not work, because this usually opens it in a WEB browser, which will not let you modify the contents.

The advanced search dialog helps you build more complex queries without memorizing the search language constructs. It can be opened through the Tools menu or through the main toolbar.

Recoll keeps a history of searches. See Advanced search history.

The dialog has two tabs:

Click on the Start Search button in the advanced search dialog, or type Enter in any text field to start the search. The button in the main window always performs a simple search.

Click on the Show query details link at the top of the result page to see the query expansion.

Recoll automatically manages the expansion of search terms to their derivatives (e.g.: plural/singular, verb inflections). But there are other cases where the exact search term is not known. For example, you may not remember the exact spelling, or only know the beginning of the name.

The search will only propose replacement terms with spelling variations when no matching document were found. In some cases, both proper spellings and mispellings are present in the index, and it may be interesting to look for them explicitly.

The term explorer tool (started from the toolbar icon or from the Term explorer entry of the Tools menu) can be used to search the full index terms list, or (later addition), display some statistics or other index information. It has several modes of operations:

Note that in cases where Recoll does not know the beginning of the string to search for (e.g. a wildcard expression like *coll), the expansion can take quite a long time because the full index term list will have to be processed. The expansion is currently limited at 10000 results for wildcards and regular expressions. It is possible to change the limit in the configuration file.

Double-clicking on a term in the result list will insert it into the simple search entry field. You can also cut/paste between the result list and any entry field (the end of lines will be taken care of).

See the section describing the use of multiple indexes for generalities. Only the aspects concerning the recoll GUI are described here.

A recoll program instance is always associated with a main index, which is the one to be updated when requested from the File menu, but it can use any number of external Recoll indexes for searching. The external indexes can be selected through the External Indexes tab in the preferences dialog, which can be reached either trough: Preferences → GUI Configuration → External Index Dialog or Query → External index dialog.

Index selection is performed in two phases. A set of all usable indexes must first be defined, and then the subset of indexes to be used for searching. These parameters are retained across program executions (there are kept separately for each Recoll configuration). The set of all indexes is usually quite stable, while the active ones might typically be adjusted quite frequently.

The main index (defined by RECOLL_CONFDIR) is always active. If this is undesirable, you can set up your base configuration to index an empty directory.

When adding a new index to the set, you can select either a Recoll configuration directory, or directly a Xapian index directory. In the first case, the Xapian index directory will be obtained from the selected configuration.

If the external index is actually located on a volume mounted from another machine, and references remote files, there may be a need to adjust the result paths so that they match the locally mounted ones (for opening documents). This can be done by using the path translation facility.

As building the set of all indexes can be a little tedious when done through the user interface, you can use the RECOLL_EXTRA_DBS environment variable to provide an initial set. This might typically be set up by a system administrator so that every user does not have to do it. The variable should define a colon-separated list of index directories, e.g.:

export RECOLL_EXTRA_DBS=/some/place/xapiandb:/some/other/db

On Windows, use semi-colons (;) as separators instead of colons.

Another environment variable, RECOLL_ACTIVE_EXTRA_DBS allows adding to the active list of indexes. This variable was suggested and implemented by a Recoll user. It is mostly useful if you use scripts to mount external volumes with Recoll indexes. By using RECOLL_EXTRA_DBS and RECOLL_ACTIVE_EXTRA_DBS, you can add and activate the index for the mounted volume when starting recoll. Unreachable indexes will automatically be deactivated when starting up.

Documents that you actually view (with the internal preview or an external tool) are entered into the document history, which is remembered.

You can display the history list by using the Tools/Doc History menu entry.

You can erase the document history by using the Erase document history entry in the File menu.

The documents in a result list are normally sorted in order of relevance. It is possible to specify a different sort order, either by using the vertical arrows in the GUI toolbox to sort by date, or switching to the result table display and clicking on any header. The sort order chosen inside the result table remains active if you switch back to the result list, until you click one of the vertical arrows, until both are unchecked (you are back to sort by relevance).

Sort parameters are remembered between program invocations, but result sorting is normally always inactive when the program starts. It is possible to keep the sorting activation state between program invocations by checking the Remember sort activation state option in the preferences.

It is also possible to hide duplicate entries inside the result list (documents with the exact same contents as the displayed one). The test of identity is based on an MD5 hash of the document container, not only of the text contents (so that e.g., a text document with an image added will not be a duplicate of the text only). Duplicates hiding is controlled by an entry in the GUI configuration dialog, and is off by default.

When a result document does have undisplayed duplicates, a Dups link will be shown with the result list entry. Clicking the link will display the paths (URLs + ipaths) for the duplicate entries.

A number of common actions within the graphical interface can be triggered through keyboard shortcuts. As of Recoll 1.29, many of the shortcut values can be customised from a screen in the GUI preferences. Most shortcuts are specific to a given context (e.g. within a preview window, within the result table).

Most shortcuts can be changed to a preferred value by using the GUI shortcut editor: Preferences → GUI configuration → Shortcuts. In order to change a shortcut, just click the corresponding cell in the Shortcut column, and type the desired sequence.

Both simple and advanced query dialogs save recent history, but the amount is limited: old queries will eventually be forgotten. Also, important queries may be difficult to find among others. This is why both types of queries can also be explicitly saved to files, from the GUI menus: File → Save last query / Load last query

The default location for saved queries is a subdirectory of the current configuration directory, but saved queries are ordinary files and can be written or moved anywhere.

Some of the saved query parameters are part of the preferences (e.g. autophrase or the active external indexes), and may differ when the query is loaded from the time it was saved. In this case, Recoll will warn of the differences, but will not change the user preferences.

You can customise some aspects of the search interface by using the GUI configuration entry in the Preferences menu.

There are several tabs in the dialog, dealing with the interface itself, the parameters used for searching and returning results, and what indexes are searched.

Most GUI settings are global and do not depend on the index in use. As of Recoll 1.42, it is possible to specify that some settings will depend on the index. At the moment, only the result table configuration can be saved in such a way.

<a href="F%N">%P</a>

"<table class=\"respar\">\n"
"<tr>\n"
"<td><a href='%U'><img src='%I' width='64'></a></td>\n"
"<td>%L &nbsp;<i>%S</i> &nbsp;&nbsp;<b>%T</b><br>\n"
"<span style='white-space:nowrap'><i>%M</i>&nbsp;%D</span>&nbsp;&nbsp;&nbsp; <i>%U</i>&nbsp;%i<br>\n"
"%s %A %K</td>\n"
"</tr></table>\n"

<u><b><a href="P%N">%T</a></b></u><br>
%A<font color=#008000>%U - %S</font> - %L

<img src="%I" align="left">%L <font color="#900000">%R</font>
&nbsp;&nbsp;<b>%T&</b><br>%S&nbsp;
<font color="#808080"><i>%U</i></font>
<table bgcolor="#e0e0e0">
<tr><td><div>%A</div></td></tr>
</table>%K

The recoll command has a number of useful command line options.

-c configdir specifies a non-default configuration directory.

-L lang can be used to use a different language for the GUI labels than the one which would be chosen according to the system locale. Some translations are quite incomplete and you may prefer to see the English messages, even if your machine is generally setup for, e.g. Spanish. Example:

recoll -L en

-q query specifies a query to be run when the program starts. It takes a single argument, which must be quoted if it contains white space.

-o/-l/-f/-a specify the type of query. The default is to interpret the -q argument as a query language string. You can use these options to interpret the argument as an Any Term, File Name or All Terms query instead.

The -t option will tell the program to behave exactly like the recollq command, printing the results to the standard output (terminal) instead of starting a graphical window.

Here follows a sample request that we are going to explain:

        author:"john doe" Beatles OR Lennon Live OR Unplugged -potatoes
      

This would search for all documents with John Doe appearing as a phrase in the author field (exactly what this is would depend on the document type, e.g.: the From: header, for an email message), and containing either beatles or lennon and either live or unplugged but not potatoes (in any part of the document).

An element is composed of an optional field specification, and a value, separated by a colon (the field separator is the last colon in the element). Examples:

The colon, if present, means "contains". Xesam defines other relations, which are mostly unsupported for now (except in special cases, described further down).

All elements in the search entry are normally combined with an implicit AND. It is possible to specify that elements be OR'ed instead, as in Beatles OR Lennon. The OR must be entered literally (capitals), and it has priority over the AND associations: word1 word2 OR word3 means word1 AND (word2 OR word3) not (word1 AND word2) OR word3.

You can use parentheses to group elements (from version 1.21), which will sometimes make things clearer, and may allow expressing combinations which would have been difficult otherwise.

An element preceded by a - specifies a term that should not appear.

By default, words inside double-quotes define a phrase search (the order of words is significant), so that title:"prejudice pride" is not the same as title:prejudice title:pride, and is unlikely to find a result. This can be changed by using modifiers.

Words inside phrases and capitalized words are not stem-expanded. Wildcards may be used anywhere inside a term. Specifying a wildcard on the left of a term can produce a very slow search (or even an incorrect one if the expansion is truncated because of excessive size). Also see More about wildcards.

To save you some typing, Recoll versions 1.20 and later interpret a field value given as a comma-separated list of terms as an AND list and a slash-separated list as an OR list. No white space is allowed. So

author:john,lennon

will search for documents with john and lennon inside the author field (in any order), and

author:john/ringo

would search for john or ringo. This behaviour is only triggered by a field prefix: without it, comma- or slash- separated input will produce a phrase search. However, you can use a text field name to search the main text this way, as an alternate to using an explicit OR, e.g. text:napoleon/bonaparte would generate a search for napoleon or bonaparte in the main text body.

Modifiers can be set on a double-quote value, for example to specify a proximity search (unordered). See the modifier section. No space must separate the final double-quote and the modifiers value, e.g. "two one"po10

Recoll currently manages the following default fields:

You can define aliases for field names, in order to use your preferred denomination or to save typing (e.g. the predefined fn and cfn aliases defined for filename and containerfilename). See the section about the fields file.

The document input handlers have the possibility to create other fields with arbitrary names, and aliases may be defined in the configuration, so that the exact field search possibilities may be different for you if someone took care of the customisation.

The field syntax also supports a few field-like, but special, criteria, for which the values are interpreted differently. Regular processing does not apply (for example the slash- or comma- separated lists don't work). A list follows.

Recoll 1.24 and later support range clauses on fields which have been configured to support it. No default field uses them currently, so this paragraph is only interesting if you modified the fields configuration and possibly use a custom input handler.

A range clause looks like one of the following:

myfield:small..big
myfield:small..
myfield:..big
        

The nature of the clause is indicated by the two dots .., and the effect is to filter the results for which the myfield value is in the possibly open-ended interval.

See the section about the fields configuration file for the details of configuring a field for range searches (list them in the [values] section).

Some characters are recognized as search modifiers when found immediately after the closing double quote of a phrase, as in "some term"modifierchars. The actual "phrase" can be a single term of course. Supported modifiers:

The following only make sense on indexes which are capable of case and diacritics sensitivity (not the default):

All words entered in Recoll search fields will be processed for wildcard expansion before the request is finally executed.

The wildcard characters are:

You should be aware of a few things when using wildcards.