What is Apache Solr?
Solr is an open-source software that delivers extremely fast and high-quality search results. Even with millions of documents, the search engine delivers results in just a few milliseconds. Solr is a project of the Apache Foundation and is based on Lucene technology. The Solr software is written in Java and runs outside TYPO3 on its own server (e.g. Tomcat or Jetty). TYPO3 communicates with this search engine server via extensions.
The most important functions include
- Full text search
- Highlighting of the search term in the results
- Faceted search (grouping of hits according to categories)
- Suggestions as soon as the search term is entered (auto-suggest)
- Suggestions for misspelled terms (did you mean...?)
- Fine-tuning of search results (e.g. boost, preference for current content)
- Synonyms (search for cell phone also returns results with smartphone)
- Stop words (no results appear for these words)
- Search in the content of files (over 100 file types)
- Update in real time
- Simple connection to database fields
- Consideration of authorizations
Why is a powerful search important for the website?
Information on a website can often only be found via nested menus. On mobile devices in particular, the search function is often preferred to menu navigation. Here it is important that the search delivers the right result very quickly.
TYPO3 already has an integrated search function with indexed_search. This only has a limited range of functions and the response time increases with the number of indexed documents.
How do I use Solr with TYPO3?
Several extensions are available for the use of Solr with TYPO3, e.g:
- The solr extension offers the basic functions
- solr_fal implements the access to files
- File contents can be indexed with the tika extension
- The solrmlt (more like this) extension enables the display of related search results
- For projects with many domains in a database (multisite), indexing is simplified with the jwtools extension
When using the extension, care must be taken to ensure that the versions of the individual software components are compatible. For TYPO3 version 12.4, a server with Apache Solr 9.3 and the Solr extension 12.x is required. Sounds confusing? A version matrix in the documentation shows the connections.
Some hosting providers also offer Solr hosting. Solr is already included in the price of our hosting packages from the Premium tariff upwards. Alternatively, you can set up your own Solr server.
Book Solr Cores
To use the search, you need a Solr server based on Java. If you would like to set up, operate and update such a server yourself, you can find the relevant instructions in the Solr documentation. It is much easier and faster with our hosting tariffswhere the use of our Solr servers is already included from the Premium tariff upwards.
A separate Solr core is required for each language, i.e. 2 cores for a bilingual website.
The search indices within a core are stored separately for each domain, so one core can be used for several domains.
From our Premium hosting tariff, two cores are already included in the price, additional cores (languages) can be ordered in packages of 5. With our Solr hosting, there are no limits on the number of indexed documents, storage space or search queries ("fair use" principle).
In our cloud tariffs, Solr cores are booked via the contract management. In the Classic Hosting tariffs, the order is placed using the following form:
Development participation
The Solr extensions are developed under the leadership of d.k.d. in Frankfurt. Some of the extensions are freely available in the TYPO3 Extension Repository (TER) or on Github. By participating in the partner programof dkd you get access to all extensions as well as the current development versions.
A matrix compares the functions of the various Solr extensions and indexed_search.
Installation, basic configuration
Only the solr extension is required for the basic Solr search function; further extensions for additional functions can be added later. A suitable Solr core is required for each website language configured in TYPO3.
On the root page of the project, the option "Use as start of website" must be activated in the page properties on the Behavior tab.
The Solr server is configured under Page administration->Pages in the Solr tab. After ordering the Solr cores using the form above, you will receive the access data in the form username (usually the customer number), password and address of the Solr server. These are entered in the form username:password@server. The connection is made via https:// on port 443, the target directory is /
Connection data to the Solr server (TYPO3 up to version 8)
After ordering a Solr Core for a hosting package, the customer receives the corresponding access data from us. These can be copied/pasted directly. Here is an example:
plugin.tx_solr.solr.host = 123456.solr-01.jweiland.net plugin.tx_solr.solr.username = 123456 plugin.tx_solr.solr.password = secret plugin.tx_solr.solr.port = 443 plugin.tx_solr.solr.path = /solr/core_german/ plugin.tx_solr.solr.scheme = https
Where 123456 corresponds to the customer number, the host name can also vary. 'scheme' specifies that the connection between TYPO3 and the Solr server is made via an encrypted connection.
For multiple languages, the associated Solr Core is configured via a condition:
[globalVar = GP:L = 1] plugin.tx_solr.solr.path = /solr/core_english/ [GLOBAL]
In the documentation for the Solr extension all options for the TypoScript configuration are described.
After configuring the connection data, the connection to the Solr server is established via the lightning bolt symbol. If the "Initialize Solr Connections" option is not visible here, you must log out of the backend and log in again. You can then check whether the connection is successful via the Admin tools->Search menu item:
Static TypoScript template
The Solr extension provides a series of static TypoScript templates. At least the "Search - Base Configuration (solr)" module must be included in the root template.
Solr indexes pages and the regular content elements by default. Extensions with their own data records each require their own configuration via TypoScript. Corresponding examples are already provided for the news and tt_news extensions; these can also serve as examples for other extensions.
Functionality
The elements to be indexed (pages with content and extension data records) are entered in the database table tx_solr_indexqueue_item. This is done automatically when creating, inserting and changing data.
A scheduler task (Index Queue_Worker) searches for new or changed entries in this table with each execution and initiates indexing on the Solr server for these. Indexing therefore only takes place if the scheduler is executed regularly, e.g. every 5 minutes. In this case, new or changed content can be found via the search after 5 minutes at the latest.
If linked files are found during indexing, they are entered in an additional queue in the database (tx_solr_indexqueue_file) (when using the solrfal extension). Indexing then takes place via another scheduler task (File Index Queue Worker).
If content is deleted, the deletion request is sent directly to the Solr server; execution of the scheduler task is not required for this.
Preparation
The HTML template of the website must contain <!--TYPO3SEARCH_begin--> and <!--TYPO3SEARCH_end--> markers. This marks the content areas to be indexed. For example, menus and list views of data records should not be included in the search. Instead of the list views, the single view is used for the search index.
For each extension (except news and tt_news), a configuration of the index_queue must be defined via TypoScript. This defines which fields with which properties are to be included in the search index.
In general, the search must be activated in TypoScript:
config.index_enable = 1The index queue is then initialized once via the menu item Admin tools->Search. Indexing is then carried out by executing the scheduler task.
