- August 12–22: Online Training (EU shift)
- FOSDEM Impressions
- June 8–18: Online Training (US shift)
- Non-monetary micro sponsoring
- November 11-12 in Frankfurt am Main
- Online-Training in July and August
- Oracle + PostgreSQL
- Party time
- PostgreSQL Performance Event
- SQL Server Performance Kurs in Stuttgart
- SQL Server performance training in London
- Shipping Terms
- The two top performance problems caused by ORM tools
- Top Tweets January 2013
- Training Survey
- Training and Conference Dates
- Use The Index, Luke
2012-04-06Indexing LIKE Filters
LIKE operator very often causes unexpected performance behavior because some search terms prevent efficient index usage. That means that there are search terms that can be indexed very well, but others can not. It is the position of the wildcard characters that makes all the difference.
The following example uses the
% wildcard in the middle of the search term:
SELECT first_name, last_name, date_of_birth FROM employees WHERE UPPER(last_name) LIKE 'WIN%D' --------------------------------------------------------------- |Id | Operation | Name | Rows | Cost | --------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 4 | | 1 | TABLE ACCESS BY INDEX ROWID| EMPLOYEES | 1 | 4 | |*2 | INDEX RANGE SCAN | EMP_UP_NAME | 1 | 2 | ---------------------------------------------------------------
LIKE filters can only use the characters before the first wildcard during tree traversal. The remaining characters are just filter predicates that do not narrow the scanned index range. A single
LIKE expression can therefore contain two predicate types: (1) the part before the first wildcard as an access predicate; (2) the other characters as a filter predicate.
For the PostgreSQL database, you might need to specify an operator class (e.g.,
varchar_pattern_ops) to use
LIKE expressions as access predicates. Refer to “Operator Classes and Operator Families” in the PostgreSQL documentation for further details.
The more selective the prefix before the first wildcard is, the smaller the scanned index range becomes. That, in turn, makes the index lookup faster. Figure 2.4 illustrates this relationship using three different
LIKE expressions. All three select the same row, but the scanned index range—and thus the performance—is very different.
Figure 2.4. Various
The first expression has two characters before the wildcard. They limit the scanned index range to 18 rows. Only one of them matches the entire
LIKE expression—the other 17 are fetched but discarded. The second expression has a longer prefix that narrows the scanned index range down to two rows. With this expression, the database just reads one extra row that is not relevant for the result. The last expression does not have a filter predicate at all: the database just reads the entry that matches the entire
Only the part before the first wildcard serves as an access predicate.
The remaining characters do not narrow the scanned index range—non-matching entries are just left out of the result.
The opposite case is also possible: a
LIKE expression that starts with a wildcard. Such a
LIKE expression cannot serve as an access predicate. The database has to scan the entire table if there are no other conditions that provide access predicates.
Tweet this tip
LIKE expressions with leading wildcards (e.g.,
The position of the wildcard characters affects index usage—at least in theory. In reality the optimizer creates a generic execution plan when the search term is supplied via bind parameters. In that case, the optimizer has to guess whether or not the majority of executions will have a leading wildcard.
Most databases just assume that there is no leading wildcard when optimizing a
LIKE condition with bind parameter, but this assumption is wrong if the
LIKE expression is used for a full-text search. There is, unfortunately, no direct way to tag a
LIKE condition as full-text search. The box “Labeling Full-Text
LIKE Expressions” shows an attempt that does not work. Specifying the search term without bind parameter is the most obvious solution, but that increases the optimization overhead and opens an SQL injection vulnerability. An effective but still secure and portable solution is to intentionally obfuscate the
LIKE condition. “Combining Columns” explains this in detail.
For the PostgreSQL database, the problem is different because PostgreSQL assumes there is a leading wildcard when using bind parameters for a
LIKE expression. PostgreSQL just does not use an index in that case. The only way to get an index access for a
LIKE expression is to make the actual search term visible to the optimizer. If you do not use a bind parameter but put the search term directly into the SQL statement, you must take other precautions against SQL injection attacks!
Even if the database optimizes the execution plan for a leading wildcard, it can still deliver insufficient performance. You can use another part of the where clause to access the data efficiently in that case—see also “Index Filter Predicates Used Intentionally”. If there is no other access path, you might use one of the following proprietary full-text index solutions.
DB2 supports the contains keyword. See “DB2 Text Search tutorial“ at IBM developerWorks.
MySQL offers the match and against keywords for full-text searching. Starting with MySQL 5.6, you can create full-text indexes for InnoDB tables as well—previously, this was only possible with MyISAM tables. See “Full-Text Search Functions” in the MySQL documentation.
The Oracle database offers the contains keyword. See the “Oracle Text Application Developer’s Guide.”
PostgreSQL offers the @@ operator to implement full-text searches. See “Full Text Search” in the PostgreSQL documentation.
Another option is to use the WildSpeed extension to optimize
LIKEexpressions directly. The extension stores the text in all possible rotations so that each character is at the beginning once. That means that the indexed text is not only stored once but instead as many times as there are characters in the string—thus it needs a lot of space.
- SQL Server
SQL Server offers the contains keyword. See “Full-Text Search” in the SQL Server documentation.
Think About It
How can you index a
LIKE search that has only one wildcard at the beginning of the search term (