AUTO_LEXER with XMLData is not returning results but returns if using the WORLD_LEXER (Doc ID 1378269.1)

Last updated on NOVEMBER 21, 2011

Applies to:

Oracle Text - Version: 11.1.0.6 to 11.2.0.2 - Release: 11.1 to 11.2
Information in this document applies to any platform.

Symptoms

The AUTO_LEXER is not returning some values after migration from WORLD_LEXER.
We have a table with a CLOB column where we store some data in XML format and we search via INPATH, ie

WHERE CONTAINS (XML,'(( (2005) INPATH(/record/extend/identifier) ))' ) > 0

For the WORDL_LEXER I get results but the AUTO_LEXER returns zero rows.

-- WORLD Lexer

SQL> select token_text, token_type from dr$WORLDIDX$i;

TOKEN_TEXT      TOKEN_TYPE
-------------- -----------
192.168.0.1              0
2005                     0 <<<<<<<<<<<<<<<<---------(!)
C                        0 <<<<<<<<<<<<<<<<---------(!)
patha                    7
pathb                    7
pathc                    7
pathd                    7

-- AUTO Lexer
SQL> select token_text, token_type from dr$AUTOIDX$i;

TOKEN_TEXT      TOKEN_TYPE
-------------- -----------
192.168.0.1              0
C-2005                   0 <<<<<<<<<<<<<<<<---------(!)
patha                    7
pathb                    7
pathc                    7
pathd                    7

Looks clear why the INPATH in case of term '2005' and with AUTO_LEXER is not returning rows.
WORLD_LEXER is creating the tokens '2005' & 'C' when the AUTO_LEXER just only 'C-2005'.

Changes

Switching from WORLD_LEXER to AUTO_LEXER.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms