Some pages are always unindexed by Solr, how to fix it?

Help / Discuss
#1

Some pages are always unindexed by Solr, how to fix it?

XWiki: …/xwiki/bin/view/Main/AllDocs
It says there are 11673 pages in my xwiki.

Solr stats:
Last Modified: 3 months ago
Num Docs: 24482
Max Doc: 30640
Heap Memory Usage: -1
Deleted Docs: 6158
Version: 682965
Segment Count: 14

But there are some pages I can not search it out by title nor pagename。
For example, a new page title=家庭数字医生 pagename=DigitalFamilyDoctor。
I can not search it out one day after it was created.

I tried run reindex through XWiki Admin, restart XWiki and Solr.
Nothing changed.

#2

Hi. Maybe there are errors when indexing these pages? could you check the xwiki logs? (see https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Logging#HLogLocation ).

#3

I found that, the title_ field is not searched by default, but do not know how to fix it.
I query by solr admin page:

q=:, qf=space:DigitalFamilyDoctor,
get numFound = 1, content is:
response":{“numFound”:1,“start”:0,“maxScore”:1.0,“docs”:[
{
“id”:“xwiki:DigitalFamilyDoctor.WebHome_zh_CN”,
“hidden”:false,
“wiki”:“xwiki”,
“name”:“WebHome”,
“space”:“DigitalFamilyDoctor”,
“spaces”:[“DigitalFamilyDoctor”],
“locale”:“zh_CN”,
“language”:“zh”,
“type”:“DOCUMENT”,
“fullname”:“DigitalFamilyDoctor.WebHome”,
“title_”:“数字家庭医生”,
…
}

q=数字家庭医生, qf=space:DigitalFamilyDoctor,
get numFound = 0

q=*:数字家庭医生, qf=space:DigitalFamilyDoctor,
get numFound = 0

q=title_*:数字家庭医生, qf=space:DigitalFamilyDoctor,
get numFound = 1

q=title_:数字家庭医生, qf=space:DigitalFamilyDoctor,
get numFound = 1

image
image
image

#4

More Information:
I use debug=true on search, found that, XWiki DO NOT search the tilte_ field, but “tilte__” (double underline) instead.
Even with “tilte__”, it split my Chinese word into individual chars. It prevents SOLR search engine to get good result.

Search: title:搜索中台

Debug Information

Query Parser

  • ExtendedDismaxQParser

Parsed Query

  • +((title_en:搜 title_en:索 title_en:中 title_en:台) | (title__:搜 title__:索 title__:中 title__:台) | title_zh_CN:搜索中台 | title_zh:搜索中台)

Filter Queries

  • {!tag=hidden}hidden:(false)
  • {!tag=type}type:(“DOCUMENT”)
  • {!tag=locale}locale:(“zh_CN” OR “”)

The document of “搜索中台” in Solr Admin:

{
“id”:“xwiki:SearchDev.SearchMiddleEnd.WebHome_zh”,
“hidden”:false,
“wiki”:“xwiki”,
“name”:“WebHome”,
“space”:“SearchDev.SearchMiddleEnd”,
“spaces”:[“SearchDev”, “SearchMiddleEnd”],
“locale”:“zh”,
“language”:“zh”,
“type”:“DOCUMENT”,
“fullname”:“SearchDev.SearchMiddleEnd.WebHome”,
“title_”:“搜索中台”,
“doccontentraw_”:"……",
“doccontent_”:"……",
“version”:“35.1”,
“doclocale”:"",
“locales”:["", “zh_CN”, “zh”,“en”],
“lang”:["", “zh_CN”, “zh”, “en”],
“author”:“xwiki:XWiki.kirbyzhou”,
“author_display”:"-",
“creator”:“xwiki:XWiki.kirbyzhou”,
“creator_display”:"-",
“creationdate”:“2019-05-23T08:42:35Z”,
“date”:“2020-02-22T06:57:05Z”,
“version”:1659219083949768704,
“score”:15.423278}

#5

See https://jira.xwiki.org/browse/XWIKI-14091 .