Lucene error after upgrading xwiki docker installation from 11.5 to 11.9

Help / Discuss
#1

Hi all,
after upgrading my xwiki docker container (from https://hub.docker.com/_/xwiki) I get an empty extension list. The logfiles show the following error:

xwiki-mysql-tomcat-web | org.apache.lucene.index.IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="/usr/local/tomcat/work/Catalina/localhost/ROOT/xwiki-temp/segments_1"))): 6 (needs to be between 7 and 9). This version of Lucene only supports indexes created with release 6.0 and later.

Any ideas about how to resolve that problem are very appreciated!

With kind regards
Ben

#2

Hi Ben, you need to remove the solr directory in your xwiki permanent directory (it’s indicated in the release notes :)).

#3

Note that we’ve also automated this now so I’m surprised, see http://jira.xwiki.org/browse/XWIKI-13639

@tmortagne any idea?

#4

I’ve done that, but it didn’t fix that issue.

#5

By any chance, could it be that you’re using a remote solr instance?

#6

The error does not seems to be related to the solr search. Are you using https://extensions.xwiki.org/xwiki/bin/view/Extension/Extension%20Repository%20Connector%20-%20Pypi/ ?

#7

Yes!

#8

This extension is using Lucene to maintain an index of the pypi packages and it probably did not like the Lucene upgrade, I guess all it needs is to clean it’s index but not sure where it’s located. The error suggest it’s directly in /usr/local/tomcat/work/Catalina/localhost/ROOT/xwiki-temp/(which is not great, should be in a subfolder, will take a look at that). Try to stop XWiki and clean everything that looks like Lucene stuff (like segments* folders) in `/usr/local/tomcat/work/Catalina/localhost/ROOT/xwiki-temp/.

#9

The /usr/local/tomcat/work/Catalina/localhost/ROOT/xwiki-temp/ folder is not inside of any persistent storage of that docker container, so it’s not possible to just stop xwiki, remove the segments_1 file and restart xwiki…

#10

Sorry not a docker image expert. @vmassol XWiki temporary files end up inside the docker image ? Sounds like a problem.

#11

Maybe the problematic files get created just at container startup - so they won’t be part of the image itself (maybe someone could check this?)

#12

Not sure it’s a problem. I don’t think they should be persistent since they’re temporary. So I feel it’s ok that they have the same lifecycle as the docker container.

You just need to log into that container: docker exec -it <container id> bash -l. And then remove the files.

#13

So what files exactly to remove? The directory listing looks something like

_0.cfe _0.cfs _0.si _1.cfe _1.cfs _1.si cache segments_1 write.lock

#14

Since we’re talking about a temporary directory, I don’t think it would harm to remove all files in that directory and restart xwiki.

#15

Now there’s something weird… Since you upgraded XWiki it means you created a new docker container from the XWiki image… so you shouldn’t have any old lucene index files…

#16

If I remove all files in that directory (inside the container…) and after that restart the container (the only way to restart xwiki…) - the files just are back there.

#17

That’s right…!

#18

I looked at the code and there is actually nothing to can do except uninstalling it for now. The extension come with a default index (so that you get extension listed without waiting for the remote index) which unfortunately does not work with the version of Lucene packaged in XWiki 11.9 and it would need to be fixed to generate this index from some list of packaging it as Lucene index.

#19

ok, mystery solved.

#20

Will try to find some quickfix for that ASAP since it’s a pity to have this extension being unusable.