Search (SOLR) - Search Index Wont Update Anymore

Our search index seems to be not updating anymore. Newly created wikipages wont show up in the search-results (and suggestions).

Looking at the catalina.out shows error messages whenever an indexing is triggered (from the admin menu or when a user creating/editing a page):

2020-01-23 12:24:39,748 [XWiki Solr index thread] ERROR o.x.s.s.i.DefaultSolrIndexer - Failed to process entry [INDEX Object_property OurWikiName:AppWithinMinutes.LiveTableTemplate^XWiki.UIExtensionClass[0].async_cached]
org.apache.solr.common.SolrException: Exception writing document id OurWikiName:AppWithinMinutes.LiveTableTemplate^XWiki.UIExtensionClass[0].async_cached to the index; possible analysis error.
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:178)
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:955)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1110)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:736)
at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:250)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:166)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2299)
at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
at org.xwiki.search.solr.internal.AbstractSolrInstance.add(AbstractSolrInstance.java:61)
at org.xwiki.search.solr.internal.DefaultSolrIndexer.processBatch(DefaultSolrIndexer.java:413)
at org.xwiki.search.solr.internal.DefaultSolrIndexer.run(DefaultSolrIndexer.java:377)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:753)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:767)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1571)
at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:279)
at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:211)
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:166)
… 20 common frames omitted
Caused by: java.nio.file.NoSuchFileException: /srv/www/virtual/data/solr/xwiki/data/index/write.lock
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at org.apache.lucene.store.NativeFSLockFactory$NativeFSLock.ensureValid(NativeFSLockFactory.java:177)
at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:43)
at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.(CompressingStoredFieldsWriter.java:109)
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
at org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:83)
at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:331)
at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:368)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1575)
… 23 common frames omitted

We have already deleted …/xwiki/data/solr/xwiki/data/index , restarted Tomcat and triggered a reindexing but the old index seems to persist and we still get these error-messages for every page during the reindexing.

Could anyone point us in some directions where this issue could be created? It seems to us that it is permission-related but the above mentioned folder has the same user set as the wiki.
We are using LTS 10.11.8 with Tomcat and MySQL.

According to the error you get that’s not where the solr index is stored, seems you configured it it be located on /srv/www/virtual/data/solr/.

Thank you for your reply,
I had copied that line from a thread here in the forum. On our system we used the correct path.

I’ve also read to delete the complete …/data/solr/ folder. Would that make a difference? We only deleted the …/solr/index/ folder

You should try to delete the folder /var/lib/xwiki/data/solr and restart to let XWiki recreate it.

It seemed to be a permission-related problem and we got it solved by killing running processes and restarting the whole machine. Thx for your help.