Xwiki no longer edits pages (errors in web console)

Hello, I installed XWiki about eight months ago on a Centos 8 VPS and everything was functioning perfectly (creating pages, editing, etc) until about a month ago when after restarting the VPS (due to a large system update), XWiki was crashing due to a lack of Java memory.

After increasing the memory to 2GB, Xwiki starts fast and loads up no problem.

Now the new issue is that I no longer can edit pages. Clicking on “edit” yields an empty space where the page editor should be, and a loading cursor that never ends.

Using Firefox’s web console I see these error messages. They may or may not be related to my problem but here they are anyways:

Loading failed for the script with source “http://centosvps:8080/xwiki/webjars/wiki%3Axwiki/appl…r-webjar/plugins/xwiki-resource/resourcePicker.bundle.min.js”.

Error: Script error for “resource”
http://requirejs.org/docs/errors.html#scripterror

Loading failed for the with source “[http://centosvps:8080/xwiki/webjars/wiki%3Axwiki/appl…keditor-webjar/plugins/xwiki-macro/macroWizard.bundle.min.js]

Error: Script error for “macroWizard”
http://requirejs.org/docs/errors.html#scripterror

Error: Load timeout for modules: ckeditor,resourcePicker
http://requirejs.org/docs/errors.html#timeout

There are several other platforms running on this VPS, and xwiki is the only one having issues right now, so I am pretty sure the system update is not responsible but I may be wrong.

I hope someone can help!

Thanks

An obvious question, but is it a cache problem? Open a private browsing session to your wiki, log in and see if you are still having the same problem.

You should check on the Network tab (Firefox’s developer tools) the HTTP requests for resourcePicker.bundle.min.js and macroWizard.bundle.min.js. See if they fail or not (what’s the status code?) and check if the response looks like valid JavaScript.

Hey guys,

@pdwalker: Yes, I have tried with different browsers (even with chromium which has NO exrtensions) and same issue nbo matter which browser or computer I use to access Xwiki.

@mflorea: I’m way out of my comfort zone but I managed to get their status with this Network tool. The following are ALL 404:

  • resourcePicker.min.css
  • macroWizard.min.css
  • resourcePicker.bundle.min.js
  • macroWizard.bundle.min.js

Everything else is 200, 204 and a few 304.

I understanding is that some critical files are missing???

Is your xwiki instance behind a proxy?

[edit] we really need a lot more information in order to diagnose your problem correctly.

OK I managed to find out what happened. Turns out, some critical extensions were missing from the install… For example, CKEditor Integration v1.48 was somehow de-installed.

Now I need to find out why that happened… That’s the first time this kind of issue happens to me on a Centos VPS. I say this is the first time this happens to me because it seems that some files were actually deleted from the hard drive of the server…

I will investigate if it is the installation that caused this issue, and report back. Perhaps Xwiki is not installed in the proper folders and after rebooting the VPS, some folders are emptied?

You need to check the permanentDirectory configuration. This is were the index of installed extensions is kept and if this folder ends up in some temporary folder of the OS then you may have problems like the one you described.

Yes this is my guess too (that you haven’t set its location and that it defaults to a tmp directory).

See https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Configuration/#HPermanentDirectory

PS: This is why I’ve proposed (don’t remember where :)) to fail XWiki when the perm dir is not set instead of fallbacking to the tmp directory.

1 Like

ooff… I may have several issues.

@vmassol: The link you provided says to either specify the permanent directory by configuring the servlet directly (not sure how). On this Centos VPS, the tomcat service is started with the following systemd file (comments removed for clarity):

[Unit]
Description=Apache Tomcat Web Application Container
After=syslog.target network.target
[Service]
Type=simple
EnvironmentFile=/etc/tomcat/tomcat.conf
Environment=“NAME=”
EnvironmentFile=-/etc/sysconfig/tomcat
ExecStart=/usr/libexec/tomcat/server start
SuccessExitStatus=143
User=tomcat
[Install]
WantedBy=multi-user.target

and the tomcat.conf file (/etc/tomcat/tomcat.conf) (comments removed for clarity)

TOMCAT_CFG_LOADED="1"
TOMCATS_BASE="/var/lib/tomcats/"
JAVA_HOME="/usr/lib/jvm/jre"
CATALINA_HOME="/usr/share/tomcat"
CATALINA_TMPDIR="/var/cache/tomcat/temp"
JAVA_OPTS=
SECURITY_MANAGER="false"
export CATALINA_OPTS="-Xmx2048m -Xms1024m"
export JAVA_OPTS="-Xmx2048m -Xms1024m"

Alternatively, the permanent directory property can be set in “xwiki.properties”. Mine resides in “/var/lib/tomcat/webapps/xwiki/WEB-INF/xwiki.properties”

This is how I would like to do it. Looking into this file, I dont see where the property “environment.permanentDirectory” could have been setup. Every line containing “environment.permanentDirectory” is currently commented out. So my understanding is that XWiki uses the temp folder specified in the Systemd script (i.e. /var/cache/tomcat/temp)/.

Looking in that temp folder I see what may be xwiki extensions.

> [root@centosvps sshuser]# ls -l /var/cache/tomcat/temp
> total 0
> drwxr-xr-x 3 tomcat tomcat 16 Jun 29 13:05 1593450338908-0
> drwxr-xr-x 3 tomcat tomcat 16 Jun 30 15:24 1593545080442-0
> drwxr-xr-x 4 tomcat tomcat 26 Jun 30 15:47 1593546464788-0
> drwxr-xr-x 4 tomcat tomcat 26 Jun 30 15:47 1593546464799-0
> drwxr-xr-x 4 tomcat tomcat 26 Jun 30 15:55 1593546909871-0
> drwxr-xr-x 3 tomcat tomcat 16 Jun 30 15:55 1593546911418-0
  1. If this is indeed the temp folder xwiki uses, how do I change it without wiping the entire install and restarting fresh? I’d like to store permanent data under /var/www/html/ as I already backup this folder containing other websites on a daily basis.

  2. To backup xwiki completely and restore should this VPS crash entirely, should I also backup the following folders in addition to the extensions folder (the one discussed on #1)?

/var/lib/tomcats/
/usr/share/tomcat

  1. What about the database? It uses MySQL but is there anything other than the permanent directory where wiki page data may be stored?

  2. The Admin Guide (link provided by @vmassol) indicates that looking at the logs should help in determining the current permanent directory. I cannot look at the logs. “/var/log/tomcat/catalina.out” is empty, and within xwiki, under “Global Administration > Logging”, I dont see the actual logs, only what appears to be a page to set the log levels (ERROR-DEBUG…) of different log factories. Am I missing something?

Sorry for being such a hassle… I feel if I can move the install to a more permanent setup that will be better.

Correct.

Doesn’t seem so. Again please see https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Configuration/#HPermanentDirectory which explains everything, including the directory structure for the permanent directory. Clearly what you’re showing isn’t the permanent directory.

The doc also says how to find what is your permanent directory (the blue info box).

Once you’ve found it, then move it to wherever you wish and set environment.permanentDirectory to point to it.

The doc is here: https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Backup

Then it means your tomcat is configured to output its logs somewhere else. You’re in control of your tomcat setup so you need to figure out how it’s configured. You can use the alternate method indicated on https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Configuration/#HPermanentDirectory (the snippet to put in a wiki page).

Definitely, you absolutely need a fixed permanent directory since any tmp dir can be removed/cleaned at any time and you’ll loose lots of important/crucial data (see https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Configuration/#HPermanentDirectory for the detailed list of what’s contained in the perm dir).

We’d really like that https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Configuration/#HPermanentDirectory is enough to understand everything there’s to know about the perm dir so if you see something missing please let us know. Now it seems it already contains everything to answer all your questions above, is it not? :slight_smile:

PS: I’m back from holidays and answering late, I hope you were able to fix this already. Answering just in case.