Migrating from installed fess to Docker fess

English:Fess
1

(from github.com/pcolmer)
I want to start using the Docker version of fess instead of the manually installed version. I’ve built a new AWS server for this purpose and I’m using this command:

docker run -d -p 8080:8080 --name fess \
    -v $PWD/data/fess/config:/opt/fess \
    -v $PWD/data/es/data:/var/lib/elasticsearch codelibs/fess:latest

Note that I’m using the default setting for Elasticsearch so, as per the documentation, I’ve removed the -v option to map that directory.

I’m having a couple of problems:

  1. I’ve tried downloading each of the backup files from the old server and then uploading them to the Dockerised server. However, at the end of that process, no web crawlers are listed. How do I get ALL of the configuration from the old server to the new one?

  2. Despite mapping the directories, it doesn’t look like fess is saving anything to the host directory. Stopping and restarting the container results in a blank configuration state again. What have I misunderstood about saving the configuration and data for this?

Thanks.

2

(from github.com/marevol)
What version did you use?

3

(from github.com/pcolmer)
@marevol It is “latest”. I think that is 13.0.2.

4

(from github.com/marevol)
latest is 13.2.0 at the moment.
It’s better to use Backup/Restore on Backup page.

5

(from pcolmer (Philip Colmer) · GitHub)

latest is 13.2.0 at the moment.

Sorry - typo on my part.

It’s better to use Backup/Restore on Backup page.

That’s what I tried to do. I downloaded the files from the old server and uploaded them to the new one. The only file that seemed to make any difference was system.properties (because we turn off the Admin link) but it didn’t survive the Docker container being stopped and restarted (i.e. nothing was written to the mapped volume). I don’t know which file is supposed to contain the web crawling configuration but I cannot find one that restores those settings.

6

(from github.com/marevol)
Web crawling configurations are in .fess_config.* indices.
You can download fess_basic_config.bulk, and then upload it.

7

(from github.com/pcolmer)
OK - these are the steps I’m taking, just in case I’m missing something out.

  • On a new system, start the Docker container:
docker run -d -p 8080:8080 --name fess \
    -v $PWD/data/fess/config:/opt/fess \
    -v $PWD/data/es/data:/var/lib/elasticsearch codelibs/fess:latest

  • Log in on the new (13.2) system.

  • On the old (11.4) system, download fess_basic_config.bulk and upload it to the new system. The new system says Bulk process is started. but I cannot find any way to check the progress of the process.

If I now browser to /admin/webconfig/, nothing is listed under Crawler > Web.

  • On the old system, download system.properties and upload it to the new system. Confirm that the upload has worked by navigating to /admin/general/ - Login Link is not selected, so I know that the old properties have been correctly applied.

  • On the new system, stop the Docker container and restart it. Browse to /admin/general on the new system. Login Link is selected again. On the host system, there is nothing under $PWD/data/fess/config.

8

(from github.com/marevol)
The data format is different between major versions because of changes of elasticsearch.
So, you need to upgrade it by 11.4 -> 12.7 -> 13.2.

For fess_basic_config.bulk, download it from an old system and then upload it to a new system.
For system.properties, put it to $PWD/data/fess/config.
If you change fess_config.properties, merge the latest one and also put it to $PWD/data/fess/config

9

(from github.com/pcolmer)
@marevol thank you for the additional information.

The properties are now surviving the container being stopped and started.

I am having trouble upgrading, though. I’m starting off with the 11.4 container and importing fess_basic_config.bulk from the old 11.4 server. That works fine - I see web crawlers and I can start the crawl process and ES shows that it has got some documents.

I’m not sure what to do next. I’ve tried the following but it hasn’t worked:

  1. Download .fess_basic_config.bulk from the 11.4 container server.
  2. Stop the 11.4 container.
  3. Change the command line to reference the 12.7 version and start the container.

After everything starts up, there are NO web crawlers listed. If I go to “/admin/upgrade”, the Version From starts at 12.0 and goes to 12.6, so I cannot upgrade from 11.4. I’ve tried importing the .fess_basic_config.bulk file from 11.4 but that hasn’t restored the web crawlers either.

Thank you for your help.

10

(from marevol (Shinsuke Sugaya) · GitHub)

Download .fess_basic_config.bulk from the 11.4 container server.

For upgrading to 12, it’s .fess_basic_config.12.bulk.

11

(from pcolmer (Philip Colmer) · GitHub)

For upgrading to 12, it’s .fess_basic_config.12.bulk.

:+1:

Thanks.

12

(from github.com/pcolmer)
What do I use to get from 12 to 13? There only seems to be .fess_basic_config.bulk and importing that into a 13.2 system doesn’t restore the crawlers.

(I can see that /admin/upgrade still includes the 12.x versions but I think re-uploading the 12.x bulk file may have broken something so I’m going to start from scratch and see how that works out)

(Just as an added note, it is necessary to delete the indices before 13.x will run after using 12.x)

13

(from github.com/pcolmer)
OK - I’ve got up to 13.2 now.

Thank you for your help with this. I realise that I’ve made life harder for myself by not keeping up to date with the various versions in between.