Thumbnail recreate/refresh?

English:Fess
21

(from github.com/hottwerk)
SOLVED! As a last try I deleted all docs from .fess_config.thumbnail_queue , restarted the thumbnail generator job, … and new thumbnails where being made again!!
I guess the queue (800.000) was just to big. (dont know what started it to get so big).

Solution: I used kibana dev tools console to empty the thumbnail queue index:
POST .fess_config.thumbnail_queue/_delete_by_query
{
“query” : {
“regexp” : {
“thumbnail_id” : “.*” }
}
}

22

(from github.com/hottwerk)
I cheered to soon… same problem again. I narrow pointed it to the htmlThumbnailGenerator, because all of the thumbs in the queue (50000 now) have generator: htmlThumbnailGenerator…

23

(from github.com/marevol)
What is a problem? need more details…
If thumbnail is enabled, Crawler adds thumbnail generate requests to .fess_config.thumbnail_queue and then Thumbnail Generator job deals with the queue.

24

(from github.com/hottwerk)
First, I installed fess and elastic on a windows server. I think the problem starts when I run 2 or more crawl jobs together. When I do only one crawl job I see in fess.log there are about 4-5 thumbs created per second. Once I start more crawl jobs the creation of thumb drops to about 4-5 thumbs each 3 minutes! Then the queue starts growing until the thumbnail creation job stops working.
(Maybe I can somehow prioritize the thumbnail creation job but I don’t know how. I see in windows task manager for each crawling job a new java process is started. But the thumbnail job process I can not find. )

25

(from github.com/hottwerk)
Windows 2008 , 16GB Ram ,2 CPU (E5-2650 v2 2.6Ghz)

26

(from marevol (Shinsuke Sugaya) · GitHub)

But the thumbnail job process I can not find.

Thumbnail Generator is a thread, not a process.
Did you check cpu utilization and Load average?
I think that 2 cpu is low for multiple crawler processes.

27

(from github.com/hottwerk)
Yes, I watched it for a while. But the average is never that high, some quick 100% Peaks but the average is good. Memory use about 40-60% .

28

(from marevol (Shinsuke Sugaya) · GitHub)

Once I start more crawl jobs the creation of thumb drops to about 4-5 thumbs each 3 minutes!

What is a log message?

29

(from github.com/hottwerk)
Its in the same state now. 300 thumb in the queue, but only about 4 made every 5 minutes or so. What is strange is that I see thumbnails are added to the queue, even though no crawling job is running…?
Logging is in debug, but this is the logging for an hour. I only get these logging because kerberos is not implemented (so NTLM is used).

2018-02-05 08:01:00,257 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials provided (Mechanism level: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt))
2018-02-05 08:07:01,331 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:16:00,281 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:17:48,459 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:25:01,351 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:00,812 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:00,854 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:01,371 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:01,441 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:01,977 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:02,524 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:03,591 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:03,639 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:27:04,184 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:34:00,271 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:42:01,455 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 08:44:00,827 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 09:05:43,152 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 09:05:43,180 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 09:05:43,212 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 09:05:43,242 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 09:05:43,802 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…
2018-02-05 09:05:43,856 [job_thumbnail_generate] WARN NEGOTIATE authentication error: No valid credentials…

30

(from github.com/biligee)
Testing Fess 13.0.1 in two different Linux. Centos7 generates thumbnails for PDF files successfully. Not generating for MS Office files(.docx,etc). As another distribution, I installed Fess 13.0.1.deb in Ubuntu 18.04. thumbnails are not generated at all, but empty folders in /var/lib/fess/thumbnails. Should I install some additional packages?