Is this because you have the library on a network drive?

Yes

Ok, makes sense

I have a more general question about updating the library... After a few months with forked-daapd, I'm really happy but updating was always problematic. I was using it with network mounted library at first and attributed constant problems to that, so I switched the installation to the server where the library actually is.

It seemed to be better and realtime updating mostly worked (when I added music to the server with rsync, changes were reflected). I thought I noticed that sometimes it didn't work though and I had to issue mpc update or even empty the database... I wasn't sure if I'm doing something wrong, but today I have a report what happened and I think it confirms some bugs with this functionality for sure.

I rsynced some changes again after some time and they weren't reflected via mpc listall, I ran mpc update and they were shown, tried rsync again with more stuff, again no update. Then I just restarted forked-daapd and now changes are reflected correctly just by file system monitoring.

So I guess the code for monitoring the fs changes gets stuck somewhere and then restart helps.

Is this known and maybe planned to be solved in near future? If no, I could possibly try to figure out in time how and why it happens. I started looking into the source for other reasons but if other people don't have these issues and I do, it would make sense for me to try to find the exact problem. It would be tricky because it only happens sometimes and also it's hard to get the exact state of complex multithreaded program while running... I would still rather like it if it was known and being worked on by people who implemented this since I'd rather examine the source for planned enhancements I need than trying to solve this bug... please let me know, thank you! Maybe at least I can help with additional diagnostics.. There was nothing in log at log level now.. I'll switch back to debug now for help in the future.

I can only speak for myself, and I am not aware this bug. If you can figure out how to reproduce it, or get a log output on debug level when it happens, then it would be a tremendous help. Then I will give fixing it high priority. Without that I don't really have anything to go on, however.

I don't personally update my library very often, so a bug in this could possibly go unnoticed for me. I will try to test a bit, but of course hard to say if the bug shows up.

The key thing here will be to check that the library directories at all times are registered for watching with inotify.

If you are able to produce some clues on the bug then please open a new issue here on github.

An additional note that could be helpful in tracking the error: If you run "sqlite3 [your-database]", and then "select * from inotify;" you will get a list of the directories that forked-daapd should have registered with inotify for watching. So if a directory is missing there it is a clue.

Another thing: You said that forked-daapd gets "stuck" - how is that precisely? I'm thinking there are two possibilities here, either the filescanner thread deadlocks, or something in the watching of the dirs is failing. If the thread is deadlocked the symptoms would be that there is no reaction in the log to any new files (even control files like ".init-rescan"), and it will probably also be difficult to stop forked-daapd.

Hi! "So I guess the code for monitoring the fs changes gets stuck somewhere and then restart helps." --> wasn't really sure what I was talking about here :) Actually I don't have a deeper insight why it would fail suddenly but maybe some state (boolean variable etc) was "stuck" at wrong value and notifications from inotify didn't trigger the update routines anymore. or maybe notifications didn't come through at all anymore.

It's just guessing, the only thing I'm sure are the symptoms.. I didn't actually look into the monitoring + updating code yet, but I will do that today so that I have a vague idea that can help troubleshoot in the future.

Regarding debug log, I enabled it but there seems to be some issue with special characters in some of the filenames of my songs, the output gets borked at some point:

[2017-07-29 23:18:43] [DEBUG]      └⎻␍: L␋⎽├␊┼␊⎼ ␌▒┌┌␉▒␌┐ ␌▒┌┌␊␍ ┬␋├␤ ␊┴␊┼├ ├≤⎻␊ 1.
[2017-07-29 23:18:43] [DEBUG]   ⎻┌▒≤␊⎼: C▒┌┌␉▒␌┐ °⎼⎺└ A␋⎼P┌▒≤ ├⎺ ␍␊┴␋␌␊ ␌⎺└└▒┼␍ ␌␉
[2017-07-29 23:18:43] [DEBUG]     └▒␋┼: C⎺└└▒┼␍ ␤▒⎽ 1 ⎻␊┼␍␋┼± ␊┴␊┼├⎽
[2017-07-29 23:18:43] [DEBUG]   ⎻┌▒≤␊⎼: C▒┌┌␉▒␌┐ °⎼⎺└ A␋⎼P┌▒≤ ├⎺ ␍␊┴␋␌␊ ␌⎺└└▒┼␍ ␌␉
[2017-07-29 23:18:43] [DEBUG]     └▒␋┼: C⎺└└▒┼␍ ␤▒⎽ 0 ⎻␊┼␍␋┼± ␊┴␊┼├⎽
[2017-07-29 23:18:43] [DEBUG]   ⎻┌▒≤␊⎼: P┌▒≤␊⎼ ⎽├▒├┤⎽: ⎽├⎺⎻⎻␊␍
[2017-07-29 23:18:43] [DEBUG]      └⎻␍: N⎺├␋°≤ ␌┌␋␊┼├⎽ ┬▒␋├␋┼± °⎺⎼ ␋␍┌␊ ⎼␊⎽┤┌├⎽: 1
[2017-07-29 23:18:43] [DEBUG]      └⎻␍: MPD └␊⎽⎽▒±␊: ␌⎺└└▒┼␍ ┌␋⎽├ ⎺┐ ␉␊±␋┼
[2017-07-29 23:18:43] [DEBUG]      └⎻␍: MPD └␊⎽⎽▒±␊: ⎽├▒├┤⎽
[2017-07-29 23:18:43] [DEBUG]   ⎻┌▒≤␊⎼: P┌▒≤␊⎼ ⎽├▒├┤⎽: ⎽├⎺⎻⎻␊␍
[2017-07-29 23:18:43] [DEBUG]       ␍␉: R┤┼┼␋┼± ─┤␊⎼≤ 'SELECT ┴▒┌┤␊ FROM ▒␍└␋┼ ▒ WHERE ▒.┐␊≤ = '─┤␊┤␊ ┴␊⎼⎽␋⎺┼';'
[2017-07-29 23:18:43] [DEBUG]       ␍␉: R┤┼┼␋┼± ─┤␊⎼≤ 'SELECT COUNT(*) FROM ─┤␊┤␊;'
[2017-07-29 23:18:43] [DEBUG]      └⎻␍: MPD └␊⎽⎽▒±␊: ␌┤⎼⎼␊┼├⎽⎺┼±
[2017-07-29 23:18:43] [DEBUG]   ⎻┌▒≤␊⎼: P┌▒≤␊⎼ ⎽├▒├┤⎽: ⎽├⎺⎻⎻␊␍
[2017-07-29 23:18:43] [DEBUG]      └⎻␍: MPD └␊⎽⎽▒±␊: ␌⎺└└▒┼␍ ┌␋⎽├ ␊┼␍
[2017-07-29 23:18:53] [DEBUG]     ⎼▒⎺⎻: F┌┤⎽␤ ├␋└␊⎼ ␊│⎻␋⎼␊␍; ├␊▒⎼␋┼± ␍⎺┬┼ RAOP ⎽␊⎽⎽␋⎺┼⎽
[2017-07-29 23:18:53] [DEBUG]     ⎼▒⎺⎻: B┤␋┌␍␋┼± TEARDOWN °⎺⎼ 'L␋┴␋┼±R⎺⎺└'
[2017-07-29 23:18:53] [DEBUG]     ⎼▒⎺⎻: B┤␋┌␍␋┼± TEARDOWN °⎺⎼ 'K␋├␌␤␊┼'
[2017-07-29 23:18:53] [DEBUG]   ⎻┌▒≤␊⎼: C▒┌┌␉▒␌┐ °⎼⎺└ A␋⎼P┌▒≤ ├⎺ ␍␊┴␋␌␊ ⎽├⎼␊▒└␋┼± ␌␉
[2017-07-29 23:18:53] [ INFO]   ⎻┌▒≤␊⎼: T␤␊ A␋⎼P┌▒≤ ␍␊┴␋␌␊ 'L␋┴␋┼±R⎺⎺└' ⎽├⎺⎻⎻␊␍
[2017-07-29 23:18:53] [DEBUG]   ⎻┌▒≤␊⎼: C▒┌┌␉▒␌┐ °⎼⎺└ A␋⎼P┌▒≤ ├⎺ ␍␊┴␋␌␊ ⎽├⎼␊▒└␋┼± ␌␉
[2017-07-29 23:18:53] [ INFO]   ⎻┌▒≤␊⎼: T␤␊ A␋⎼P┌▒≤ ␍␊┴␋␌␊ 'K␋├␌␤␊┼' ⎽├⎺⎻⎻␊␍

␍▒┴␋␍@␊␌┌␋⎻⎽␊:·$

This could be problematic when I actually need to see... any ideas or easy possible improvements here?

Among songs in my library is this stuff "cached" from youtube:

Beautiful Romantic Music - Relaxing Music, Piano Music, Violin Music, Guitar Music, Sleep Music ★101-ZR38JWEsSD0.mp3
Saxophone ❤️ Romantic Music Mix-dG6W5Qihtgc.mp3

the names have some unicode characters: ★ / ❤️

There are other songs with more usual special characters like á etc...

Straying a bit off topic here, can you open a new issue? Or possibly two, one on the log output and another on the scanning, when you have details on that. Not sure the two are related. Also, try viewing the log with another editor - maybe it is just your text viewer that is messing up the charset. If that's not it, please share one of the mp3's with me ([email protected]).

Any info when this will be included in an upcoming version?
Will there be an option to do a regular library scan, f.e. to run once a day?