Error when accesing a file from different processes

[juanep97]

I'm trying to access (read) a file from different processes, but httpdirfs (-f --cache) crashes:

thread 140111547192896: pthread_mutex_lock() failed, 22, Invalid argument

Oops! HTTPDirFS crashed! :(
backtrace() returned the following 12 addresses:
httpdirfs(exit_failure+0x32)[0x555acc1c78e2]
httpdirfs(+0x4996)[0x555acc1c7996]
httpdirfs(+0x4f1f)[0x555acc1c7f1f]
httpdirfs(Cache_read+0x168)[0x555acc1cb148]
httpdirfs(+0x81fe)[0x555acc1cb1fe]
/lib/x86_64-linux-gnu/libfuse.so.2(fuse_fs_read_buf+0x1e8)[0x7f6f24cea2f8]
/lib/x86_64-linux-gnu/libfuse.so.2(+0xd4c6)[0x7f6f24cea4c6]
/lib/x86_64-linux-gnu/libfuse.so.2(+0x16f1f)[0x7f6f24cf3f1f]
/lib/x86_64-linux-gnu/libfuse.so.2(+0x1769c)[0x7f6f24cf469c]
/lib/x86_64-linux-gnu/libfuse.so.2(+0x1421d)[0x7f6f24cf121d]
/lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7f6f24707ac3]
/lib/x86_64-linux-gnu/libc.so.6(+0x126a40)[0x7f6f24799a40]

PS: thanks for the nice software!

[fangfufu]

Could you describe what you did in a bit more detail please?

[juanep97]

I mounted a directory with some 100mb-300mb files in it and executed a program that needs to read them / load several of them. When this program executes in a single thread, it just takes time and they are downloaded and cached normally. But when this program executes in parallel (~6 processes), I get the error message above (using httpdirfs -f --cache).

[fangfufu]

What program is this? Which HTTP server are you using? I need to be able to reproduce your problem at my local end.

[fangfufu]

Also could you compile HTTPDirFS with this SHA 1e80844 and see if the problem still occurs.

[juanep97]

Hi, sorry for the long delay, I've been quite busy with academic life. I use HTTPDirFS as a workaround in https://github.com/juanep97/iop4 to mount a remote archive, and avoid having to download ~35GB of files during the CI. But as you can see, tests fail most of the time, because the program does not find the file where it should be / it stopped being mounted at some point (maybe after a connection error?). The version I use is the one in the ubuntu repo. I will try to check if the problem still occurs using that commit you mention.

[fangfufu]

No worries, I am a busy person too. I actually have a rough suspicion of what might be causing the problem. Since you are using this for science, I am going to put this on higher priority. When do you need this to be done by?

Edit: Apologies, I got confused this issues with the other two issues that's due to curl_easy_escape. Debugging lock problem is not exactly easy.

Is this issue only observed within the CI, or does it happen in real life too? Could you point me to a CI run where this failure occur?

You could try and run httpdirfs in single thread mode by passing in -s. This is mentioned in the README under "Useful FUSE options"