start.sh stops

[johnwkeller54]

I am attempting to install CalcUS on fresh Fedora 37 system. Docker Engine and Compose are installed and working. G16 and ORCA 5 are installed as noted in the .env file. But the start.sh script stops at celery_main_1 lines. See attached files. Any suggestions on how to proceed?

calcus-startsh-output-7-21-2023.txt
env.backup.txt

[RaphaelRobidas]

Based on your Docker logs, it seems like there is a problem with the database scheme. If you attempted to launch CalcUS previously and encountered another error, it's possible that the database was only partially initialized. You can try deleting the data folder (which contains the database) and restarting CalcUS. That sort of error has happened before and was solved this way.

If that doesn't fix it, let me know and I'll investigate further into the issue.

[johnwkeller54]

Hi Raphael, Thanks for info. I deleted all docker containers and images, and the CalcUS folder. Ran git clone again. Ran generate_env.py. ran start.sh. The terminal paused on a "celery" line for several minutes, then the "web_1" lines appeared. The terminal window for start.sh command remained active - it did not go back to a terminal prompt. At that point the localhost:8080 web page was visible. But any attempt to login or register gives a "Content could not be loaded: internal server error" Have you run CalcUS on a Fedora 37 system? If not, which distro do you use? I chose this because it was one of the few that Docker says is compatible w Docker. John ************************ John W. Keller, Ph.D. Professor of Chemistry Emeritus Department of Chemistry & Biochemistry University of Alaska Fairbanks Fairbanks, AK 99775-6160 ***@***.*** *****************************

…

On Fri, Jul 21, 2023 at 1:27 PM Raphaël Robidas ***@***.***> wrote: Based on your Docker logs, it seems like there is a problem with the database scheme. If you attempted to launch CalcUS previously and encountered another error, it's possible that the database was only partially initialized. You can try deleting the data folder (which contains the database) and restarting CalcUS. That sort of error has happened before and was solved this way. If that doesn't fix it, let me know and I'll investigate further into the issue. — Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A24YV5QKUHEMGPACATY2QY3XRLX4DANCNFSM6AAAAAA2THSTXM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[RaphaelRobidas]

Hello John,

Thanks for reporting this and bearing with me. I reproduced your issue and it was linked with the outdated prebuilt Docker image. I updated the code on the master branch and updated the prebuilt Docker image. On my end, with the new code, new image and a fresh directory, it starts up and allows me to login normally. Can you try doing the same to make sure it works now? When using start.sh, the new image will be downloaded automatically.

Regarding your question with the distribution, I use Debian 12 myself, but Fedora should work great too.

[johnwkeller54]

OK. Removed old files. re-loaded CalcUS as above. Web page starts and su now can log in. Is it by design that celery is running at 100% on 1 cpu, even after I log out and close the localhost:8080 page?
After logging in, I can enter a structure by name, the 3D structure is shown, choose energy with xtb method. Job is queued, but does not start. (Is xtb installed with CalcUS, or do I install that separately?)

[RaphaelRobidas]

Great, that's some progress!

xtb (and related binaries) is redistributed with CalcUS for convenience, yes. If the job stays queued for more than a few seconds, there should be some error message in the Docker logs. In your case, do you have an error message?

As for the 100% CPU usage, that is a bit odd. Perhaps that's linked to the calculation that failed; if xtb spawned a process then crashed, the process can sometimes remain. It's normal that logging out and closing the page doesn't change anything though. This doesn't change much for the processes running inside Docker. Restarting the Docker containers (e.g., by triggering a smooth shutdown with Ctrl-C in the console where CalcUS is running, then running start.sh again) will get rid of any zombie process though.

[johnwkeller54]

Ran start.sh again. Submitted another energy job (xtb). Again, queued but not running (now several jobs queued). The terminal shows
celery_comp_1 | PermissionError: [Errno 13] Permission denied: '/calcus/scr/BmVrnvm6zd6ON'

[RaphaelRobidas]

Ah, I see. We've encountered this issue a couple times before and it seems like I haven't managed to fix it permanently. It's related to the file permissions in Linux and how Docker handles them inside the containers. Let me run some more tests and I'll get back to you soon.

[johnwkeller54]

I would like this project to work. The benefits are, compared to WebMO is its free, and it should be able to manage jobs on a local server, a local cluster, and remote clusters.

[johnwkeller54]

<!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; font-size:11.0pt; font-family:"Calibri",sans-serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.WordSection1 {page:WordSection1;} -->Thanks, Raphael!JK  Sent from Mail for Windows From: Raphaël RobidasSent: Saturday, July 22, 2023 1:00 PMTo: cyllab/CalcUSCc: johnwkeller54; AuthorSubject: Re: [cyllab/CalcUS] start.sh stops (Discussion #35) Ah, I see. We've encountered this issue a couple times before and it seems like I haven't managed to fix it permanently. It's related to the file permissions in Linux and how Docker handles them inside the containers. Let me run some more tests and I'll get back to you soon.—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***> 

[RaphaelRobidas]

Hi John,

I think I managed to fix the issue. I created another user on my local machine and got the same error as you. This error only occurs when the user is not the first user of the system, which is your case, judging by your .env file. Of course, this shouldn't happen either way, but it explains why it did not always occur.

I updated docker-compose.yml in order to fix it. You can download only this file and put it in your CalcUS folder, you don't need to start entirely fresh. Here a direct link to the new version: https://raw.githubusercontent.com/cyllab/CalcUS/master/docker-compose.yml

Let me know if that solves your issue and if you encounter any further problem!

Also, for some CPU architectures, it is necessary to specify PGI_FASTMATH_CPU=sandybridge in your .bashrc in order for Gaussian to work probably (not just in CalcUS, but in general). If that is your case, you can add the line in your .env file so that this setting is applied inside the Docker containers as well.

[johnwkeller54]

Hi Raphael, Downloaded the docker-compose.yml file replaceing the old one. Ran ./start.sh (how else to start CalcUS?) Deleted old project. Started new project1. Acetophenone structure with default energy calculation. Calculation started. but gave error shown. John

On Sun, Jul 23, 2023 at 5:57 AM Raphaël Robidas ***@***.***> wrote: Hi John, I think I managed to fix the issue. I created another user on my local machine and got the same error as you. This error only occurs when the user is not the first user of the system, which is your case, judging by your .env file. Of course, this shouldn't happen either way, but it explains why it did not always occur. I updated docker-compose.yml in order to fix it. You can download only this file and put it in your CalcUS folder, you don't need to start entirely fresh. Here a direct link to the new version: https://raw.githubusercontent.com/cyllab/CalcUS/master/docker-compose.yml Let me know if that solves your issue and if you encounter any further problem! Also, for some CPU architectures, it is necessary to specify PGI_FASTMATH_CPU=sandybridge in your .bashrc in order for Gaussian to work probably (not just in CalcUS, but in general). If that is your case, you can add the line in your .env file so that this setting is applied inside the Docker containers as well. — Reply to this email directly, view it on GitHub <#35 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A24YV5RK6XBCTLMRMMARZATXRUUWBANCNFSM6AAAAAA2THSTXM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

web_1 | 172.19.0.8 - - [23/Jul/2023:12:26:38 -0500] "GET /projects/ HTTP/1.0" 200 15286 "http://localhost:8080/calculations/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" web_1 | 172.19.0.8 - - [23/Jul/2023:12:26:44 -0500] "GET /calculations/ HTTP/1.0" 200 19662 "http://localhost:8080/projects/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" web_1 | 172.19.0.8 - - [23/Jul/2023:12:26:44 -0500] "GET /list/?csrfmiddlewaretoken=J9yf5YEZXYWdRxnoREGGr5c255lR1i58Xwhu9EdOYFPZBAXCGZl1Ly9IAaSMrgnZ&project=All%20projects&page=1&user_id=OXJ2QMPVxjqzr&type=All%20steps&status=All%20statuses&mode=Workspace HTTP/1.0" 200 43 "http://localhost:8080/calculations/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" web_1 | 172.19.0.8 - - [23/Jul/2023:12:27:04 -0500] "GET /calculations/ HTTP/1.0" 200 19662 "http://localhost:8080/calculations/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" web_1 | 172.19.0.8 - - [23/Jul/2023:12:27:04 -0500] "GET /list/?csrfmiddlewaretoken=rgVkEAnnJGvNh9q48dQsN8uK3tcmzR49FDEzIgWcKnoz1c0iXyvN7BrqyyJhZPm0&project=All%20projects&page=1&user_id=OXJ2QMPVxjqzr&type=All%20steps&status=All%20statuses&mode=Workspace HTTP/1.0" 200 43 "http://localhost:8080/calculations/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" web_1 | 172.19.0.8 - - [23/Jul/2023:12:27:07 -0500] "GET /launch/ HTTP/1.0" 200 79281 "http://localhost:8080/calculations/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" web_1 | 2023-07-23 12:28:17,164 [INFO] tasks: exe_dir: (calc -1) web_1 | [2023-07-23 17:28:47 +0000] [29] [CRITICAL] WORKER TIMEOUT (pid:30) web_1 | [2023-07-23 12:28:47 -0500] [30] [INFO] Worker exiting (pid: 30) web_1 | [2023-07-23 17:28:47 +0000] [29] [ERROR] Worker (pid:30) exited with code 1 web_1 | [2023-07-23 17:28:47 +0000] [29] [ERROR] Worker (pid:30) exited with code 1. web_1 | [2023-07-23 17:28:47 +0000] [39] [INFO] Booting worker with pid: 39

[RaphaelRobidas]

Hello John,

These errors shouldn't be too problematic. the web_1 | 2023-07-23 12:28:17,164 [INFO] tasks: exe_dir: (calc -1) line tells me that the calculation has been attempted, so that's a good sign. Is there any different error message after this? Also, does the calculation appear as failed in the interface? Here is an example of a failed and a successful calculation side by side:

[johnwkeller54]

Hi again, Gaussian 16 is set up per directions on WebMO site. A test calculation runs fine per the WebMO directions (without the sandybridge env line you mentioned.) Why isnt G16 or ORCA appearing in CalcUS? John

…

On Sun, Jul 23, 2023 at 5:57 AM Raphaël Robidas ***@***.***> wrote: Hi John, I think I managed to fix the issue. I created another user on my local machine and got the same error as you. This error only occurs when the user is not the first user of the system, which is your case, judging by your .env file. Of course, this shouldn't happen either way, but it explains why it did not always occur. I updated docker-compose.yml in order to fix it. You can download only this file and put it in your CalcUS folder, you don't need to start entirely fresh. Here a direct link to the new version: https://raw.githubusercontent.com/cyllab/CalcUS/master/docker-compose.yml Let me know if that solves your issue and if you encounter any further problem! Also, for some CPU architectures, it is necessary to specify PGI_FASTMATH_CPU=sandybridge in your .bashrc in order for Gaussian to work probably (not just in CalcUS, but in general). If that is your case, you can add the line in your .env file so that this setting is applied inside the Docker containers as well. — Reply to this email directly, view it on GitHub <#35 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A24YV5RK6XBCTLMRMMARZATXRUUWBANCNFSM6AAAAAA2THSTXM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[RaphaelRobidas]

If Gaussian 16 works fine, then you can safely ignore the sandybridge line!

As for G16 and ORCA, it must be because of the new simplified interface. It does not show explicitely some options (like software) but instead chooses the options that make the most sense. It's mostly for cases where G16 and ORCA aren't available and for more beginner users. You might want to enable the full interface in your user profile:

You can tick the first checkbox (and untick the other two, if you want), then update your settings. The advanced interface will then be shown in the "Launch Calculation" page. You should then see the "Software" option.