feat(schematic): added nginx to schematic docker container

[linglp]

Context

To make sure that schematic APIs could be deployed to production, we have to add nginx. This is similar but different from what I did here in this PR: Sage-Bionetworks/schematic#1202

Changes

Some configuration files are the same as before. For example, self-signed.conf, ssl-params.conf, uwsgi.ini.. but there are also some changes:

1. I remembered that I had to modify "entrypoint.sh" in the original image. But that wasn't a clean solution, and it is not a good practice to modify the original nginx.conf. To avoid that, I ended up adding redirect.conf
2. since the base image that we are using is still: tiangolo/uwsgi-nginx-flask:python3.10, the python version that we use in schematic API has to match the python version allowed by the base image. I ended up having to change python version from 3.10.12 to 3.10.13 in mono repo. If you do nx prepare schematic-api again, you should be able to get 3.10.13
3. When updating the file permission, for simplicity, I just updated the file permission of root folder and app folder. But this is NOT recommended. Please see "future items" for more details.

To test this PR:

1. cd apps/schematic/api
2. In apps/schematic/api folder, make sure that you have renamed .env.example to .env or you already have a .env file
3. run nx prepare schematic-api to get the correct version of Python
4. Run poetry shell to make sure that you are in an virtual environment. And to prepare for certificate and private key locally, run prepare_key_certificate.py This script will generate private_localhost_certificate.crt and private_localhost.key files and save the content of private key and certificate as environment variables in .env
5. Run the build command to build a schematic image: docker build -t schematic-api .
   And run the schematic command by doing: docker run -d --name mycontainer --env-file .env -p 7443:7443 schematic-api

OR
nx build-image schematic-api
when running docker images, you should be able to see something like:

REPOSITORY                               TAG           IMAGE ID       CREATED          SIZE
ghcr.io/sage-bionetworks/schematic-api   local         3d4d4e9fd980   57 minutes ago   2.3GB

And then you could create a docker container by doing something like: docker run -p 7443:7443 -t <docker image id>
or you could go back to the root folder where you ran "docker build-image" command and ran: nx serve-detach schematic-api

And if you are using VS code, you should be able to open the docker container like this:

When seeing the page below:

make sure you click on: Advanced -> "Proceed to localhost(unsafe)"

Test by curl

Using curl to test and ensure that everything is working as expected. The actual port number depends on your setting. Below are examples:

1. curl -I https://localhost:7443/api/v1/ui/
   I saw:

curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

This makes sense because we are using a self-signed certificate

5. curl -I -k https://localhost:7443/api/v1/ui/
   I saw:

HTTP/2 200 
server: nginx/1.23.2
date: Thu, 11 Jan 2024 20:42:57 GMT
content-type: text/html; charset=utf-8
content-length: 1488
access-control-allow-origin: *
x-frame-options: DENY
x-content-type-options: nosniff
x-xss-protection: 1; mode=block

This is also expected because -k allow connections to SSL sites without certificates or to ignore certificate validation errors.

6. curl -I -k http://localhost:7443/api/v1/ui/:

HTTP/1.1 302 Moved Temporarily
Server: nginx/1.23.2
Date: Thu, 11 Jan 2024 20:44:19 GMT
Content-Type: text/html
Content-Length: 145
Connection: close
Location: https://localhost:60110/api/v1/ui/
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block

This makes sense because we redirect http to https

Future items

This Dockerfile is not a final product. There are some important issues that need to be address before actually using it for production deployment:

1. .synapseCache is stored in /root
   This is not a good practice. But currently schematic doesn't allow cache of .synapseCache folder to be configurable. This work is captured in ticket: FDS 1446
   After we move .synapseCache out of root folder, we need to revisit this dockerfile and update the file permission according to the new location of cache.

2. Run the dockerfile as a non-root user
   I tried adding USER www-data in dockerfile but it didn't work because the base image: tiangolo/uwsgi-nginx-flask:python3.10 that we are using is using supervisord to bind pots. And binding pots in docker requires super user permission. I can't have www-data to bind pots or work with supervisord for now. This will require additional tickets and attention if we want to meet the best practice.

3. purging cache function needs to be moved out of schematic library
   The purging cache function in schematic library currently calculate the cache size of .synapseCache and if it meets a certain criteria, the cache will get purge. We should move that logic to the mono repo (see issue FDS 1445)

4. Modify check_synapse_cache_size function to deal with empty .synapseCache folder
   For the container built in schematic library, even when .synapseCache is empty, du -sh .synapseCache returns 4KB. But this is not the case when I tried running the same command in this docker container for some reasons. When .synapseCache folder is empty, du -sh .synapseCache returns 0 without a unit. We have to modify check_synapse_cache_size to deal with 0. (See FDS 1445)

Note:

This should be merged after: #2417

[tschaffter]

This PR introduces changes that break best practices (security) and standards that the monorepo aims to establish. I suggest to schedule a call to clarify the content of this review.

[milen-sage]

@tschaffter a nice overview - adding @andrewelamb for awareness.

[linglp]

@tschaffter Do you mind reviewing this PR again? I have stored SSL private key and certificate on AWS, and after more trouble shooting, the deployment seems to be working now: https://schematic-dev-refactor.api.sagebionetworks.org/api/v1/ui/. If the Dockerfile looks good to you, I think I could work on resolve conflicts and merge this PR.

[tschaffter]

I skimmed thought the PR and made a few comment. I can make a formal review once the PR is ready (after cleanup and conflict resolution).

Does the command nx serve schematic-api listen to the new port? This is a convention of the monorepo so that developers can run the REST API either manually or with Docker while keeping the same behavior for the REST API.

[tschaffter]

@linglp You check is failing. The error message should point to the solution :-)

[sonarcloud]

Quality Gate passed for 'schematic-api'

Issues
16 New issues
0 Accepted issues

Measures
4 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[linglp]

@tschaffter If the PR looks good, maybe you could merge it on your end? I am not authorized to merge the PR. Thank you again for reviewing it