[DISCUSSION]: deploying an NX monorepo, CI/CD and documentation.

[SSmale]

Description

Having a read though the documentation there seems to be a real lack of examples and best practices on how to deploy your shiny new project and searching around for articles is a sea of outdated or hacky looking workarounds.

I would like to discuss how the community is deploying their repos and some best practice input from the Dev team and then either help write some docs or at least curate them into an article. I am really enjoying developing with nx but I find the docs lacking.

A secondary thing would be some examples of CI/CD configs for the main providers for different configs.

Lastly, is there a NestJS version of these angular.json deployers

Suggested areas of documentation

preparing to deploy

• environment files or .env for api url
• deployment types
  • docker
  • ui from backend
  • ui and backend separately

Configuring the deployer

running your first deploy

NX CICD Pipeline by example

[MikeAliG]

Hello! Any update on this?

[SSmale]

No, I was hoping for some discussion with the community. I have got mine deploying via circle ci to heroku for the nestjs api and netlify for the angular front end but it feels a little hacky.

[subramanya4shenoy]

yes it feels hacky and feels like missing the party. need a proper documentation and example to deploy NX monorepo to AWS

[MikeAliG]

Thank you for your response. I tried deploying my express app to Heroku but the main issue is that we don't have a package.json for the built Node apps and that's really hard to maintain especially with Docker. Hopefully, we'll see better documentation soon since I have not yet seen a real world deployable and stable solution to tackle this issue.

[SSmale]

My current approach is to send the whole NX project to Heroku and then Taylor the build and deploy npm commands to only deploy the nest backend

[MikeAliG]

I also do the same thing but it's just CI/CD commands stacked together just to deploy and this whole process feels wrong for some reason because if any update and symlinks are added to the project, this whole chain won't work anymore and we have to write another stack of CI/CD. Hopefully, we find a real solution soon because this project has increased our productivity so much and it's such a shame if we cannot deploy stuff properly.

[Delapouite]

If I'm not mistaken, a similar pain point was raised in this issue #3817

[MikeAliG]

Yes. Unfortunately that's the same issue.

[mikecann]

I too am trying to use nx with Heroku and running into issues. So far im trying to follow this guide which basically involves building docker first then pushing to heroku: https://hackmd.io/US2eky6lSMecuHxegFZCbA?view but it doesnt seem to work for me

Just a quick edit I was able to follow a suggestion on this discussion: #1777 which mentioned using https://github.com/vercel/ncc which seems to work pretty well

[github-actions[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[Arsenalist]

Is there any example of guidance here by nrwl on how to deploy these apps?

[github-actions[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[SSmale]

If we missed this issue please reply to keep it active.

a reply from a maintainer with their recommendations would be great

[kasius]

any update on it?

[lampewebdev]

Some best practices or some basic guidance would be nice here :)

[Arsenalist]

I ended up deploying the front-end to Netlify and the back-end to Heroku off of the same repo.

Basically, configure two different commands in package.json to build stuff. For me, the front-end command was this:

"build:ranker:prod": "node node_modules/nx/bin/nx build ranker --configuration=production --prod"

The back-end APIs were:

"serve:api:prod": "node node_modules/nx/bin/nx serve api"

Depending on where you host your FE or BE, you may have to edit your boot files (e.g., for Heroku it was Procfile) to start the app.

[wiredmatt]

I think tools like this and turborepo are the future of development. But Nx plays horribly deployment-wise. It doesn't make sense at all that when you deploy, let's say a React app to Vercel, it also installs Nest and Bcrypt that you use in your microservices, and it's the same the other way around.

What I found was best is to just use a standard monorepo with yarn workspaces, and in your CD just delete the packages that have dependencies you don't care about. I'm doing so here

You could also create a Dockerfile for each service / frontend:

• service-a.Dockerfile

FROM node:lts-alpine
WORKDIR /app
COPY . .
RUN rm -rf frontend
RUN yarn install
RUN yarn workspace local-lib build
RUN yarn workspace service-a build

CMD yarn workspace service-a start:prod

• frontend-a.Dockerfile

FROM node:lts-alpine
WORKDIR /app
COPY . .
RUN rm -rf backend
RUN yarn install
RUN yarn workspace ui-lib build
RUN yarn workspace dashboard build

CMD yarn workspace dashboard start:prod

And then you would build the docker images for those, later pushing them to the image registry, and finally pulling the image from your EC2 / DigitalOcean box, or whatever, you can use this as reference on how to automate that, you could also have a husky hook to force the developer to build the image locally and allow to push only if it succeeds.

Finally I want to say that the build tool of nx itself is actually fine, you could just use it for that, building packages.

Hope my comments helped, cheers.

[dan-cooke]

Use the generatePackageJson option in your webpack builder options. This will allow you to only install the dependencies required for each project

[wiredmatt]

Great solution for packages that use webpack, but I don't use it

[dan-cooke]

Oh. Then you can use the devkit as another commenter suggested.

NX exposes the dependency graph to you via the readCachedDependencyGraph function.

So you can run a quick script before your build to generate your own package.json.

ideally this functionality could be added to a custom executor for your bundler

[hsellik]

Vite build executor now seems to have the capability to generate a package.json file: https://nx.dev/nx-api/vite/executors/build#generatepackagejson

Didn't test if it also creates a pruned lockfile.

[rfprod]

In my experience, Nx works great both in terms of development and deployment.
If workspace dependencies are cached between workflow runs, there is no need to reinstall the dependencies every time, i.e. the installation step will be very fast unless one of them has changed. Changing any dependency creates a new version of the workspace cache that is used for subsequent runs of the workflows/pipelines.
With test production builds whose results are also cached, there is no need to rebuild anything from scratch if nothing has changed, Nx handles this, one just needs to properly manage the cache in the CI/CD environment.

To make docker containers take up less space, I would recommend thinking about how to organize project dependencies.
I would suggest only declaring the API packages as dependencies and the rest as devDependencies.
In such way, one can create a base image for deploying all applications in a monorepository, e.g.

# NodeJS base app image.
# This image is rebuilt whenever any of the project dependencies change.

# Define image.
FROM node:16.15.1-alpine
# Set environment variables.
ENV DEBIAN_FRONTEND=noninteractive
# Define app directory.
WORKDIR /app
# Copy dist.
COPY /package.json .
COPY /yarn.lock .
# Install dependencies and create a user.
RUN npm i -g npm ; \
  npm i --production --ignore-scripts --legacy-peer-deps ; \
  npm cache clean --force; \
  addgroup -S appgroup && \
  adduser -S appuser -G appgroup

Then the base image mentioned above can be used to build an API app image, e.g.

# NodeJS API app image.

# Define image.
FROM rfprod/nx-ng-starter:base-latest
# Set environment variables.
ENV DEBIAN_FRONTEND=noninteractive
# Define app directory.
WORKDIR /app
# Copy sources.
COPY /tools/proto ./tools/proto
COPY /dist/apps/api ./dist
# Set user.
USER appuser
# Configure exposed port.
EXPOSE 8080 8082
# Define startup command.
CMD [ "node", "./dist/main.js" ]

or a client app image, e.g.

# NodeJS client app image.

# Define image.
FROM rfprod/nx-ng-starter:base-latest
# Set environment variables.
ENV DEBIAN_FRONTEND=noninteractive
# Define app directory.
WORKDIR /app
# Copy dist.
COPY /dist/apps/server-prod ./dist
COPY /dist/apps/client ./dist/assets
# Set user.
USER appuser
# Configure exposed port.
EXPOSE 8080
# Define startup command.
CMD [ "node", "./dist/main.js" ]

To make client application images even lighter, instead of using NodeJS to serve client applications, Nginx can be used, something like

# Nginx client app image.

# Define image.
FROM nginx:alpine
# Copy Nginx configuration.
COPY ./.docker/nginx_default.conf /etc/nginx/conf.d/default.conf
# Remove default nginx index page.
RUN rm -rf /usr/share/nginx/html/*
# Copy dist.
COPY /dist/apps/client /usr/share/nginx/html
# Expose port.
EXPOSE 8010 80
# Define entrypoint.
ENTRYPOINT ["nginx", "-g", "daemon off;"]

With this approach, application distributions are built on the runner and cached, and docker uses build artifacts.
Using this approach, one can build multiple applications in parallel.

Also, again in my experience, Nx works pretty well with serverless platforms like Firebase.
Automating API deployment to Firebase Functions and client application deployment to Firebase Hosting is pretty straightforward.
Firebase Functions uses a separate package.json that is used by the platform when deployments are triggered. One just has to keep it in sync with the main package.json.

[dan-cooke]

This answer does not help in the real world. It works for your simplistic example of having only a single API

What if your monorepo consists of thousands of projects, with different teams?

[rfprod]

This answer does not help in the real world. It works for your simplistic example of having only a single API

What if your monorepo consists of thousands of projects, with different teams?

It helps if you can scale it properly. Take your time to think how.

[dan-cooke]

@rfprod i don’t See how it could possibly scale.

if I have a GraphQL API and various scheduled tasks that utilise the MongoDb node driver for example, how can I ensure the Scheduled tasks do not include any GraphQL deps using your method of putting “api dependencies in production”?

your method works if all of your node API code share the same dependencies , but that is highly unlikely in the real world. There are better methods for ensuring only the exact dependencies each module needs are included in image

A Dockerfile with multi stage and the generatePackageJson webpack works well for me. I’m going to do a write up on this discussion to demonstrate when I find some time.

Ps I wrote this on mobile so apologies for the formatting

[rfprod]

@dan-cooke Exactly as you described, but not necessarily with multi-stage builds.
Multiple base images and/or multi-stage builds is a way to go.

For example you can remove dependencies using an executor before building a base image. Should work well if you do it in an isolated context like a CI/CD runner that builds only one base image. Several runners can be spinned up to build different base images in parallel.

1. download workspace artifacts (all dependencies that were cached previously)
2. run an executor

schema.json

{
  "$schema": "http://json-schema.org/schema",
  "type": "object",
  "cli": "nx",
  "properties": {
    "projectName": {
      "type": "string",
      "enum": ["api1", "api2", "api3"],
      "description": "An API project name."
    }
  },
  "required": ["projectName"]
}

executor.json

{
  "executors": {
    "configure": {
      "implementation": "./configure",
      "schema": "./schema.json",
      "description": "Remove unneeded dependencies to build production APIs.."
    }
  }
}

schema.d.ts

export type TSupportedProjectName = 'api1' | 'api2' | 'api3';

export interface IExecutorOptions {
  projectName: TSupportedProjectName;
}

configure.ts

import { ExecutorContext, removeDependenciesFromPackageJson } from '@nrwl/devkit';
import { flushChanges, FsTree } from 'nx/src/generators/tree';

import { IExecutorOptions } from './schema';

const userExclusivelyBy = {
  api1: ['dep1', 'dep2'],
  api2: ['dep3', 'dep4'],
  api3: ['dep5', 'dep6'],
};

export default async function cleanupPackageJson(options: IExecutorOptions, context: ExecutorContext): Promise<{ success: boolean }> {
  const name = options.projectName;
  const tree: FsTree = new FsTree(context.root, false);

  switch (name) {
    case 'api1':
      removeDependenciesFromPackageJson(tree, [...userExclusivelyBy.api2, ...userExclusivelyBy.api3], []);
      break;
    case 'api2':
      removeDependenciesFromPackageJson(tree, [...userExclusivelyBy.api1, ...userExclusivelyBy.api3], []);
      break;
    case 'api3':
      removeDependenciesFromPackageJson(tree, [...userExclusivelyBy.api1, ...userExclusivelyBy.api2], []);
      break;
    default:
      break;
  }

  const changes = tree.listChanges();

  flushChanges(context.root, changes);

  return { success: true };
}

4. start building a docker base image for an api, copy package.json, install production dependencies etc.

The same executor can be used with a multi-stage build approach.

[dan-cooke]

Another issue here is that you are building files locally on your system, as oppossed to in a docker layer, this will not work for libraries that use native binding like mongodb.

Its generally best practices to run your build inside docker, and not on the host.

This leads us back to one of the most annoying issue with NX + Docker.

You need to run npm ci for the dev deps required to actually build your app, and then npm prune --production later to remove these dev dependencies from the final image.

[itsmelion]

Any ideas of how to run lint+typescript checks on PR's?
i know how to lint, but i'm not getting all TS errors, especially on NextJS projects.

[itsmelion]

yes, thats exactly what i'm doing. but i was looking for more granular steps.

reason is we don't really want to build on GH Actions, takes a lot of time, so we just check. the build goes somewhere else.

[rfprod]

@itsmelion I am not sure if the below can be considered a suitable alternative, but here's what can be done to catch compiler errors using tsc (you'll need a custom executor to do that):

1. print affected app names, save output as an array of strings, map to get relative paths to tsconfig.app.json files

nx print-affected --type=app --select=projects

2. print affected lib names, save output as an array of strings, map to get relative paths to tsconfig.lib.json files

nx print-affected --type=lib --select=projects

3. merge two arrays, iterate over all elements of the final array executing

tsc -p ${path} # where path is ./libs/lib-name/tsconfig.lib.json or ./apps/app-name/tsconfig.app.json

A brief sketch of what I mean. I haven't tested it. Most likely, it will be necessary to filter stdouts to remove unnecessary information before converting to arrays of project names. And there is a better way to implement execution of multiple asynchronous tsc commands than doing it inside a loop.

import type { ExecutorContext, logger } from '@nrwl/devkit';
import { exec } from 'child_process';
import { promisify } from 'util';

export interface ExecutorOptions {
  apps: boolean;
  libs: boolean;
}

export default async function tscCheck(
  options: ExecutorOptions,
  context: ExecutorContext
): Promise<{ success: boolean }> {
  let apps: string[] = [];
  let libs: string[] = [];

  if (options.apps) {
    const { stdout, stderr } = await promisify(exec)(
      `nx print-affected --type=app --select=projects`
    );
    logger.log(stdout);

    if (stderr) {
      logger.error(stderr);
      process.exit(1);
    }

    apps = stdout.split(' ').map(item => `./apps/${item}/tsconfig.app.json`);
  }

  if (options.libs) {
    const { stdout, stderr } = await promisify(exec)(
      `nx print-affected --type=lib --select=projects`
    );
    logger.log(stdout);

    if (stderr) {
      logger.error(stderr);
      process.exit(1);
    }

    libs = stdout.split(' ').map(item => `./libs/${item}/tsconfig.lib.json`);
  }

  const projects = [...apps, ...libs];

  for (let i = 0, max = projects.length; i < max; i += 1) {
    const project = projects[i]; // ./libs/lib-name/tsconfig.lib.json or ./apps/app-name/tsconfig.app.json
    const { stdout, stderr } = await promisify(exec)(
       `tsc -p ${project}`
    );

    // stdout is empty if there is no issues
    if (stdout) {
      logger.log(stdout);
      process.exit(1);
    }

    if (stderr) {
      logger.error(stderr);
      process.exit(1);
    }
  }

  return { success: true };
}

Custom executor docs ref https://nx.dev/executors/creating-custom-builders#creating-custom-executors

PS: see my next message for the full executor implementation.

[rfprod]

@itsmelion here's something that might possibly help:

• feat(tsccheck): implement a workspace executor - tsc-check rfprod/nx-ng-starter#535
• https://github.com/rfprod/nx-ng-starter/tree/a526880e8847f3a197541846aa037245d569c790/tools/executors/tsc - the executor
• usage:

npx nx affected --target tsc-check

or

npx nx run-many --target tsc-check --all

[itsmelion]

Seems legit. ill give it a shot.

[rfprod]

@itsmelion let me know if you need help configuring the executor. However, the PR that I shared has everything needed.

You can just copy/paste the executor.
Then add this target to the tools project configuration if you have one (this is handy if you have local executors/generators that are meant to be used globally across the entire workspace rather than per project).
Also, if you have local executors, adding this command to the postinstall hook helps, see here https://github.com/rfprod/nx-ng-starter/blob/main/package.json#L70. Local executors will be compiled/recompiled each time the project dependencies are installed.
Finally, use the following commands to configure your workspace projects to use the executor:

• add the tsc-check target to all eligible projects

npx nx run tools:tsc-configure

• remove the tsc-check target from all projects

npx nx run tools:tsc-configure --cleanup

[kitsune7]

The company I'm at isn't small and has several apps that we threw into a single Nx monorepo. Deployment was a bit tricky because of the tools we already had set up require that an app deployment happens in its own unique repository. For us, the super hacky solution we eventually came to was building a custom executor that grabs all the assets needed to deploy a specific app and then overrides the majority of the content of another git repository in the same organization on Github (a repository specifically for deployment needed to be bootstrapped for each app). Github Actions can kick off a deployment automatically when something new is pushed.

It definitely works, but it feels like there should be a more effective way of doing things 😅

[ehaynes99]

Effectively, what's needed is functionality like this: https://turbo.build/repo/docs/reference/command-line-reference#turbo-prune---scopetarget

For each application, it creates a directory containing:

• The full source code of all internal workspaces that are needed to build the target
• A new pruned lockfile that only contains the pruned subset of the original root lockfile with the dependencies that are actually used by the workspaces in the pruned workspace.
• A copy of the root package.json

It's effectively just tree-shaking. I would go a step further, though, and say I only really want the build output and a pruned package file with only the prod dependencies. With remote caching enabled, though, it shouldn't really matter. NX might have to do a bit more work to reliably determine the external dependencies that are needed, but it's definitely possible to crawl through an AST and work that out, and the library templates already seem to do this reliably.

[dan-cooke]

After your comment here. I reluctantly decided to try out turbo. And oh boy is it good.

Took about 2 days to fully refactor all my Nx code over but wow is it better.

here are the advantages already:

• no more deploying every app when anything in the root package json changes
• my Ui docker image size has decreased by 50% because I no longer have to bundle api production dependencies
• no need to write verbose project.json files just to opt in to the caching mechanism, turbo just reads your package json

Turbo is much lower level than NX so you will definetly have to customise it for your needs to match some of the built in NX plugins, but I’m really happy with it so far

[hsellik]

Turborepo also doesn't have a definitive guide on setting it up with Docker and CI/CD. The CI/CD is awkward to set up with GitLab if you want to dynamically show tasks only for the affected apps since you can't use dotenv variables created in job scripts in rules, because rules are evaluated before any jobs run and creating child pipelines dynamically is a hassle.

With Vite and Turborepo, the pruned lockfile creation did work very well. Nx Vite build executor also has generatePackageJson argument, but not sure if it has functional parity with webpack executor and prunes the dependencies / generates a lockfile. The docs on ci deployment say otherwise, but maybe it's outdated?

Turborepo has a convenient way to host your own cache though.

[gregg-cbs]

That cache / self build and deploy solution is magical because you dont need CI/CD pipelines, you can build locally and store that to the cache for your hosting provider to use. I did not find turborepo seamless though, i found NX better in getting up and running but at the point of build and deploy I have hit a brick wall. I agree with the deployment size, it is huge because you have to install node modules in the root project, build and then install node modules in the dist. There seems no way to deploy an app on its own without having the other packages and apps lying around. We have also noticed our dist folder has a strange super nested folder structure which is awkward and does not make sense.

[sloanstewart]

I haven't setup a full pipeline this way yet, but I suspect it is possible to get a Docker/podman image of each app and push it to a container registry/elsewhere using the nx-container plugin and GitHub Actions.

nx-container docs: https://github.com/gperdomor/nx-tools/blob/main/packages/nx-container/README.md

1. Install NX Container
   npm install -D @nx-tools/nx-container
2. Install Container Metadata
   npm install -D @nx-tools/container-metadata
3. Init app with container
   nx g @nx-tools/nx-container:init appName
4. Setup app project.json - webpack options

   sourceMap: false
   generatePackageJson: true
   

The GH Action would be similar to this from the docs, but add an affected container command to create an image for each affected app. Then you would just need to get the output from that portion and get to your container registry.

https://nx.dev/recipes/ci/monorepo-ci-github-actions

name: Containerize Apps
on:
  push:
    branches: main

jobs:
  checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - uses: nrwl/nx-set-shas@v3
      - run: npm ci
      - run: 'git branch --track main origin/main'
      - run: npx nx workspace-lint
      - run: npx nx format:check
      - run: npx nx affected --target=lint --parallel=3
      - run: npx nx affected --target=test --parallel=3 --ci --code-coverage
      - run: npx nx affected --target=build --parallel=3
      - run: INPUT_GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} npx nx affected --target=container --parallel=3
      -  GET YOUR CONTAINER TAG OUTPUT AND CONTINUE YOUR CONTAINER REGISTRY PUSH / DEPLOY HERE

[pedro-mass]

- run: 'git branch --track main origin/main'
This line fails for me when it merges back into main

[ChristopherCapito]

Have there been any updates regarding examples?

[ehaynes99]

https://nx.dev/recipes/ci/ci-deployment#prepare-applications-for-deployment-via-ci

Perhaps this is the answer? Honestly, I've already moved on at this point. The whole thing has just approached the "endless backlog syndrome" of new features that it's become more complex than the problem that it ostensibly solves for me...

[gregg-cbs]

Yeah i see the code in that example and i go cross eyed... We are 2 days into NX and we are thinking of leaving it, its becoming a time sink at this point. Some things are really well done but some require are too ambigous.

[filipefborba]

After successfully implementing my (hacky) CI/CD with an integrated Nx repo, I am now hacking my way at another CI/CD for a package-based Nx repo.

To me, it's very frustrating that Nx doesn't provide any documentation on deployments. Deploying is one, if not the most, important aspects of your application, and Docker is a pretty solid standard right now.

All we want is a working example of an Nx workspace with a nest app and a next app, both with their respective Dockerfiles and a possible target to run with nx affected (ex: nx affected --target=docker-build).

I love the tool, and I am really enjoying working with monorepos, but if things keep getting on the way, it gets hard to justify its use in our companies, especially when it involves working in a different way that people know and are used to (polyrepo).

PS: thank you for all the work! I'd be happy to share my experience with Nx if anyone's interested.

[JayAhn2]

Do you have any public repo to get some insight? (Dockerfile, project.json, and so on)

[ruifelizardo-unlocking]

I would also like to know your solution for the integrated NX repo.
Thanks!

[naviat]

@filipefborba I would greatly appreciate it if you could share your solution. Thank you very much.

[filipefborba]

Sorry guys, only saw that because @naviat mentioned me (@JayAhn2 @ruifelizardo-unlocking). I can't share the repo but I can tell you what I did. It's a good idea to create a test repo to share with everyone here if you can! I used "nx": "14.4.3" with Next.js, Nestjs and Prisma.

There are some steps to my solution:

• NX
  Inside project.json I added a new target command to be run when deploying. It would be responsible for building and publishing Docker.

# project.json

"deploy": {
      "executor": "@nrwl/workspace:run-commands",
      "defaultConfiguration": "development",
      "options": {
        "commands": [
          "docker build -f ./apps/core/Dockerfile . -t {args.DockerContainerRegistry}/your-registry",
          "docker push {args.DockerContainerRegistry}/your-registry"
        ],
        "parallel": false
      },
      "configurations": {
        "development": {
          "DockerContainerRegistry": "your-registry-dev.azurecr.io"
        },
        "production": {
          "DockerContainerRegistry": "your-registry-prod.azurecr.io"
        }
      }
    },

• GH Action
  Used the new target command to deploy.

      - name: Deploy affected projects - DEV
        if: github.event.inputs.environment == 'development' || github.ref_name != 'main'
        run: npx nx affected --target=deploy --configuration=development

      - name: Deploy affected projects - PROD
        if: github.event.inputs.environment == 'production' || github.ref_name == 'main'
        run: npx nx affected --target=deploy --configuration=production

• Dockerfile
  This is the Dockerfile that I was using. Notice that I use prisma inside a common library so backend and frontend can share it. I created the folder core inside prisma with the core-schema.prisma (IMPORTANT: output = "../generated/core-client"). Did this to be able to use multiple prisma schemas/DBs, but didn't end up using.

# Builder -> used to build the image
FROM node:lts-alpine as builder

# Create app directory, this is in our container/in our image
WORKDIR /app

# Copy builded app to container
COPY ./dist/apps/core .
COPY ./yarn.lock .
COPY ./libs/common/src/prisma/core/core-schema.prisma ./prisma/schema.prisma

# Fix OpenSSL when alpine was bumped to version 17
RUN apk add --update --no-cache openssl1.1-compat

# Install dependencies
RUN yarn install --frozen-lockfile
RUN npx prisma generate

# Runner image -> used to copy things from builder and run the app
FROM node:lts-alpine as runner
WORKDIR /app

# Fix OpenSSL when alpine was bumped to version 17
RUN apk add --update --no-cache openssl1.1-compat

COPY --from=builder ./app ./
COPY --from=builder ./app/generated/core-client ./

# Run app
ENV PORT=3333
EXPOSE ${PORT}
CMD node ./main.js

I think this will help you all =)

[andreapigatto]

Please correct me if I'm wrong (I'm still experimenting with nx).
I think integrated repo is a great idea but I don't understand why nx want to sell it as the best approach.
Actually I see more cons than pros to have a single package.json.
One for sure is related to CICD and deployment. Just hacking as in previous posts is possible to run multiple pipelines one x project.
At same time having all dependencies in one package.json will make the list grow and be hard maintainable.
I don't really see how this can scale. Any idea? Am I missing something?

[dan-cooke]

It’s not the best approach, it’s just an approach. Of course NX are going to sell it, it’s a product at the end of the day. You have to weigh up the pros and cons yourself. My personal opinion is that NX just has far too much overhead for monorepo and if you are going to go down that route I would strongly recommend turbo instead. But I personally prefer multi repo these days anyway, just because google use a monorepo doesn’t mean your startup has to. Multi repo has much nicer separation of concerns, especially for distributed systems. It’s much harder to enforce code owners and permissions in a monorepo. Don’t default to Monorepo, use multi repo until you find it’s slowing you down or you would gain something from a monorepo. Not worth the complexity - especially early on

…

On Fri, 6 Oct 2023 at 17:02, Andrea Pigatto ***@***.***> wrote: Please correct me if I'm wrong (I'm still experimenting with nx). I think integrated repo is a great idea but I don't understand why nx want to sell it as the best approach. Actually I see more cons than pros to have a single package.json. One for sure is related to CICD and deployment. Just hacking as in previous posts is possible to run multiple pipelines one x project. At same time having all dependencies in one package.json will make the list grow and be hard maintainable. I don't really see how this can scale. Any idea? Am I missing something? — Reply to this email directly, view it on GitHub <#9183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFOCQ5ZYH4V4O4FKP5X64ZLX6AMQ5AVCNFSM5SRMZLIKU5DIOJSWCZC7NNSXTOKENFZWG5LTONUW63SDN5WW2ZLOOQ5TOMRRGA3DQMA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreapigatto]

@dan-cooke personally I liked the idea of monorepo and was playing a lot with lerna even in the past in projects now in production.
I like the idea to have all in one repo. I found it interesting for example if you have a landing page developed in next.js, a web app in react and a design system with Storybook shared between the 2. Or even if you have a node backend and react frontend. It's nice that you can have some peer dependencies, optimize node_modules, share configs like eslint, jest, etc.
Plus you are giving visibility to other teams and reuse more the code. It's true that you need to have all development department aligned on this mindset and remove any code ownership or permissions that you can have with multi-repo approach.
But to be honest as frontend developer is still hard to understand for me why if I'm working in the same company I should not be able to see other repos sometimes even directly impacting my code (imagine an API repo for a web app).

I like the effort from nx to have some kind of monorepo boilerplate ready to use but it's good just at beginning or if you are not that skilled developer.
Personally I would call it monolith and not monorepo having just one package.json with dependencies on root.
More your app will grow and scale more you will need to touch those configs and the time saved at beginning will be lost later on finding workarounds or trying to make scaling something that is not done for that.
Then as usual you will find someone in the company that want to use some specific react version or other frameworks and you will end up to spend your time to solve conflicts between dependencies.
One of benefits nx tried to sell at beginning will disappear going down the way of integrated monorepo.
I had experience in my last project about this. Project it's 6 months old and these scaling issues were already hitting us.

P.S. Post was not a complain for nx that I find it great especially together with lerna but more to realise if it was just me or someone else is feeling the same about integrated monorepo :)

[dan-cooke]

I too have used monorepos for the exact use cases you are describing, it’s definetly useful for that kind of thing.

And you are right the single package json is a major flaw, turbo also suffers similar issues, although because of its prune command for deploying individual apps from the overall repo - it is much better all round.

See this vercel/turborepo#2862 issue I raised over there

i just don’t think the existing monorepo solutions are good for Javascript. It would be interesting to see some companies that use NX / Turbo - how large their codebase are etc.

[gregg-cbs]

I was using turbo repo but we kept running into issues with commonjs and modules and importing packages into different projects... it just was not working out for us.

NX was a learning curve but eventually we got things working together. Now we are strugggglinngggg with deployments, we want to cry.

In turbo you can build the project locally and ship that build. In NX i am finding it difficult to run tests, build and deploy without having github actions and pipleines and circle ci and all this stuff. This stuff is slow. My pc builds my entire project in seconds. These online solutions take minutes.

There is no concrete examples on how to deploy this stuff and because NX brags about doing everything for you it seems this is not something they do for you or help with.

[zackarydev]

What kind of git branching strategies do you use? Do you just tag main for your deployments? How do you keep microservices independently deployable while using a monorepo solution?

[pedro-mass]

@akv-mshin Any chance we can see the yml?
Curious how it's setup

[fpaz]

interested in this strategy as well

[naviat]

@akv-mshin I am also interested.

[JesseDavda]

@akv-mshin How do you handle version rollbacks? Are you rebuilding the artifacts on rollback?

[david1542]

I'm also interested in this yaml!

[phihochzwei]

I just wanted to share our approach to this matter. As others said: It might not be perfect but it's a process/setup that is working for us.

We have a NX project which currently consists of:

• 2 Nest.js Applications
• 3 Angular Applications, 2 of them are different Webshops, approx. 6-8 will follow within the next 2 years
• 13 Libraries shared across everything (Well, some, not all, apparently)

Our department (16 devs, 3 Project Owners and 6 QA Engineers) is divided into 3 Team: one for each Webshop and one for what we call "Core Development". This "Core" consists of the 3rd Angular Application (a configuration utility), the two Nest.js applications and several libraries that are used by both Webshops.

The whole project is living inside Gitlab. We assigned all files to the different members of the teams by creating a CODEOWNERS file. That way, when a merge request is created and you have changed something outside your "Domain", at least one member of the respective team has to approve. Codereviews and Approvals can and should be made by everyone, merging is restricted only to the team leads.

Since both shops deploy at different times and in different cycles, at one one point once in a while, the respective team lead cries:

and creates a release branch. When released, the branch is merged into master via merge request (squash commits) and receives Tag. Once that has been done, no new features/task will be added, only fixes

As a branching strategy we went more of less for the "classic" git flow approach with some naming conventions:

• "develop" (the root of all evil)
• "staging" (finished features that are deployed to our QA systems. Develop gets merged manually in here by the team leads)
• "release/XXX" (state of code branched from staging at the moment the code freeze is declared)
• "feature/XXX" (new feature/task named with the corresponding Jira Ticket. branched from develop and MR´s go back to develop)
• "bugfix/XXX" (fixes for bugs found by QA during the release phase. gets branched of release-branch and merged back into it)
• "hoffix/XXX" (we all know, once in a while, shit hits the fan. In such cases, hotfix gets branched from master)
• "master" (yeah, its the released stuff :) )

To build/deploy the whole story, we have parent-child pipelines set up in Gitlab, which can be started either by branch or by Tag. Once building is done we create docker containers and push them into the Gitlab registry. In parallel, there is a connection to Azure (don´t ask me about that, this is all black magic to me) which checks every 10 min for new containers in the registry and if something new is available it gets rolled out to our respective environments.

So far, we do not face any issues where I would say "Okay, we are missing something, THIS is a problem". We did not do any hacky stuff, just the usually builds and all is fine. Working in a monorepo for such a project is really nice and smooth since you dont have to struggle with library publishing and, and that is the part I like the most about out it, the teams can easily help each other out.

[AlanGRutter]

The issue of how to deploy your Nx applications using Docker seems to crop up constantly on the Nx Discord channel, but so far there is still no up-to-date, definitive examples that illustrate all the necessary steps. I have a simple Host and Remote Angular app with some libraries and I cannot find a canonical example of deploying this using a Dockerfile.

Nx is a great tool but the lack of concrete examples for everyday tasks such as dockerizing host and remote apps into a single container will inevitably lead to user frustration and abandonment in favour of something else.

[gregg-cbs]

This lack of documentation on deployment is as relevant today as it was in 2020.. We are struggling to figure out how to deploy our apps from our monorepo to digital ocean.

We run tests and build locally and now we want to deploy the build but seems we cant without docker and tons of other interventions. Its really confusing. And the dude in the youtube videos about NX is so chill and makes it look so easy, its really misleading.

[andreapigatto]

Yes for sure in 2024 you need a lot of gitops to properly deploy in production some applications in a scalable and maintainable way. Terraform, Kubernetes, Docker, Helm, Vault, etc. are just some of the tools that are becoming mandatory. Nx is just a tool to manage your monorepo architecture and max to deploy your apps to npm or verdaccio. Deploy all to a cluster in digitalocean is out of the scope of nx. It's normal that you don't find any nx videos explaining fully what you are trying to achieve. You need to check all other pieces and after a lot of research put the puzzle all together. Or add a DevOps team in your company. I'm in the same journey and quite close to have a MVP of all architecture required.