All projects too often affected

[cwmrowe]

I have introduced an Nx monorepo to our organisation and on the whole its going really well. Thank you Nrwl 🚀! My only issue is with how often the affected command seems to rebuilding all the apps. I think it boils down to everything getting rebuilt when any change to one of the following implicit dependencies is made.

  // nx.json
  "implicitDependencies": {
    "angular.json": "*",
    "package.json": "*",
    "tslint.json": "*",
    "nx.json": "*",
    "tsconfig.base.json": "*"
  },

So even if we add a library that is not yet referenced anywhere, everything is affected and rebuilt.

I was lead to believe from my research that Nx was clever enough only to rebuild what is "really" affected.

• In the Nx docs it suggested that updating package.json will only affect your app if your app depends on a package whose version has changed. But it seems that any change at all to package.json causes everything to get rebuilt.
• And on the Nrwl blog it suggests that I should never have to rebuild another teams app unless I change something that affects that app. But unfortunately this is not my current experience.

It would be really useful to know if there something wrong with my setup or my understanding here. Thank you.

[wrslatz]

I'm also looking for guidance here, specifically on what files should be included in global implicitDependencies and which have opt-in for more intelligent affected analysis. I'd be a lot more comfortable trying to opt into smarter analysis if I could read more documentation on the feature and any best practices.

#3033 (comment) mentions the ability to opt into more intelligent processing, but I don't see that feature documented anywhere in the Nx computation caching docs where global implicitDependencies are mentioned. I found the original PR that introduced this (#2210) but don't see a good answer there either.

[wrslatz]

Looks like these are also related (and don't provide much more insight):
#2338
#3768

I'm trying to figure out which files are suitable for removal from global implicitDependencies based on smarter intelligent analysis. I'm currently on Nx 12.7.0 on one repo and 12.5.9 on another.

[brandonroberts]

@cwmrowe What version of Nx are you on? One option is to make the implicit dependencies for the package.json more granular.

// nx.json

  "implicitDependencies": {
    "angular.json": "*",
    "package.json": {
      "dependencies": "*",
      "devDependencies": "*"
    },
    "tslint.json": "*",
    "nx.json": "*",
    "tsconfig.base.json": "*"
  },

This only looks to see if the dependencies or devDependencies have changed, rather than the whole file itself. It should also only reflect what projects were affected by those changes.

[cwmrowe]

@brandonroberts I am on Nx 12.6.5.

After making the suggested change to the base branch, I am getting the same behaviour as before. The only way Nx lists the correct projects affected by the NPM dependency change is if I remove "package.json" from "implicitDependencies" completely.

[brandonroberts]

@cwmrowe if you can create a GitHub repo with some steps to reproduce, we'll convert this to an issue to investigate further.

[wrslatz]

Coming back to this now that Nx 13 is here (:tada:) I'd be happy to put together a reproduction for this @brandonroberts. We're seeing two specific cases I'd like more guidance on:

1. Global file changes (nx.json, workspace.json, tsconfig.base.json) affect everything
2. Package dependency changes affect everything

For 1, I think the new project.json option may help make this analysis easier? I also haven't gotten to test the selectivelyHashTsConfig option yet but that seems promising.

For 2, I saw Nx is now doing some analysis / caching of npm dependencies, but I don't see any docs nor recommendations on how smart any analysis that exists there might be.

Really I'm looking for guidance on if we can remove some of the global files from implicitDependencies and reduce how often we affect the entire monorepo.

[wrslatz]

FYI @cwmrowe @brandonroberts I did some manual testing on this today (running different changes against affected analysis) and here's what I found (I'm currently on Nx 13.1.1 btw):

1. I seem to be able to remove tsconfig.base.json from global implicitDependencies. Even without it there, Nx calculates a change to the file (like a change to paths when adding or moving a project) as affecting all projects in the repository.
2. Using selectivelyHashTsConfig I was able to validate that changing a single project's path only affects that project (and invalidates the cache for all dependent projects). So if you're using caching in CI (we can't currently 😞) you would see big benefits by reducing unnecessary cache invalidations.

As for package.json changes, I saw the Nx 14 roadmap has some future changes and documentation coming in this area, which I asked about in #7602 (comment)

[cwmrowe]

I have created a GitHub repo with npx create-nx-workspace --preset=angular and added two apps; app1 and app2.

https://github.com/cwmrowe/nx-affected

In this repo I have created a branch npm-dep where all I have done install @angular/material. Neither app1 or app2 reference @angular/material but when I run nx affected:apps --base=master --plain on this branch it lists them both when I would expect none to be listed.

[cwmrowe]

On a separate note I see that nx.json file for the newly created project has much less listed under implicitDependencies than my current project.

  "implicitDependencies": {
    "package.json": {
      "dependencies": "*",
      "devDependencies": "*"
    },
    ".eslintrc.json": "*"
  },

With the most notable omissions being angular.json, nx.json and tsconfig.base.json. If I were to remove these from my repos implicitDependencies, this would mean I would avoid a lot of unnecessary application rebuilds, such as when I add a new library. Which is exactly what I was questioning in my original post.

I think what @wrslatz and myself are after is confirmation that:

1. Its a good idea to remove angular.json, nx.json and tsconfig.base.json from implicitDependencies, and what the implications might be of doing so.
2. Why we still have package.json in implicitDependencies if Nx can already figure our which apps/libs are dependent a package.

[wrslatz]

@Ezard thanks for the detailed reply! That tracks with what I understood so far.

[leggomuhgreggo]

For anyone else, I believe this note:

Nx isn't aware of the mapping between @types/* dependencies and their origin dependency

was addressed here #10083

[wrslatz]

Are those features documented anywhere? That should let us remove the dependency entries in implicitDependencies if I'm reading the changes correctly, but it still seems like scripts are not intelligent enough to know if a change to a package affects the script and there's no way to mark implicitDependencies by task type.

I may need to play with this further to really understand it. For example, I'm not sure that updating eslint would cause all projects' lint tasks that use ESLint to be affected. That still seems like a potential gap.

[leggomuhgreggo]

Are those features documented anywhere?

I wasn't able to find anything in my cursory search, earlier today.

For example, I'm not sure that updating eslint would cause all projects' lint tasks that use ESLint to be affected

Yeah, wouldn't be surprised if that kinda mapping isn't accounted for yet. Seems like a daunting feature to support. Not sure how it'd work without adding formal config support for task- / executor-level implicitDeps definitions, as you say.

[miluoshi]

Not sure how it'd work without adding formal config support for task- / executor-level implicitDeps definitions, as you say.

Nx has recently added a support for task-level cache inputs that allow to configure a glob of files that are inputs of a task cache key computation. If package.json's fields could be entered similarly to implicitDependencies config, it could be used to configure packages that invalidate cache per a specific Nx task.

[xpolb01]

Thank you! This a great thread and everything is now working as expected :)

[LongLiveCHIEF]

it's still broken for me

[dan-cooke]

So one gotcha with how the dependencies are determined https://nx.dev/more-concepts/how-project-graph-is-built

I had the following in my tsconfig.base.json

"~/*": ["./apps/ui/app/*", "/apps/api/src/*"],

Because I wanted to use the same relative paths for my UI and API apps ~

But I checked my nx affected:graph and I noticed that my API depended on my UI. Strange...

That is because they use your paths mapping to determine part of the local dependencies, so because all of my API had imports beggining with ~/ in it - that resolved to the first parameter in the mapping array, which was the UI project.

This caused my API to depend on my UI project.

Fix

I just created a seperate path mapping for my API project

"@api/*": ["./apps/api/src/*"],

[PowerSupply]

I also have this problem in 2024 with Nx 19.0.4 (also 20.0.3)

Removing a path in tsconfig.base.json leads to ALL my projects being affected.

I have a very ordinary tsconfig.base.ts with just a list of paths for every lib project. There are no "sharedGlobals" in nx.json and none of my inputs for targets or namedInputs includes tsconfig.base.json.

How can I find out why Nx thinks all my projects are affected?

// nx.json
"targetDefaults": {
        "build": {
            "dependsOn": ["^build"],
            "inputs": ["production", "^production"],
            "cache": true
        },
        "ci-build": {
            "dependsOn": ["build", "lint", "test", "build-migrations"]
        },
        "ci-build-app": {
            "dependsOn": ["build", "lint", "generate-example-data"]
        },
        "ci-cypress": {
            "dependsOn": ["build"]
        },
        "generate-example-data": {
            "inputs": ["{projectRoot}/src/examples/**/*", "excludingTestFiles"],
            "cache": true
        },
        "build-migrations": {
            "inputs": ["{projectRoot}/src/migrations/**/*", "{projectRoot}/tsconfig.migrations.json"],
            "cache": true
        },
        "npm-link": {
            "dependsOn": ["build"]
        },
        "e2e": {
            "cache": true,
            "inputs": ["default", "^production"]
        },
        "@nx/eslint:lint": {
            "inputs": ["default", "{workspaceRoot}/.eslintrc.json", "{workspaceRoot}/package.json", "^packageJson"],
            "cache": true
        },
        "@nx/angular:webpack-browser": {
            "dependsOn": ["^build"],
            "inputs": [
                "production",
                "^production",
                {
                    "env": "NX_MF_DEV_SERVER_STATIC_REMOTES"
                }
            ],
            "cache": true
        },
        "@nx/angular:package": {
            "cache": true,
            "dependsOn": ["^build"],
            "inputs": ["production", "^production"]
        },
        "@nx/vite:test": {
            "cache": true,
            "inputs": ["default", "^production"]
        },
        "@angular-devkit/build-angular:application": {
            "cache": true,
            "dependsOn": ["^build"],
            "inputs": ["production", "^production"]
        }
    },
    "namedInputs": {
        "default": ["{projectRoot}/**/*", "sharedGlobals"],
        "sharedGlobals": [],
        "production": [
            "default",
            "excludingTestFiles",
            "!{projectRoot}/.eslintrc.json",
            "!{projectRoot}/**/?(*.)+(spec|test).[jt]s?(x)?(.snap)",
            "!{projectRoot}/tsconfig.spec.json",
            "!{projectRoot}/scripts/**/*"
        ],
        "excludingTestFiles": [
            "!{projectRoot}/**/?(*.)+(spec|test|cy).[jt]s?(x)?(.snap)",
            "!{projectRoot}/tsconfig.spec.json",
            "!{projectRoot}/vite.config.[jt]s"
        ],
        "excludingPackageJson": ["!{projectRoot}/package.json"],
        "packageJson": ["{projectRoot}/package.json"]
    },