feat: add incremental mode support for Dialyzer (OTP 26+)

[Ch4s3]

PR description

Summary

This PR introduces support for Dialyzer’s incremental mode (OTP 26+) in Dialyxir, including the ability to run incremental analysis based on passed in OTP applications.

The goal is to make Dialyxir work naturally with Dialyzer’s newer incremental workflow while remaining backwards compatible with existing setups.

---

What is this for

Dialyzer’s incremental mode is designed around:

• long-lived, incremental PLTs
• app-based analysis using --apps / --warning_apps
• dependency-aware re-analysis

Dialyxir historically drives Dialyzer using classic PLTs and a whole app analysis. Incremental mode can significantly improve analysis of smaller changes on large code bases especially in CI environments.

---

What this PR adds

1. Incremental mode integration

Dialyxir can now pass incremental: true through to Dialyzer, enabling OTP 26+ incremental analysis.

2. Application-based analysis support

New support for Dialyzer’s:

• --apps
• --warning_apps

via Dialyxir configuration.

3. Higher-level configuration options

New symbolic configuration values:

dialyzer: [
  incremental: true,
  core_apps: [:erts, :kernel, :stdlib, :crypto, :public_key, :ssl, :elixir, :logger, :mix],
  apps: :transitive,
  warning_apps: :project
]

Supported behaviors:

• apps: :project → current project / umbrella apps
• apps: :transitive → core_apps ++ deps ++ project_apps
• warning_apps: :project → only project apps
• warning_apps: :all → all resolved apps

This allows large projects to opt into app-based incremental mode without manually maintaining long, fragile app lists.

---

Implementation overview

• Adds app-resolution logic in Dialyxir.Project to compute:

  • project apps (umbrella or single)
  • runtime dependency apps (via public Mix project config)
  • user-defined core apps (mostly things from OTP plus elixir itself)

• Introduces resolution helpers:

  • resolve_apps/1
  • resolve_warning_apps/2
  • resolve_app_args/2

• Updates the Mix task to:

  • resolve symbolic config into concrete app lists
  • automatically switch Dialyzer into app-mode when appropriate
  • drop :files when running app-mode to avoid mixed invocation types.

---

Backwards compatibility

This change is opt-in:

• If incremental mode is not flagged/configured, Dialyxir behaves exactly as before.
• Existing plt based Dialyzer workflows are unaffected.
• App-based incremental mode only activates when the new config/flag values are used.

---

How to use

Example umbrella setup:

dialyzer: [
  incremental: true,
  core_apps: [:erts, :kernel, :stdlib, :crypto, :public_key, :ssl, :elixir, :logger],
  apps: :transitive,
  warning_apps: :project
]

Run:

mix dialyzer --incremental

This enables Dialyzer’s native incremental mode

Preliminary results

Large codebase initial run

our_app
  Dialyzer exited with code 2
  Incremental mode enabled; skipping PLT check step
  Will use PLT file: ops/plts/dialyxir_erlang-28.1.1_elixir-1.19.3_deps-test_incremental.plt
  ignore_warnings: .dialyzer_ignore.exs
  Starting Dialyzer
  [
    analysis_type: :incremental,
    warning_apps: [:monitoring, :api, :another_app, :another_app_2,
     :another_app_3, :another_app_4, ...],
    ...
  ]
  Total errors: 68, Skipped: 65, Unnecessary Skips: 5
  done in 25m3.7s
  Warning: The created anonymous function has no local return.
  Warning: The function call update! will not succeed.
  Warning: Invalid type specification for function changeset.
  done (warnings were emitted)
  Halting VM with exit status 2
Error: Process completed with exit code 1.

Large app 2nd run with error fix

our_app
  Dialyzer exited with code 0
  Incremental mode enabled; skipping PLT check step
  Will use PLT file: ops/plts/dialyxir_erlang-28.1.1_elixir-1.19.3_deps-test_incremental.plt
  ignore_warnings: .dialyzer_ignore.exs
  Starting Dialyzer
  [
    analysis_type: :incremental,
    warning_apps: [:monitoring, :api, :another_app, :another_app_2,
     :another_app_3, :another_app_4, ...],
    ...
  ]
  Total errors: 65, Skipped: 65, Unnecessary Skips: 5
  done in 0m23.13s

You can see that the second run with a +1/-0 diff ran in 23.13s. This application has around 700k lines of elixir code.

Demonstration App

github
ci run

Refs: https://www.erlang.org/doc/apps/dialyzer/dialyzer.html#incremental-mode
Closes: #498

[Ch4s3]

I'm still working on the tests.

[Ch4s3]

This is a related issue to a test failure christhekeele/erlex#6

[oliver-kriska]

first of all thanks you are working in it. I think it's important to say in documentation/readme or somewhere that it use _build folder for cache. So when developers use common approach with cache they cache deps and _build folder before dialyzer step/job. But common practice is to use cache versioning based on mix lock file or something like that. In this case it would mean that dialyzer cache will be stored only in case deps are changed. So not big benefits for incremental feature. I think this is important to explain that developers have to check properly their cache logic what they have. I would suggest to introduce new cache which will directly cache folder where new file dialyzer files are stored. Because this new cache has to be updated everytime with some logic. For example you don't want to store new dialyzer's files from some branch same way as you store main branch to avoid usage branch's cache in main. But you want to store branch files due to multiple runnings of ci/cd if it's common for your development flow to have multiple runs of ci/cd per branch. So for example in GitHub Action you can use branch name in cache name with fallback from main branch, for runs from main only cache with main cache. Also I would suggest to use ci/cd runs number which should be incremental every run, because thanks to that it should store cache everytime when it runs, not only when key is changed. But this has to be checked because last time (a few months ago) when I checked GH Action Cache app it had bug/removed feature for flagging cache to be stored everytime when it runs, because we can assume that every run (often) should contains some code change. Is it know exact folder path for cached files?

[Ch4s3]

I would suggest to introduce new cache which will directly cache folder where new file dialyzer files are stored. Because this new cache has to be updated everytime with some logic. For example you don't want to store new dialyzer's files from some branch same way as you store main branch to avoid usage branch's cache in main. But you want to store branch files due to multiple runnings of ci/cd if it's common for your development flow to have multiple runs of ci/cd per branch. So for example in GitHub Action you can use branch name in cache name with fallback from main branch

I was thinking about this. Is you thought to just document this behavior or to add some mechanism to the library to better facilitate this caching approach?

Also I would suggest to use ci/cd runs number which should be incremental every run

this is on my todo list.

[oliver-kriska]

I think adding good documentation is base. Right now I don't see how library can give some tool or approach for it. Because it depends on own CI/CD configurations how people use cache. Something can be handled by library only in case library is able to set how to store these incremental files. Thanks library can partly allow to configure it for developers or use some common cache so developers will not have to configure new cache or something.

[Ch4s3]

@jeremyjh do you have any pointers for getting the tests to not time out in CI? They all work for me locally.

[jeremyjh]

@jeremyjh do you have any pointers for getting the tests to not time out in CI? They all work for me locally.

@Ch4s3 Thanks for working on this!

The problem must be in this branch; I reran master - it passed and test times haven't changed. #578

I don't immediately see the issue; there are tests timing out that aren't really doing much. I think I'd start by commenting all the new tests; if it passes then uncomment just the new output test.

[oliver-kriska]

from documentation:

--incremental - The analysis starts from an existing incremental PLT, or builds one from scratch if one does not exist, and runs the minimal amount of additional analysis to report all issues in the given set of apps. Notably, incremental PLT files are not compatible with "classic" PLT files, and vice versa. The initial incremental PLT will be updated unless an alternative output incremental PLT is given.

it means that file location is same but content is different. I tried your PR and when I use plt_local_path and plt_core_path it use same paths so basically no changes for CI/CD behavior I guess. Or do you have different experience? I think only important part is to remove them before running incremental, because these files are not compatibile with each other.
BTW: I can not confirm that it makes some speed up, it felt same with flag or without flag, with true or false in config.

BTW: You wrote you fixed Fix critical API usage bug in Dialyzer.Runner are you sure you put that change in this module in this PR?

[Ch4s3]

BTW: You wrote you fixed Fix critical API usage bug in Dialyzer.Runner are you sure you put that change in this module in this PR?

I think that’s leftover from an incremental step in some commits I squashed. I need to rewrite the PR description.

[Ch4s3]

it means that file location is same but content is different. I tried your PR and when I use plt_local_path and plt_core_path it use same paths so basically no changes for CI/CD behavior I guess. Or do you have different experience? I think only important part is to remove them before running incremental, because these files are not compatibile with each other.

Let me double check that I didn't have some environment config for erlang causing an issue here

BTW: I can not confirm that it makes some speed up, it felt same with flag or without flag, with true or false in config.

I can definitely confirm that on my production code base there's a speedup the measurement I included is from that app. I should clarify it is NOT faster to general the PLT and initial run, it is only faster for subsequent runs.

[Ch4s3]

@oliver-kriska once I have tests working, I'm going to test this in CI on my app and gather timing info as well as double checking the path information and I'll update docs here.

[Ch4s3]

I think I found the main problem

[Ch4s3]

I'm still working on this, I just had to take a break do to work/life business.

[Ch4s3]

I think the issue here was trying to test the system halt call on older OTP versions. Since the other code path that calls a halt isn't tested, I'm removing the test that verifies a halt on OTP < 26. Hopefully that's an acceptable tradeoff.

[Ch4s3]

I'm fiddling with how to correctly use this in CI. It seems like just mix dialyzer --incremental plus any formatting should be run without a discrete --plt step. It doesn't seem like OTP generates an incremental plt if you use both flags.

[michalmuskala]

Yes, with the new incremental mode you don't need a separate PLT step - an incremental run is both generating a new PLT and checking your code - internally it was always the same operation, this just explicitly combines them.

[Ch4s3]

@michalmuskala thanks for weighing in. Am I understanding correctly that incremental mode doesn't build the core language PLT so you need to also have that if you want to avoid getting errors from missing language functions?

Or should we be using --apps and --warning-apps to deal with this?

[TD5]

Am I understanding correctly that incremental mode doesn't build the core language PLT so you need to also have that if you want to avoid getting errors from missing language functions?
Or should we be using --apps and --warning-apps to deal with this?

Incremental mode analyses whatever modules you give it, including OTP modules (they don't get any special treatment). That means that you do want to use --apps and --warning-apps to make sure everything is included. For code that you don't own (OTP, libraries made by others), I'd suggest using --apps, so Dialyzer knows about them and can comment on their usage, but won't complain about them directly. Then use --warning-apps for code you actually want issues to be raised for.

Sadly, it's not quite as clean and orthogonal as it sounds, because if there is some sort of discrepancy in your --apps, you won't get a direct error reported for it, but your usages of that code from your --warning-apps might end up generating warnings due to your calls into broken code. As such, it's worth having that in the back of your mind when investigating Dialyzer warnings.

For the mental model for how to use --incremental mode, try to forget your existing knowledge of how Dialyzer works, and instead just list all the code you care about as either --apps or --warnings-apps (including OTP and libraries). You call the same command each time, asking Dialyzer to analyse your--apps and --warnings-apps. The incremental command is given a path to a PLT file, and it will worry about building, changing and updating the cache (PLT) file to minimise the work it needs to use between runs. You don't need to do anything explicitly to manage that lifecycle apart from making sure you point it to a consistent PLT file (and as someone pointed out above, perhaps you want to point to a different one per build configuration/branch if you're often moving between them).

[TD5]

Strictly, in incremental mode, Dialyzer analyses the modules that were changed since the PLT was last used (if there is one), plus the set of modules which Dialyzer thinks might depend on it. This perhaps sounds like a trivial difference, but if you have a core module that is used by a lot of your codebase, changing that will cause all the modules that depend on it to need to be re-analysed too (since you might have fixed/broken them with your change!).

In my experience, this leads to people asking: "I changed this one file but Dialyzer analyses hundreds of files - wasn't incrementality supposed to fix this?", with the answer being that incrementally tries to reduce re-analysing irrelevant files, but files can be relevant because they're in a dependency chain with something that has changed.

[Ch4s3]

This perhaps sounds like a trivial difference

No, this makes perfect sense. I was struggling with how to explain this well, perhaps even to myself.

but files can be relevant because they're in a dependency chain with something that has changed

Right, the classic problem that changing a config file causes a massive recompilation.

[TD5]

This allows you to analyze entire applications, which can be more efficient for large codebases.

Strictly, this is true, but can lead to misleading results if you include modules to be analysed with the modules they (transtively) use function or type definitions from. My suggestion would be to analyse all your code in a repo, including any dependencies it has, in one go. Incrementality itself will do the heavy lifting to soundly work out what code isn't affected by a change, avoiding the risk of the mistake above. Of course, at a certain scale you may really have no choice (e.g. if your codebase is so large the Dialyzer analysis for the entire codebase becomes too big to fit into RAM).

[TD5]

In my mind, this is the primary motivation for apps and warning_apps, rather than trying to only analyse a subset of a codebase (since that's so error prone).

[Ch4s3]

these are great comments, I think I can do a lot better with the readme now. thanks!

[TD5]

Hi there 👋 I made Dialyzer's incremental mode. I tried to add a bit of context where I could on this PR. I hope that helps.

[Ch4s3]

Am I understanding correctly that incremental mode doesn't build the core language PLT so you need to also have that if you want to avoid getting errors from missing language functions?
Or should we be using --apps and --warning-apps to deal with this?

Incremental mode analyses whatever modules you give it, including OTP modules (they don't get any special treatment). That means that you do want to use --apps and --warning-apps to make sure everything is included. For code that you don't own (OTP, libraries made by others), I'd suggest using --apps, so Dialyzer knows about them and can comment on their usage, but won't complain about them directly. Then use --warning-apps for code you actually want issues to be raised for.

This is what I'm figured out after some trial and error and the latest changes are an attempt to make passing those through as straightforward as possible. Thanks for further clarification.

Sadly, it's not quite as clean and orthogonal as it sounds, because if there is some sort of discrepancy in your --apps, you won't get a direct error reported for it, but your usages of that code from your --warning-apps might end up generating warnings due to your calls into broken code. As such, it's worth having that in the back of your mind when investigating Dialyzer warnings.

Makes sense.

For the mental model for how to use --incremental mode, try to forget your existing knowledge of how Dialyzer works, and instead just list all the code you care about as either --apps or --warnings-apps (including OTP and libraries). You call the same command each time, asking Dialyzer to analyse your--apps and --warnings-apps. The incremental command is given a path to a PLT file, and it will worry about building, changing and updating the cache (PLT) file to minimise the work it needs to use between runs. You don't need to do anything explicitly to manage that lifecycle apart from making sure you point it to a consistent PLT file (and as someone pointed out above, perhaps you want to point to a different one per build configuration/branch if you're often moving between them).

Awesome, this is such helpful context! I think I'm pretty close on this, I have the branch working against a production repo and giving me meaningful output. Thanks again, and also for your work on incremental mode.

[Ch4s3]

@TD5 could you weigh in again? I'm now just passing --apps and --warning-apps but not the files in incremental mode and I'm not seeing errors for code with trivial dialyzer errors picked up by classic mode.

Based on this test setup from OTP, and the original commit message I'm assuming --apps and --warning-apps have to overlap. I'm not sure how I missed this before.

[TD5]

Based on this test setup from OTP, and the original commit message I'm assuming --apps and --warning-apps have to overlap. I'm not sure how I missed this before.

That sounds plausible. Honestly, I wrote the code long enough ago that I can't remember the motivation there.

[Ch4s3]

That sounds plausible. Honestly, I wrote the code long enough ago that I can't remember the motivation there.

I know the feeling! Thanks for the input.

[Ch4s3]

I'm not crazy about this and may need to change it.

[Ch4s3]

this feels a bit defensive but I tripped myself up on this a few times so I added a check.

[Ch4s3]

@jeremyjh this should basically work now and I like the api, with the caveat that core_apps feels a bit hacky. I'm putting it down until after US Thanksgiving, but it should be in a reviewable state even though I plan to refactor a bit next week.

[jeremyjh]

@Ch4s3 I haven't really had a chance to dig into the details here. Will the new core_apps key only be used for incremental mode or would it apply to normal builds as well? If we want a separate apps list for incremental maybe we should namespace the options so thats more clear.

[Ch4s3]

That's the idea, but I really don't love it. I can definitely namespace it, but I'm not 100% sure yet that its the right options api for this problem. I just couldn't find a reliable way to programatically get these.

[jeremyjh]

Why do we need a different app list for incremental mode? Could we not use whatever the current options API resolves ?

[Ch4s3]

we could use :apps but I wanted to support apps: :transitive and couln't figure out a good way to automatically pick up things like :erts, and :elixir but I'm afraid it's too clever by half. I'm exploring other options.

[Ch4s3]

without :elixir in apps you end up with missing function warnings for thing like String.t().

[Ch4s3]

I have some new changes shaping up, and I'll publish a test app with this running in CI perhaps later this week.

[Ch4s3]

@jeremyjh I have a coworker sanity checking the code, and I'm doing a little doc cleanup and refactoring, but we've confirmed this works locally on our macs, in ci for our large app, and in my test repo on GHA.

[notactuallytreyanastasio]

We ran into a REALLY annoying issue that was coming from default low ulimits on Mac OS. I made a PR to @Ch4s3's branch that, hopefully, will make it so no one else loses hours on this wondering why it passes on CI but not on some random Mac lol

Ch4s3#1

[Ch4s3]

We ran into a REALLY annoying issue that was coming from default low ulimits on Mac OS. I made a PR to @Ch4s3's branch that, hopefully, will make it so no one else loses hours on this wondering why it passes on CI but not on some random Mac lol

Ch4s3#1

I think this may be best to handle in the readme rather than in code.

[Ch4s3]

@jeremyjh I think this is reviewable now. Below is a sketch of how to read through the changes or as best I could reason about how to approach it.

1) Start with the intent and surface changes

• Changelog: CHANGELOG.md — confirms incremental mode feature addition and headline behavior.
• README incremental section: README.md — updated guidance for apps/warning_apps, incremental usage, caching, and macOS ulimit tip.

2) Core entry points (behavioral changes)

• Mix task front door: lib/mix/tasks/dialyzer.ex
  • CLI parsing and incremental gating: R226.
  • Selection assembly and resolver wiring: R252.
  • Argument assembly for Dialyzer: R319 and dialyzer_args/4 around R488.
• Dialyzer runner: lib/dialyxir/dialyzer.ex
  • Incremental analysis toggle and formatter parsing: R12.

3) Centralized app/warning_app resolution

• New resolver module: lib/dialyxir/app_selection.ex
  • Single place that merges CLI + config, expands flags, normalizes atom/string/charlist, filters non-project warning_apps with warnings, and merges warning_apps into apps.
  • Side-effect scope: only emits warnings via Dialyxir.Output.
• Project-level helpers: lib/dialyxir/project.ex
  • resolve_apps/1 and resolve_warning_apps/1 unify config resolution; see R307 and R337.
  • fallback_list/3 and resolve_list_value/2 handle list configs and backward compatibility; see R354.
  • expand_transitive_apps/1 extracts :transitive expansion to avoid duplication; see R375.
  • Semantics: apps expands :transitive (deps + project) but still requires explicit OTP apps; warning_apps lists are project-only, merged into apps later; nil → empty list for backward compatibility.

4) Incremental PLT handling

• PLT file handling: lib/dialyxir/plt.ex
  • Confirm file enumeration and incremental PLT handling are compatible with incremental analysis (no regressions in classic PLTs): see R244 for classic vs incremental PLT file handling.

5) Docs and CI caching

• CI examples: docs/github_actions.md, docs/gitlab_ci.md, docs/circleci.md
  • Clarified cache keys (OTP/Elixir/mix.lock, optional branch keys) and guidance that incremental artifacts live in priv/plts.
• MacOS ulimit note: README.md
  • EMFILE avoidance tip for large codebases during MD5 hashing.

6) Tests (behavioural coverage)

• Resolver unit tests: test/dialyxir/app_selection_test.exs
  • Covers incremental gating, CLI overrides, filtering, and merging.
• Project helpers: test/dialyxir/project_test.exs
  • Expanded scenarios for apps/warning_apps resolution and :transitive expectations.
• Task integration: test/mix/tasks/dialyzer_test.exs
  • Ensures flags flow into dialyzer args correctly, including incremental/app-mode paths.
• Dialyzer runner: test/dialyxir/dialyzer_test.exs
  • Confirms runner behavior in incremental contexts.

7) Fixtures and samples

• Incremental/app-mode fixtures: under test/fixtures/* (e.g., apps_transitive, warning_apps_project, incremental_*)
  • Check they mirror the intended semantics (OTP apps explicit, warning_apps limited to project).

8) Thing to look out for

• CLI vs config precedence in AppSelection: CLI should win; warning_apps filtered to project apps only; apps expanded for :transitive.
• Incremental vs classic: incremental should skip classic PLT build when --incremental; classic flow unchanged when not incremental.
• Caching guidance: priv/plts is the cache root for both classic and incremental artifacts; keys tied to OTP/Elixir/mix.lock (optional branch suffix).
• ulimit/EMFILE: documented; no code changes, just awareness for large repos.

[jeremyjh]

Both :transitive and :project modes are deprecated in favor of :app_tree (the default, equivalent to mix app.tree) and :apps_direct (direct app dependencies). In most cases we only want to include only apps that would be deployed in the application, and not other project dependencies.

#223

[Ch4s3]

not sure how I missed this, I'll get it addressed

[Ch4s3]

I think in most cases we want to add some_otp_stuff ++ [:app_tree] to :apps

[Ch4s3]

my one concern is that :apps_direct includes deps and for the way :warning_apps is used in incremental mode you need just a list of your own apps and not the deps.

I really need to either have another flag here or to just require users to declare their app/apps. It's mostly only an issue for umbrella apps though.

[Ch4s3]

I've added a new key, :apps_project that uses Mix.Project.apps_paths() (if defined) to get top level project applications. It only works in :warning_apps and is documented as such.

[jeremyjh]

If this key is only meaningful for incremental mode we need to indicate that in the option name, or nest it e.g. incremental: [apps: [...]].

Where possible - where the semantics are the same - we should use the same config names as https://github.com/jeremyjh/dialyxir?tab=readme-ov-file#dependencies - and let them optionally nested to override incremental mode. This will make adoption easier for people.

[Ch4s3]

I think I'll try nesting it. I looked at using plt_add_deps and plt_add_app but the semantics were a bit different and it made the existing code really hard to follow.

[Ch4s3]

@jeremyjh there are some sort of tricky tradeoffs to consider here. The existing code for resolving dependencies is built around the way classic PLTs work and assumes that things like :erts, :elixir, :kernel, and so on are already in the PLT. This means that trying to add those items in automatically through the old code causes runtime warnings about loading deps like the following:

Error loading sasl, dependency list may be incomplete.
 {~c"no such file or directory", ~c"sasl.app"}

I can solve that by resolving deps in a less efficient way in a new code path, or by requiring the end user to declare them explicitly in apps: [...]. Any other approach would have to cut into a lot of older code in a way that feels dicey in terms of broad ecosystem support (to me at least).

Update

I think this is ready for re-review. My compromise here is that users have to declare core language dependencies like [:erts, :kernel, :stdlib, :elixir, :logger]. I don't love it but trying to make it work automatically with existing dialixyr code using an existing flag would require a lot of changes to dependency resolution. You can see it in use here

[jeremyjh]

@Ch4s3 is there an issue with always adding [:erts, :kernel, :stdlib, :crypto, :elixir] to the app list when :app_tree or :apps_direct are used? If a user doesn't want some of those, they could just provide a complete list of the apps they want instead of using either of those methods. I'm not sure that is an edge case that even exists.

[Ch4s3]

@Ch4s3 is there an issue with always adding [:erts, :kernel, :stdlib, :crypto, :elixir] to the app list when :app_tree or :apps_direct are used? If a user doesn't want some of those, they could just provide a complete list of the apps they want instead of using either of those methods. I'm not sure that is an edge case that even exists.

@jeremyjh if the hard coded value is good with you I can absolutely add it to app_tree. I was just trying to avoid hard coding if I could.

[jeremyjh]

@jeremyjh if the hard coded value is good with you I can absolutely add it to app_tree. I was just trying to avoid hard coding if I could.

It is already hard coded, those are the default apps that go into the Erlang/Elixir PLTs and are included in the project one.

[Ch4s3]

@jeremyjh I assumed since it was deprecated that I should do something different. I'll add it into my code path.

If you're you ok using the new apps_project flag for the warning apps then it's read for review.