Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Why Mill? #3880

Merged
merged 15 commits into from
Oct 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions docs/modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Or if you prefer a video introduction:

Mill is used to build many real-world projects, such as the
https://github.com/swaldman/c3p0[C3P0 JDBC Connection Pool],
https://github.com/coursier/coursier[Coursier JVM dependency resolver],
https://github.com/coursier/coursier[Coursier dependency resolver],
https://github.com/com-lihaoyi/Ammonite[Ammonite REPL], and the
https://github.com/SpinalHDL/SpinalHDL[SpinalHDL] and
https://github.com/chipsalliance/chisel[Chisel] hardware design frameworks.
Expand Down Expand Up @@ -124,4 +124,10 @@ they are interesting:
To engage Mill with the community, you can use the channels below:

* https://github.com/com-lihaoyi/mill/discussions[Mill Github Discussions]
* https://discord.com/invite/scala[Scala Discord], in the TOOLING#mill channel
* https://discord.com/invite/scala[Scala Discord], in the TOOLING#mill channel

Mill maintains an open list of issue bounties below, with payouts ranging from
500USD to 3000USD per issue. Feel free to look through the list and try your
hand at any bounties that may interest you

* https://github.com/orgs/com-lihaoyi/discussions/6[Open Issue Bounties]
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/javalib/intro.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
// what Mill is, hopefully have downloaded an example to try out, and be
// interested in learning more about the Mill build tool

= Building Java Projects with Mill
= Building Java with Mill
:page-aliases: Intro_to_Mill.adoc, Intro_to_Mill_for_Java.adoc, Java_Intro_to_Mill.adoc

include::partial$gtag-config.adoc[]
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/scalalib/intro.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
// what Mill is, hopefully have downloaded an example to try out, and be
// interested in learning more about the Mill build tool

= Building Scala Projects with Mill
= Building Scala with Mill
:page-aliases: Intro_to_Mill_for_Scala.adoc, Scala_Intro_to_Mill.adoc

include::partial$gtag-config.adoc[]
Expand Down
141 changes: 79 additions & 62 deletions docs/modules/ROOT/pages/why-mill.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -42,19 +42,16 @@ for the respective build tool go into more detail (for xref:comparisons/maven.ad
xref:comparisons/gradle.adoc[Gradle], and xref:comparisons/sbt.adoc[SBT]), but at a high level
these advantages are:

1. Performance
* *Mill offers a 2-10x speedup means less time waiting for your build tool*, meaning
less time waiting for your build and more time doing useful work
1. *Performance*: Mill offers a 2-10x speedup means less time waiting for your build tool,
meaning less time waiting for your build and more time doing useful work

2. Ease of Use
* *Mill offers better IDE support in IntelliJ and VSCode and rich visualization tools*,
to help understand your build and what it is doing
2. *Ease of Use*: Mill has better IDE support in IntelliJ and VSCode and richer
visualization tools that other tools, to help understand your build and what it is doing

3. Extensibility
* *Directly write code or use any Java library from Maven Central in your build*,
without being beholden to third party plugins that may not exist or be well maintained.
3. *Extensibility*: Mill lets you write code or use any published JVM library in your build,
customizing it to your needs without being limited by third-party plugins

We will discuss each one in turn:
We will discuss each one in turn.

## Performance

Expand Down Expand Up @@ -90,8 +87,11 @@ For the purposes of this page, we will focus on two benchmarks


This benchmark involves running `clean` to delete all generated files and re-compiling
everything in parallel. For Maven, parallelism is opt-in via `-T 10`, while for Mill it
is enabled by default.
everything in parallel.

* For Maven, parallelism is opt-in via `-T 10`, while for Mill it is enabled by default.
* For Maven, tests and linters are opt-out via the `-D` properties, while in Mill
tests an linters are opt-in

Mill sees a significant ~8x speedup over Maven for this benchmark.

Expand All @@ -117,16 +117,17 @@ compiling 1 Java source to /Users/lihaoyi/Github/netty/out/common/compile.dest/c
This benchmark involves making a single edit to a single already-compiled file in `common` -
adding a single newline to the end of the file - and re-compiling `common` and `common.test`.

Mill sees a huge ~39x speedup for this benchmark, because Mill's incremental compiler (Zinc)
is able to detect that only one file in one module has changed, and the change is small enough
Mill sees a huge ~39x speedup for this benchmark, because Mill's incremental compiler
(https://github.com/sbt/zinc[Zinc]) is able to detect that only one file in one module
has changed, and that the change is small enough
to not require other files to re-compile. In contrast, Maven re-compiles all files in both
modules, even though only one file was touched and the change was trivial.


### Gradle

The comparison with Gradle is less stark, but still significant. Mill is 3-4x faster than Gradle
for clean compiles, both parallel and sequential, and for many modules or for a single module:
across the various workflows:


|===
Expand Down Expand Up @@ -179,7 +180,8 @@ root module - adding a single newline to the end of the file - and re-compiling
its tests.

Both Gradle and Mill appear to do a good job limiting the compilation to only the changed
file, but Mill has less fixed overhead than Gradle does.
file, but Mill has less fixed overhead than Gradle does, finishing in about ~0.5s
rather than ~1.5 seconds.


## Ease of Use
Expand All @@ -205,25 +207,27 @@ The Chrome profiling UI is interactive, so you can zoom in and out, or click on
tasks to show the exact duration and other metadata.

But the real benefit of the Chrome profile isn't the low-level data it provides, but the
high-level view: in the profile above, it is clear that for the first ~700ms, Mill is able
to use all cores on 10 cores on my laptop to do useful work. But after that, utilization is
much more sparse: `common.compile`, `buffer.compile`, `transport.compile`, `codec.compile`,
appear to be "long polls" in the build. These 4 tasks wait for one another to run sequentially,
likely due to dependencies between them, and they take long enough that all the other tasks
depending on them get held up. For example, when `codec.compile` finishes above, we can see
a number of downstream tasks immediately start running.
high-level view:

* In the profile above, it is clear that for the first ~700ms, Mill is able
to use all cores on 10 cores on my laptop to do useful work.
* But after that, utilization is
much more sparse: `common.compile`, `buffer.compile`, `transport.compile`, `codec.compile`,
appear to wait for one another and run sequentially one after another.


This waiting is likely due to dependencies between them, and they take long enough that all
the other tasks depending on them get held up. For example, when `codec.compile` finishes
above, we can see a number of downstream tasks immediately start running.

This understanding of your build's performance profile is not just an academic exercise, but
provides actionable information:

* If I wanted to speed up the Netty clean compile timings,
speeding up `common.compile`, `buffer.compile`, `transport.compile`, or `codec.compile` would
make the most impact: perhaps some of these modules could be broken up into smaller sub-modules
that would compile in parallel!
* If I wanted faster Netty clean compiles, speeding up `common.compile`, `buffer.compile`,
`transport.compile`, or `codec.compile` would make the most impact.

* On the other hand, time speeding up the various `codec-*.compile` tasks would help not at all:
these tasks are already running at a time where the CPUs are mostly idle, and speeding them up
would just result in more idle time while waiting for the the "long poll" tasks to complete
these tasks are already running at a time where the CPUs are mostly idle.

Most build tools do provide some way of analyzing build performance, but none of them provide
it as easily as Mill does: any Mill run generates a profile automatically, and any computer
Expand All @@ -240,6 +244,8 @@ show the dependency graph between the modules of the Netty build below:

image::comparisons/NettyCompileGraph.svg[]

(_Right-click open-image-in-new-tab to see full size_)

In this graph, we can clearly see that `common.compile`, `buffer.compile`,`transport.compile`,
and `codec.compile` depend on each other in a linear fashion. This explains why they each must
wait for the prior task to complete before starting, and cannot run in parallel with one another.
Expand Down Expand Up @@ -303,21 +309,19 @@ image::comparisons/IntellijMockitoMillCompile.png[]
From there, if you are curious about any of the other tasks used alongside `javacOptions`, it's
easy for you to pull up _their_ documentation, jump to _their_
definition, or find _their_ usages. For example we can pull up the docs of
`compileClasspath()` below:

image::comparisons/IntellijMockitoMillCompileClasspath.png[]

Or we can use _find usages_ on `def compile` to see where it is used, both in this build
and upstream in the Mill libraries:
`compileClasspath()` below, jump to _its_ implementation, and continue
interactively exploring your build logic:

image::comparisons/IntellijMockitoMillCompileUsages.png[]
image::comparisons/IntellijMockitoMillCompileClasspath.png[]

Unlike most other build tools, Mill builds are extremely easy to explore interactively in your
Unlike most other build tools, Mill builds can be explored interactively in your
IDE. If you do not know what something does, it's documentation, definition, or usages is always
one click away in IntelliJ or VSCode. That's not to say Mill builds aren't complex - as
we saw above, compilation has to deal with upstream outputs, classpaths, flags, reporters, and so on -
but at least in Mill your IDE can help you explore, understand and manage the complexity in a way
that no other build tool supports.
one click away in IntelliJ or VSCode. This isn't a new experience for Java developers, as it
is what you would be used to day-to-day in your application code! But Mill brings that same
polished experience to your build system - traditionally some that that has been opaque
and hard to understand - and does so in a way that no other build tool does.


## Extensibility

Expand All @@ -327,15 +331,16 @@ from Maven Central.
Most build tools need plugins to do anything: if you want to Foo you need a
Foo plugin, if you want to Bar you need a Bar plugin, for any possible Foo or Bar. These could
be simple tasks - zipping up files, pre-rendering web templates, preparing static assets for
deployment - but even tasks that would be trivial to implement in a few lines of code need you
to google for third-party plugins, dig through their Github to see which one is best maintained,
and hope for the best when you include it in your build.
deployment - but even a tasks that would be trivial to implement in a few lines of code requires
you to Google for third-party plugins, dig through their Github to see which one is best
maintained, and hope for the best when you include it in your build. And while you could
write plugins yourself, doing so is usually challenging and non-trivial.

Mill is different. Although it does have plugins for more advanced integrations, for most
simple things you can directly write code to achieve what you want, using the bundled
filesystem, subprocess, and dependency-management libraries. And even if you need third-party
libraries from Maven Central, you can directly import the "Foo" library and use it programmatically,
without having to find a "Foo build plugin" wrapper.
libraries from Maven Central to do Foo, you can directly import the "Foo" library and use it
directly, without having to find a "Foo build plugin" wrapper.

### Simple Custom Tasks

Expand All @@ -356,7 +361,7 @@ Compiling 1 Java source...
```

If you want to add a custom task, this is as simple as defining a method e.g.
`def lineCount = Task`. The body of which performs the action we want, and
`def lineCount = Task { ... }`. The body of `Task` performs the action we want, and
can depend on other tasks such as `allSourceFiles()` below:

```scala
Expand Down Expand Up @@ -422,8 +427,13 @@ Line Count: 17
Most developers do not need to embed the line-count of their codebase in a resource
file to look up at runtime, but nevertheless this example shows how easy it is to write
code to perform ad-hoc tasks without needing to pull in and configure some third-party
plugin. Next, we'll look at a more realistic example, which includes usage of third-party
libraries in the build.
plugin. And we get full IDE support with autocomplete/navigation/documentation/etc.
while we are writing our custom task.


While most build tools do allow writing and wiring up custom tasks, none of them
have a workflow as simple as Mill. Next, we'll look at a more realistic example,
which includes usage of third-party libraries in the build.

### Using Libraries from Maven Central in Tasks

Expand Down Expand Up @@ -462,11 +472,13 @@ object foo extends JavaModule {
}
```

Once we have run `import $ivy`, we can `import org.thymeleaf.TemplateEngine`,
`import org.thymeleaf.context.Context`, and replace our `def lineCount` with a `def htmlSnippet`
task that instead of counting lines uses Thymeleaf to render some HTML. Again, it is inspectable
from the Mill command line via `show`, and we wire it up into `def resources` so it can be
inspected and used (in this case just printed out) at runtime by the application:
Once we have run `import $ivy`, we can import `TemplateEngine`, `Context`, and replace our
`def lineCount` with a `def htmlSnippet` task that uses Thymeleaf to render HTML. Again,
we get full IDE support for working with the Thymeleaf Java API, the new `htmlSnippet` task
is inspectable from the Mill command line via `show`, and we wire it up into
`def resources` so it can be inspected and used at runtime by the application
(in this case just printed out):


```bash
> mill show foo.htmlSnippet
Expand All @@ -480,13 +492,13 @@ compiling 1 Java source...
generated snippet.txt resource: <h1>hello</h1>
```

Rendering HTML using the Thymelead templating engine is not rocket science, but what is
Rendering HTML using the Thymeleaf templating engine is not rocket science, but what is
interesting here is what we did _not_ need to do:

* We did not need to find a Thymeleaf-Mill plugin in order to include Thymeleaf in our
* We did _not_ need to find a Thymeleaf-Mill plugin in order to include Thymeleaf in our
build

* We did not need to learn a special API or framework for authoring build plugins ourselves
* We did _not_ need to learn a special API or framework for authoring build plugins ourselves
to write a plugin to include Thymeleaf in our build

Instead, we could simply import Thymeleaf directly from Maven Central and use it just
Expand All @@ -497,14 +509,19 @@ non-experts to configure their build to do exactly what they need, rather than b
limited by what some unmaintained third-party plugin might support. And although
the configuration is done in the Scala language, the syntax should be relatively
familiar ("Java without Semicolons" it is sometimes called) and the JVM libraries
and tools (e.g. IntelliJ, VSCode) are the exact same libraries and tools you are
already used to.
and tools (e.g. Thymeleaf, IntelliJ, VSCode) are the exact same libraries and tools
you are already used to.

## Conclusion

To wrap up, Mill does all the same things that other build tools like Maven or Gradle do,
but aims to do them better: faster, easier to user, and easier to extend. All together,
this means both time saved waiting for your build tool to run in day-to-day work, as well
but aims to do them better: faster, easier to use, and easier to extend.
This means both time saved waiting for your build tool to run in day-to-day work, as well
as time saved when you inevitably need to evolve or adjust your build system to accommodate
changing requirements. With Mill, you can therefore spend less time waiting for or fighting
with your build tool, and more time on the actual work you are trying to accomplish.
changing requirements.

With Mill, you can therefore spend less time waiting for or fighting with your build tool,
and more time on the actual work you are trying to accomplish. And while traditionally build
systems were often mysterious black boxes that only experts could work with, Mill's ease
of use and ease of extension democratize the build system so any developer can figure out
what its doing or extend it to do exactly what they need.
1 change: 1 addition & 0 deletions docs/package.mill
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ object `package` extends RootModule {
Jvm.runSubprocess(
commandArgs = Seq(
npmExe,
"--no-audit",
"install",
"@antora/[email protected]",
"@antora/[email protected]",
Expand Down
2 changes: 1 addition & 1 deletion docs/supplemental-ui/partials/nav-tree.hbs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This lets us open up the first-level tree nodes by default, letting us use tree
nodes to organize things while keeping them open and easily browsable/discoverable.
We leave the first 3 items (0, 1, 2) to only open on demand, since the
`Build $LANGUAGE Projects with Mill` entries are pretty repetitive and we do not need to
`Build $LANGUAGE with Mill` entries are pretty repetitive and we do not need to
show all of them all the time
}}
{{#if navigation.length}}
Expand Down
Loading