Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kotlin Web (Ktor & KotlinJS) examples (1650USD Bounty) #3611

Closed
5 of 6 tasks
lihaoyi opened this issue Sep 26, 2024 · 12 comments
Closed
5 of 6 tasks

Add Kotlin Web (Ktor & KotlinJS) examples (1650USD Bounty) #3611

lihaoyi opened this issue Sep 26, 2024 · 12 comments
Labels
Milestone

Comments

@lihaoyi
Copy link
Member

lihaoyi commented Sep 26, 2024


From the maintainer Li Haoyi: I'm putting a 1650USD bounty on this issue, payable by bank transfer on a merged PR implementing this.


The goal of this ticket is to implement support for example/kotlinlib/web example tests for using Kotlin for web development, mirroring the example/javalib/web and example/scalalib/web, and support for KotlinJS in the spirit of scalajslib/. I would want the following example tests, broken down into smaller groups each with its own bounty:

650USD:

  • example/kotlinlib/web/1-hello-ktor
    • A hello-world example using Kotlin and KTor to set up a web server serving a single "<h1>Hello, World!</h1>" web page, in the spirit of example/javalib/web/2-hello-spring-boot
  • example/kotlinlib/web/2-todo-ktor
    • An implementation of the common TodoMVC demo application, including HTML templating, JS/CSS resources, database access and unit/integration/runBackground+curl tests, in the spirit of example/javalib/web/3-todo-spring-boot

650USD:

  • example/kotlinlib/web/3-kotlinjs-module (325USD)
    • How to define a simple KotlinJS module that can be compiled to Javascript, testing, and run using node, in the spirit of example/scalalib/web/3-scalajs-module
  • example/kotlinlib/web/4-webapp-kotlinjs (325USD)
    • How to wire up a KotlinJS module with a Kotlin/Ktor server to serve the generated Javascript, in the spirit of example/scalalib/web/4-webapp-scalajs

700USD:

  • example/kotlinlib/web/5-webapp-kotlinjs-shared (350USD)
    • How to implement code sharing between a JVM KTor server and a KotlinJS javascript module, such that common code and utilities can be used by both. In the spirit of example/scalalib/web/5-webapp-scalajs-shared,
  • example/kotlinlib/web/6-cross-platform-publishing (350USD)
    • How to define a library with both Kotlin-JVM and Kotlin-JS versions and publishing it to maven central. In the spirit of example/scalalib/web/6-cross-version-platform-publishing
    • Moving this to a separate bounty

Each of these examples above has a corresponding javalib/scalalib equivalent, and it should generally follow the style of those equivalents (scope, testing, etc.) but translated to Kotlin (Java/Scala -> Kotlin, Scalajs -> Kotlinjs, Cask/SpringBoot -> KTor, etc.)

For now the Kotlin-JS logic should live in kotlinlib/ together with the rest of the kotlin-specific configuration.

@lihaoyi lihaoyi changed the title Add Kotlin Web (Ktor & KotlinJS) examples (1500USD Bounty) Add Kotlin Web (Ktor & KotlinJS) examples (2000USD Bounty) Sep 26, 2024
@lihaoyi lihaoyi added the bounty label Sep 26, 2024
lihaoyi pushed a commit that referenced this issue Sep 28, 2024
Part of #3611

Working in the TodoMVC demo application. I'll raise a follow up PR for
that.
@c0d33ngr
Copy link

Let me try if I can do anyone

@c0d33ngr
Copy link

c0d33ngr commented Sep 29, 2024

Hello, I'm new here.
How do I run the tests in the examples directory? mill testin where build.mill file is located output

build.sc file not found. Are you in a Mill project folder?

@lihaoyi
Copy link
Member Author

lihaoyi commented Sep 29, 2024

@c0d33ngr please see the readme.md at the root of the repo

@0xnm
Copy link
Contributor

0xnm commented Sep 30, 2024

I did a quick overview of the things which should be done here, and it seems that the work can be sliced in a bit more granular way.

The main complexity here comes actually from implementing Kotlin/JS support (similar to the Scala/JS) which is itself a quite big task, because:

  • There are 2 modes here: only module content output (may be the case for libraries) or a complete output with Kotlin runtime and other libraries included (basically output with linking applied) - this is the case for the executable output.
  • What about the NPM modules support (Kotlin/JS supports it)? I don't think it should come in the initial version, but it is a quite handy feature.
  • Different module kind support (should be just a configuration flag, I hope, though).

So I think maybe implementing Kotlin/JS support is worth to be extracted into a different task? Making examples is easier than that.

Also this raises a design question: say there is a code which should be compiled to both JVM and JS targets. How this could be done with existing Scala modules, for example? I believe ScalaModule and ScalaJSModule cannot be declared for the same module at the same time, because of the conflicting declarations (ScalaJSModule extends ScalaModule).

@lihaoyi
Copy link
Member Author

lihaoyi commented Oct 1, 2024

@0xnm as written, the KotlinJS stuff (bullets 3-6) is already in separate milestones from the pure Ktor/web stuff (bullets 1-2). Bullets 3 and 6 seem pretty independent of Ktor and should be doable without any pre-requisites, and bullets 1-2 should be independent of KotlinJS. I don't mind adding more non-KotlinJS web stuff, just not sure what that would entail.

In Scala-land, cross-platform Scala code is normally handled via PlatformScalaModule, which defines multiple sub-modules for .jvm .js (and maybe .native) that share the same code. Thus the code ends up being compiled multiple times by the different versions of the Scala compiler. KotlinJS/KotlinJVM could use the same approach, or if they have any other techniques for cross-platform code we could use their thing as well.

I'm hoping the KotlinJS tooling already has a decent API/CLI, and most of the work would just be wiring it into Mill. I'm not sure how true that actually is, but as a point of reference Mill's ScalaJS support is 14 files and 1600 lines of code with another 1000 lines of tests for basically 100% coverage of ScalaJS' functionality. I'm hoping KotlinJS won't be that much more work, but I don't know

The bounty amounts are pretty arbitrary TBH and are just my best guess w.r.t. the amount of work necessary. For the KTor stuff they should be pretty reasonable, since those bullets shouldn't require any special tooling background. But if we think the KotlinJS bounties aren't proportionate to the amount of work necessary, I can totally bump them up, or if someone wants a different arrangement (e.g. some payment up front to de-risk the effort )

@0xnm
Copy link
Contributor

0xnm commented Oct 5, 2024

I added a very basic PR with a foundation for Kotlin/JS support in #3678, but there are many issues/missing functionality. See that PR for more details.

lihaoyi added a commit that referenced this issue Oct 7, 2024
Another part for #3611.

This PR adds a very basic foundation for the Kotlin/JS support. The code
provided allows to:

* Build code with `org.jetbrains` (most likely) dependencies only
* Run it with Node (no browser support)
* Execute tests (using `kotlinx-test` so far), but without any test
results collection/tests selector, etc.

However, I think that full Kotlin/JS support **will take a lot of
time**, because even if there is a Scala.JS counterpart available, the
way these technologies work is very different (distribution, format,
etc.)

Issues I've encountered:

* Kotlin 2+ is not supported, because with Kotlin 2+ jars are not
published anymore and there is only `klib` file available (see
https://repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-js/2.0.0/).
When Gradle is used, it is able to fetch it using attributes declared in
`.module` file, but Coursier is not able to recognize and fetch it.
* Kotlin versions below 1.8.20 are not supported, because of the
different set of compiler arguments. With the certain effort it is
possible to go further in supporting older versions, but I'm not sure if
it is needed: since Mill is somewhat experimental, probably there is no
need for the users to use old Kotlin versions.
* Even if some Kotlin/JS library has `jar` file published with Kotlin/JS
library inside, it may be rejected by the compiler. For example, this
https://repo1.maven.org/maven2/org/jetbrains/kotlinx/kotlinx-html-js/0.8.0/
has the necessary `jar` file with `.meta.js` / `.kjsm` files inside, but
it is rejected by the compiler of Kotlin 1.8/1.9. Here
https://repo1.maven.org/maven2/org/jetbrains/kotlin/kotlin-stdlib-js/1.9.24/,
for example, nothing is rejected, so I suppose there is an issue in the
ABI/metadata version. Not sure how it can be solved (maybe by relying
only on `klib`? But `klib` cannot be fetched by Coursier).
* Gradle Kotlin plugin is utilizing NPM dependencies to generate
`package.json` and add the necessary JS test frameworks/runners there if
executed in Node environment, or even webpack for Browser environment.
This is also a big chunk of work to be done. For now I've added only
test binary execution, but there is no test results collection / test
selector.
* Kotest cannot be used, because with version 5 only `klib` is
published, and `jar` of version 4 is not compatible with the 1.8/1.9 IR
compiler.
* ~~Kotlin/JS IR compiler has different modes: it can output either
IR/Klib or can produce final JS (basically IR+linking). Kotlin Gradle
plugin is using 2 passes: to generate IR and then produce final JS. I
guess it is done for the better performance / better incremental
support, but I, for the initial drop, rely on a single pass (IR+linking
in a single compiler invocation) => **need to make `compile` task to
produce only IR code and add kind of `link` task to produce executable
in the future.**~~ => addressed in the 2nd commit.

---------

Co-authored-by: 0xnm <[email protected]>
Co-authored-by: Li Haoyi <[email protected]>
@lihaoyi
Copy link
Member Author

lihaoyi commented Oct 9, 2024

First 3 bullets (mostly) work. Further progress is blocked by #3695

@lihaoyi
Copy link
Member Author

lihaoyi commented Oct 11, 2024

@0xnm Now that we can resolve klib files, the subsequent two bullets should be doable if you're up for taking a crack at them, since the only thing blocking them before was accessing third party library for DOM interactions. The other limitations around module file support and test framework setup are issues but not blockers, at least for demo purposes. Using the same approach as the scala examples using PlatformScalaModule should work i think

The last bullet may require more work, I'm not sure how much work the gradle kotlin/kmp plugin does to support publishing. We can investigate when we get to that

lihaoyi added a commit that referenced this issue Oct 14, 2024
Point 4 in #3611: add a sample of Kotlin/JS frontend+ Ktor backend.

---------

Co-authored-by: 0xnm <[email protected]>
Co-authored-by: Li Haoyi <[email protected]>
lihaoyi added a commit that referenced this issue Oct 14, 2024
This PR addresses task 5 from #3611 by adding and example of code
sharing between JVM and JS Kotlin targets.

I also simply copied `PlatformScalaModule` -> `PlatformKotlinModule`,
because type aliasing won't work: doc is different (and need to extend
from `KotlinModule` still).

---------

Co-authored-by: 0xnm <[email protected]>
Co-authored-by: Li Haoyi <[email protected]>
@lihaoyi
Copy link
Member Author

lihaoyi commented Oct 14, 2024

Thanks @0xnm, I merged the two kotlinjs/webapp examples with some minor fixes and closed out the bounty for 4-webapp-kotlinjs.

@lihaoyi
Copy link
Member Author

lihaoyi commented Oct 14, 2024

For example/kotlinlib/web/6-cross-platform-publishing, I think this would require code changes (in PlatformKotlinModule to properly generate the module files and upload them. We also should add support in Coursier for resolving those, but I think that can be done separately.

@0xnm
Copy link
Contributor

0xnm commented Oct 14, 2024

Yes, I would suggest to split bounty for bullets 5/6 and maybe extract bullet 6 into a new issue (to have a clean space free of previous discussions), because it will definitely require some exploration work to be done and it won't contain only sample migration.

@lihaoyi
Copy link
Member Author

lihaoyi commented Oct 14, 2024

@0xnm sounds good. I'll split it out into a separate bounty and pay out the first bullet first

@lihaoyi lihaoyi changed the title Add Kotlin Web (Ktor & KotlinJS) examples (2000USD Bounty) Add Kotlin Web (Ktor & KotlinJS) examples (1650USD Bounty) Oct 29, 2024
@lihaoyi lihaoyi closed this as completed Oct 29, 2024
@lefou lefou added this to the 0.12.2 milestone Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants