Skip to content

Conversation

@KristofferC
Copy link
Member

@KristofferC KristofferC commented Nov 5, 2025

The Pkg build functionality was made for a different era in Julia package management. It gets run whenever a package is downloaded (which is a weird thing to trigger on where packages are considered immutable and multiple environments share the same package downloads), it requires the package to recompile when build options are changed etc. There are now better options based on e.g. scratch spaces.

Instead of using Pkg.build to install the default kernel, this instead auto installs it if needed.
It provides a separate function to update the jupyter path because tying kernel installation and updating of jupyter path together felt wrong.

It moves to using the Preference system over using DEPOT_PATH[1]/prefs (but reads the old file as a fallback)

It changes some things that previously had to be done at build time (like setting the debug mode) to now just be a runtime thing.

Using IJulia from scratch (by using an empty depot and clearing out all kernels) looks like:

JULIA_DEPOT_PATH="$(mktemp -d):" julia +1.12 --project -q

(IJulia) pkg> instantiate 
...
  Installing artifacts ━━━━━━━━━━ 2/2
    Building Conda → `/var/folders/2c/rk2fkhgn6c35lr6qp7cll4t00000gn/T/tmp.8kRd4JRUqk/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/8f06b0cfa4c514c7b9546756dbae91fcfbc92dc9/build.log`

julia> using IJulia
Precompiling IJulia finished.
  13 dependencies successfully precompiled in 11 seconds. 30 already precompiled.

julia> notebook()
┌ Info: No default Julia kernel found for Julia 1.12.
│ Installing kernel automatically. You can reinstall or update the kernel
│ anytime by running: IJulia.installkernel()
│ 
│ To disable this auto-installation, set the environment variable:
└     ENV["IJULIA_NODEFAULTKERNEL"] = "true"
[ Info: Installing 'Julia 1.12' kernelspec in /Users/kc/Library/Jupyter/kernels/julia-1.12
...
install Jupyter via Conda, y/n? [y]: y
[ Info: Downloading miniconda installer ...
[ Info: Installing miniconda ...
PREFIX=/var/folders/2c/rk2fkhgn6c35lr6qp7cll4t00000gn/T/tmp.8kRd4JRUqk/conda/3/aarch64
Unpacking bootstrapper...
Unpacking payload...
...
...
[ Info: running setenv(`/var/folders/2c/rk2fkhgn6c35lr6qp7cll4t00000gn/T/tmp.8kRd4JRUqk/conda/3/aarch64/bin/jupyter notebook`)
...

@codecov
Copy link

codecov bot commented Nov 5, 2025

Codecov Report

❌ Patch coverage is 10.00000% with 63 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.53%. Comparing base (f504ce0) to head (9eb22e3).

Files with missing lines Patch % Lines
src/config.jl 6.66% 42 Missing ⚠️
src/jupyter.jl 0.00% 16 Missing ⚠️
src/IJulia.jl 20.00% 4 Missing ⚠️
src/kspec.jl 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1215      +/-   ##
==========================================
- Coverage   72.36%   70.53%   -1.83%     
==========================================
  Files          18       20       +2     
  Lines        1317     1446     +129     
==========================================
+ Hits          953     1020      +67     
- Misses        364      426      +62     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Member

@JamesWrigley JamesWrigley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making it so quickly ❤️

Also CC @goerz and @stevengj, in case you have any suggestions.

@stevengj
Copy link
Member

stevengj commented Nov 5, 2025

(This still won't fix external jupyter programs from breaking every time juliaup does a patch update unless the user manually re-installs the kernel, unfortunately. The root of this problem is that juliaup keeps moving the path underneath us, and gives us no official way to tell the actual command that was used to launch julia. JuliaLang/juliaup#857 … but at least we can fix running IJulia.jupyter(), I guess.)

@KristofferC
Copy link
Member Author

KristofferC commented Nov 5, 2025

Just FYI, while doing this I was not aware of the juliaup issue being referenced (and this PR was not meant to address that) but from the comment I can sort of imagine what the problem is. Pkg Apps have the same problem in that we store the absolute path to the Julia executable that installed the path. When juliaup does GC that exe, it's sad times.

My plan to help with the Pkg App problem is by using JuliaLang/juliaup#1315 in combination with a Pkg App specific env variable to override the executable that launch the app to be just julia. The automatic julia version selection will then pick the correct julia version by looking at the manifest, if needed download the needed julia version and launch the app

@KristofferC
Copy link
Member Author

Changed to using Preferences and did some updates based on review and own reading of code. I think the update_jupyter_path is quite important to look at now.

README.md Outdated
- Prompt you to install Jupyter if you don't already have it (hit enter to install via [Conda.jl](https://github.com/Luthaf/Conda.jl), which creates a minimal Python+Jupyter distribution private to Julia)
- Automatically install the Julia kernel for your current Julia version

If you already have Jupyter installed and prefer to use it, you can launch it from the terminal with `jupyter notebook` instead. The Julia kernel will be automatically installed the first time you run `IJulia.notebook()` or `IJulia.jupyterlab()`.
Copy link
Member

@stevengj stevengj Nov 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to just tell people to run installkernel() if they want to use an externally installed Jupyter front-end of some kind.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ab9137c.

@JamesWrigley
Copy link
Member

@KristofferC, do you think you'll have time to finish this in the next week or two? No worries if not, I'm happy to take over if you like.

@JamesWrigley
Copy link
Member

(bump)

1 similar comment
@JamesWrigley
Copy link
Member

(bump)

@JamesWrigley
Copy link
Member

@KristofferC I'll finish this off next week if that's alright with you, feel free to comment if you'd like to do it.

KristofferC and others added 2 commits December 31, 2025 18:37
…ding a jupyter path update function

The Pkg build functionality is not good. It gets run whenever a package is *downloaded* (which is a weird thing to trigger on where packages are considered immutable and multiple environments share the same package downloads), it requires the package to recompile when build options are changed etc.

This instead auto installs the kernel if needed and also provides a separate function to update the jupyter path.

It also moves to using Scratch.jl system over using `DEPOT_PATH[1]/prefs` (but reads the old file as a fallback)

It also changes some things that had to be done at build time (like setting the debug mode) to now just be a runtime thing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants