Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport ebae716 to v1.10 #56354

Closed
wants to merge 427 commits into from
Closed

Backport ebae716 to v1.10 #56354

wants to merge 427 commits into from

Conversation

mofeing
Copy link
Contributor

@mofeing mofeing commented Oct 27, 2024

Pangoraw and others added 30 commits October 24, 2023 10:44
[Docs](https://docs.julialang.org/en/v1/manual/control-flow/#else-Clauses)
state:

> The try, catch, else, and finally clauses each introduce their own
> scope blocks.

But it is currently not the case for `else` blocks

```julia
julia> try
       catch
       else
           z = 1
       end
1

julia> z
1
```

This change actually makes `else` blocks have their own scope block:

```julia
julia> try
       catch
       else
           z = 1
       end
1

julia> z
ERROR: UndefVarError: `z` not defined
```

(cherry picked from commit 17a36ee)
Fixes #51771

The convert method that asserts in #51771 is arguably still faulty
though.

(cherry picked from commit cf00550)
Otherwise `--heap-size-hint` will become a no-op.

Likely a merge bug from #51661.
It seems this case has already been fixed by other improvements, so we
no longer need this hack, which is now causing problems.

Fixes #51694
…ion. (#51840)

This allows other users of LLVM to use opaque pointers with their
contexts.

Co-authored-by: Jameson Nash <[email protected]>
Restores the method whose removal was probably causing problems.

(cherry picked from commit 14d9c7c)
Restores the method whose removal was probably causing problems.

(cherry picked from commit f6f1ee9)
This shouldn't be needed because `ldd` should do it itself.

(cherry picked from commit 5b34cdf)
Backported PRs:
- [x] #50932 <!-- types: fix hash values of Vararg -->
- [x] #50975 <!-- Use rr-safe `nopl; rdtsc` sequence -->
- [x] #50989 <!-- fix incorrect results in `expm1(::Union{Float16,
Float32})` -->
- [x] #51284 <!-- Avoid infinite loop when doing SIGTRAP in arm64-apple
-->
- [x] #51332 <!-- Add s4 field to Xoshiro -->
- [x] #51397 <!-- call Pkg precompile hook in latest world -->
- [x] #51405 <!-- Remove fallback that assigns a module to inlined
frames. -->
- [x] #51491 <!-- Throw clearer ArgumentError for strip with two string
args -->
- [x] #51531 <!-- fix `_tryonce_download_from_cache` (busybox.exe
download error) -->
- [x] #51541 <!-- Fix string index error in tab completion code -->
- [x] #51530 <!-- Don't mark nonlocal symbols as hidden -->
- [x] #51557 <!-- Fix last startup & shutdown precompiles -->
- [x] #51512 <!-- avoid limiting Type{Any} to Type -->
- [x] #51595 <!-- reset `maxprobe` on `empty!` -->
- [x] #51582 <!-- Aggressive constprop in LinearAlgebra.wrap -->
- [x] #51592 <!-- correctly track element pointer in heap snapshot -->
- [x] #51326 <!-- complete false & true more generally as vals -->
- [x] #51376 <!-- make `hash(::Xoshiro)` compatible with `==` -->
- [x] #51557 <!-- Fix last startup & shutdown precompiles -->
- [x] #51845 
- [x] #51840 
- [x] #50663 <!-- Fix Expr(:loopinfo) codegen -->
- [x] #51863 <!-- LLVM 15.0.7-9 -->

Contains multiple commits, manual intervention needed:

- [ ] #51035 <!-- refactor GC scanning code to reflect jl_binding_t are
now first class -->
- [ ] #51092 <!-- inference: fix bad effects for recursion -->

Non-merged PRs with backport label:
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
- [ ] #51414 <!-- improvements on GC scheduler shutdown -->
- [ ] #51366 <!-- Handle infix operators in REPL completion -->
- [ ] #50919 <!-- Code loading: do the "skipping mtime check for stdlib"
check regardless of the value of `ispath(f)` -->
- [ ] #50824 <!-- Add some aliasing warnings to docstrings for mutating
functions in Base -->
- [ ] #49805 <!-- Limit TimeType subtraction to AbstractDateTime -->
`jl_errorexception_type` is undefined at the point we (fail to) load a
sysimg.

(cherry picked from commit 20a5fa7)
This is required now once Distributed is not in the sysimage.

Fixes https://github.com/JuliaLang/julia/issues/51756

(cherry picked from commit 795d8d7)
This aligns their behavior with manual calls to `finalize(o)`, and
prepares for a future time in which these functions are always run on a
separate thread. This means that they can wait to acquire locks in this
context, which otherwise would have been denied to them.

(cherry picked from commit c54a3f2)
Can cause spurious warnings about not closing these properly and
unexpected events to appear after `close` returns.

(cherry picked from commit d0c4284)
…dispatch (#51995)

The artifacts dict is not lowered to ensure_artifact_installed which
causes to load the ".toml" during runtime for lazy artifacts

(cherry picked from commit 9bc6994)
This bumps Statistics to the latest commit of the release-1.10 branch in
order to backport JuliaStats/Statistics.jl#153.

See JuliaData/DataFrames.jl#3383. Cc: @bkamins
@George9000
This avoids a crashes where we run the destructors because C++ is fun
and runs destructors before thread exit.

(cherry picked from commit 3d88550)
…#50207)

Suggested by @vchuravy.

---------

Co-authored-by: Jameson Nash <[email protected]>
(cherry picked from commit 2adf54a)
Two chagnes wrapped into one `Base.copymutable` => `Base.copymutable` &
`collect` and `Base.copymutable` => `similar` & words.

Followup for #52086 and #46104; also fixes #51932 (though we still may
want to make `copymutable` public at some point)

---------

Co-authored-by: Jameson Nash <[email protected]>
(cherry picked from commit 42c088b)
This fixes a whole bunch of small but annoying bugs, as described in the
JuliaSyntax-0.4.7 release notes

https://github.com/JuliaLang/JuliaSyntax.jl/releases/tag/v0.4.7

I've been careful about cutting the JuliaSyntax-0.4.7 release from
nonbreaking changes, so we should be able to backport this to 1.10.

---

Extended notes about compatibility
* The public keyword in
JuliaLang/JuliaSyntax.jl#320 is released in
JuliaSyntax-0.4.7 but JuliaSyntax is multi-version aware so this is
disabled when used as the default parser in Julia 1.10, but is enabled
in 1.11-DEV. So should be backportable.
* We aim for parsing to `Expr` to always be stable in JuliaSyntax and
independent of the host Julia `VERSION`, but we're not fully there yet
for 1.11 / 1.10 due to
JuliaLang/JuliaSyntax.jl#377. Thus some
careful management of the JuliaSyntax-0.4.x branch for now.

(cherry picked from commit 85d7cca)
…aces (#51520)

On AMDGPU, this was generating a `addrspace(10)` pointer to an `alloca`
which is illegal and lead to other issues.

(cherry picked from commit af9a7af)
Fixes #51985

Ensure that the REPL completions escape and unescape text correctly,
using the correct functions, and accounting for exactly what the user
has currently typed.

The old broken method is left around for Pkg, since it has an
over-reliance on it returning incorrect answers. Once Pkg is fixed, we
can delete that code.

Co-authored-by: Jameson Nash <[email protected]>
(cherry picked from commit 5edcdc5)
palday and others added 29 commits July 24, 2024 12:10
This release fixes issues with complex valued returns from functions
such as `cdotc` on Windows x64. See this discussion [0] for initial
diagnosis, and this PR [1] for the relevant fixes.

[0] JuliaLinearAlgebra/BLISBLAS.jl#15
[1] JuliaLinearAlgebra/libblastrampoline#129

(cherry picked from commit 3054c00)
Per
JuliaCI/julia-buildkite#224 (comment),
the path needs to be updated so that `llvm-size` can find `libLLVM.dll`.

(cherry picked from commit 657ce04)
…dir (#52820)

Fix #51740

Since we are providing libssp.dll on Windows and we want to dynamically
link to it, exposing libssp.dll.a is necessary. The inconsistency is
that libjulia-codegen.so looks in build_libdir and build_private_libdir
while standard library precompilation looks in build_shlibdir and
build_private_dir.

(cherry picked from commit c3836e1)
)

Backport of #54871 onto 1.10

We don't have the luxury of modifying the C-API. This pass surfaces LLVM
missed transformation warnings,
which is surprising to users. It can be very helpful, but it can also
add unactionable noise to the output
of Julia programs.

This PR simply removes it from the pipeline.
LLVM generates calls to math intrinsics like `trunc` and `rint` (at
least in my local i686 mingw) build, so linking to openlibm is required.
We already have this on the sysimg link line in `Base.link_image`, so
this aligns those options.

---------

Co-authored-by: Elliot Saba <[email protected]>
(cherry picked from commit 320366b)
After #55180 we implicitly require an LLVM built with Zlib support, but
compiling Julia with `make USE_BINARYBUILDER_LLVM=0` builds an LLVM
without Zlib support, despite the fact we attempt to request it at

https://github.com/JuliaLang/julia/blob/996351f5f6651d1508aef3c35c7d37eb22a0fb1e/deps/llvm.mk#L97
This was first identified in #55337.

`ZLIB_LIBRARY` must be the path to the zlib library, but we currently
set it to the libdir where the library is installed (introduced in

https://github.com/JuliaLang/julia/blob/996351f5f6651d1508aef3c35c7d37eb22a0fb1e/deps/llvm.mk#L97
which is wrong. However, CMake is actually able to find Zlib correctly,
but then the check at
https://github.com/llvm/llvm-project/blob/46425b8d0fac3c529aa4a716d19abd7032e452f3/llvm/cmake/config-ix.cmake#L139-L141
uses the value of `ZLIB_LIBRARY` to list the Zlib to link for the test,
but being `ZLIB_LIBRARY` a directory, CMake doesn't see any valid Zlib
and thus tries to run the test without linking any Zlib, and the test
silently fails (they're silent only when `LLVM_ENABLE_ZLIB=ON`),
resulting in no usable Zlib available, even if found.

`ZLIB_ROOT` is the only [hint recommended by the CMake module
`FindZLIB`](https://cmake.org/cmake/help/latest/module/FindZLIB.html#hints).
This PR replaces a broken `ZLIB_LIBRARY` with an appropriate
`ZLIB_ROOT`. Also, we set `LLVM_ENABLE_ZLIB=FORCE_ON` which is the only
way to make CMake fail loudly if no usable Zlib is available, and avoid
going on with a non-usable build.

I confirm this fixes #55337 for me, it should likely address
JuliaCI/julia-buildkite#373 as well.

Also, options `COMPILER_RT_ENABLE_IOS`, `COMPILER_RT_ENABLE_WATCHOS`,
`COMPILER_RT_ENABLE_TVOS`, and `HAVE_HISTEDIT_H` don't exist anymore,
and they are removed.
…data (#55306) (#55397)

…data (#55306)

Co-authored-by: Valentin Churavy <[email protected]>
Co-authored-by: Mosè Giordano <[email protected]>
Co-authored-by: Gabriel Baraldi <[email protected]>
The docstring of `LinearAlgebra.reflectorApply!` is incorrect. It says
the function is applying `(I - τ*[1; x] * [1; x]')*A`, but in reality it
is applying `(I - conj(τ)*[1; x[2:end]]*[1; x[2:end]]')*A`. You can
check that by looking at the code, or running for example
```julia
using LinearAlgebra
T=ComplexF64;d=5;
τ=randn(T);x=randn(T,d);A=randn(T,d,d);
(I - conj(τ)*[1; x[2:end]]*[1; x[2:end]]')*A
LinearAlgebra.reflectorApply!(x,τ,A)
```

(cherry picked from commit 1ece299)
`stack` does not require that the inner iterator defines `axes`, but the
code to assemble an error message assumed this.

Found here:
https://discourse.julialang.org/t/reduce-hcat-is-type-unstable/112800/3

(cherry picked from commit ae483c3)
It can be very helpful, when struggling with LBT forwarding, to see what
functions were actually forwarded to a new library. This utility
function makes it easy to query which functions are forwarded to that
library.

(cherry picked from commit e0f2e29)
In 1.10 we have

```jl
julia> isleapyear(Year(1992))
false
```

which is semantically incorrect because `Year(1992)` is a duration (1992
years), not an instant. This PR restricts the currently unrestricted
argument to integers.

(cherry picked from commit fdecc59)
(cherry picked from commit b759fe2)
This includes support to properly forward MKL v2024's ILP64 CBLAS
symbols, which fixes this [Enzyme
issue](EnzymeAD/Enzyme.jl#1683)

(cherry picked from commit 602b582)
This ensures that `tr` for a block `SymTridiagonal` symmetrizes the
diagonal elements.

(cherry picked from commit a163483)
…1a6e8 (#55437)

Stdlib: Pkg
URL: https://github.com/JuliaLang/Pkg.jl.git
Stdlib branch: release-1.10
Julia branch: backports-release-1.10
Old commit: 9cb6d35f8
New commit: 45521a6e8
Julia version: 1.10.4
Pkg version: 1.10.0(Does not match)
Bump invoked by: @IanButterworth
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)

Diff:
JuliaLang/Pkg.jl@9cb6d35...45521a6

```
$ git log --oneline 9cb6d35f8..45521a6e8
45521a6e8 backport extension dependency modeling from Base 1.11 to Pkg 1.10 (#3986)
1810b1123 don't swallow internal errors (#3987)
```

Co-authored-by: Dilum Aluthge <[email protected]>
…6cff6 (#55463)

Stdlib: Pkg
URL: https://github.com/JuliaLang/Pkg.jl.git
Stdlib branch: release-1.10
Julia branch: backports-release-1.10
Old commit: 45521a6e8
New commit: a4f26cff6
Julia version: 1.10.4
Pkg version: 1.10.0(Does not match)
Bump invoked by: @IanButterworth
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)

Diff:
JuliaLang/Pkg.jl@45521a6...a4f26cf

```
$ git log --oneline 45521a6e8..a4f26cff6
a4f26cff6 [release-1.10] Pkg.precompile: Handle when the terminal is very short (#3988)
```

Co-authored-by: Dilum Aluthge <[email protected]>
Backported PRs:
- [x] #51351 <!-- Remove boxing in pinv -->
- [x] #52678 <!-- Profile: Improve module docstring -->
- [x] #54201 <!-- Fix generic triangular solves with empty matrices -->
- [x] #54605 <!-- Allow libquadmath to also fail as it is not available
on all systems -->
- [x] #54634 <!-- Fix trampoline assembly for build on clang 18 on apple
silicon -->
- [x] #54635 <!-- Aggressive constprop in trevc! to stabilize triangular
eigvec -->
- [x] #54645 <!-- ensure we set the right value to gc_first_tid -->
- [x] #54671 <!-- Add boundscheck in bindingkey_eq to avoid OOB access
due to data race -->
- [x] #54672 <!-- make: Fix `sed` command for LLVM libraries with no
symbol versioning -->
- [x] #54704 <!-- LazyString in reinterpretarray error messages -->
- [x] #54713 <!-- make: use `readelf` for LLVM symbol version detection
-->
- [x] #54781 <!-- [LinearAlgebra] Improve resilience to unknown
libblastrampoline flags -->
- [x] #54837 <!-- Do not add type tag size to the `alloc_typed` lowering
for GC allocations -->
- [x] #54815 <!-- add sticky task warning to `@task` and `schedule` -->
- [x] #55141 <!-- Update the aarch64 devdocs to reflect the current
state of its support -->
- [x] #55178 <!-- Compat for `Base.@nospecializeinfer` -->
- [x] #55197 <!-- compat notice for a[begin] indexing -->
- [x] #55209 <!-- correction to compat notice for a[begin] -->
- [x] #55203 <!-- document mutable struct const fields -->
- [x] #54769 <!-- add missing compat entry to edit -->
- [x] #54791 <!-- Bump libblastrampoline to v5.10.1 -->
- [x] #55070 <!-- LinearAlgebra: LazyString in error messages for
Diagonal/Bidiagonal -->
- [x] #54624 <!-- more precise aliasing checks for SubArray -->
- [x] #54690 <!-- Fix assertion/crash when optimizing function with dead
basic block -->
- [x] #55084 <!-- Use triple quotes in TOML.print when string contains
newline -->


Need manual backport:
- [ ] #52505 <!-- fix alignment of emit_unbox_store copy -->
- [ ] #53373 <!-- fix sysimage-native-code=no option with pkgimages -->
- [ ] #53984 <!-- Profile: fix heap snapshot is valid char check -->
- [ ] #54276 <!-- Fix solve for complex `Hermitian` with non-vanishing
imaginary part on diagonal -->
- [ ] #54669 <!-- Improve error message in inplace transpose -->
- [ ] #54871 <!-- Make warn missed transformations pass optional -->

Contains multiple commits, manual intervention needed:
- [ ] #52854 <!-- Change to streaming out the heap snapshot data -->
- [ ] #53218 <!-- Fix interpreter_exec.jl test -->
- [ ] #53833 <!-- Profile: make heap snapshots viewable in vscode viewer
-->
- [ ] #54303 <!-- LinearAlgebra: improve type-inference in
Symmetric/Hermitian matmul -->
- [ ] #52694 <!-- Reinstate similar for AbstractQ for backward
compatibility -->
- [ ] #54737 <!-- LazyString in interpolated error messages involving
types -->
- [ ] #54738 <!-- serialization: fix relocatability bug -->
- [ ] #55052 <!-- Fix `(l/r)mul!` with `Diagonal`/`Bidiagonal` -->

Non-merged PRs with backport label:
- [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows -->
- [ ] #55169 <!-- `propertynames` for SVD respects private argument -->
- [ ] #55013 <!-- [docs] change docstring to match code -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
- [ ] #50813 <!-- More doctests for Sockets and capitalization fix -->
- [ ] #50157 <!-- improve docs for `@inbounds` and
`Base.@propagate_inbounds` -->
- [ ] #41244 <!-- Fix shell `cd` error when working dir has been deleted
-->
@mofeing mofeing closed this Oct 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment