Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Significant regression in TTFX, Julia 1.11 #649

Open
jakobnissen opened this issue Jul 24, 2024 · 1 comment
Open

Significant regression in TTFX, Julia 1.11 #649

jakobnissen opened this issue Jul 24, 2024 · 1 comment

Comments

@jakobnissen
Copy link
Contributor

With a nearly empty dummy package Foo, and running:

julia> using JET, Foo

julia> @time @eval report_package(Foo)

I get the following timings:
1.10.4: 11.770015 seconds (13.73 M allocations: 977.822 MiB, 10.09% gc time, 99.85% compilation time)
1.11-rc2: 33.809627 seconds (51.71 M allocations: 2.676 GiB, 4.95% gc time, 99.86% compilation time)

The high level of compilation and allocations suggests perhaps something could be done to reduce this?

@lgoettgens
Copy link

lgoettgens commented Oct 15, 2024

Now that julia 1.11 is out, the issue seems to have gotten even worse. I've done some timings on @report_opt and @report_call, as these are the things from JET that I used to use on a daily basis. But with these regressions, that seems no longer viable on 1.11.

All timings have been performed with JET v0.9.10
1.10.5:

julia> using JET

julia> @time @eval @report_opt 2+2
  0.246439 seconds (308.67 k allocations: 20.559 MiB, 99.73% compilation time)
No errors detected


julia> @time @eval @report_opt 2+2
  0.000364 seconds (488 allocations: 31.539 KiB, 0.14% compilation time)
No errors detected


julia> @time @eval @report_call 2+2
  0.106172 seconds (40.82 k allocations: 2.802 MiB, 99.48% compilation time)
No errors detected


julia> @time @eval @report_call 2+2
  0.000817 seconds (207 allocations: 12.602 KiB, 0.13% compilation time)
No errors detected

1.11.0:

julia> using JET

julia> @time @eval @report_opt 2+2
 13.131656 seconds (30.70 M allocations: 1.605 GiB, 6.32% gc time, 100.00% compilation time)
No errors detected


julia> @time @eval @report_opt 2+2
  0.000365 seconds (653 allocations: 29.164 KiB, 0.13% compilation time)
No errors detected


julia> @time @eval @report_call 2+2
 14.425664 seconds (24.85 M allocations: 1.291 GiB, 3.23% gc time, 100.00% compilation time)
No errors detected


julia> @time @eval @report_call 2+2
  0.000318 seconds (266 allocations: 12.039 KiB, 0.18% compilation time)
No errors detected

1.11.1:

julia> using JET

julia> @time @eval @report_opt 2+2
 13.215292 seconds (30.75 M allocations: 1.607 GiB, 9.01% gc time, 100.00% compilation time)
No errors detected


julia> @time @eval @report_opt 2+2
  0.000359 seconds (653 allocations: 29.164 KiB, 0.15% compilation time)
No errors detected


julia> @time @eval @report_call 2+2
 13.560778 seconds (24.86 M allocations: 1.292 GiB, 4.25% gc time, 99.99% compilation time)
No errors detected


julia> @time @eval @report_call 2+2
  0.000314 seconds (266 allocations: 12.039 KiB, 0.16% compilation time)
No errors detected

Edit: Re-ran everything on a different device, and added 1.11.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants