Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trait implementation causes Object File Too Large error and compilation crash #135849

Open
benwis opened this issue Jan 21, 2025 · 5 comments
Open
Labels
A-trait-system Area: Trait system C-bug Category: This is a bug. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. I-compiletime Issue: Problems and improvements with respect to compile times. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver)

Comments

@benwis
Copy link

benwis commented Jan 21, 2025

For Leptos 0.7, we've switched our HTML rendering engine to use large recursive static types to represent the HTML tree, giving us significant benefits to final binary size and runtime speed, but unfortunately not to compile time.

To fix that, we've brought back the old behavior where we type erase the HTML components in the tree behind a feature flag for users to develop faster. However, we also added generic attribute spreading in this release to make our maintenance and user's code nicer and easier to write.

The crux of the issue here is implementing the trait that does that causes an error: cannot encode offset of relocations; object file too large on Mac after compiling for 10ish minutes and using 50GB of ram, with other issues on other platforms, but only with the AddAnyAttr trait. Not having attribute spreading in the type erased version severely limits its usefulness.

The trait can be found here but is basically this:

// Pre-erases output to reduce compile-time explosions
impl AddAnyAttr for AnyView {
    type Output<SomeNewAttr: Attribute> = AnyView;

    #[inline(never)]
    fn add_any_attr<NewAttr: Attribute>(
        self,
        attr: NewAttr,
    ) -> Self::Output<NewAttr>
    where
        Self::Output<NewAttr>: RenderHtml,
    {
        let attr = attr.into_cloneable_owned();
        let any_attr = attr.into_any_attr();
        self.value.dyn_add_any_attr(any_attr)
    }
}

A similar issue was discussed broadly here, but I wanted to provide a specific reproducer for a more targeted issue in the hopes of a solution or a workaround. Please let me know if this is out of line.

This can be reproduced by running cargo +stable build --bin demo --features ssr in the main branch of this repo.

To fix(and disable attribute spreading), in the workspace Cargo.toml change the leptos branch to "any-attr-disabled", see the extra commit on that branch for the minimal change that allows compilation to succeed in about 1m30s. This leads us to the idea that this specific trait implementation brakes everything, even though it seems to be perfectly valid Rust.

Thanks to @zakstucke and all the other Leptos users who have worked on this and tested various workarounds

@benwis benwis added the C-bug Category: This is a bug. label Jan 21, 2025
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Jan 21, 2025
@workingjubilee
Copy link
Member

fascinating.

@workingjubilee workingjubilee added I-compiletime Issue: Problems and improvements with respect to compile times. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. A-trait-system Area: Trait system WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver) I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Jan 21, 2025
@benwis
Copy link
Author

benwis commented Jan 22, 2025

@workingjubilee This is not limited to Mac, it also crashes on Linux and probably Windows. Here is the stack trace for Linux on Fedora 16:

benwis in 🌐 seldon in thaw on  main via 🦀 v1.81.0 
❯ cargo +stable build --bin demo --features ssr
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /home/benwis/Downloads/thaw/demo/Cargo.toml
workspace: /home/benwis/Downloads/thaw/Cargo.toml
   Compiling demo v0.1.0 (/home/benwis/Downloads/thaw/demo)
error: rustc interrupted by SIGSEGV, printing backtrace

/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-3f4ebb066deec3c0.so(+0x32b0323) [0x71230b4b0323]
/usr/lib/libc.so.6(+0x3d1d0) [0x71230804c1d0]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.81.0-stable(+0x62be442) [0x712306abe442]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.81.0-stable(+0x62b2853) [0x712306ab2853]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.81.0-stable(_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE+0x136f) [0x71230650f5af]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.81.0-stable(_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE+0x2f) [0x71230650e185]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.81.0-stable(_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE+0x260) [0x712306ab0ea0]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-3f4ebb066deec3c0.so(LLVMRustWriteOutputFile+0x190) [0x71230d6624d0]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-3f4ebb066deec3c0.so(+0x54620df) [0x71230d6620df]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-3f4ebb066deec3c0.so(+0x545f5f0) [0x71230d65f5f0]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-3f4ebb066deec3c0.so(+0x545f25e) [0x71230d65f25e]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-3f4ebb066deec3c0.so(+0x545d6b4) [0x71230d65d6b4]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-3f4ebb066deec3c0.so(+0x545d1a9) [0x71230d65d1a9]
/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libstd-1c4b19562077c20d.so(rust_metadata_std_1f2242ed6435445e+0xc75fb) [0x71230ed165fb]
/usr/lib/libc.so.6(+0x9439d) [0x7123080a339d]
/usr/lib/libc.so.6(+0x11949c) [0x71230812849c]

note: we would appreciate a report at https://github.com/rust-lang/rust
help: you can increase rustc's stack size by setting RUST_MIN_STACK=16777216
error: could not compile `demo` (lib)

Caused by:
  process didn't exit successfully: `/home/benwis/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name demo --edition=2021 demo/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=150 --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 --cfg 'feature="csr"' --cfg 'feature="default"' --cfg 'feature="ssr"' --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values("csr", "default", "hydrate", "nightly", "ssr", "tracing"))' -C metadata=74f9f6f82da900f9 -C extra-filename=-74f9f6f82da900f9 --out-dir /home/benwis/Downloads/thaw/target/debug/deps -C incremental=/home/benwis/Downloads/thaw/target/debug/incremental -L dependency=/home/benwis/Downloads/thaw/target/debug/deps --extern cfg_if=/home/benwis/Downloads/thaw/target/debug/deps/libcfg_if-6afc45ef9c22aff9.rmeta --extern chrono=/home/benwis/Downloads/thaw/target/debug/deps/libchrono-a071519ce9432b48.rmeta --extern console_error_panic_hook=/home/benwis/Downloads/thaw/target/debug/deps/libconsole_error_panic_hook-def73760e3f683ed.rmeta --extern console_log=/home/benwis/Downloads/thaw/target/debug/deps/libconsole_log-e4bceb6fdbd9a06e.rmeta --extern demo_markdown=/home/benwis/Downloads/thaw/target/debug/deps/libdemo_markdown-585762e4e2e4faed.so --extern gloo_net=/home/benwis/Downloads/thaw/target/debug/deps/libgloo_net-c33a3db1dbd03c13.rmeta --extern icondata=/home/benwis/Downloads/thaw/target/debug/deps/libicondata-24a55c09ece64b6d.rmeta --extern js_sys=/home/benwis/Downloads/thaw/target/debug/deps/libjs_sys-a4a6c58932579698.rmeta --extern leptos=/home/benwis/Downloads/thaw/target/debug/deps/libleptos-03dc1254bb730a49.rmeta --extern leptos_meta=/home/benwis/Downloads/thaw/target/debug/deps/libleptos_meta-c8d114563dc0e233.rmeta --extern leptos_router=/home/benwis/Downloads/thaw/target/debug/deps/libleptos_router-5104eaa327e4c49e.rmeta --extern log=/home/benwis/Downloads/thaw/target/debug/deps/liblog-1141fc38e8923d7e.rmeta --extern palette=/home/benwis/Downloads/thaw/target/debug/deps/libpalette-fa9fe91759050312.rmeta --extern send_wrapper=/home/benwis/Downloads/thaw/target/debug/deps/libsend_wrapper-86494fb825b44e31.rmeta --extern thaw=/home/benwis/Downloads/thaw/target/debug/deps/libthaw-85a16e32ef484ed4.rmeta --extern uuid=/home/benwis/Downloads/thaw/target/debug/deps/libuuid-1d67e255752c3984.rmeta` (signal: 11, SIGSEGV: invalid memory reference)

@workingjubilee workingjubilee removed the O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) label Jan 22, 2025
@workingjubilee
Copy link
Member

@benwis huh. what is the actual output on macOS?

@benwis
Copy link
Author

benwis commented Jan 23, 2025

@benwis huh. what is the actual output on macOS?

cargo +stable build --bin demo --features ssr
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /Users/zak/z/code/thaw/demo/Cargo.toml
workspace: /Users/zak/z/code/thaw/Cargo.toml
   Compiling thaw v0.4.1 (/Users/zak/z/code/thaw/thaw)
   Compiling demo v0.1.0 (/Users/zak/z/code/thaw/demo)
error: cannot encode offset of relocations; object file too large

error: could not compile `demo` (lib) due to 1 previous error

@luxalpa
Copy link

luxalpa commented Jan 27, 2025

Just confirming that this is also an issue under Windows 11 / msvc:

stable-x86_64-pc-windows-msvc (default)
rustc 1.84.0 (9fc6b43 2025-01-07)

rustc-LLVM ERROR: out of memory
Allocation failed
error: could not compile `demo` (lib)

Caused by:
  process didn't exit successfully: `C:\Users\Luxalpa\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin\rustc.exe --crate-name demo --edition=2021 demo\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=119 --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 --cfg "feature=\"csr\"" --cfg "feature=\"default\"" --cfg "feature=\"ssr\"" --check-cfg cfg(docsrs) --check-cfg "cfg(feature, values(\"csr\", \"default\", \"hydrate\", \"nightly\", \"ssr\", \"tracing\"))" -C metadata=5e2bde8af5739bef -C extra-filename=-5e2bde8af5739bef --out-dir C:\projects_from_github\thaw\target\debug\deps -C incremental=C:\projects_from_github\thaw\target\debug\incremental -L dependency=C:\projects_from_github\thaw\target\debug\deps --extern cfg_if=C:\projects_from_github\thaw\target\debug\deps\libcfg_if-64bb2cffd428e39a.rmeta --extern chrono=C:\projects_from_github\thaw\target\debug\deps\libchrono-4997604bfdea4c35.rmeta --extern console_error_panic_hook=C:\projects_from_github\thaw\target\debug\deps\libconsole_error_panic_hook-e0851fc201ef9f25.rmeta --extern console_log=C:\projects_from_github\thaw\target\debug\deps\libconsole_log-cb9dfba97e19e4c9.rmeta --extern demo_markdown=C:\projects_from_github\thaw\target\debug\deps\demo_markdown-845ecd812a754256.dll --extern gloo_net=C:\projects_from_github\thaw\target\debug\deps\libgloo_net-99fd98a4ffb8597a.rmeta --extern icondata=C:\projects_from_github\thaw\target\debug\deps\libicondata-6e5c728b6b7b6956.rmeta --extern js_sys=C:\projects_from_github\thaw\target\debug\deps\libjs_sys-2b9929c1cb3fee66.rmeta --extern leptos=C:\projects_from_github\thaw\target\debug\deps\libleptos-1b52bd18fff8d698.rmeta --extern leptos_meta=C:\projects_from_github\thaw\target\debug\deps\libleptos_meta-07ae17ef97db0b63.rmeta --extern leptos_router=C:\projects_from_github\thaw\target\debug\deps\libleptos_router-341af47491526d64.rmeta --extern log=C:\projects_from_github\thaw\target\debug\deps\liblog-f90f0a800d58277c.rmeta --extern palette=C:\projects_from_github\thaw\target\debug\deps\libpalette-37de36958c75ce26.rmeta --extern send_wrapper=C:\projects_from_github\thaw\target\debug\deps\libsend_wrapper-0b6434e45d809341.rmeta --extern thaw=C:\projects_from_github\thaw\target\debug\deps\libthaw-ec8fa3c0c0c5e056.rmeta --extern uuid=C:\projects_from_github\thaw\target\debug\deps\libuuid-91c0ae681be859a7.rmeta -L native=C:\Users\Luxalpa\.cargo\registry\src\index.crates.io-6f17d22bba15001f\windows_x86_64_msvc-0.52.6\lib` (exit code: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN)

libthaw-ec8fa3c0c0c5e056.rlib being 1.5 GB in size:

Image

Also, it looks like my system is running out of memory:

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-trait-system Area: Trait system C-bug Category: This is a bug. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. I-compiletime Issue: Problems and improvements with respect to compile times. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver)
Projects
None yet
Development

No branches or pull requests

4 participants