[email protected] and @swc/[email protected] cannot run Wasm plugins that uses SourceFile

[fpapado]

Describe the bug

Follow the reproduction at https://github.com/fpapado/swc-1.8.0-and-plugin-styled-components-4.0.0.

Using @swc/[email protected] together with @swc/[email protected] and then attempting to transpile a file leads to a panic. This seems contrary to https://plugins.swc.rs/versions/range/19, which shows @swc/[email protected] and @swc/[email protected] as compatible.

SWC_DEBUG=1 pn swc --sync index.js

thread '<unnamed>' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/swc_plugin_proxy-3.0.0/src/memory_interop/read_returned_result_from_host.rs:110:10:
Returned value should be serializable: wasm plugin bytecheck failed "check bytes error: check failed for tuple struct member 0: check failed for struct member source_file: check failed for enum tuple variant Some: check failed for tuple struct member 0: check failed for struct member lazy: invalid tag for enum: 131"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Panic: PanicHookInfo { payload: Any { .. }, location: Location { file: "/Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/swc-4.0.0/src/plugin.rs", line: 169, col: 14 }, can_unwind: true, force_no_backtrace: false }
Backtrace:    0: <unknown>
   1: <unknown>
   2: <unknown>
   3: _napi_register_module_v1
   4: _napi_register_module_v1
   5: _napi_register_module_v1
   6: _napi_register_module_v1
   7: _wasmer_vm_imported_memory32_atomic_notify
   8: _wasmer_vm_imported_memory32_atomic_notify
   9: _napi_register_module_v1
  10: _napi_register_module_v1
  11: _napi_register_module_v1
  12: _napi_register_module_v1
  13: _napi_register_module_v1
  14: _napi_register_module_v1
  15: _napi_register_module_v1
  16: <unknown>
  17: <unknown>

failed to handle: failed to invoke plugin: failed to invoke plugin on 'Some("index.js")'

Caused by:
    0: failed to invoke `@swc/plugin-styled-components` as js transform plugin at @swc/plugin-styled-components
    1: failed to run Wasm plugin transform. Please ensure the version of `swc_core` used by the plugin is compatible with the host runtime. See the documentation for compatibility information. If you are an author of the plugin, please update `swc_core` to the compatible version.

                       Note that if you want to use the os features like filesystem, you need to use `wasi`. Wasm itself does not have concept of filesystem.

                       https://swc.rs/docs/plugin/selecting-swc-core

                       See https://plugins.swc.rs/versions/from-plugin-runner/3.0.0 for the list of the compatible versions.

                       Build info:
                           Date: 2024-11-04
                           Timestamp: 2024-11-04T01:54:05.821856000Z

                       Version info:
                           swc_plugin_runner: 3.0.0
                           Dependencies: anyhow 1.0.92,codspeed-criterion-compat 2.7.2,criterion 0.5.1,enumset 1.1.5,futures 0.3.31,once_cell 1.20.2,parking_lot 0.12.3,serde 1.0.214,serde_json 1.0.132,swc_atoms 2.0.0,swc_common 3.0.0,swc_css_ast 3.0.0,swc_css_parser 3.0.0,swc_ecma_ast 3.0.0,swc_ecma_loader 3.0.0,swc_ecma_parser 4.0.0,swc_ecma_visit 3.0.0,swc_malloc 1.0.0,swc_plugin_proxy 3.0.0,testing 3.0.0,tokio 1.41.0,tracing 0.1.40,vergen 9.0.1,virtual-fs 0.16.0,wasmer 4.3.7,wasmer-cache 4.3.7,wasmer-compiler-cranelift 4.3.7,wasmer-wasix 0.27.0

    2: RuntimeError: unreachable
Error: Failed to compile 1 file with swc.
    at Object.assertCompilationResult (/Users/fotis/swc-1.8.0-and-plugin-styled-components-4.0.0/node_modules/.pnpm/@[email protected]_@[email protected]/node_modules/@swc/cli/lib/swc/util.js:165:15)
    at files (/Users/fotis/swc-1.8.0-and-plugin-styled-components-4.0.0/node_modules/.pnpm/@[email protected]_@[email protected]/node_modules/@swc/cli/lib/swc/file.js:205:19)
    at async _default (/Users/fotis/swc-1.8.0-and-plugin-styled-components-4.0.0/node_modules/.pnpm/@[email protected]_@[email protected]/node_modules/@swc/cli/lib/swc/file.js:224:9)

Input code

import {styled} from 'styled-components';

const MyComponent = styled.div`
  color: purple;
`

Config

{
  "$schema": "https://swc.rs/schema.json",
  "jsc": {
    "experimental": {
      "plugins": [
        ["@swc/plugin-styled-components", {}]
      ]
    }
  }
}

Playground link (or link to the minimal reproduction)

https://github.com/fpapado/swc-1.8.0-and-plugin-styled-components-4.0.0

SWC Info output

/Users/fotis/Library/Application Support/fnm/node-versions/v20.18.0/installation/lib/node_modules/corepack/dist/lib/corepack.cjs:22373
throw new UsageError(This project is configured to use ${result.spec.name} because ${result.target} has a "packageManager" field);
^

UsageError: This project is configured to use pnpm because /Users/fotis/swc-1.8.0-and-plugin-styled-components-4.0.0/package.json has a "packageManager" field
at Engine.findProjectSpec (/Users/fotis/Library/Application Support/fnm/node-versions/v20.18.0/installation/lib/node_modules/corepack/dist/lib/corepack.cjs:22373:21)
at async Engine.executePackageManagerRequest (/Users/fotis/Library/Application Support/fnm/node-versions/v20.18.0/installation/lib/node_modules/corepack/dist/lib/corepack.cjs:22404:24)
at async Object.runMain (/Users/fotis/Library/Application Support/fnm/node-versions/v20.18.0/installation/lib/node_modules/corepack/dist/lib/corepack.cjs:23096:5) {
clipanion: { type: 'usage' }
}

Node.js v20.18.0

Operating System:
    Platform: darwin
    Arch: arm64
    Machine Type: arm64
    Version: Darwin Kernel Version 24.0.0: Mon Aug 12 20:51:54 PDT 2024; root:xnu-11215.1.10~2/RELEASE_ARM64_T6000
    CPU: (10 cores)
        Models: Apple M1 Pro

Binaries:
    Node: 20.18.0
    npm: 10.8.2
    Yarn: N/A
    pnpm: 9.12.3

Relevant Packages:
    @swc/core: 1.8.0
    @swc/helpers: N/A
    @swc/types: N/A


SWC Config:
    output: N/A
    .swcrc path: N/A

Next.js info:
    output: N/A

Expected behavior

I expected swc to transpile the code without errors around the plugin's compatibility with core, as listed in https://plugins.swc.rs/versions/range/19.

Actual behavior

SWC exits with a panic

Version

1.8.0

Additional context

Please let me know if there is any other information that I can provide, and I will get back to you promptly 😌

[kdy1]

I think there's a UB in the code related to rkyv. When I run it via cargo test, it works.

If I add logging code and run it via node.js, it hangs.

Log

❯ npx swc index.js
try_serialize: [0, 0, 0, 0, 9, 0, 0, 0, 15, 0, 0, 0, 9, 0, 0, 0, 15, 0, 0, 0, 2, 0, 0, 0, 115, 116, 121, 108, 101, 100, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 115, 116, 121, 108, 101, 100, 45, 99, 111, 109, 112, 111, 110, 101, 110, 116, 115, 39, 115, 116, 121, 108, 101, 100, 45, 99, 111, 109, 112, 111, 110, 101, 110, 116, 115, 39, 22, 0, 0, 0, 41, 0, 0, 0, 17, 0, 0, 0, 212, 255, 255, 255, 1, 0, 0, 0, 19, 0, 0, 0, 217, 255, 255, 255, 77, 121, 67, 111, 109, 112, 111, 110, 101, 110, 116, 0, 14, 0, 0, 0, 64, 0, 0, 0, 70, 0, 0, 0, 2, 0, 0, 0, 115, 116, 121, 108, 101, 100, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 64, 0, 0, 0, 74, 0, 0, 0, 180, 255, 255, 255, 0, 0, 0, 0, 71, 0, 0, 0, 74, 0, 0, 0, 100, 105, 118, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 9, 99, 111, 108, 111, 114, 58, 32, 112, 117, 114, 112, 108, 101, 59, 10, 10, 9, 99, 111, 108, 111, 114, 58, 32, 112, 117, 114, 112, 108, 101, 59, 10, 0, 0, 75, 0, 0, 0, 92, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 17, 0, 0, 0, 204, 255, 255, 255, 17, 0, 0, 0, 213, 255, 255, 255, 74, 0, 0, 0, 93, 0, 0, 0, 180, 255, 255, 255, 0, 0, 0, 0, 208, 255, 255, 255, 1, 0, 0, 0, 0, 0, 0, 0, 17, 0, 0, 0, 64, 0, 0, 0, 93, 0, 0, 0, 0, 0, 0, 0, 80, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 200, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 50, 0, 0, 0, 93, 0, 0, 0, 0, 0, 0, 0, 50, 0, 0, 0, 61, 0, 0, 0, 2, 0, 0, 0, 11, 0, 0, 0, 188, 254, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 144, 255, 255, 255, 0, 0, 0, 0, 44, 0, 0, 0, 93, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 180, 255, 255, 255, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 42, 0, 0, 0, 228, 253, 255, 255, 1, 0, 0, 0, 76, 254, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 17, 0, 0, 0, 2, 0, 0, 0, 164, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 93, 0, 0, 0, 140, 255, 255, 255, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
try_serialize: [99, 111, 109, 109, 111, 110, 95, 112, 108, 117, 103, 105, 110, 95, 116, 114, 97, 110, 115, 102, 111, 114, 109, 44, 101, 99, 109, 97, 95, 97, 115, 116

If I run it without logging code, it crashes with the error message in the issue body.

@kwonoj What do you think? Should we update rkyv (and wasmer) first?

[kwonoj]

Do we have exact change caused this regression? If we revert those what we'll lose?

I'm in for rkyv / wasmer update and it should happen, but rkyv had major breaking so update would require some code changes - also we are't 100% sure if update will fix this issue.

[kdy1]

The problematic patch is #9696, which makes source file analysis of swc_common lazy, improving the total runtime by 2% ~ 5%.
I introduced CacheCell with the PR, and it seems like it's problematic. Maybe code for rkyv-impl at

swc/crates/swc_common/src/cache.rs

Lines 35 to 115 in 499c803

	#[cfg(feature = "rkyv-impl")]
	mod rkyv_impl {
	use std::{hint::unreachable_unchecked, ptr};
	use rkyv::{
	option::ArchivedOption, out_field, Archive, Archived, Deserialize, Fallible, Resolver,
	Serialize,
	};
	use super::*;
	#[allow(dead_code)]
	#[repr(u8)]
	enum ArchivedOptionTag {
	None,
	Some,
	}
	#[repr(C)]
	struct ArchivedOptionVariantNone(ArchivedOptionTag);
	#[repr(C)]
	struct ArchivedOptionVariantSome<T>(ArchivedOptionTag, T);
	impl<T> Archive for CacheCell<T>
	where
	T: Archive,
	{
	type Archived = Archived<Option<T>>;
	type Resolver = Resolver<Option<T>>;
	unsafe fn resolve(&self, pos: usize, resolver: Self::Resolver, out: *mut Self::Archived) {
	match resolver {
	None => {
	let out = out.cast::<ArchivedOptionVariantNone>();
	ptr::addr_of_mut!((*out).0).write(ArchivedOptionTag::None);
	}
	Some(resolver) => {
	let out = out.cast::<ArchivedOptionVariantSome<T::Archived>>();
	ptr::addr_of_mut!((*out).0).write(ArchivedOptionTag::Some);
	let v = self.0.get();
	let value = if let Some(value) = v.as_ref() {
	value
	} else {
	unreachable_unchecked();
	};
	let (fp, fo) = out_field!(out.1);
	value.resolve(pos + fp, resolver, fo);
	}
	}
	}
	}
	impl<T: Serialize<S>, S: Fallible + ?Sized> Serialize<S> for CacheCell<T> {
	#[inline]
	fn serialize(&self, serializer: &mut S) -> Result<Self::Resolver, S::Error> {
	self.0
	.get()
	.map(|value| value.serialize(serializer))
	.transpose()
	}
	}
	impl<T: Archive, D: Fallible + ?Sized> Deserialize<CacheCell<T>, D> for ArchivedOption<T::Archived>
	where
	T::Archived: Deserialize<T, D>,
	{
	#[inline]
	fn deserialize(&self, deserializer: &mut D) -> Result<CacheCell<T>, D::Error> {
	match self {
	ArchivedOption::Some(value) => {
	let v = value.deserialize(deserializer)?;
	Ok(CacheCell::from(v))
	}
	ArchivedOption::None => Ok(CacheCell::new()),
	}
	}
	}
	}

is problematic, but I failed to debug.

[kwonoj]

Isn't rspack attempt to upgrade to latest swc in the next release? Blocking all plugin sounds like a deal breaker; maybe cost those perf improvement and revert, then retry after rkyv / wasmer update?

[kdy1]

It's not all Wasm plugin, but I think we should revert it anyway. I left a question on rkyv Discord, so if it's a real UB, I'll revert the PR and reapply it after updating rkyv