Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip javascriptcore's first parse step for ES Modules #15758

Draft
wants to merge 61 commits into
base: main
Choose a base branch
from

Conversation

pfgithub
Copy link
Contributor

@pfgithub pfgithub commented Dec 14, 2024

Not ready yet.

First half of #7384. Does not fix that problem, but gets closer to fixing it.

This pr:

  • Adds a test to prepare for when 7384 is fixed (marked with {todo: true} (Adds support to mark a test file as --todo with {todo: "true"} #15633)
  • Constructs ModuleInfo when printing with all the information needed to create a JavaScriptCore JSModuleRecord
  • Replaces javascriptcore's first parsing step to generate the module record with our own generated module record
  • Adds an env var COMPILE_ERRORS_ONLY=1 bun run build for building bun that skips codegen. This shows errors ~50% faster and can sometimes reveal an error that crashed zig without the flag

TODO:

  • Cache ModuleInfo in RuntimeTranspilerCache
  • Store ModuleInfo in the source provider, and get it from there rather than from the comment
  • Make sure cached ModuleInfo in RuntimeTranspilerCache is used
  • Remove the json comment hack
  • Only use SourceProviderSourceType::BunTranspiledModule when ModuleInfo is available for the module
  • Enable module record comparison in debug builds
  • Measure performance (should be faster unless there's a problem or if jsc's parseRootNode is really fast)
  • Test all the types of module imports and exports and re-exports and star exports and make sure they still work
  • Consider whether to run the module_info building as a seperate pass at the end of printAst rather than integrating it into js_printer
  • Consider using Ast.import records, export records, scope, ... to find the needed items rather than finding them in printer. There are a few special cases like the jest thing (but maybe could be solved by moving that transformation to a step before printing?)
  • Audit utf-8 ←→ utf-16 conversions
  • Discard memory holding ModuleInfo when it is no longer needed (after creating the real record and in other cases without introducing bugs)
  • Is this needed in standalone_module_graph? module_loader.zig line 2633?
  • Fix failing tests:
    • resolve.test.ts: 'StrContainsNullByte' but used to have an actual error message. not sure what to do about this one.
    • require-cache.test.ts: detects a memory leak, which makes sense because the memory isn't freed yet
    • mock-module.test.ts segfault?
    • serve-body-leak.test.ts: detects a memory leak. maybe because this doesn't free memory? or maybe some other cause

Performance impact:

  • large file, with runtime transpiler cache: ~14% faster
  • large file, without runtime transpiler cache: ~6% faster
  • small file: ~6% slower with imports, ~4% slower without imports :/

@robobun
Copy link

robobun commented Dec 14, 2024

Updated 8:51 PM PT - Dec 19th, 2024

@pfgithub, your commit 0652407 has 2 failures in #8302:


🧪   try this PR locally:

bunx bun-pr 15758

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants