Replies: 18 comments 36 replies
-
This is what the Parcel transformer side would look like: const { default: ThrowableDiagnostic } = require("@parcel/diagnostic");
const { Transformer } = require("@parcel/plugin");
module.exports = new Transformer({
async transform({ asset, logger, inputFs }) {
let {
jsResult,
wasmResult,
error,
invalidateOnFileChange,
invalidateOnFileCreate,
invalidateOnEnvChange,
} = await something({
filePath: asset.filePath,
inputCode: await asset.getCode(),
readFile: (...args) => fs.readFile(...args),
});
/*
jsResult should be something like
```js
const wasmURL = new URL("asc-wasm-module", import.meta.url);
export async function instantiate(imports = {}) {
const { exports } = await WebAssembly.instantiateStreaming(fetch(wasmURL), imports);
// AssemblyScript probably has more code in here for bindings
return exports;
}
```
wasmResult should be a binary buffer containing the compiled Wasm
*/
if (error) {
throw new ThrowableDiagnostic({
diagnostic: {
message: error.message,
codeFrames: [
{
language: "asc",
filePath: error.filePath,
codeHighlights: [
{
message: error.detailedMessage,
start: { line: error.start.line, column: error.start.column },
end: { line: error.end.line, column: error.end.column },
},
],
},
],
},
});
}
for (let file of invalidateOnFileChange) {
asset.invalidateOnFileChange(file);
}
for (let file of invalidateOnFileCreate) {
asset.invalidateOnFileCreate({ filePath: file });
}
for (let envvar of invalidateOnEnvChange) {
asset.invalidateOnEnvChange(envvar);
}
asset.type = "js";
asset.setCode(result);
return [
asset,
// uniqueKey is however the Wasm module was imported on the JS side.
{ type: "wasm", content: wasmResult, uniqueKey: "asc-wasm-module" },
];
},
}); What you need to figure out is the function I can help you if you have that Here is a repo of a working transformer (as a template): https://github.com/mischnic/parcel-transformer-wat/
Have you seen https://parceljs.org/plugin-system/transformer/ ? |
Beta Was this translation helpful? Give feedback.
-
Hey @mischnic, thanks a lot for your detailed answer, I highly appreciate it! 🙏 |
Beta Was this translation helpful? Give feedback.
-
I was thinking of building and running parcel-transformer-wat locally to have a working point of reference. However, Do you have any quick tips to work around the problem? (if not, it's totally okay :) ) Click to see an excerpt from the error printout``` gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ] make: Entering directory '/parcel-transformer-wat/node_modules/msgpackr-extract/build' CXX(target) Release/obj.target/msgpackr-extract/src/extract.o In file included from ../../nan/nan.h:178, from ../src/extract.cpp:11: ../../nan/nan_callbacks.h:55:23: error: ‘AccessorSignature’ is not a member of ‘v8’ 55 | typedef v8::Local Sig; | ^~~~~~~~~~~~~~~~~ ../../nan/nan_callbacks.h:55:40: error: template argument 1 is invalid 55 | typedef v8::Local Sig; | ^ In file included from ../src/extract.cpp:11: ../../nan/nan.h: In function ‘void Nan::SetAccessor(v8::Local, v8::Local, Nan::GetterCallback, Nan::SetterCallback, v8::Local, v8::AccessControl, v8::PropertyAttribute, Nan::imp::Sig)’: ../../nan/nan.h:2551:16: error: no matching function for call to ‘v8::ObjectTemplate::SetAccessor(v8::Local&, void (*&)(v8::Local, const v8::PropertyCallbackInfo&), void (*&)(v8::Local, v8::Local, const v8::PropertyCallbackInfo&), v8::Local&, v8::AccessControl&, v8::PropertyAttribute&, Nan::imp::Sig&)’ 2551 | , signature); | ^ In file included from /.cache/node-gyp/19.3.0/include/node/v8-function.h:15, from /.cache/node-gyp/19.3.0/include/node/v8.h:33, from ../src/extract.cpp:8: /.cache/node-gyp/19.3.0/include/node/v8-template.h:807:8: note: candidate: ‘void v8::ObjectTemplate::SetAccessor(v8::Local, v8::AccessorGetterCallback, v8::AccessorSetterCallback, v8::Local, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)’ 807 | void SetAccessor( | ^~~~~~~~~~~ /.cache/node-gyp/19.3.0/include/node/v8-template.h:812:22: note: no known conversion for argument 7 from ‘Nan::imp::Sig’ {aka ‘int’} to ‘v8::SideEffectType’ 812 | SideEffectType getter_side_effect_type = SideEffectType::kHasSideEffect, ``` |
Beta Was this translation helpful? Give feedback.
-
Hey @mischnic , yet another question. Update: Please see my latest comment(s) , this question is probably irrelevant.It appears as if my
|
Beta Was this translation helpful? Give feedback.
-
Please never mind! I tried copying my configuration to your project and added the corresponding files. The
I surely get an error since the WAT transformer won't understand AssemblyScript 😆 but at least it shows the right line where it stumbles upon "the incorrect" code. I must have simply missed something in my set-up. |
Beta Was this translation helpful? Give feedback.
-
Hey @mischnic, I believe I have all the essentials operational now and I can finally work on the logic of the transformer. Thanks a lot for your support, time and patience! I highly appreciate that 🙏 |
Beta Was this translation helpful? Give feedback.
-
Hi @mischnic , I hope you are well. It took me some time, but I have some results. I bumped into a problem and I don't seem to find a solution quickly. If you have some time, could you point me in the right direction? Thanks in advance! Good news:
So-so news:
I tried to guess some settings in My poor guesses in config that did not work:
Here is the log with some debug output, statistics, and the error message:
|
Beta Was this translation helpful? Give feedback.
-
Hi @mischnic , thanks for your comments! To expand the context a little, there's a setting in Anyway, below are those two kinds of glue per different value of "esm"
{
/// Omitted
"options": {
"bindings": "esm"
}
} results in: async function instantiate(module, imports = {}) {
const adaptedImports = {
env: Object.assign(Object.create(globalThis), imports.env || {}, {
abort(message, fileName, lineNumber, columnNumber) {
// ~lib/builtins/abort(~lib/string/String | null?, ~lib/string/String | null?, u32?, u32?) => void
message = __liftString(message >>> 0);
fileName = __liftString(fileName >>> 0);
lineNumber = lineNumber >>> 0;
columnNumber = columnNumber >>> 0;
(() => {
// @external.js
throw Error(`${message} in ${fileName}:${lineNumber}:${columnNumber}`);
})();
},
"console.log"(text) {
// ~lib/bindings/dom/console.log(~lib/string/String) => void
text = __liftString(text >>> 0);
console.log(text);
},
}),
};
const { exports } = await WebAssembly.instantiate(module, adaptedImports);
const memory = exports.memory || imports.env.memory;
function __liftString(pointer) {
if (!pointer) return null;
const
end = pointer + new Uint32Array(memory.buffer)[pointer - 4 >>> 2] >>> 1,
memoryU16 = new Uint16Array(memory.buffer);
let
start = pointer >>> 1,
string = "";
while (end - start > 1024) string += String.fromCharCode(...memoryU16.subarray(start, start += 1024));
return string + String.fromCharCode(...memoryU16.subarray(start, end));
}
return exports;
}
export const {
memory,
add,
} = await (async url => instantiate(
await (async () => {
try { return await globalThis.WebAssembly.compileStreaming(globalThis.fetch(url)); }
catch { return globalThis.WebAssembly.compile(await (await import("node:fs/promises")).readFile(url)); }
})(), {
}
))(new URL("output.wasm", import.meta.url)); "raw"
{
/// Omitted
"options": {
"bindings": "raw"
}
} results in export async function instantiate(module, imports = {}) {
const adaptedImports = {
env: Object.assign(Object.create(globalThis), imports.env || {}, {
abort(message, fileName, lineNumber, columnNumber) {
// ~lib/builtins/abort(~lib/string/String | null?, ~lib/string/String | null?, u32?, u32?) => void
message = __liftString(message >>> 0);
fileName = __liftString(fileName >>> 0);
lineNumber = lineNumber >>> 0;
columnNumber = columnNumber >>> 0;
(() => {
// @external.js
throw Error(`${message} in ${fileName}:${lineNumber}:${columnNumber}`);
})();
},
"console.log"(text) {
// ~lib/bindings/dom/console.log(~lib/string/String) => void
text = __liftString(text >>> 0);
console.log(text);
},
}),
};
const { exports } = await WebAssembly.instantiate(module, adaptedImports);
const memory = exports.memory || imports.env.memory;
function __liftString(pointer) {
if (!pointer) return null;
const
end = pointer + new Uint32Array(memory.buffer)[pointer - 4 >>> 2] >>> 1,
memoryU16 = new Uint16Array(memory.buffer);
let
start = pointer >>> 1,
string = "";
while (end - start > 1024) string += String.fromCharCode(...memoryU16.subarray(start, start += 1024));
return string + String.fromCharCode(...memoryU16.subarray(start, end));
}
return exports;
} By all means, we need I spent some time trying to make it all work, but so far to no avail. I am expecting a very busy week ahead, so I might not have time to work on this further in the upcoming days, but I am definitely planning to get back to it as soon as I have a chance to. |
Beta Was this translation helpful? Give feedback.
-
Hi @mischnic , I hope you are well. I finally had the time to try your latest suggestion. I have a working POC, it's in master
See the screenshots below: The solution is still super hairy and needs to be refactored quite a bit. I hope I'll have the time for it sometime soon. Would you recommend keeping the transformer in JS or is it a better practice to use TS? If you have any feedback or suggestions for improvements, please let me know! That would be highly appreciated. Kind regards. |
Beta Was this translation helpful? Give feedback.
-
I still need to work on populating |
Beta Was this translation helpful? Give feedback.
-
Inside the transformer function logger.log(">>> log");
logger.info(">>> info");
logger.warn(">>> warn");
logger.error(">>> error"); From the calls above only
What am I missing, please? I was planning to print the outputs and statistics from ASC to the terminal window using |
Beta Was this translation helpful? Give feedback.
-
I was trying to make the source maps work but to no avail so far. Judging by the look of the maps file, I assumed that ASC generates VLQ-maps. That, however, did not help me either. You can find a map generated by my example project here. If you have any ideas on how to make the maps work, please push me in the right direction. Thank you! |
Beta Was this translation helpful? Give feedback.
-
Hey @mischnic , I've been improving the quality of my TS code and bumped into a need to make these types manually. Is there maybe some package with TS-types already? Maybe there exists an automation with jscodeshift or something along those lines? If not, it's okay, I can live with a bunch of manually converted types :) |
Beta Was this translation helpful? Give feedback.
-
There does not seem to be an easy way to populate the fields below while handling an error from ASC: codeHighlights: [
{
message: "My message",
start: { line: 12, 12 },
end: { line: 12, column: 19 },
},
], ASC sends a pretty error message to A developer can see what exactly and where went wrong in the AssemblyScript code, so it should be good enough for now. If I ever find a way to get the compilation error details as an object, then I'll rework this solution in the proper Parcel way. |
Beta Was this translation helpful? Give feedback.
-
@mischnic , hi again. There's a problem which I don't yet understand but I am suspecting Parcel. Happens in the same branch we looked into earlier today. Once I run On NodeJS v19.3.0 and v19.8.1 that looks like this:
or like this:
Just in case I also tried it on NodeJS v18.15.0 (LTS). There it's even better:
Just in case I tried it with the Transformer in pure JS (checked out the last commit before I migrated to TypeScript) but the problem persisted. It's likely to be about this line. Do you by chance have any thoughts on what's wrong? Thank you. |
Beta Was this translation helpful? Give feedback.
-
Hi @mischnic , I hope you are well. Could you tell whether this line in the user's configuration file will need to stay and always be used to make the AssemblyScript transformer work? Or was it a temporary work-around and a proper solution will follow? Also, could you tell if you know when the next release of Parcel might happen? I am looking forward to having #8990 and hopefully #9009 available in a regular release version (i.e. not a nightly build). I understand it can depend on many things, but if you have any general indication, I'd highly appreciate that. Thank you. |
Beta Was this translation helpful? Give feedback.
-
Hey @mischnic , I hope you are doing well. Thank you for your answers, they really helped! I published parcel-transformer-assemblyscript-codument and created a template project to speed up things for the users. If I have time I'll see if I can work on a scaffolding tool to spare developers the need to download the template manually. Anyway, this is how things are at the moment. If you have time to check how it works, you're very welcome to do so. If not, it's also totally fine 🙂 Thanks once again for all your time and efforts! I don't know why https://www.npmjs.com/package/parcel-transformer-assemblyscript-codument shows just me as a collaborator, while github page shows you as a contributor. Anyway, I mentioned your role in the README under Contributors, I hope you don't mind. And as always, I am always open to any feedback should you happen to have any. |
Beta Was this translation helpful? Give feedback.
-
Hi @mischnic , I hope you are well. Could you tell whether there is a chance to get PR #9009 merged any time soon? Or was there maybe an alternative solution implemented? Thank you in advance. |
Beta Was this translation helpful? Give feedback.
-
Dear fellow developers,
May I humbly propose an idea that could benefit the community of TypeScript and AssemblyScript developers and WebAssembly enthusiast? It's about a plugin that would allow integration of AssemblyScript code into web projects using Parcel.
Some background info and my motivation
I experienced that firsthandedly: AssemblyScript can be a a very smooth and natural way for a TypeScript developer to step into the world of WebAssembly.
As they say on assemblyscript.org:
That definitely worked for me. In 2022 I managed to develop a WASM demo (as in the Demoscene) using AssemblyScript, there were some challenges along the way, but the familiar TS-like syntax and style of development helped greatly! If you have a couple of spare minutes, please feel free to see the demo (it contains some flashing images and music): https://wasm-demo.codument.com
I made my own build system for that project using
esbuild
,asc
(the AssemblyScript compiler) and good old shell scripts. I simply could not find an existing building solution that would suit my needs back then. But well, my build system is not too portable and it is suitable only for a small project, so I believe that a Parcel plugin could be a game changer for TS/AS/WASM developers like myself.Why I want it in Parcel?
Parcel already supports WebAssembly modules, having an "AssemblyScript to WebAssembly" compilation step added would make it just fantastic! I'd love to be able to start working on a new web project with AssemblyScript just by
yarn add
-ing Parcel and maybe a plugin or two.What I tried
Unfortunately, I could not find any tutorials on how to write plugins for Parcel 2. I tried looking at the source code of some other plugins, that gave me some idea but as @hirasso mentioned in their post:
I am feeling the same, being a complete newbie to Parcel I have no idea where to start
What I'm looking for here
I believe I know my way around the AssemblyScript compiler, its options and some not-so-obvious details on how to build WASM modules with it, so I can cover that part. I am hoping that someone with experience in writing plugins for Parcel 2 could lend me a hand in getting this project off the ground.
Thank you for your time, patience, and consideration. I look forward to reading your thoughts here.
Kind regards,
Leo
Beta Was this translation helpful? Give feedback.
All reactions