Support wasm files in Cloudflare build plugin

[wesbos]

I'm using a package - Shiki - which requires me to import a .wasm file since Cloudflare workers doesn't support inlined WASM binaries.

I've tried using this package: https://github.com/hadeeb/vite-plugin-wasm/ but since there seems to be multiple Vite build steps as part of the Waku build, the file gets put in the dist/public/assets/ directory, and it's not found in the dist/assets directory where it's needed for the server component.

Cloudflare recently added support for wasm files in their own vite plugin. So I am wondering if Waku can also get this functionality, or somehow use the cloudflare vite plugin?

cloudflare/workers-sdk#8031

[dai-shi]

Thanks for reporting. So, this is a Cloudflare-specific issue.

@rmarscher Can you help on this?

[dai-shi]

since there seems to be multiple Vite build steps as part of the Waku build

Yeah, it's the tricky part. This might be mitigated with Vite Environment API in the future, but not sure.
Our workaround for now is unstable_viteConfigs in waku.config.ts, but it may not work for all vite plugins which assume the single vite instance. cc @tylersayshi

[wesbos]

It looks like Cloudflare's own vite config is using the environment API.

Is there a way I can detect which build is running inside a vite plugin? Since I only need mine on the server, maybe just running it on that vite config would fix it

[dai-shi]

Is there a way I can detect which build is running inside a vite plugin?

Yes, for now, we have an unstable API.

import { unstable_getBuildOptions } from 'waku/server';

const phase = unstable_getBuildOptions().unstable_phase;

waku/packages/waku/src/server.ts

Lines 73 to 96 in bfae4ea

	type BuildOptions = {
	deploy?:
	| 'vercel-static'
	| 'vercel-serverless'
	| 'netlify-static'
	| 'netlify-functions'
	| 'cloudflare'
	| 'partykit'
	| 'deno'
	| 'aws-lambda'
	| undefined;
	unstable_phase?:
	| 'analyzeEntries'
	| 'buildServerBundle'
	| 'buildSsrBundle'
	| 'buildClientBundle'
	| 'buildDeploy'
	| 'emitStaticFiles';
	};
	// TODO tentative name
	export function unstable_getBuildOptions(): BuildOptions {
	return ((globalThis as any).__WAKU_BUILD_OPTIONS__ ||= {});
	}

[wesbos]

great thank you!

[rmarscher]

I use an argon2 wasm file for password hashing but I may have never actually tested it with Waku. I haven't been using password hashing much lately. I'll give a try when I have a moment. I trust that there's probably an issue though.

Let us know if you are able to find a workaround @wesbos. You can probably deploy a separate worker with the functions that need wasm and invoke it from your waku app worker using service bindings. That's a bunch of hoops to jump through but if find yourself fighting with Vite for server-side stuff, then it might be a good route to try. https://developers.cloudflare.com/workers/runtime-apis/bindings/service-bindings/

[wesbos]

Thanks - I guess I could put the renderer in another worker and call it via a service binding.

Though, I'd like to keep everything as "RSC" as possible without making it dependent on Cloudflare.

I was able to switch to the Regex renderer for now, but would like to switch back to the WASM renderer when possible.

[rmarscher]

I'd like to keep everything as "RSC" as possible without making it dependent on Cloudflare.

👍 I'll report back here with my findings.

[rmarscher]

When I tried my hashing function in the dev server, I get this error:

"ESM integration proposal for Wasm" is not supported currently. Use vite-plugin-wasm or other community plugins to handle this. Alternatively, you can use .wasm?init or .wasm?url. See https://vite.dev/guide/features.html#webassembly for more details.

I tried that workaround but I get an error with the dev server:

Error: Failed to parse URL from /src/server/lib/password/argon2-wasi.wasm?init

I also tried building it and running with wrangler dev and that gives a similar error:

✘ [ERROR] TypeError: Invalid URL: /assets/argon2-wasi-Caz-hM6E.wasm?init

      at initWasm

That is a known issue but the workaround uses node:fs and Cloudflare Workers can't access the filesystem.

We can use our Cloudflare Workers static assets binding to fetch from the public assets dir. It's hard to avoid some amount of Cloudflare-specific stuff in your app when deploying there because of the filesystem restrictions.

const c = getHonoContext();
const wasmModule = await c.env.ASSETS.fetch("argon2.wasm");

I have to put this down for a bit and focus on other deadlines that I have... but I'll circle back later. I'm curious if I can get it working with assets fetch.

This won't help people that are using a WASM file that is hidden away in another NPM dependency though - so we should also figure out vite-plugin-wasm. I'll try that too when I get a chance.

[hi-ogawa]

Currently cloudflare's wasm import semantics is a bit unique, so I'd imagine the support for this would require a special setup for now (also assuming their vite plugin is still not general enough for framework usage).

I have shiki wasm example in my rsc demo https://rsc-experiment.hiro18181.workers.dev/test/wasm and made a vite plugin for that purpose https://github.com/hi-ogawa/vite-plugins/tree/main/packages/server-asset#vitepluginwasmmodule. I ported my example to Waku and I think the same idea can work here too. Here is a code with some notes https://github.com/hi-ogawa/reproductions/tree/main/waku-1245-cloudflare-wasm and the deployed app https://waku-1245-cloudflare-wasm.hiro18181.workers.dev/.

One thing I found is that Waku currently moves .wasm assets from rsc build to client build, so I need to patch that part of the code for now to keep .wasm asset accessible for server code:

waku/packages/waku/src/lib/builder/build.ts

Lines 392 to 394 in 29af481

	const nonJsAssets = serverBuildOutput.output.flatMap(({ type, fileName }) =>
	type === 'asset' && !fileName.endsWith('.js') ? [fileName] : [],
	);

[dai-shi]

so I need to patch that part of the code for now to keep .wasm asset accessible for server code:

That makes sense. Would you please send a PR?

[hi-ogawa]

so I need to patch that part of the code for now to keep .wasm asset accessible for server code:

That makes sense. Would you please send a PR?

I think another safer way is to use copyFile instead rename since it's possible that server code wants to let client code fetch wasm via url for other cases. Also probably the same can be said for other assets than .wasm. What do you think @dai-shi?

waku/packages/waku/src/lib/builder/build.ts

Lines 441 to 445 in ec0c419

	for (const nonJsAsset of nonJsAssets) {
	const from = joinPath(rootDir, config.distDir, nonJsAsset);
	const to = joinPath(rootDir, config.distDir, DIST_PUBLIC, nonJsAsset);
	await rename(from, to);
	}

I tested with this patch and the demo is working for me.

[dai-shi]

Yeah, I'm fine with the safer approach (copy instead of rename) for all non-js assets!