using simulacrum from a compiled binary #72
Comments
|
This is excellent research @jbolda! It sounds like there is a clear winner in nexe.
I don't think that we need to do this yet. It's something we can consider down the road, especially around a given product. I say we get cracking with |
|
It seems that Through finding that out, I did find out about https://github.com/leafac/caxa and https://github.com/mongodb-js/boxednode. The former seems promising as well and does support ESM, but it newer relative to the others.
|
|
Looks like it doesn't have a problem using test file: binary creation: test run on test package.json: test run on file in unrelated folder: |
|
@jbolda What about using |
|
What do you guys think about using
|
|
This will definitely be a blocker at some point since we'll surely want a plugin system denoland/deno#8655 We could hope that it gets addressed by the time we need it, but the biggest concern I have is that in order to use |
|
Maybe we can just keep it in mind. |
Good question. I presume they should work, but it's easy to test so I'll try on Monday to be certain. |
|
Confirmed that they work. With this code: const fs = require("fs/promises");
const path = require("path");
const interact = async (args) => {
switch (args[2]) {
case "read":
return readFile(args);
case "require":
return requireFile(args);
case "import":
return importFile(args);
default:
return readFile(args);
}
};
const readFile = async (args) => {
const filePath = path.join(process.cwd(), args[3]);
const content = await fs.readFile(filePath, "utf-8");
console.log(content);
};
const requireFile = async (args) => {
const filePath = path.join(process.cwd(), args[3]);
const script = require(filePath);
script();
};
const importFile = async (args) => {
const filePath = `file://${path.join(process.cwd(), args[3])}`;
const { script } = await import(filePath);
script();
};
interact(process.argv);And running them:
|
Use Case
We are investigating building simulacrum as a binary. I believe this serves two primary purposes. The first is using without specific dev tools installed (or more specifically without nodejs). This could be used in the context of QA running a test, and starting a server for the tests to run against. The second use case would be to run the server without installing it as part of your nodejs app dev dependencies. It could reduce compile time or alleviate issues with transitive dependencies (although likely quite a rare use case).
Options
The are two primary paths that we can take in creating these. One option is a strictly terminal-based binary that is invoked and runs a server. One may interact with the server to possibly adjust the server state or invoke it with options. It may be more complicated to orchestrate multiple services though with this method. The other option is some sort of GUI and/or taskbar-based app that could act much in the vein of docker desktop where we can start or run multiple services.
In all of these options, Apple is becoming more strict about these being signed which can complicate a workflow. Also, if we are compiling nodejs, it is more likely that we will need to set up a matrix build system to compile these binaries (or use a specific version of node that has a precompiled version available).
Binary
Creating a single binary appears to be the easiest start. The history of javascript means that there are more options for compiling javascript into something that can run in a browser,
rollup(and any bundler) or alternatives likencccome to mind that could possibly bundle our code includingnode_modules. We want to invoke the binary without nodejs available though which then limits our options. nexe and pkg appear to be the only options remotely capable of this.I was able to get
nexeworking both from a precompiled version of nodejs (it downloads it if available andbuild: falseis set). I was also able to build from my system version after following the nodejs steps for building to get all of the prereqs. If we are building from a precompiled version (the latest v14 is14.15.3), it can handle cross compiling (using one OS to build everything), but it will need to be run on a Mac for the mac app signature. I installednexeat the root and ran this script from the root.I did also try
pkg, but every time I try it (both here and in past projects), I run into issues with how it does the resolution. It does appear to build correctly, but it throws errors when trying to run it.With this and
pkgnot seeming to be part of vercel's main business strategy, I suspect our best path forward would be usingnexe. Also to note, any packages that have a.nodeor a gyp compiled asset start to cause issues. In this case, I did upgrade effection to a version that has[email protected]to get then-apibased binary. It did not work prior to that with the.nodefile.GUI App
The big player is obviously Electron here, but it sounds like Tauri may also be an option. I think the big benefit we may gain is that it might be easier to run multiple services including having multiple versions of a service available (various projects could have various versions). Within Tauri, we could sidecar nodejs and bundle the services "browser style" with the expectation of being invoked by a node process (rather than compiling nodejs with each binary). This may make plugins easier and also reduce the possible issues involved with relying on a resolution algorithm to package node_modules and a virtual filesystem correctly.
Even if we would go this route, there is a case to be made to still bundle all of the binaries, but I suspect an option like rollup could bundle up a package into a single js file with less issues. There is also a switch happening with the support of native es-modules, and I can only imagine that will cause issues with
pkgdependence onrequire.@cowboyd do you think it is worth investigating and setting up a test for a GUI based version of this out of the gate?
The text was updated successfully, but these errors were encountered: