what's an appropriate design pattern for combining embind + pthreads + async?

[dr-matt]

Hi, I am doing real-time data processing via wasm / c++ and looking to improve efficiency. What I'd like to have happen is this:

1. JS code instantiates stateful c++ class (wrapped using embind) at startup.
2. When a given data packet is received, JS passes it to the class instance via an embound class method.
3. That method returns a Promise immediately so as not to block the main thread.
4. Behind the scenes, the invoked c++ method spawns a thread to do the actual work (so that multiple sequential packets can be processed simultaneously), and when that work is done, it resolves the Promise object that was returned in step 3.

So in JS land, something like:

const processor = new MyModule.Processor();  // create instance of stateful class

while (getNextPacket()) {
  if (processor.isBusy()) continue;
  processor.processData(newData)  // invoke class method that returns a promise
  .then((result) => {
    doSomething(result);
  });
}

processor.delete();

In c++:

emscripten::val Processor::process_data(const emscripten::val& input_data) {
  auto future = std::async(std::launch::async, do_work, input_data);  // spawn thread to do work
  // TODO: return promise linked to the spawned task
}

emscripten::val do_work(const emscripten::val& input_data) {
  // ... heavy computation here
  // TODO: when done, somehow resolve the promise returned above
}

EMSCIRPTEN_BINDINGS(my_module) {
  emscripten::class_<Processor>("Processor")
    .constructor<>()
    .function("processData", &Processor::process_data)
  ;
}

Basically looking for pointers on how to accomplish the above in a correct and hopefully simple way, as there are interactions between JS async, pthreads, and embind class bindings.

Using the c++ future as written will block the main thread - what's the right pattern to avoid this? I'd like to keep all the thread operations in c++, rather than say creating a separate JS worker and manually passing messages back and forth; and since there is no main() here, the -sPROXY_TO_PTHREAD option isn't viable. Also, for performance reasons I'd like to avoid using asyncify if possible, and for simplicity I'd like to use the c++ standard library (std::thread, std::future, std::async, etc) rather than lower-level pthread or emscripten calls.

How can a JS Promise object be returned from process_data() ? I've had a look at @RReverser 's c++20 coroutine integration as described here, #20420 which is really interesting and possibly relevant for this use case, but I don't understand it well enough to know how/whether it can work in conjunction with threads.

Lastly - how can that Promise be resolved after the work is complete, so that the JS promise chain can continue execution?

Any help is appreciated. Thanks!

[sbc100]

You probably want to take a look the proxying API: https://emscripten.org/docs/api_reference/proxying.h.html

And also the C/C++ interface to JS promises: https://github.com/emscripten-core/emscripten/blob/main/system/include/emscripten/promise.h

In particular there is an API in promise.h called emscripten_proxy_promise which can be used to create a promise that resolves once the proxied work is complete. The return value from emscripten_proxy_promise is an integer handle but in you JS code you can call getPromise to get a the actual JS handle:

emscripten/src/library_promise.js

Line 12 in b5b7fed

$getPromise: (id) => promiseMap.get(id).promise,

@tlively am I missing anything here?

[dr-matt]

The examples in test_pthread_proxying.h contain other calls that I'm not currently including, such as emscripten_exit_with_live_runtime and emscripten_runtime_keepalive_push/pop. Are any of those needed in the above code? Also, what is the use case for the with_ctx version of emscripten_proxy_promise vs. without the context?

[tlively]

Since your worker thread is just spinning in a loop, there's no need to use those extra calls to keep the runtime alive. Those are for when you allow the worker thread main function to return so the worker returns to the event loop, but you still want to keep its runtime alive so that it will execute proxied work that comes in. It actually might be worth switching to this pattern because it is more efficient than having the worker spin in a loop.

The with_ctx versions of the proxying functions are for when the proxied work involves asynchronous JS, so it actually requires multiple turns of the worker's event loop to be finished. It lets the worker signal explicitly when the work is done rather than automatically considering the work to be done when the proxied function returns.

[dr-matt]

@tlively I think I understand. So I probably don't need the with_ctx calls, but the code may gain some efficiency from using the runtime related calls. I'll study those more to see if I can figure out how they work and how they should be incorporated in place of the spin loop.

[dr-matt]

Just getting back to this now - the one example I see of this model is in test_pthread_proxying.c:358:

em_promise_result_t
test_promise_returner(void** result, void* data, void* value) {
  widget* w16 = data;

  em_promise_t promise =
    emscripten_proxy_promise(proxy_queue, returner, do_run_widget, w16);

  *result = emscripten_promise_then(promise, check_widget, explode, w16);
  emscripten_promise_destroy(promise);

  return EM_PROMISE_MATCH_RELEASE;
}

The returner thread calls returner_main, which looks like:

void* returner_main(void* queue) {
 emscripten_err("returner_main");
 emscripten_exit_with_live_runtime();
}

As far as I can tell, the returner thread doesn't use the proxy_queue passed into emscripten_proxy_promise in this case, unlike the looper thread in that same file which is what I based my example on.

When I try this model, putting custom code in place of the emscripten_err call in returner_main, the function is called, but never returns from emscripten_exit_with_live_runtime() and that thread just hangs. So I don't quite see how all this works together. Is the thread function supposed to exit, and behind the scenes the corresponding worker is kept around for later somehow? Am I missing a function call or piece of the required logic?

[tlively]

The neat thing with this pattern is that the worker thread doesn't need to do anything to explicitly execute the proxied work. It will automatically wake up and execute the work as long as it returns to the JS event loop. emscripten_exit_with_live_runtime makes the worker thread stay alive while it exits the thread main function and returns to the event loop. When it is done with the work, it will return to the event loop and will be ready to execute the next proxied work that comes in until you explicitly force the thread to exit.

[sophonet]

Hi all,

This is a very interesting discussion. I have tried to implement the conclusions from the messages above that were posted last year, but the deref() function from @tlively did not compile: reinterpret_cast from 'val' to 'emscripten::val *' is not allowed here:

auto* ptr = reinterpret_cast<emscripten::val*>(co_await ptrPromise);

Since you @dr-matt have succeeded in implementing the deref() proposal, what have you done to avoid compilation problems and make this work? I am a little bit puzzled since the code in val.h indeed seems like co_await returns a val, and not a pointer to it (which makes it difficult to delete the val created on the heap in then's callback). Further debugging revealed that the val returned by co_await is a number and not a string, even though the populate_callback stores a string-val as result.

Thanks a lot,

Sophonet