Iterate callstack API #4033
Iterate callstack API #4033
Conversation
g0djan
requested review from
loganek,
lum1n0us,
no1wudi,
TianlongLiang,
wenyongh,
xujuntwt95329 and
yamt
as code owners
|
@loganek addressed all your comments and rebased to fix the checks. The last failing check is CI issue and there's another PR that will fix it. |
|
@lum1n0us could you take a look at this PR? |
|
@yamt could you please take a look at this PR? |
| * interruption from another thread if next variables hold valid pointers | ||
| * - exec_env | ||
| * - exec_env->module_inst | ||
| * - exec_env->module_inst->module |
There was a problem hiding this comment.
IMO, making this complex functionality async-signal-safe is too much maintenance burden, especially when wamr itself doesn't rely on the property at all.
given that you need to suspend the target thread anyway, why don't you call this from another thread?
There was a problem hiding this comment.
async-signal-safe is too much maintenance burden
Yeah, we understand that and I kept it simple as much as possible. Basically non async-signal-safe implementation would be different only in a few checks removed and there wouldn't be comments that I added in the code.
Particularly this comment about validity of pointers is a theoretical problem atm. We don't know any platform yet where updating pointer variable might be interrupted by a signal, possibly after launch we will see that it never happens.
given that you need to suspend the target thread anyway, why don't you call this from another thread?
Do you mean using wasm_cluster_suspend_thread?
I tried that and there're 2 problems for us:
- Now there’s no awaiting till thread actually gets suspended
- Suspension happens only after certain checks so we're not getting stacktraces that we need. E.g. if there's sleep somewhere, there won't be sleep in stacktrace reported, it'd report calls after sleep has finished. If there's a deadlock stacktrace won't be reported at all
New WAMR public API to iterate over the runtime call stack frames and execute a user defined callback on those.
To make the most use of it use next APIs inside of the callback.
CAUTION: this APIs is not thread safe and not intended to be. If you need to call it from another thread ensure the passed exec_env is suspended.
Our use case
Sometimes WAMR runtime gets stuck in production and we have no data where in the code compiled to WASM it happens. We currently only track such situations in a separate native thread. To increase visibility into the problem we developed internal solution that requires presence of this API in WAMR. If a separate thread finds that the WASM VM thread has stuck, it interrupts it with a user defined signal and calls this API to collect callstack. The main complexity is maintaining async-signal-safety and avoiding segfaults. For that we're maintaining atomic copies of
exec_env,exec_env->module_inst,exec_env->module_inst->module. Those copies are always set to NULL before the referenced memory is freed. Before a call to this API those copies are always checked for validity. In our use case scenario we guarantee ourselves only absence of crashes but we realize that the frame data that we collect might be invalidated due to a signal interruption. However it's highly unlikely and is not a concern for us.Have we tried existing WAMR APIs for our usecase?
Yes, we've tried suggested by maintainers
wasm_cluster_suspend_threadandwasm_runtime_terminate.wasm_cluster_suspend_threaddoesn't suit us either. Even if it did we'd still need API to iterate over stackframes.