Rewrite execution of microbatch models to avoid blocking the main thread#11332

Merged

QMalcolm merged 36 commits intomainfrom qmalcolm--11243-stop-microbatch-from-blocking-main-thread

Mar 3, 2025

+454-387

Commits on Feb 24, 2025

Add comments to help guide the work of pushing microbatch batch orchestration to a runner
QMalcolm
committed
Stub out new MicrobatchModelRunner and MicrobatchBatchRunner.
QMalcolm
committed
Make handle_job_queue of RunTask call invoke the MicrobatchModelRunner directly
QMalcolm
committed
Straight copy handle_microbatch_model into MicrobatchModelRunner
QMalcolm
committed
Convert handle_microbatch_model into runner execute type method
QMalcolm
committed
Begin getting batches in MicrobatchModelRunner
QMalcolm
committed
Begin getting _has_relation in execute method of MicrobatchModelRunner
QMalcolm
committed
Begin initializing empty RunResult for MicrobatchModelRunner
QMalcolm
committed
Make RunTask and ThreadPool available in MicrobatchModelRunner
QMalcolm
committed
Make merge_batch_results to MicrobatchModelRunner
QMalcolm
committed
Begin instantiating MicrobatchBatchRunner during _submit_batch
QMalcolm
committed
Move should_run_in_parallel to MicrobatchBatchRunner
QMalcolm
committed
Move describe_batch to MicrobatchBatchRunner
QMalcolm
committed
Move batch print eventing methods to MicrobatchBatchRunner
QMalcolm
committed
Move before_execute and after_execute to MicrobatchBatchRunner
QMalcolm
committed
Move describe_node to MicrobatchBatchRunner and MicrobatchModelRunner
QMalcolm
committed
Implement compile for MicrobatchBatchRunner and MicrobatchModelRunner
QMalcolm
committed
Implement on_skip for MicrobatchBatchRunner
QMalcolm
committed

Commits on Feb 26, 2025

Commits on Mar 3, 2025