A generic way of starting task runners

[RealOrangeOne]

It would be nice if there was a single backend-agnostic API for starting the background worker process. Switching backends will currently require changing the command to start the runner, and perhaps understanding all of its command-line configuration.

Ideally, there would be a single API to start the worker, and the backend could define how this might actually work. For example:

./manage.py start_task_runner ...
./manage.py start_task_runner --backend=backend-1 ...

[RealOrangeOne]

As an extension to this, it's likely that some backends are really more of "queue store" implementations (eg SQS). In those cases, the backend would end up duplicating much of the code for the workers, even though very little has changed.

Therefore, as part of this, there might also need to be an API (exactly how that looks I don't know) for a backend to specify that it can use the built-in worker implementation, and just pass tasks and results around. This way, there's less code to duplicate and maintain.