Create Recipes to use Feluda and its operators in elixir codebases

[dennyabrain]

Overview

We've been using Elixir for many web applications at Tattle. Being able to invoke feluda operators directly from an elixir function increases the kind of thing we can do via a web interface. Coupled with Elixir's distributed and resilient nature and support for job queues makes it possible to schedule and monitor long running tasks, retries and cancellations. It also allows us to tap into the ML capabilities of the python ecosystem into our web apps.

References

• Elixir/Erlang has a notion of Ports, that allow you to communicate with an OS process. Here's a tutorial on it. A limitation of this is that its too low level, in that one has to worry about string parsing and formatting responses. Any third party library that solves our problem will most likely use Ports in the background, so it helps to understand how it works.
• Erlport allows calling python functions from elixir.
• A blog post explaining a practical implementation using Erlport.
• There is an alternative project called Numerical Elixir, which is recreating support for numerical computing in elixir. They have equivalents of Huggingface, pandas, scikit etc in pure elixir which are apparently as performant. It makes your code base to be purely elixir, and might be worth pursuing, but its out of scope for this issue.

Scope

• Create a working prototype of invoking a simple feluda operator (one without requiring installation of system dependencies like ffmpeg) from elixir/phoenix project
• We will have to find a way to dockerize this solution as well. Given that we have functioning Dockerfile for our phoenix apps and given the simplifications we have done to feluda installation process, it might be worth it to figure out the minimal lines of (operator specific) lines of code one must add to this Dockerfile to support feluda in a phoenix app.