NeonBackend how to support multi thread parallel inference

[xiaotongnii]

NeonBackend how to support multi thread inference。
support multi thread parallel in Neon operator，like onnxruntime IntraOpNumThreads；
Or Others

NeonBackendModelContext::NeonBackendModelContext(const ModelOptions& modelOptions)
    : m_IsFastMathEnabled(false), m_NumberOfThreads(0)
{
   if (!modelOptions.empty())
   {
       ParseOptions(modelOptions, "CpuAcc", [&](std::string name, const BackendOptions::Var& value)
       {
           if (name == "FastMathEnabled")
           {
               m_IsFastMathEnabled |= ParseBool(value, false);
           }
           if (name == "NumberOfThreads")
           {
               m_NumberOfThreads |= ParseUnsignedInt(value, 0);
           }
       });
   }
}

bool NeonBackendModelContext::IsFastMathEnabled() const
{
    return m_IsFastMathEnabled;
}

unsigned int NeonBackendModelContext::GetNumberOfThreads() const
{
    return m_NumberOfThreads;
}

[orlmon01]

Hiya Xiaotongnii,
Yes, if you want to do multithread inference you can set it in the NeonBackendModelContext by changing the number of threads from the default 0 to whatever you want it to use. This will be passed down into the Neon backend and used wherever possible to multithread the ops.