Connection to Application Insights breaks on using PFClient.run() versus flow-as-a-function

[Flowhill]

Discussed in #3921

^{Originally posted by Flowhill January 14, 2025}
I'm having trouble when switching starting my flow using the Promptflow SDK from flow-as-a-function to PFClient.run()
I want to start using the PFClient because it allows for logging the system metrics such as the prompt_tokens and completions_tokens.

I initialize my Application insights through this code

if os.getenv("APPLICATIONINSIGHTS_CONNECTION_STRING"):
    configure_azure_monitor(
        connection_string=os.getenv("APPLICATIONINSIGHTS_CONNECTION_STRING")
    )

And in the instance of flow-as-a-function I run the flow like this

flow = 'path/to/flow/folder'
f = load_flow(flow)
utterance = request.message.content
try:
    if contains_json_html_structure(utterance):
        raise ValueError("Utterance contains json/html-like values")  # noqa: E501
    # post the request and get the result
    result = f(
        webhook_input=request.webhook_input,
        session_id=completion_input.context.message.session_id,
        role=completion_input.context.message.role,
        # Filter JSON elements out of utterance
        utterance=utterance,
    )

    result = result["output"]
    # Log results for application insights
    logger.info(json.dumps(result))
except Exception as e:
    <....>

In this case all my logs both in python code in the flow as well as after get neatly uploaded to Application Insights

If I replace the flow-as-a-function with a PFClient.run() like so:

flow = 'path/to/flow/folder'
pf = PFClient()
<... preparing data.jsonl file ...>
# Run the flow
run = pf.run(
    flow=flow,
    data="data.jsonl",
    column_mapping={
        "webhook_input": "${data.webhook_input}",
        "session_id": "${data.session_id}",
        "role": "${data.role}",
        "utterance": "${data.utterance}",
    },
)

logger.debug(pf.get_details(run))

Then the logs stop as soon as the line run = pf.run(... starts. What also changes in the output in terminal is the following additions:

• Multiple instances of trace detail URL being created as such: You can view the trace detail from the following URL: http://127.0.0.1:23336/v1.0/ui/traces/?#collection=src&uiTraceId=0xb0e9328dedd7df8f0e29fe305d371f24 with differing TraceId's
• At the beginning and end of a flow this ERROR appears: 2025-01-14 15:30:45 +0100 azure.monitor.opentelemetry.exporter.export._base:_transmit ERROR Non-retryable server side error: Operation returned an invalid status 'Bad Request'.

Package versions for promptflow and telemetry related packages is:

promptflow                               1.17.1
promptflow-azure                         1.17.1
promptflow-core                          1.17.1
promptflow-devkit                        1.17.1
promptflow-tools                         1.4.0
promptflow-tracing                       1.17.1
openai                                   1.59.7
azure-core-tracing-opentelemetry         1.0.0b11
azure-monitor-opentelemetry              1.6.4
azure-monitor-opentelemetry-exporter     1.0.0b32
opentelemetry-api                        1.29.0
opentelemetry-exporter-otlp-proto-common 1.29.0
opentelemetry-exporter-otlp-proto-http   1.29.0
opentelemetry-instrumentation            0.50b0
opentelemetry-instrumentation-asgi       0.50b0
opentelemetry-instrumentation-dbapi      0.50b0
opentelemetry-instrumentation-django     0.50b0
opentelemetry-instrumentation-fastapi    0.50b0
opentelemetry-instrumentation-flask      0.50b0
opentelemetry-instrumentation-psycopg2   0.50b0
opentelemetry-instrumentation-requests   0.50b0
opentelemetry-instrumentation-urllib     0.50b0
opentelemetry-instrumentation-urllib3    0.50b0
opentelemetry-instrumentation-wsgi       0.50b0
opentelemetry-proto                      1.29.0
opentelemetry-resource-detector-azure    0.1.5
opentelemetry-sdk                        1.29.0
opentelemetry-semantic-conventions       0.50b0
opentelemetry-util-http                  0.50b0

I have tried to configure Application Insights through the flow.dag.yaml as well by adding all combinations of

app_insights_enabled: true
environment_variables:
  APPLICATIONINSIGHTS_CONNECTION_STRING: <connection_string>

None of which seem to make a difference.

The flow and surrounding code is started through a call to a fastapi endpoint.

[github-actions]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!