[FEATURE]: Ability to hide queryString from generated spans

[sdwa0]

Describe the goal of the feature

When we switched from an old version of ingress-nginx instrumented with nginx-opentracing module to using nginx-datadog , we noticed query parameters are appended to the http.url span attribute. Also, there is a new span attribute http.url_details.queryString that contains the list of all query parameter.

I am not sure if this is due to the switch to nginx-datadog though. I couldn't find a related difference in code between nginx-opentracing and nginx-datadog.

Regardless, an ability to hide query string from generated spans would be useful. There could be services that write PII data on query string, even though that is a bad security practice. An ability to hide it will avoid that from surfacing on Datadog.

Is your feature request related to a problem?

No response

Describe alternatives you've considered

No response

Additional context

No response

[dmehala]

Hello @sdwa0,

Thank you for bringing this to our attention.

From what I understand, the module sends the full URL here, and the agent (or intake?) split it. I’m also surprised that query strings are not obfuscated by default, especially because the agent obfuscate some span metadata by default since v7.48.0.

I’d recommend reviewing the documentation on Trace Obfuscation here. From what I understand, you may need to update the agent configuration with the following:

apm_config:
  enabled: true
  obfuscation:
    http:
      ## Enables obfuscation of query strings in URLs. Disabled by default.
      remove_query_string: true
      remove_paths_with_digits: true

I hope this solves your issue. Keep me posted.

Thank you.

[sdwa0]

Thanks, @dmehala for the reply.

The old ingress-nginx version we were using was instrumented with nginx-opentracing module v0.19.0. From the code of the old module, it seems to have been sending the full URL too.

If that's the case, it is strange that full URLs with query strings started showing up after the switch to nginx-datadog. FWIW, no agent version upgrades were done around the switch, and obfuscation of query strings was never enabled.

What might be the cause of the difference in the behavior?

[dmehala]

You're right @sdwa0, the old module source code is indeed identical. The discrepancy may be due to how the Datadog agent or intake processes the incoming trace. Unfortunately, I don’t have the bandwidth to investigate this further at the moment.

I strongly recommend opening a support ticket to get a deeper investigation and resolution. They should be able to provide more insights into what might be causing this behavior.

Let me know if there's anything else I can help with!

[sdwa0]

I strongly recommend opening a support ticket to get a deeper investigation and resolution. They should be able to provide more insights into what might be causing this behavior.

Noted. Thanks for taking a look, @dmehala. I have opened a support request. I will update here if anything is discovered there.

[sdwa0]

I noticed dd-trace libraries for a few other languages provide a way to avoid sending query strings in http.url using the DD_HTTP_SERVER_TAG_QUERY_STRING environment variable. It would be nice to add that support to dd-trace-cpp too.

[dmehala]

You may have noticed that not all dd-trace libraries are identical in functionality. While I understand DD_HTTP_SERVER_TAG_QUERY_STRING could address your needs, I believe that adding more complex logic to the tracer itself (which has been the default too many times) isn't the best approach. Instead, the responsibility should be on the user of the tracer to adapt to different use cases. In this context, it is more appropriate for nginx-datadog to provide an option for sending URLs with or without query strings. I plan to implement this soon, I'll keep you posted :)

Thank you for bringing this up.