[stretch][spike] Node.js library HTTP2 support in core Node HTTP2

[kmudduluru]

This ticket is to do a spike on what it takes to support HTTP2 in core Node. could be a long running transaction similar to what we did in GRPC. Describe the possible approach with pros and cons and share the findings with the team.

Description

Overall intent is to ensure the Node Agent HTTP instrumentation works properly with the HTTP 2 protocol.

[workato-integration]

https://new-relic.atlassian.net/browse/NR-341470

[kmudduluru]

Check this sheet prepared by Java agent team for reference and need similar doc for Node.js
Java doc - https://docs.google.com/spreadsheets/d/150ntwH2aO9Sr6mg0lVVkNScmLKSJpxNZXhqAoTHKo5U/edit?gid=1475390298#gid=1475390298
Node.js doc - https://docs.google.com/spreadsheets/d/1eKU-VCblbjoNmsepkNzXJegAxiqDB768_uszJKuLKdY/edit?gid=0#gid=0

[kmudduluru]

Understand how java agent team supported this.

[jsumners-nr]

Unlike Java, where there are multiple discrete HTTP implementations, there is only one HTTP implementation in Node.js: the one provided by core. As a result, all web frameworks targeting Node.js are basically wrappers on top of core's HTTP implementation. This is equally true for the HTTP/2 implementation.

Known state of HTTP/2 in the ecosystem:

1. OTEL does not support it add auto instrumentation of http2 module open-telemetry/opentelemetry-js#2667
2. Sentry does not support it https://docs.sentry.io/platforms/javascript/guides/node/configuration/integrations/http/
3. DataDog supports client and server, but they utilize a DiagnosticsChannel approach that makes things easier https://github.com/DataDog/dd-trace-js/tree/9329f6a2b06dda4586f415aa34cf9b0256fa2525/packages/datadog-plugin-http2/src
4. Elastic APM supports it by wrapping the stream's end method https://github.com/elastic/apm-agent-nodejs/blob/6d1665f0f5d968b8a79234c315a3aa71abca3a7d/lib/instrumentation/modules/http2.js

Our tracing model is built around the concept of a transaction:

1. a request is received
2. a "transaction" is started
3. work is performed and data collected
4. a response is sent
5. the transaction is ended

In step 3, all tracing data being collected is done so within the application's memory. Thus, if we were to take the same approach as Elastic, we would very likely end up generating a lot of in-memory data and receive reports that we are blowing up customer applications. If we had an event based model, like the DataDog implementation, we shouldn't see this issue.

It is my recommendation that we do not try to generically instrument Node.js's HTTP/2 module. If anything, I suggest we write documentation that shows people how to instrument specific HTTP/2 scenarios with our agent API.