Cannot run pytest-cov with ddtrace installed in the same project #11256

ddarricau · 2024-10-31T16:18:28Z

Summary
Basically I am unable to run test coverage on a project that includes both ddtrace and pytest-cov

Expected vs actual result
I should be able to do so

Reproducer
In order to reproduce I have a very simple project with

[tool.poetry.dependencies]
python = "^3.12"
pytest = "^8.3.3"
pytest-cov = "^6.0.0"
ddtrace = "2.12.0"

And a test file

def test_ddtrace_coverage():
    assert 1 == 1

Whenever I run the command COVERAGE_FILE=foo COVERAGE_CORE=sysmon pytest --cov=. tests/test_ddtrace_coverage.py I get the following error

  File "/Users/didierdarricau/Library/Caches/pypoetry/virtualenvs/ddtrace-coverage-test-5TsU0QvL-py3.12/lib/python3.12/site-packages/coverage/sysmon.py", line 226, in start
    sys_monitoring.use_tool_id(self.myid, "coverage.py")
ValueError: tool 1 is already in use

Note that this only happens when using the COVERAGE_CORE=sysmon flag, it seems that both ddtrace and pytest-cov request the same tool_id in that module. Note that using this flag is important to me as it significantly shortens the CI run time at my job.

I am not sure who should resolve the issue so I am also posting an issue on the pytest-cov repo.

Also not that if I use a ddtrace version < 2.12.0 it works fine.

The text was updated successfully, but these errors were encountered:

romainkomorn-exdatadog · 2024-11-11T09:03:26Z

Thanks for the report, @ddarricau .

We're winning the race to sys_monitoring.use_tool_id() with our own call as a result of imports, but the main issue is that we shouldn't be calling use_tool_id() unless we're actually collecting coverage.

@vitor-de-araujo or I should refactor the code so that we only call use_tool_id() if instrument_all_lines() is actually called.

ddarricau · 2024-11-13T08:38:33Z

Great thank you !

ddarricau · 2025-01-13T08:52:06Z

Hello could we get an update on when you might look at this ?
Thanks !

pablogamboa · 2025-02-25T11:05:17Z

Hi @vitor-de-araujo any progress in this issue? We're also affected by it, not being able to use sysmon coverage is a bummer. It slows down test collection in a large test suite a whooping 200%. Thank you

On Python 3.12+, CI Visibility / Test Optimization uses `sys.monitoring` to gather code coverage information. The problem is that if other tools, such as `pyest-cov`, also try to register a `sys.monitoring` tool for coverage, an exception will be raised because Python only allows one handler per tool type (see issue #11256). This PR postpones tool registering until the first call to `instrument_all_lines()`, giving other tools the opportunity to register themselves; in this case, however, we do not gather code coverage information. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

vitor-de-araujo · 2025-02-28T20:04:21Z

Hi, @pablogamboa and @ddarricau! I have just merged a PR to fix this. It should come out in the next ddtrace releases, within the next couple of weeks. Sorry for the delay in addressing this issue!

On Python 3.12+, CI Visibility / Test Optimization uses `sys.monitoring` to gather code coverage information. The problem is that if other tools, such as `pyest-cov`, also try to register a `sys.monitoring` tool for coverage, an exception will be raised because Python only allows one handler per tool type (see issue #11256). This PR postpones tool registering until the first call to `instrument_all_lines()`, giving other tools the opportunity to register themselves; in this case, however, we do not gather code coverage information. - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) (cherry picked from commit 7074375)

pablogamboa · 2025-03-04T16:38:26Z

amazing news thanks @vitor-de-araujo ❤

…port 3.0] (#12577) Backport 7074375 from #12558 to 3.0. On Python 3.12+, CI Visibility / Test Optimization uses `sys.monitoring` to gather code coverage information. The problem is that if other tools, such as `pyest-cov`, also try to register a `sys.monitoring` tool for coverage, an exception will be raised because Python only allows one handler per tool type (see issue #11256). This PR postpones tool registering until the first call to `instrument_all_lines()`, giving other tools the opportunity to register themselves; in this case, however, we do not gather code coverage information. - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) (cherry picked from commit 7074375) Co-authored-by: Federico Mon <[email protected]>

…port 3.1] (#12579) Backport 7074375 from #12558 to 3.1. On Python 3.12+, CI Visibility / Test Optimization uses `sys.monitoring` to gather code coverage information. The problem is that if other tools, such as `pyest-cov`, also try to register a `sys.monitoring` tool for coverage, an exception will be raised because Python only allows one handler per tool type (see issue #11256). This PR postpones tool registering until the first call to `instrument_all_lines()`, giving other tools the opportunity to register themselves; in this case, however, we do not gather code coverage information. - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) (cherry picked from commit 7074375) Co-authored-by: Federico Mon <[email protected]>

…port 2.21] (#12578) Backport 7074375 from #12558 to 2.21. On Python 3.12+, CI Visibility / Test Optimization uses `sys.monitoring` to gather code coverage information. The problem is that if other tools, such as `pyest-cov`, also try to register a `sys.monitoring` tool for coverage, an exception will be raised because Python only allows one handler per tool type (see issue #11256). This PR postpones tool registering until the first call to `instrument_all_lines()`, giving other tools the opportunity to register themselves; in this case, however, we do not gather code coverage information. - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) (cherry picked from commit 7074375)

vitor-de-araujo · 2025-03-24T09:45:33Z

The fix is available in ddtrace >=3.2.0 and also has been backported to 2.21.4. I'm closing this issue, but let me know if you run into any issues.

ddarricau mentioned this issue Oct 31, 2024

Cannot run pytest-cov with ddtrace installed in the same project pytest-dev/pytest-cov#665

Closed

romainkomorn-exdatadog added the CI App label Nov 11, 2024

romainkomorn-exdatadog assigned vitor-de-araujo Nov 11, 2024

github-actions bot added the stale label Jan 13, 2025

github-actions bot removed the stale label Jan 14, 2025

vitor-de-araujo mentioned this issue Feb 27, 2025

fix(ci_visibility): avoid clash with other sys.monitoring tools #12558

Merged

2 tasks

vitor-de-araujo mentioned this issue Feb 28, 2025

fix(ci_visibility): avoid clash with other sys.monitoring tools [backport 3.0] #12577

Merged

2 tasks

vitor-de-araujo mentioned this issue Feb 28, 2025

fix(ci_visibility): avoid clash with other sys.monitoring tools [backport 2.21] #12578

Merged

2 tasks

vitor-de-araujo mentioned this issue Feb 28, 2025

fix(ci_visibility): avoid clash with other sys.monitoring tools [backport 3.1] #12579

Merged

2 tasks

vitor-de-araujo closed this as completed Mar 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot run pytest-cov with ddtrace installed in the same project #11256

Cannot run pytest-cov with ddtrace installed in the same project #11256

ddarricau commented Oct 31, 2024 •

edited

Loading

romainkomorn-exdatadog commented Nov 11, 2024

ddarricau commented Nov 13, 2024

ddarricau commented Jan 13, 2025

pablogamboa commented Feb 25, 2025

vitor-de-araujo commented Feb 28, 2025

pablogamboa commented Mar 4, 2025

vitor-de-araujo commented Mar 24, 2025

Cannot run pytest-cov with ddtrace installed in the same project #11256

Cannot run pytest-cov with ddtrace installed in the same project #11256

Comments

ddarricau commented Oct 31, 2024 • edited Loading

romainkomorn-exdatadog commented Nov 11, 2024

ddarricau commented Nov 13, 2024

ddarricau commented Jan 13, 2025

pablogamboa commented Feb 25, 2025

vitor-de-araujo commented Feb 28, 2025

pablogamboa commented Mar 4, 2025

vitor-de-araujo commented Mar 24, 2025

ddarricau commented Oct 31, 2024 •

edited

Loading