Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

freethreaded support #193

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

freethreaded support #193

wants to merge 5 commits into from

Conversation

15r10nk
Copy link
Owner

@15r10nk 15r10nk commented Feb 6, 2025

solves #192

@15r10nk 15r10nk force-pushed the thread-safe branch 9 times, most recently from 716e539 to 4dff78e Compare February 7, 2025 17:22
@15r10nk 15r10nk force-pushed the thread-safe branch 2 times, most recently from 10ed989 to 799728a Compare February 8, 2025 21:59
@davidhewitt
Copy link

Thanks for working on this! If you have specific things you'd like help on, myself or @ngoldbaum may be of use :)

@15r10nk
Copy link
Owner Author

15r10nk commented Feb 11, 2025

I think the two remaining problems are the following:

@davidhewitt
Copy link

I guess for black, you might be able to just put a lock around that? Sadly warnings.catch_warnings is also not thread safe (I think there might be new APIs considered for 3.14).

Pydantic support is on the way, follow pydantic/pydantic#10483 - probably will be ready in ~1 month.

@15r10nk
Copy link
Owner Author

15r10nk commented Feb 14, 2025

I guess for black, you might be able to just put a lock around that?

This does not work. The problem is that another thread is writing something to sys.stdout at the same time the CliRunner is mocking it to capture the output of black.
I will have to use the internal api functions here. I just hope that I don't have to re-implement the config option parsing of black.

The following code shows the problem with stdout.

# /// script
# dependencies = [
#   "pytest",
#   "pytest-run-parallel"
# ]
# ///


from contextlib import redirect_stdout
from io import StringIO
import time


def test_redirect():
    text=StringIO()
    with redirect_stdout(text):
        print("hello")
        time.sleep(1)
        print("hello")

    assert text.getvalue()=="hello\nhello\n"


if __name__ == "__main__":
    import pytest
    pytest.main(["--parallel-threads=5",__file__])

contextlib.redirect_stdout is also not thread save for the same reason. It is modifying global state. I don't know if stdin/stdout can be fixed in the same way.

@ngoldbaum
Copy link

Maybe open a CPython bug? It seems reasonable to me for python to add locking for global state in the C standard library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants