Add a benchmark/example for numexpr usage under free-threading conditions

[andfoy]

Fixes #503

[andfoy]

@FrancescAlted, it seems that this lock is a performance bottleneck when multiple threads are ran in parallel, is it necessary?

[andfoy]

With the lock:

Benchmarking Expression 1:
NumPy time (threaded over 32 chunks with 16 threads): 3.276298 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 9.975059 seconds
numexpr speedup: 0.33x
----------------------------------------
Benchmarking Expression 2:
NumPy time (threaded over 32 chunks with 16 threads): 18.981946 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 50.327974 seconds
numexpr speedup: 0.38x
----------------------------------------
Benchmarking Expression 3:
NumPy time (threaded over 32 chunks with 16 threads): 20.414158 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 70.900648 seconds
numexpr speedup: 0.29x
----------------------------------------
Benchmarking Expression 4:
NumPy time (threaded over 32 chunks with 16 threads): 38.012808 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 160.476216 seconds
numexpr speedup: 0.24x
----------------------------------------

[andfoy]

Without locking:

Benchmarking Expression 1:
NumPy time (threaded over 32 chunks with 16 threads): 3.415349 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 2.618876 seconds
numexpr speedup: 1.30x
----------------------------------------
Benchmarking Expression 2:
NumPy time (threaded over 32 chunks with 16 threads): 19.005238 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 12.611407 seconds
numexpr speedup: 1.51x
----------------------------------------
Benchmarking Expression 3:
NumPy time (threaded over 32 chunks with 16 threads): 20.555149 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 17.690749 seconds
numexpr speedup: 1.16x
----------------------------------------
Benchmarking Expression 4:
NumPy time (threaded over 32 chunks with 16 threads): 38.338372 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 35.074684 seconds
numexpr speedup: 1.09x
----------------------------------------

[FrancescAlted]

Uh, that was introduced long ago, so I don't remember at all; but if it is there, I'd say that it is necessary, yes. Just to double check, what happens to the test suite if you remove the lock?

[andfoy]

Tests are passing locally and on CI

[FrancescAlted]

Yep, I did my tests locally on a series of packages that depend on numexpr, and everything seems fine with removing the lock. So, feel free in proceeding with the lock removal.

[andfoy]

Maybe the lock was in place to prevent concurrent access to the caches, which shouldn't be much of an issue now, given they were made thread-local

[FrancescAlted]

FWIW, this is what I am seeing with my regular python:

(py3.13) faltet@bigdaddy:~/blosc/numexpr$ python bench/free_threading.py
Benchmarking Expression 1:
NumPy time (threaded over 32 chunks with 16 threads): 1.173090 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 0.951071 seconds
numexpr speedup: 1.23x
----------------------------------------
Benchmarking Expression 2:
NumPy time (threaded over 32 chunks with 16 threads): 10.410874 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 8.248753 seconds
numexpr speedup: 1.26x
----------------------------------------
Benchmarking Expression 3:
NumPy time (threaded over 32 chunks with 16 threads): 9.605909 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 11.087108 seconds
numexpr speedup: 0.87x
----------------------------------------
Benchmarking Expression 4:
NumPy time (threaded over 32 chunks with 16 threads): 3.836962 seconds
numexpr time (threaded with re_evaluate over 32 chunks with 16 threads): 18.054531 seconds
numexpr speedup: 0.21x
----------------------------------------

I suppose that running against Python 3.13t (with free-threading enabled), will reproduce your conditions. Could you please update the instructions at the beginning of the benchmark so as to stress this out? Also, adding a link to a free-threaded Python explanation/discussion would be great.