Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add shared config to build Python shared library #795

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

adehad
Copy link

@adehad adehad commented Jan 9, 2024

I was trying to use this devcontainer with PyInstaller, and it seems like the build fails due to the Python installation not built with the shared libraries.

Summary of Changes

  1. Provides a new option to control the build flag for the shared libraries

Concerns / Thoughts

  1. Do we want to preserve the original behaviour (ie have the option as false)
  2. Do we want to do something special regarding the MacOS specific commands

Details

Here are the relevant logs

    run()
  File "/workspaces/pyside6-boilerplate/.venv/lib/python3.12/site-packages/PyInstaller/__main__.py", line 198, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/workspaces/pyside6-boilerplate/.venv/lib/python3.12/site-packages/PyInstaller/__main__.py", line 69, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/workspaces/pyside6-boilerplate/.venv/lib/python3.12/site-packages/PyInstaller/building/build_main.py", line 1071, in main
    build(specfile, distpath, workpath, clean_build)
  File "/workspaces/pyside6-boilerplate/.venv/lib/python3.12/site-packages/PyInstaller/building/build_main.py", line 1011, in build
    exec(code, spec_namespace)
  File "app.spec", line 13, in <module>
    a = Analysis(
        ^^^^^^^^^
  File "/workspaces/pyside6-boilerplate/.venv/lib/python3.12/site-packages/PyInstaller/building/build_main.py", line 470, in __init__
    self.__postinit__()
  File "/workspaces/pyside6-boilerplate/.venv/lib/python3.12/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
  File "/workspaces/pyside6-boilerplate/.venv/lib/python3.12/site-packages/PyInstaller/building/build_main.py", line 581, in assemble
    raise PythonLibraryNotFoundError()
PyInstaller.exceptions.PythonLibraryNotFoundError: Python library not found: libpython3.12.so, libpython3.12.so.1.0
    This means your Python installation does not come with proper shared library files.
    This usually happens due to missing development package, or unsuitable build parameters of the Python installation.

    * On Debian/Ubuntu, you need to install Python development packages:
      * apt-get install python3-dev
      * apt-get install python-dev
    * If you are building Python by yourself, rebuild with `--enable-shared` (or, `--enable-framework` on macOS).

@adehad adehad requested a review from a team as a code owner January 9, 2024 02:09
@adehad
Copy link
Author

adehad commented Jan 9, 2024

@microsoft-github-policy-service agree

if [ "${SHARED_LIBS}" = "true" ]; then
shared_libs="--enable-shared"
fi
./configure --prefix="${INSTALL_PATH}" --with-ensurepip=install ${config_args} ${shared_libs}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
./configure --prefix="${INSTALL_PATH}" --with-ensurepip=install ${config_args} ${shared_libs}
./configure --prefix="${INSTALL_PATH}" --with-ensurepip=install ${config_args} --enable-shared

Perhaps this is a feature we want by default and no configuration option is needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you happen to know the performance consequences of adding --enable-shared? I suspect that this would be to have as a default (perhaps inverting the option to let us disable) as long as it doesn't significantly increase the (already long) compile time

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately I do not, the 'default' Python Docker compilation includes other flags that might improve performance however?

https://github.com/docker-library/python/blob/c007698b9f91bedba5e6300d9394c10acd194dcb/3.11/bookworm/Dockerfile#L44-L53

@joshspicer
Copy link
Member

joshspicer commented Jan 17, 2024

Would you mind adding a scenario test to exercise your new flag (docs)?

@adehad
Copy link
Author

adehad commented Jan 17, 2024

Would you mind adding a scenario test to exercise your new flag (docs)?

Of course, my bad for missing this.

any preference for using an ldd command (nix specific)

root@c098cfd1b2e0:/workspaces/python# ldd $(which python)
        linux-vdso.so.1 (0x00007fff0a3fb000)
        libpython3.12.so.1.0 => /usr/local/bin/../lib/libpython3.12.so.1.0 (0x00007f43580d5000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4357ef8000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4357ed6000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4357ed0000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f4357ecb000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4357d87000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f4358706000)

When not built with shared looks like:

['/usr/local/python/3.10.13/lib', 'libpython3.10.a']

or a Python only

root@c098cfd1b2e0:/workspaces/python# python -c "import sysconfig; print(sysconfig.get_config_vars('LIBDIR', 'INSTSONAME'))" 
['/usr/local/lib', 'libpython3.12.so.1.0']

I'll grep either with option with | grep 'libpython3.*so' in the check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants