Removing libcrypt.so.1 from the whitelist breaks many builds

[dralley]

Credit to @codonell for helping me with this issue, and offering the fixes provided below.

Background

PR where libcrypt.so.1 was removed from the whitelist: #182

Rationale behind the change: pypa/manylinux#305 (comment)

Commentary on a new proposal to solve the issue: pypa/manylinux#305 (comment)

The Problem

Certain libraries on CentOS 7 are compiled against libcrypt.so.1 provided by glibc in CentOS 7. Since libcrypt.so.1 is no longer on the whitelist, any package that links against libraries that transitively depend upon libcrypt.so.1 will be unable to be repaired by auditwheel.

A libcrypt.so.2 is available on the build path, so any Python wheels being built directly against libcrypt will use that. But that does not solve the problem presented by transitive dependencies described above, which is possibly the larger issue of the two.

How to fix this?

The original rationale for this change was that because Fedora 30 dropped libcrypt.so.1 from the base system and replaced it with a separate package, libxcrypt-compat, it could no longer be relied on to exist on "all" platforms.

There are two potential solutions to this problem.

First, if the python3-pip package on Fedora (or any other distro which migrates to libxcrypt) would "require" libxcrypt-compat for the duration of the lifespan of manylinux2014, then libcrypt.so.1 would remain available for these packages, and allow them to continue working as-is.

Second, would be to build libcrypt.so.1 from libxcrypt, and make it directly avaialble only for vendoring. The compat libcrypt.so.1 built from libxcrypt has no dependencies on GLIBC_PRIVATE symbols and can be safely bundled. Is this a viable option given the vendoring infrastructure we have?

[hroncok]

First, if the python3-pip package on Fedora (or any other distro which migrates to libxcrypt) would "require" libxcrypt-compat for the duration of the lifespan of manylinux2014

It recommends it.

[hroncok]

However, only on x86:

# Some manylinux1 wheels need libcrypt.so.1.
# Manylinux1, a common (as of 2019) platform tag for binary wheels, relies
# on a glibc version that included ancient crypto functions, which were
# moved to libxcrypt and then removed in:
#  https://fedoraproject.org/wiki/Changes/FullyRemoveDeprecatedAndUnsafeFunctionsFromLibcrypt
# The manylinux1 standard assumed glibc would keep ABI compatibility,
# but that's only the case if libcrypt.so.1 (libxcrypt-compat) is around.
# This should be solved in the next manylinux standard (but it may be
# a long time until manylinux1 is phased out).
# See: https://github.com/pypa/manylinux/issues/305
# Note that manylinux is only applicable to x86 (both 32 and 64 bits)
%global crypt_compat_recommends() %{expand:
Recommends: (libcrypt.so.1()(64bit) if python%{1}(x86-64))
Recommends: (libcrypt.so.1 if python%{1}(x86-32))
}

We would adapt that for manylinux2014 architectures.

[mayeut]

Fixed by providing lib crypt.so.1 suitable for grafting in manylinux images.