lmdb #658

egemenakbal · 2022-08-07T11:14:02Z

Building wheels for collected packages: lmdb, gfpgan, pyyaml, future, filterpy, MarkupSafe
  Building wheel for lmdb (setup.py): started
  Building wheel for lmdb (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /Users/egemenakbal/opt/miniconda3/envs/holistic/bin/python3.8 -u -S -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-install-dd5z4olu/lmdb/setup.py'"'"'; __file__='"'"'/private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-install-dd5z4olu/lmdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-wheel-fiq9m22b --python-tag cp38
       cwd: /private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-install-dd5z4olu/lmdb/
  Complete output (17 lines):
  py-lmdb: Using bundled liblmdb with py-lmdb patches; override with LMDB_FORCE_SYSTEM=1 or LMDB_PURE=1.
  patching file lmdb.h
  patching file mdb.c
  py-lmdb: Using CPython extension; override with LMDB_FORCE_CFFI=1.
  running bdist_wheel
  running build
  running build_py
  creating build/lib.macosx-10.9-x86_64-3.8
  creating build/lib.macosx-10.9-x86_64-3.8/lmdb
  copying lmdb/cffi.py -> build/lib.macosx-10.9-x86_64-3.8/lmdb
  copying lmdb/__init__.py -> build/lib.macosx-10.9-x86_64-3.8/lmdb
  copying lmdb/_config.py -> build/lib.macosx-10.9-x86_64-3.8/lmdb
  copying lmdb/tool.py -> build/lib.macosx-10.9-x86_64-3.8/lmdb
  copying lmdb/__main__.py -> build/lib.macosx-10.9-x86_64-3.8/lmdb
  running build_ext
  building 'cpython' extension
  error: CCompiler.compile: Chaquopy cannot compile native code
  ----------------------------------------
  ERROR: Command errored out with exit status 1:

  error: CCompiler.compile: Chaquopy cannot compile native code

  ERROR: Command errored out with exit status 1: /Users/egemenakbal/opt/miniconda3/envs/holistic/bin/python3.8 -u -S -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-install-dd5z4olu/lmdb/setup.py'"'"'; __file__='"'"'/private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-install-dd5z4olu/lmdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-wheel-fiq9m22b --python-tag cp38 Check the logs for full command output.
  ERROR: Command errored out with exit status 1: /Users/egemenakbal/opt/miniconda3/envs/holistic/bin/python3.8 -u -S -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-install-dd5z4olu/lmdb/setup.py'"'"'; __file__='"'"'/private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-install-dd5z4olu/lmdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/66/t9mg3jhn7k5cfq28w9nbdmgm0000gn/T/pip-wheel-fiq9m22b --python-tag cp38 Check the logs for full command output.

ERROR: Failed to install lmdb from https://files.pythonhosted.org/packages/ed/61/41f3c7cbd8a67202ef24fad3375ed936093a0547dc645581dd11c09581b7/lmdb-1.3.0.tar.gz.
For assistance, please raise an issue at https://github.com/chaquo/chaquopy/issues.
Failed to install lmdb from https://files.pythonhosted.org/packages/ed/61/41f3c7cbd8a67202ef24fad3375ed936093a0547dc645581dd11c09581b7/lmdb-1.3.0.tar.gz.

Chaquopy: Exit status 1

> Task :app:generateArm32DebugPythonRequirements FAILED
Execution failed for task ':app:generateArm32DebugPythonRequirements'.
> Process 'command '/Users/egemenakbal/opt/miniconda3/envs/holistic/bin/python3.8'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

The text was updated successfully, but these errors were encountered:

mhsmith · 2022-08-08T08:23:29Z

Thanks for the report. This package contains native components, so it would have to be built into a wheel file. If you'd like to try doing this yourself, follow the instructions here. And if you're successful, please make a pull request so we can add the package to the public repository.

If anyone else wants this package too, please let us know by clicking the thumbs-up button above.

axvx · 2023-03-23T02:33:05Z

[Moved to #828]

mhsmith · 2023-12-12T08:28:02Z

Originally posted by @YouSirY in #1036 (comment)

when I build lmdb use code:
./build-wheel.py --python 3.8 --abi arm64-v8a lmdb
have a problem：

build-wheel: /home/youzh/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy/lib/libpthread.a
build-wheel: /home/youzh/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy/lib/librt.a
build-wheel: cd /home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/src
build-wheel: Building with PEP 517
py-lmdb: Using bundled liblmdb with py-lmdb patches; override with LMDB_FORCE_SYSTEM=1 or LMDB_PURE=1.
patching file lmdb.h
patching file mdb.c
py-lmdb: Using CPython extension; override with LMDB_FORCE_CFFI=1.
running bdist_wheel
running build
running build_py
creating build/lib.linux_aarch64-cpython-38
creating build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/__init__.py -> build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/__main__.py -> build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/_config.py -> build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/tool.py -> build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/cffi.py -> build/lib.linux_aarch64-cpython-38/lmdb
running build_ext
building 'cpython' extension
creating build/temp.linux_aarch64-cpython-38
creating build/temp.linux_aarch64-cpython-38/build
creating build/temp.linux_aarch64-cpython-38/build/lib
creating build/temp.linux_aarch64-cpython-38/lmdb
/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/wrappers/aarch64-linux-android21-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy/include -idirafter /home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy/include/python3.8 -fPIC -Ilib/py-lmdb -Ibuild/lib -I/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/env/include -I/home/youzh/anaconda3/envs/build-wheel/include/python3.8 -c build/lib/mdb.c -o build/temp.linux_aarch64-cpython-38/build/lib/mdb.o -DHAVE_PATCHED_LMDB=1 -UNDEBUG -w
Chaquopy: ignored invalid -I directory: '/home/youzh/anaconda3/envs/build-wheel/include/python3.8'
Traceback (most recent call last):
  File "/home/youzh/chaquopy-master/server/pypi/compiler-wrapper.py", line 50, in <module>
    main()
  File "/home/youzh/chaquopy-master/server/pypi/compiler-wrapper.py", line 46, in main
    os.execv(args_out[0], args_out)
PermissionError: [Errno 13] Permission denied
error: command '/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/wrappers/aarch64-linux-android21-clang' failed with exit code 1
build-wheel: Error: Backend subprocess exited when trying to invoke build_wheel

meta.yaml

package:
  name: lmdb
  version: "1.4.1"

build:
  number: 0

How to solve this problem？

mhsmith · 2023-12-12T08:36:45Z

There may be something wrong with your NDK installation, perhaps because you interrupted it while it was installing. Try deleting /home/youzh/android-sdk/ndk/22.1.7171670, then running build-wheel again.

If that doesn't work, try editing compiler-wrapper.py like this to add some additional logging:

    try:
        os.execv(args_out[0], args_out)
    except Exception:
        print("execv failed", args_out)
        raise

That should print the path of the executable it's failing to run, and you can check whether it actually does have execute permission.

YouSirY · 2023-12-12T09:39:05Z

The error now is that ndk does not exist

Collecting setuptools>=40.8.0
  Obtaining dependency information for setuptools>=40.8.0 from https://files.pythonhosted.org/packages/bb/e1/ed2dd0850446b8697ad28d118df885ad04140c64ace06c4bd559f7c8a94f/setuptools-69.0.2-py3-none-any.whl.metadata
  Using cached setuptools-69.0.2-py3-none-any.whl.metadata (6.3 kB)
Collecting wheel
  Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/c7/c3/55076fc728723ef927521abaa1955213d094933dc36d4a2008d5101e1af5/wheel-0.42.0-py3-none-any.whl.metadata
  Using cached wheel-0.42.0-py3-none-any.whl.metadata (2.2 kB)
Using cached setuptools-69.0.2-py3-none-any.whl (819 kB)
Using cached wheel-0.42.0-py3-none-any.whl (65 kB)
Installing collected packages: wheel, setuptools
Successfully installed setuptools-69.0.2 wheel-0.42.0
build-wheel: abi=arm64-v8a; api_level=21; prefix=/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy; . /home/youzh/chaquopy-master/server/pypi/../../target/build-common.sh; export
Installing NDK: this may take several minutes
/home/youzh/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/*/bin/llvm-ar does not exist
build-wheel: Error: Command 'abi=arm64-v8a; api_level=21; prefix=/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy; . /home/youzh/chaquopy-master/server/pypi/../../target/build-common.sh; export' returned non-zero exit status 1.

mhsmith · 2023-12-18T20:23:24Z

What does exist under /home/youzh/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt? Or if that directory doesn't exist, how much of its path does exist?

We did have a previous report (#862) where the NDK installation wasn't happening for some reason. If that's the case, try running this command:

$ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager "ndk;22.1.7171670"

Then run build-wheel again.

EDIT: For further discussion of this issue, see #1084.

YouSirY · 2023-12-21T09:16:36Z

A new problem has arisen? LMDB, do you have a plan for this package?

py-lmdb: Using CPython extension; override with LMDB_FORCE_CFFI=1.
running egg_info
writing lmdb.egg-info/PKG-INFO
writing dependency_links to lmdb.egg-info/dependency_links.txt
writing top-level names to lmdb.egg-info/top_level.txt
reading manifest file 'lmdb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'lmdb/__pycache__'
adding license file 'LICENSE'
writing manifest file 'lmdb.egg-info/SOURCES.txt'
build-wheel: abi=arm64-v8a; api_level=21; prefix=/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy; . /home/youzh/chaquopy-master/server/pypi/../../target/build-common.sh; export
build-wheel: /home/youzh/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy/lib/libpthread.a
build-wheel: /home/youzh/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy/lib/librt.a
build-wheel: cd /home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/src
build-wheel: Building with PEP 517
py-lmdb: Using bundled liblmdb with py-lmdb patches; override with LMDB_FORCE_SYSTEM=1 or LMDB_PURE=1.
patching file lmdb.h
patching file mdb.c
py-lmdb: Using CPython extension; override with LMDB_FORCE_CFFI=1.
running bdist_wheel
running build
running build_py
creating build/lib.linux_aarch64-cpython-38
creating build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/__init__.py -> build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/__main__.py -> build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/_config.py -> build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/tool.py -> build/lib.linux_aarch64-cpython-38/lmdb
copying lmdb/cffi.py -> build/lib.linux_aarch64-cpython-38/lmdb
running build_ext
building 'cpython' extension
creating build/temp.linux_aarch64-cpython-38
creating build/temp.linux_aarch64-cpython-38/build
creating build/temp.linux_aarch64-cpython-38/build/lib
creating build/temp.linux_aarch64-cpython-38/lmdb
/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/wrappers/aarch64-linux-android21-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy/include -idirafter /home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/requirements/chaquopy/include/python3.8 -fPIC -Ilib/py-lmdb -Ibuild/lib -I/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/env/include -I/home/youzh/anaconda3/envs/build-wheel/include/python3.8 -c build/lib/mdb.c -o build/temp.linux_aarch64-cpython-38/build/lib/mdb.o -DHAVE_PATCHED_LMDB=1 -UNDEBUG -w
Chaquopy: ignored invalid -I directory: '/home/youzh/anaconda3/envs/build-wheel/include/python3.8'
build/lib/mdb.c:4898:53: error: use of undeclared identifier 'PTHREAD_MUTEX_ROBUST'
                if (!rc) rc = pthread_mutexattr_setrobust(&mattr, PTHREAD_MUTEX_ROBUST);
                                                                  ^
1 error generated.
error: command '/home/youzh/chaquopy-master/server/pypi/packages/lmdb/build/1.4.1/cp38-cp38-android_21_arm64_v8a/wrappers/aarch64-linux-android21-clang' failed with exit code 1

mhsmith · 2023-12-22T11:34:29Z

It looks like this PTHREAD_MUTEX_ROBUST API isn't supported by Android. The code does make some attempt to handle this, but obviously it isn't working. Maybe it should be using __ANDROID__ rather than ANDROID.

To experiment with this, you can use the --no-unpack option, as described in the README.

BobbyRaduloff · 2024-08-20T14:13:00Z

It looks like this PTHREAD_MUTEX_ROBUST API isn't supported by Android. The code does make some attempt to handle this, but obviously it isn't working. Maybe it should be using __ANDROID__ rather than ANDROID.

To experiment with this, you can use the --no-unpack option, as described in the README.

Can confirm the change to __ANDROID__ fixes the build step.

mhsmith changed the title ~~building 'cpython' extension error: CCompiler.compile: Chaquopy cannot compile native code~~ lmdb Aug 8, 2022

mhsmith added the packages label Aug 8, 2022

mhsmith mentioned this issue Mar 23, 2023

tiktoken #828

Open

mhsmith mentioned this issue Dec 6, 2023

basicsr install error #1036

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lmdb #658

lmdb #658

egemenakbal commented Aug 7, 2022 •

edited by mhsmith

Loading

mhsmith commented Aug 8, 2022 •

edited

Loading

axvx commented Mar 23, 2023 •

edited by mhsmith

Loading

mhsmith commented Dec 12, 2023

mhsmith commented Dec 12, 2023

YouSirY commented Dec 12, 2023

mhsmith commented Dec 18, 2023 •

edited

Loading

YouSirY commented Dec 21, 2023

mhsmith commented Dec 22, 2023

BobbyRaduloff commented Aug 20, 2024

lmdb #658

lmdb #658

Comments

egemenakbal commented Aug 7, 2022 • edited by mhsmith Loading

mhsmith commented Aug 8, 2022 • edited Loading

axvx commented Mar 23, 2023 • edited by mhsmith Loading

mhsmith commented Dec 12, 2023

mhsmith commented Dec 12, 2023

YouSirY commented Dec 12, 2023

mhsmith commented Dec 18, 2023 • edited Loading

YouSirY commented Dec 21, 2023

mhsmith commented Dec 22, 2023

BobbyRaduloff commented Aug 20, 2024

egemenakbal commented Aug 7, 2022 •

edited by mhsmith

Loading

mhsmith commented Aug 8, 2022 •

edited

Loading

axvx commented Mar 23, 2023 •

edited by mhsmith

Loading

mhsmith commented Dec 18, 2023 •

edited

Loading