|
| 1 | +# Contribution Guidelines |
| 2 | + |
| 3 | +python-for-android is part of the [Kivy](https://kivy.org) ecosystem - a large group of |
| 4 | +products used by many thousands of developers for free, but it |
| 5 | +is built entirely by the contributions of volunteers. We welcome (and rely on) |
| 6 | +users who want to give back to the community by contributing to the project. |
| 7 | + |
| 8 | +Contributions can come in many forms. See the latest |
| 9 | +[Contribution Guidelines](https://github.com/kivy/kivy/blob/master/CONTRIBUTING.md) |
| 10 | +for how you can help us. |
| 11 | + |
| 12 | +.. warning:: |
| 13 | + The python-for-android process differs in small but important ways from the |
| 14 | + Kivy framework's process. See below. |
| 15 | + |
| 16 | +## Development model |
| 17 | + |
| 18 | +Unlike the Kivy framework, python-for-android is developed using the following |
| 19 | +model: |
| 20 | + |
| 21 | +- The `master` branch always represents the latest stable release. |
| 22 | +- The `develop` branch is the most up to date with new contributions. |
| 23 | +- Releases happen periodically, and consist of merging the current `develop` |
| 24 | + branch into `master`. |
| 25 | + |
| 26 | +This means pull requests for python-for-android code and documentation |
| 27 | +submissions should be made to the `develop` branch, not the `master` branch. |
| 28 | + |
| 29 | +For reference, this is based on a |
| 30 | +[Git flow](https://nvie.com/posts/a-successful-git-branching-model/) model, |
| 31 | +although we don't follow this religiously. |
| 32 | + |
| 33 | +## Versioning |
| 34 | + |
| 35 | +python-for-android releases currently use |
| 36 | +[calendar versioning](https://calver.org/). Release numbers are of the form |
| 37 | +YYYY.MM.DD. |
| 38 | + |
| 39 | +We use calendar versioning because in practice, changes in |
| 40 | +python-for-android are often driven by updates or adjustments in the |
| 41 | +Android build tools. It's usually best for users to be working from |
| 42 | +the latest release. We try to maintain backwards compatibility even |
| 43 | +while internals are changing. |
| 44 | + |
| 45 | +## History |
| 46 | + |
| 47 | +In 2015, these tools were rewritten to provide a new, easier-to-use and |
| 48 | +easier-to-extend interface. If you'd like to browse the old toolchain, |
| 49 | +its status is |
| 50 | +[recorded for posterity](https://github.com/kivy/python-for-android/tree/old_toolchain). |
| 51 | + |
| 52 | +In the last quarter of 2018, the Python recipes were changed. The |
| 53 | +new recipe for Python3 (3.7.1) had a new build system which was |
| 54 | +applied to the ancient Python recipe, allowing us to bump the Python2 |
| 55 | +version number to 2.7.15. This change unified the build process for |
| 56 | +both Python recipes, and probably solved various issues detected over the |
| 57 | +years. These **unified Python recipes** require a **minimum target api level of 21**, |
| 58 | +*Android 5.0 - Lollipop*. If you need to build targeting an |
| 59 | +api level below 21, you should use an older version of python-for-android |
| 60 | +(<=0.7.1). |
| 61 | + |
| 62 | +On March 2020, we dropped support for creating apps that use Python 2. The |
| 63 | +latest python-for-android release that supported building Python 2 was version |
| 64 | +2019.10.6. |
| 65 | + |
| 66 | +On August 2021, we added support for Android App Bundle (aab). As a |
| 67 | +collateral benefit, we now support multi-arch apk. |
| 68 | + |
| 69 | +## Creating a new release |
| 70 | + |
| 71 | +(These instructions are for core developers, not casual contributors.) |
| 72 | + |
| 73 | +New releases follow these steps: |
| 74 | + |
| 75 | +- Create a new branch `release-YYYY.MM.DD` based on the `develop` branch. |
| 76 | + - `git checkout -b release-YYYY.MM.DD develop` |
| 77 | +- Create a Github pull request to merge `release-YYYY.MM.DD` into `master`. |
| 78 | +- Complete all steps in the [release checklist](#Release_checklist), |
| 79 | + and document this in the pull request (copy the checklist into the PR text) |
| 80 | + |
| 81 | +At this point, wait for reviewer approval and conclude any discussion that |
| 82 | +arises. To complete the release: |
| 83 | + |
| 84 | +- Merge the release branch to the `master` branch. |
| 85 | +- Also merge the release branch to the `develop` branch. |
| 86 | +- Tag the release commit in `master`, with tag `vYYYY.MM.DD`. Include a short |
| 87 | + summary of the changes. |
| 88 | +- Release distributions and PyPI upload should be |
| 89 | + [handled by the CI](https://github.com/kivy/python-for-android/blob/v2020.04.29/.travis.yml#L60-L70). |
| 90 | +- Add to the GitHub release page (see e.g. [this example](https://github.com/kivy/python-for-android/releases/tag/v2019.06.06): |
| 91 | + - The python-for-android README summary |
| 92 | + - A short list of major changes in this release, if any |
| 93 | + - A changelog summarising merge commits since the last release |
| 94 | + - The release sdist and wheel(s) |
| 95 | + |
| 96 | +## Release checklist |
| 97 | + |
| 98 | + - [ ] Check that the builds are passing |
| 99 | + - [ ] [GitHub Action](https://github.com/kivy/python-for-android/actions) |
| 100 | + - [ ] Run the tests locally via `tox`: this performs some long-running tests that are skipped on github-actions. |
| 101 | + - [ ] Build and run the [on_device_unit_tests](https://github.com/kivy/python-for-android/tree/master/testapps/on_device_unit_tests) app using buildozer. Check that they all pass. |
| 102 | + - [ ] Build (or download from github actions) and run the following [testapps](https://github.com/kivy/python-for-android/tree/master/testapps/on_device_unit_tests) for arch `armeabi-v7a` and `arm64-v8a`: |
| 103 | + - [ ] on_device_unit_tests |
| 104 | + - [ ] `armeabi-v7a` (`cd testapps/on_device_unit_tests && PYTHONPATH=.:../../ python3 setup.py apk --ndk-dir=<your-ndk-dir> --sdk-dir=<your-sdk-dir> --arch=armeabi-v7a --debug`) |
| 105 | + - [ ] `arm64-v8a` (`cd testapps/on_device_unit_tests && PYTHONPATH=.:../../ python3 setup.py apk --ndk-dir=<your-ndk-dir> --sdk-dir=<your-sdk-dir> --arch=arm64-v8a --debug`) |
| 106 | + - [ ] Check that the version number is correct |
| 107 | + |
| 108 | +## How python-for-android uses `pip` |
| 109 | + |
| 110 | +*Last update: July 2019* |
| 111 | + |
| 112 | +This section is meant to provide a quick summary how |
| 113 | +python-for-android uses pip and Python packages in |
| 114 | +its build process. |
| 115 | +**It is written for a Python |
| 116 | +packager's point of view, not for regular end users or |
| 117 | +contributors,** to assist with making pip developers and |
| 118 | +other packaging experts aware of p4a's packaging needs. |
| 119 | + |
| 120 | +Please note this section just attempts to neutrally list the |
| 121 | +current mechanisms, so some of this isn't necessarily meant |
| 122 | +to stay but just how things work inside p4a in |
| 123 | +this very moment. |
| 124 | + |
| 125 | + |
| 126 | +### Basic concepts |
| 127 | + |
| 128 | +*(This part repeats other parts of the docs, for the sake of |
| 129 | +making this a more independent read)* |
| 130 | + |
| 131 | +p4a builds & packages a Python application for use on Android. |
| 132 | +It does this by providing a Java wrapper, and for graphical applications |
| 133 | +an SDL2-based wrapper which can be used with the Kivy framework if |
| 134 | +desired (or alternatively just plain PySDL2). Any such the Python application |
| 135 | +will likely have further library dependencies to do its work. |
| 136 | + |
| 137 | +p4a supports two types of package dependencies for a project: |
| 138 | + |
| 139 | +**Recipe:** Install a script in custom p4a format. Can either install |
| 140 | +C/C++ or other software that cannot be pulled in via pip, or software |
| 141 | +that can be installed via pip but break on Android by default. |
| 142 | +These are maintained primarily inside the p4a source tree by p4a |
| 143 | +contributors and interested folks. |
| 144 | + |
| 145 | +**Python package:** any random pip python package can be directly |
| 146 | +installed if it doesn't need adjustments to work for Android. |
| 147 | + |
| 148 | +p4a will map any dependency to an internal recipe if present, and |
| 149 | +otherwise use pip to obtain it regularly from whatever external source. |
| 150 | + |
| 151 | + |
| 152 | +### Install process regarding packages |
| 153 | + |
| 154 | +The install/build process of a p4a project, as triggered by the |
| 155 | +`p4a apk` command, roughly works as follows in regards to Python |
| 156 | +packages: |
| 157 | + |
| 158 | +1. The user has specified a project folder to install. This is either |
| 159 | + just a folder with Python scripts and a `main.py`, or it may |
| 160 | + also have a `pyproject.toml` for a more standardized install. |
| 161 | + |
| 162 | +2. Dependencies are collected: they can be either specified via |
| 163 | + ``--requirements`` as a list of names or pip-style URLs, or p4a |
| 164 | + can optionally scan them from a project folder via the |
| 165 | + pep517 library (if there is a `pyproject.toml` or `setup.py`). |
| 166 | + |
| 167 | +3. The collected dependencies are mapped to p4a's recipes if any are |
| 168 | + available for them, otherwise they're kept around as external |
| 169 | + regular package references. |
| 170 | + |
| 171 | +4. All the dependencies mapped to recipes are built via p4a's internal |
| 172 | + mechanisms to build these recipes. (This may or may not indirectly |
| 173 | + use pip, depending on whether the recipe wraps a python package |
| 174 | + or not and uses pip to install or not.) |
| 175 | + |
| 176 | +5. **If the user has specified to install the project in standardized |
| 177 | + ways,** then the `setup.py`/whatever build system |
| 178 | + of the project will be run. This happens with cross compilation set up |
| 179 | + (`CC`/`CFLAGS`/... set to use the |
| 180 | + proper toolchain) and a custom site-packages location. |
| 181 | + The actual comand is a simple `pip install .` in the project folder |
| 182 | + with some extra options: e.g. all dependencies that were already |
| 183 | + installed by recipes will be pinned with a `-c` constraints file |
| 184 | + to make sure pip won't install them, and build isolation will be |
| 185 | + disabled via ``--no-build-isolation`` so pip doesn't reinstall |
| 186 | + recipe-packages on its own. |
| 187 | + |
| 188 | + **If the user has not specified to use standardized build approaches**, |
| 189 | + p4a will simply install all the remaining dependencies that weren't |
| 190 | + mapped to recipes directly and just plain copy in the user project |
| 191 | + without installing. Any `setup.py` or `pyproject.toml` of the user |
| 192 | + project will then be ignored in this step. |
| 193 | + |
| 194 | +6. Google's gradle is invoked to package it all up into an `.apk`. |
| 195 | + |
| 196 | + |
| 197 | +### Overall process / package relevant notes for p4a |
| 198 | + |
| 199 | +Here are some common things worth knowing about python-for-android's |
| 200 | +dealing with python packages: |
| 201 | + |
| 202 | +- Packages will work fine without a recipe if: |
| 203 | + |
| 204 | + * they would also build on Linux ARM, |
| 205 | + * don't use any API not available in the NDK if they use native code, and |
| 206 | + * don't use any weird compiler flags the toolchain doesn't like if they use native code. |
| 207 | + * works with cross compilation. |
| 208 | + |
| 209 | +- There is currently no easy way for a package to know it is being |
| 210 | + cross-compiled (at least that we know of) other than examining the |
| 211 | + `CC` compiler that was set, or that it is being cross-compiled for |
| 212 | + Android specifically. If that breaks a package, it currently needs |
| 213 | + to be worked around with a recipe. |
| 214 | + |
| 215 | +- If a package does **not** work, p4a developers will often create a |
| 216 | + recipe instead of getting upstream to fix it because p4a simply |
| 217 | + is too niche. |
| 218 | + |
| 219 | +- Most packages without native code will just work out of the box. |
| 220 | + Many with native code tend not to, especially if complex, e.g. numpy. |
| 221 | + |
| 222 | +- Anything mapped to a p4a recipe cannot be just reinstalled by pip, |
| 223 | + specifically also not inside build isolation as a dependency. |
| 224 | + (It *may* work if the patches of the recipe are just relevant |
| 225 | + to fix runtime issues.) |
| 226 | + Therefore as of now, the best way to deal with this limitation seems |
| 227 | + to be to keep build isolation always off. |
| 228 | + |
| 229 | + |
| 230 | +### Ideas for the future regarding packaging |
| 231 | + |
| 232 | +- We in overall prefer to use the recipe mechanism less if we can. |
| 233 | + Overall, the recipes are just a collection of workarounds. |
| 234 | + It may look quite hacky from the outside, since p4a |
| 235 | + version pins recipe-wrapped packages usually to make the patches reliably |
| 236 | + apply. This creates work for the recipes to be kept up-to-date, and |
| 237 | + obviously this approach doesn't scale too well. However, it has ended |
| 238 | + up as a quite practical interim solution until better ways are found. |
| 239 | + |
| 240 | +- Obviously, it would be nice if packages could know they are being |
| 241 | + cross-compiled, and for Android specifically. We aren't currently aware |
| 242 | + of any good mechanism for that. |
| 243 | + |
| 244 | +- If pip could actually run the recipes (instead of p4a wrapping pip and |
| 245 | + doing so) then this might even allow build isolation to work - but |
| 246 | + this might be too complex to get working. It might be more practical |
| 247 | + to just gradually reduce the reliance on recipes instead and make |
| 248 | + more packages work out of the box. This has been done e.g. with |
| 249 | + improvements to the cross-compile environment being set up automatically, |
| 250 | + and we're open for any ideas on how to improve this. |
0 commit comments