A place to experiment with getting GitHub Actions to build like we’d like it to. If successful, we might even leave this repo around as an example/reference for those with similar build tastes.
-
Run tests on pushes everywhere. Forks or not. This will allow folks to verity their work before submitting PRs.
-
Avoid duplicate runs of same work
-
Avoid confusion. A specific case that is especially annoying is that when goal 1 is enabled, we get a duplicate test run for a push to a PR. These duplicate runs are both registered as checks for the PR.
-
Avoid unecessary compute. The planet is suffering, we don’t want to make it suffer more.
-
-
Trigger release on new version tag. Again, without duplication of work.
I don’t think GitHub Actions was designed with care for this combination of goals.
Can we bend it to our will? Or is this a quixotic whack-a-mole situation?
GitHub Actions must be enabled by the user for forks. Our premise that folks are typically using GHA on forks to verify before submitting PRs might be incorrect. But that does not invalidate our approach. They could be using GHA on forks. If they want to.
GitHub actions allows some control over what invokes a workflow. But some controls are only available at the job level within a workflow. So sometimes we’ll trigger a workflow but skip all the jobs in that workflow. Just because that’s the way GitHub Actions currenltly works.
The idea is that duplicate tests runs are being triggered by the following events:
- push
- pull_request’s `synchronize activity
If we remove synchronize activity, would this do the trick?
No. This works for PRs created from branches off the repository. But for forks, tests are run on the fork only and not as checks for PR.
If we can check that a commit is in a PR for a push event, can we simply supress the running of tests?
I can use gh, the github command line tool to search if the current commit sha is part of a PR.
So if the triggering event is push and the sha is in a PR, tests should be skipped.
Look into:
-
When opening a PR the commit is not yet part of a PR. So duplicate runs occur. Anything we can do that would work for both local branch and fork?
-
In my testing I got occassional 403 errors, indicating rate limiting.
You have exceeded a secondary rate limit. Please wait a few minutes before you try again.
Not sure if this is an artifact of using
gh. Or just me pushing frequently.
While researching I stumbled upon https://github.com/fkirc/skip-duplicate-actions. This might be a good general solution to avoid duplicate work.
If I understand it correctly, it goes even further than I was thinking.
It can, for example, avoid re-running tests on merge that were just successfully run as part of a PR, if all files are identical. I suppose it could be smart enough to skip those tests on release as well.
Another scenario it handles is cancelling running jobs if a new job run is launched. This can happen for long-running jobs that have not completed before a new commit is pushed. Apparently GitHub Actions itself now has some support for this scenario.
Probably worth a try.
We think that tests should be run on release. But why? If they were already successfully run, why do we need to repeat them?
-
To double-check?
-
To handle case where a PR was merged that had failing tests?
-
To handle the case where the environment/os/build-tools might have changed since the same tests were last run?
Trigger tests for:
-
push to master or any other branch
-
push to fork of this repo
-
push to PR
-
from local branch
-
from fork
-
-
a release
Some specific desired behaviour:
-
Trigger tests to run only once when working from a PR. We need the tests to be perceived by GitHub as checks for the PR.
-
On publish, triggered by a version tag push, trigger a test run, then follow that up with a release work. An additional test run should not be triggered here. Any commit solely related to publishing should not trigger a test run (i.e. version bumps).
-
Publish work should not be executed when on a fork.