Auto-merge not adhering to Branch Protection Rules

[smcvb]

First and foremost, I am not 100% confident this is the right place to report this in.
So, if not, feel free to move my request or me somewhere else.

Now, to the predicament at hand.
I maintain an open-source organization with several repositories utilizing Dependabot to ensure all our dependencies stay up to date.
As manual merging became too much of a burden to the team, I investigated how to employ auto-merge in our environment.

This investigation landed me on the Automating Dependabot with GitHub Actions page that describes how to achieve just that.
After going through the description step by step, I assumed everything was in place.
Thus I set an (in my case, dedicated) YAML for the auto-approval and auto-merge procedures and added branch protection rules for dependabot specific pull requests.

Although the GitHub configuration matches the branch protection rules with the dependabot PRs, some are merged regardless of whether the task succeeds.

After noticing this predicament, I went on a second investigation to figure out what I was missing.
This eventually led me to the README.md of this project, which also describes the auto-merge capabilities.

Furthermore, I noticed a slight discrepancy between this project's readme and the GitHub docs page.
The latter states the following on auto-merge:

If you want to allow maintainers to mark certain pull requests for auto-merge, you can use GitHub's auto-merge functionality.
This enables the pull request to be merged when any tests and approvals required by the branch protection rules are successfully met.
For more information, see "Automatically merging a pull request" and "Managing a branch protection rule."
You can instead use GitHub Actions and the GitHub CLI. Here is an example that auto merges all patch updates to my-dependency:

Whereas this project's readme states:

If you are using the auto-merge feature on your repository, you can set up an action that will enable Dependabot PRs to merge once CI and other branch protection rules are met. (Note that you must use a personal access token (PAT) when executing the merge instruction.)

The former seems to state that GitHub Action can be used instead of branch protection rules, whereas the latter describes the necessity of combining both.
Aside from being interested in the actual approach, I'm also wondering why my branch protection rules do not seem to block the auto-merge.

If anybody reading this can help me with that or guide me to somebody who does know, that would be very much appreciated.

[jeffwidman]

Given this is happening on an open source project, can you link to it as well as provide screenshots of the branch protection rules you've set in the repo settings?

[smcvb]

But of course, @jeffwidman. Thanks for chipping in.

Here's a recent pull request showing this behavior.
It upgrades Spring Boot from 2.7.10 to 2.7.11, wherein the project's GitHub Action performs three builds to test compatibility.
These are a JDK8, JDK11, and JDK17 build.
These are called Test and Build on JDK # for each respective version.

The branch protection rules expect all three JDK builds to succeed.
Or, that's the intention, at least.

Here is the branch name pattern and several branches it currently applies to:

And here are the rules I've set up:

None of the remaining rules below "Require signed commits" have been toggled, so I left those out.

[smcvb]

@jeffwidman, I don't want to push, but I am wondering whether there's any information I can go on concerning this predicament.
Do you per chance have an update for me? Thanks in advance for your time! 🙇