9.0.0 to 9.3.0 sendMulticastAsync vs sendEachForMulticastAsync. Server crashes when sending too many pushes

[SmikeSix2]

[REQUIRED] Step 2: Describe your environment

• Operating System version: debian / grails / groovy / java 11
• Firebase SDK version: 9.3.0
• Library version: _____
• Firebase Product: admin

[REQUIRED] Step 3: Describe the problem

We switched out the firebase admin version from 9.0.0 to 9.3.0 yesterday. Sending out a few pushes works fine, but we sometimes send out a few millions in a "batch". There never were issues with the old sendMulticastAsync

    ApiFuture<BatchResponse> apiFuture = FirebaseMessaging.getInstance().sendEachForMulticastAsync(multicastMessage)

    apiFuture.addListener({
        def batchResponse = apiFuture.get()
        if (batchResponse != null && batchResponse.failureCount > 0) {
            for (int i = 0; i < batchResponse.responses.size(); i++) {
                def response = batchResponse.responses.get(i)
                if (response != null && !response.successful && response.exception != null && (
                        (ErrorCode.NOT_FOUND == response.exception.errorCode && MessagingErrorCode.UNREGISTERED == response.exception.messagingErrorCode) ||
                                (ErrorCode.INVALID_ARGUMENT == response.exception.errorCode && MessagingErrorCode.INVALID_ARGUMENT == response.exception.messagingErrorCode))) {
                    UidRemovalJob.removalCandidates.add(new UidRemovalCandidate(phoneId: tokenList[i], os: 2))
                }
            }
        }
    }, executorServiceAndroidResponses)

Does this need to be switched to another implementation?

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[lahirumaramba]

This is a known limitation in sendEach* methods (see #941). Should be addressed in #979

[lahirumaramba]

Could also be due to #985, which will be fixed in the upcoming release.

[jonathanedey]

Both of those changes are now available with the release of v9.4.0.
See our wiki for more information on HTTP Transport change.

[fwadnjar142]

#1001 (comment)