Helm install failed for release XYZ with chart [email protected]: context deadline exceeded

[vijayansarathy]

Hi Everyone,

When I am deploying workloads using HelmRelease, I have observed that from time to time, the status of the HelmRelease reads like this:

Helm install failed for release NAMESPACE/RELEASE_NAME with chart [email protected]: context deadline exceeded

The HelmRelease status is set to Status:False, Reason:InstallFailed. However, when I check the status of the underlying K8s Deployment resource created by the Helm install, the Pods are up and running and are in a healthy state. Even if it were the case that they did not reach the healthy state the first time around, I would expect that when the next reconciliation loop runs, it should update the status to Status:True, Reason:InstallSucceeded. This does not seem to happen and the HelmRelease finally reaches the Status:True, Reason:RetriesExceeded state.

Can someone please explain why this happens? I have attached the output obtained from running kubectl describe command on one such HelmRelease object.

Thanks,
Viji

HelmRelease.txt

[dart-ops]

Hi

not sure, but: hr waiting for jobs for some time

maybe jobs or post hooks are stuck?

[hiddeco]

You need to increase the timeout you have configured, as this error is a clear signal Helm times out for some reason.

[vijayansarathy]

But why does it not realize that everything is in a Ready state on the next reconciliation run and set status to status to Status:True, Reason:InstallSucceeded ?

[hiddeco]

Because that's not how the automation process works. It runs a helm install or helm upgrade to reach the desired state, not a simple apply.

Which means that on the next attempt, it will time out again unless you set the right timeout settings.

[vijayansarathy]

The HelmController executes the steps listed here at the interval defined by spec.interval.
So, when it runs those steps the next time, I would think that it will observe that the underlying resources created from the HelmRelease are all now in a healthy state and therefore update the status of the HelmRelease to Status:True, Reason:InstallSucceeded. Why would it timeout again given that it is not doing an helm install or an helm upgrade and is merely performing a health assessment of the deployed workloads?