Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discover RDS deploys version out of sync with deployment updater #46625

Open
GavinFrazar opened this issue Sep 14, 2024 · 3 comments
Open

Discover RDS deploys version out of sync with deployment updater #46625

GavinFrazar opened this issue Sep 14, 2024 · 3 comments
Assignees
Labels
aws Used for AWS Related Issues. bug database-access Database access related issues and PRs discover Issues related to Teleport Discover

Comments

@GavinFrazar
Copy link
Contributor

I had an ECS cluster deployed via the integration already, but I had manually scaled the service tasks to 0.

In the background, the AWS OIDC integration updater checks if it should update ECS task definitions to a newer version of Teleport. It checks for a cluster maintenance window every 30 minutes, and if it's in the window then it updates ECS deployments.

In this case, this morning the auto-updater created a new task revision for v16.3.0 and updated my ECS cluster to use the new revision. The prior revision used v16.2.2.

Today, I went through the discover flow again, and when I clicked "Deploy Teleport Service" it updated my ECS cluster service to run 2 tasks again, but it also created a new task definition that again used v16.2.2.

So the ECS task definition revisions look like this:
revision 1: teleport v16.2.2 (from when I created it the first time)
revision 2: teleport v16.3.0 (auto-updater created this morning)
revision 3: teleport v16.2.2 (redeployed teleport service this afternoon)

I found it quite surprising that my ECS service was downgraded.

We need to make sure the service deployment version matches what we use for the auto-updater.

Bug details:

  • Teleport version: 16.3.0
@GavinFrazar GavinFrazar added bug aws Used for AWS Related Issues. discover Issues related to Teleport Discover labels Sep 14, 2024
@GavinFrazar
Copy link
Contributor Author

cc @marcoandredinis

@marcoandredinis
Copy link
Contributor

marcoandredinis commented Sep 16, 2024

The version check has slightly different conditions, but they seem to resolve to the same version 🤔

stableVersion, err := updater.UpgradeChannel.GetVersion(ctx)

cloudStableVersion, err := h.cfg.AutomaticUpgradesChannels.DefaultVersion(ctx)

Was this a self-hosted tenant? Maybe that's why

@GavinFrazar
Copy link
Contributor Author

The version check has slightly different conditions, but they seem to resolve to the same version 🤔

stableVersion, err := updater.UpgradeChannel.GetVersion(ctx)

cloudStableVersion, err := h.cfg.AutomaticUpgradesChannels.DefaultVersion(ctx)

Was this a self-hosted tenant? Maybe that's why

it was a cloud staging tenant

@greedy52 greedy52 added the database-access Database access related issues and PRs label Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aws Used for AWS Related Issues. bug database-access Database access related issues and PRs discover Issues related to Teleport Discover
Projects
None yet
Development

No branches or pull requests

4 participants