Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ES index upgrades when using index aliases #3863

Merged
merged 2 commits into from
Sep 10, 2024

Conversation

tillprochaska
Copy link
Contributor

@tillprochaska tillprochaska commented Sep 9, 2024

ES supports fetching the index config for multiple indices in a single request. We don’t make use of this, but for this reason the ES responses are an object with a key for each requested index. For example:

{
  "aleph-entity-company-v1": {
    "settings": { ... },
    "mappings: { ... }
  }
}

In case an index is requested using an alias, the key will be the original index name and not the alias, i.e. we cannot rely on the key being the name that was used to request the index, as this would fail in case aliases are used.

However, as we’re always requesting only a single index at a time, the response should always contain data for one index, so we can extract that without relying on the name of the index.

You can test the following scenarios:

  • Creating an index from scratch:
    1. Delete one of the indexes: curl -X "DELETE" "http://localhost:9200/aleph-entity-company-v1"
    2. Run aleph upgrade.
    3. The logs should indicate that the index was created.
  • Update an aliased index with incompatible mappings:
    1. Follow instructions in JIRA to reproduce the issue.
    2. Run aleph upgrade.
    3. This shouldn’t error and the logs should indicate that the configuration has been updated.

ES supports fetching the index config for multiple indices in a single request. We don’t make use of this, but for this reason the ES responses are an object with a key for each requested index. For example:

```json
{
  "aleph-entity-company-v1": {
    "settings": { ... },
    "mappings: { ... }
  }
}
```

In case an index is requested using an alias, the key will be the original index name and *not* the alias, i.e. we cannot rely on the key being the name that was used to request the index, as this would fail in case aliases are used.

However, as we’re always requesting only a single index at a time, the response should always contain data for one index, so we can extract that without relying on the name of the index.
@tillprochaska tillprochaska marked this pull request as ready for review September 9, 2024 13:28
f"ES response does not contain data for index {index}"
)

if len(res) > 1:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity: how do you feel about merging these checks into a if len != 1?

Copy link
Contributor Author

@tillprochaska tillprochaska Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, I’ve changed this.

Copy link
Contributor

@stchris stchris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks and works great, thanks for the fix!

@tillprochaska tillprochaska merged commit 4b523d9 into release/4.0.0 Sep 10, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants