sprint production release 2022.02.14

[harshad16]

Hello, Thoth-station!

This Issue would be used for the current sprint cycle production release.
By the end of the sprint cycle, we will consolidate the information of thoth-station components features upgrade and fixes in this issue.

/kind documentation
/area release-eng
/triage accepted
/milestone 2022.02.14
/priority important-soon

[fridex]

Optimized syncs across deployments in document-sync-job

Our last release introduced a new component called document-sync-job which can sync documents created in one Thoth deployment to another. This release comes with an optimized implementation of document-sync-job which can sync documents concurrently. This optimization was required as the number of documents we need to handle is too large.

• Use concurrency to optimize copying documents document-sync-job#11

[fridex]

Pedantic thamos run feature

Our command line interface, Thamos, now performs pedantic application runs. As thamos run acts as a wrapper for Python interpreter for spawning Python applications (thamos manages virtual environments automatically for users if configured so), it can now also check that the configuration supplied by the user matches the configuration of the runtime environment used. An example can be a requirement on CUDA, eventually in a specific version. If CUDA in the specified version is not present in the runtime environment, thamos run refuses to run the application (by default). This way users can ensure that the requirements on the runtime environment they use match expectations recorded in the Thoth's configuration file. Moreover, thamos run newly also checks that the lockfile respects direct dependencies (no re-locking is needed).

These checks can be suppressed by using --no-pedantic option or by providing THAMOS_RUN_NO_PEDANTIC=1 environment variable to the runtime environment.

• Provide pedantic thamos run  thamos#1010
• Introduce pedantic runs thamos#1036

[fridex]

thamos run allows specifying environment variables to the Python process run

Starting this release, users can declare their environment variables in .env files configured per overlay. This configuration follows Pipenv's .env file configuration, except Thoth users can declare different environment variables to be supplied to the process based on overlay/runtime environment used.

• Introduce pedantic runs thamos#1036

[pacospace]

Pulp Grafana Dashboard

A new Grafana Dashboard for Pulp deployed on Operate First is avalable at Grafana Operate First: https://grafana.operate-first.cloud/d/B7DLT7anz/pulp-metrics?orgId=1

[pacospace]

graph-backup-job
New metrics were added to monitor the creation of pg dumps and if any issue appears creating pg dumps.

• Add metrics about pg dump creation status graph-backup-job#215

[pacospace]

graph-metrics-exporter
Introduce a new task for monitoring pg_dumps on Ceph. In this way we can verify if all backups are created correctly.

• Introduce pg dump metrics graph-metrics-exporter#69

[harshad16]

we have completed the release for v2022.02.14 🎉 🎊 🥳

Features

Optimized syncs across deployments in document-sync-job

Our last release introduced a new component called document-sync-job which can sync documents created in one Thoth deployment to another. This release comes with an optimized implementation of document-sync-job which can sync documents concurrently. This optimization was required as the number of documents we need to handle is too large.

• Use concurrency to optimize copying documents document-sync-job#11

Pedantic thamos run feature

Our command-line interface, Thamos, now performs pedantic application runs. As thamos run acts as a wrapper for Python interpreter for spawning Python applications (thamos manages virtual environments automatically for users if configured so), it can now also check that the configuration supplied by the user matches the configuration of the runtime environment used. An example can be a requirement on CUDA, eventually in a specific version. If CUDA in the specified version is not present in the runtime environment, thamos run refuses to run the application (by default). This way users can ensure that the requirements on the runtime environment they use match expectations recorded in Thoth's configuration file. Moreover, thamos run newly also checks that the lockfile respects direct dependencies (no re-locking is needed).

These checks can be suppressed by using --no-pedantic option or by providing THAMOS_RUN_NO_PEDANTIC=1 environment variable to the runtime environment.

• Provide pedantic thamos run  thamos#1010
• Introduce pedantic runs thamos#1036

thamos run allows specifying environment variables to the Python process run

Starting this release, users can declare their environment variables in .env files configured per overlay. This configuration follows Pipenv's .env file configuration, except Thoth users can declare different environment variables to be supplied to the process based on overlay/runtime environment used.

• Introduce pedantic runs thamos#1036

Pulp Grafana Dashboard

A new Grafana Dashboard for Pulp deployed on Operate First is available at Grafana Operate First: https://grafana.operate-first.cloud/d/B7DLT7anz/pulp-metrics?orgId=1

graph-backup-job
New metrics were added to monitor the creation of pg dumps and if any issue appears creating pg dumps.

• Add metrics about pg dump creation status graph-backup-job#215

graph-metrics-exporter
Introduce a new task for monitoring pg_dumps on Ceph. In this way, we can verify if all backups are created correctly.

• Introduce pg dump metrics graph-metrics-exporter#69

Component Updates

• user-api: v0.34.4

  • Fix mercator OpenAPI schema user-api#1662

• adviser: v0.51.0

  • Respect also runtime environment when linking packages in Search adviser#2252
  • Get environments from solvers configmap and fix tests for solved software environments adviser#2253
  • Adjust tests when checking for solvers configured adviser#2257
  • Skip empty lines in solvers configmap adviser#2256

• solver: v1.11.1

  • Remove Fedora that is EOL solver#5091
  • Look for dependency versions on separate dependency indexes solver#5116
  • Raise an error if no artifact hashes are found solver#5129
  • Remove old solver-fedora-31-py36 solver#5140

• graph-metrics-exporter: v0.6.0

  • Introduce pg dump metrics graph-metrics-exporter#69

• graph-backup-job: v0.9.0

  • Add metrics about pg dump creation status graph-backup-job#215

Thanks for the amazing work everyone. 💯