Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APISIX integration #2061

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open

APISIX integration #2061

wants to merge 17 commits into from

Conversation

mbertrand
Copy link
Member

@mbertrand mbertrand commented Feb 19, 2025

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/6756

Related infrastructure PR: mitodl/ol-infrastructure#2983

Description (What does it do?)

  • Makes mit-learn compatible with APISIX, using the code from unified-ecommerce and learn-ai as a template.
  • Adds two optional docker containers to start up local instances of Keycloak and APISIX.
  • Removes LiteLLM proxy service code (that is now in learn-ai)

How can this be tested?

Follow instructions in the new README. You should be able to log in and log out.

Copy link

gitguardian bot commented Feb 19, 2025

⚠️ GitGuardian has uncovered 4 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
13777606 Triggered Generic Private Key dab67da config/keycloak/tls/tls.key.default View secret
13777608 Triggered Generic High Entropy Secret dab67da config/keycloak/realms/default-realm.json View secret
13777609 Triggered Generic Password dab67da config/keycloak/realms/default-realm.json View secret
13777610 Triggered Generic High Entropy Secret dab67da config/keycloak/realms/default-realm.json View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

main/settings.py Outdated
"authentication.pipeline.user.user_created_actions",
# redirect new users to onboarding
"authentication.pipeline.user.user_onboarding",
)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This didn't seem to be getting used even without keycloak and apisix

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's definitely being used, should be left in.

Copy link
Member Author

@mbertrand mbertrand Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, adding it back, must have misunderstood yesterday's slack chat, I thought apisix was supposed to be a replacement for the pipeline. Currently both the apisix middleware and the pipeline trigger profile creation for new users, should I remove from one or the other?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PS the final pipeline step is supposed to redirect the user to "/onboarding" but that isn't currently happening on RC with new users I created there, even though MITOL_NEW_USER_LOGIN_URL=https://rc.learn.mit.edu/onboarding - should it be? Haven't tried it on production yet.

@mbertrand mbertrand added the Needs Review An open Pull Request that is ready for review label Feb 19, 2025
@jkachel jkachel self-assigned this Feb 20, 2025
Copy link
Contributor

@jkachel jkachel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 was able to log in successfully via Keycloak. It also attached the keycloak user to the [email protected] superuser I created via the management command.

Comment on lines 117 to 121
profiles:
- keycloak
image: apache/apisix:latest
depends_on:
- keycloak
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove profiles and depends_on here, because otherwise I'm forced to spend resources running a duplicate keycloak instance I won't use if I'm not running the keycloak container defined above.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might it be useful to still keep the profiles part, but change it to its own value so that apisix (and keycloak) don't start automatically, only when explicitly requested? So:

COMPOSE_PROFILES=backend,frontend,keycloak,apisix starts everything
COMPOSE_PROFILES=backend,frontend,apisix starts apisix but not keycloak
etc

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just made that change (removed depends_on, gave keycloak and apisix containers their own separate profiles)

Copy link
Contributor

@jkachel jkachel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simpler workflow seems to work pretty well. I have one nit regarding logging. (I am also only evaluating the APISIX/Keycloak integration parts of this.)

I had some issues getting to the onboarding until I figured out that the onboarding URL is configurable. May be a good idea to add that to the docs (or update the pipeline or default to use the app base URL) - it defaults to just /onboarding which ends up sending the user to the API service and results in an error.

"""
if request.META.get(self.header):
new_header = decode_apisix_headers(request, self.header)
log.error("FOUND APISIX HEADER: %s", new_header)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is logging as error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review An open Pull Request that is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants