Onboarding/Offboarding users to data platform realm

[shaidar]

User Story

• As a user, I want access to apps in the data platform realm.
• As an admin, I want to invite someone to the data platform realm.
• As an admin, I want to ensure that only users no longer have access to the data platform realm once they leave the institute and/or no longer require access to any of the apps in that realm.

Description/Context

Currently we have no process in place for inviting users or allowing a user to request access to some of the apps under the data platform realm. Additionally, there is no clear method of off boarding users who no longer need or are no longer affiliated with the Institute.

Acceptance Criteria

• A clear process (ideally automated, but can be manual for now) to grant someone access to apps in the realm
  • MIT users
  • Non-MIT users
• A clear process (ideally automated, but can be manual for now) to revoke a users access to apps in the realm
  • MIT users
  • Non-MIT users

Limitations/Considerations

• Keycloak does not have an email invite feature out of the box
• SAML claims don't include all the groups that an MIT user belongs to and the only ones being sent as part of a successful auth are names with ol- prefix
• There are different clients in the realm and we need to ensure that granting a user access to an app in the realm, does not necessarily give them access to the other apps within that realm
• A user won't have any roles mapped to them and thus access would be limited. Once an invite is accepted and a user account is created, that user can be manually assigned roles depending on the requirements.

Plan/Design

• Touchstone integration offers us the following:
  • User familiarity with authentication flow
  • User account management

However, those advantages, at this point don't seem to outweigh the limitations/issues we're running into trying to integrate it with how we would like to manage users in the data platform realm.

I propose the following:

• Remove Touchstone Integration
  • If down the line, Keycloak makes changes that remove some of the stated blockers, we should be able to re-enable the Touchstone integration and link existing MIT accounts with the IdP.
• Enable organizations in the data-platform realm and initially setup two orgs (MIT, non-MIT)
  • At this stage, the org feature is mainly used to handle email invites (no other Keycloak built-in feature for that) and group users for potentially later filtering/targeting
• Pre-create user accounts through the Pulumi Keycloak provider
  • We would need to maintain a private username list that Pulumi can operate on and a commit to that list would trigger a concourse pipeline to add/remove user.
• Scheduled task that uses the MIT People API as part of the off boarding process. The API will be queried to verify MIT account status and if a user has become inactive, that account should be disabled in Keycloak.
• To aid with the user experience, enable passwordless login
  • Require some testing to come up with a process on handling account resets and such.

[shaidar]

Version 25 of Keycloak introduced a new preview feature (has to be enabled through the cli) called organizations which enables multi-tenancy within a realm in addition to email invites.
I tried the following test use cases to see if that would help solve this issue:

• Scenario 1:
  • No account
  • IDP under org is configured to be hidden on login page and redirect when email domain matches is OFF
  • Outcome:
    • User can't login as it says there's no account and there's no visible way to create one
• Scenario 2:
  • Pre-create MIT account
  • IDP under org is configured to be hidden on login page and redirect when email domain matches is OFF
  • Outcome:
    • Identifies that there's an account, but asks for password which doesn't make sense in this case.

Based on the above testing, orgs can potentially be used for non-MIT users where an account would need to be pre-created in the Keycloak UI, added as a member of an org, and an email invite sent from within the UI. The challenge with non-MIT users will be off-boarding.

I also posted a question about it in case someone has a similar use case and can recommend a solution - https://groups.google.com/g/keycloak-user/c/9W3xXV0g3CE

[shaidar]

A few tweaks worth recording here to the proposed plan/design:

• Not using Pulumi for user account creation as the current provider does not offer any support for handling Keycloak Organizations
• The on-boarding script should proceed as follows:
  • Add a realm user
  • Add user to appropriate group(s)/role(s)
  • Enable WebAuthn required action
  • Invite user to org (matching domain) which would trigger the email invite

[shaidar]

Setup private repo and a Github Action to handle onboarding users to Keycloak along with assigning them to pre-created roles and sending an email invite. For off-boarding, we have two scenarios:

1. Update the csv file in the private repo and remove a user. This applies to all users including non-MIT ones.
2. Run the off-boarding script (applies to only MIT users) as it checks the People API to verify whether a user account is active or not and disables the Keycloak account based on the response.

Regarding (1), I just need work on adding that functionality to the now existing workflow.
As far as (2) I already wrote the off-boarding script and just need to add it to the private repo along with a Github Action to trigger it periodically.

[shaidar]

So far, I've been testing on CI, but it's now worth deploying changes to QA and verifying the workflow's working as designed. Would be good at this point to invite someone (data-platform folks) to test it out and make sure I'm not missing anything. To get things deployed to QA, I will need to:

• Merge Changes for onboarding data-platform users #2944 and deploy to QA
• Add GitHub action keycloak QA secrets
• Test a commit and ensure it's triggering the workflow and sending the email invite
• Look into the options of shipping GitHub action logs to Grafana

On another note, after a conversation with Tobias last week, he recommended a change/addition to the scope/design. He proposed using the data we already have in the data lake of instructors currently teaching to create user accounts as they would need access to Superset. I briefly looked at ol_warehouse_production_intermediate and appears that we have a few tables that have the data needed. Thinking of pulling the needed data and writing it to the csv file which should then trigger the existing onboarding workflow.

[pdpinch]

Another scenario to consider: on the UAI project, we will be offering collections of courses to partner colleges and corporations. Administrators from those partners will need access to student data -- at least certificates, maybe more. Should/can we give them access through the data platform? How would we handle onboarding and authorization?

[shaidar]

@pdpinch The idea is to setup a Keycloak Org in the data-realm for each one of those partners and basically follow the same workflow that we will have for MIT folks. The two things I need to test/hash out are:

• Superset authorization functionality is working as expected and that entails both ensuring that Keycloak role mapping to Superset roles is being mapped as expected. Additionally, row level security is being applied as designed
• Limit access to the Superset client in that realm for those users as we have other apps in the realm that they wouldn't/shouldn't need to access.

[pdpinch]

Additionally, row level security is being applied as designed

Currently, our row-level access is based on a mapping of users to course runs. This could be unwieldy, especially if want to give admins access to data about all users from a particular partner.