[BUG] uncaughtException: Cannot read properties of undefined (reading 'match') #169
Comments
|
This one will be tricky. Indeed I (from another bug) already have code queued up to detect a null state and return an 403 immediately so don’t worry about that one. The question is assuming a relatively quick login process why is the state not there? |
|
Wait is this occurring after a logout process has occurred? |
|
To the error occur when:
|
|
Does it do that same behavior on fresh incognito window? Looking through the code the only scenario I saw where the state data gets stored as not the state_id is during a logout situation (another bug). |
|
I just double checked...
I'm quite sure that its the setup of the provider, in relation to the config token that is the issue. Since the everything works, to the point of the redirect back to the original site, where the EAS is blowing up, I think its the config token that must be the issue here |
|
One sec.... ill check if incognito will help :) |
|
I can confirm.... the problem is not persistent. Any way to work around it, other then clearing cookies and app-data...? Or even better a permanent fix? |
|
Yeah, I'll fix it up and snap a new release. |
|
Cool, thanks.... Need any help with that? or anything i can do i the meantime? |
|
Nah, might be a day or 2 though. |
|
Fair enough! :) Thanks agian for the quick response |
|
LOL. Can you send your cleansed config token? You're using some pretty advanced stuff (that I don't test often, but also doesn't change often) so I want to make sure I'm covering the use-case. |
|
This here is the template that we have made.... We are using it to programmatically generate config_tokens so we can have it as IAC: {
"eas": {
"plugins": [{
"type": "oidc",
"issuer": {
"discover_url": f"{discovery_url_base}/auth/realms/{client.realm}/.well-known/openid-configuration"
},
"client": {
"client_id": client.id,
"client_secret": client.secret
},
"scopes": ["openid", "email", "profile"],
"features": {
# how to expire the cookie
# true = cookies expire will expire with tokens
# false = cookies will be 'session' cookies
# num seconds = expire after given number of second
"cookie_expiry": True,
# how frequently to refresh userinfo data
# true = refresh with tokens(assuming they expire)
# false = never refresh
# num seconds = expire after given number of seconds
"userinfo_expiry": True,
# how long to keep a session (server side) around
# true = expire with tokenSet (if applicable)
# false = never expire
# num seconds = expire after given number of seconds (enables sliding window)
#
# sessions become a floating window *if*
# - tokens are being refreshed
# or
# - userinfo being refreshed
# or
# - session_expiry_refresh_window is a positive number
# 48 hours
"session_expiry": 172800,
# window to update the session window based on activity if
# nothing else has updated it (ie: refreshing tokens or userinfo)
#
# should be a positive number less than session_expiry
#
# For example, if session_expiry is set to 60 seconds and session_expiry_refresh_window value
# is set to 20 then activity in the last 20 seconds (40-60) of the window will 'slide' the window
# out session_expiry time from whenever the activity occurred
# 24 hours
"session_expiry_refresh_window": 86400,
# will re-use the same id (ie: same cookie) for a particular client if a session has expired
"session_retain_id": True,
# if the access token is expired and a refresh token is available, refresh
"refresh_access_token": True,
# fetch userinfo and include as X-Userinfo header to backing service
# only helpful if your specific provider has been implemented
"fetch_userinfo": True,
"introspect_access_token": False,
# which token (if any) to send back to the proxy as the Authorization Bearer value
# note the proxy must allow the token to be passed to the backend if desired
#
# possible values are access_token, or refresh_token
"authorization_token": "access_token"
},
"assertions": {
# assert the token(s) has not expired
"exp": True,
"nbf": True,
"iss": True,
# custom userinfo assertions
"userinfo": [],
"id_token": []
},
"cookie": {
"name": f"_{client.id}_session", # default is _oeas_oauth_session
# domain: "example.com", # defaults to request domain, could do sso with more generic domain
"path": "/"
# httpOnly: true,
# secure: false,
# sameSite: lax,
},
# xhr detection is determined by the presence of an 'origin' header OR X-Requested-With: XMLHttpRequest
"xhr": {
# defaults to 302 but could be set to anything
# if set to 401 the response will include a WWW-Authenticate header with proper realm/scopes
"redirect_http_code": 401,
# if set to true, the browser will be redirected to the referer
"use_referer_as_redirect_uri": True
},
"csrf_cookie": {
# TODO - this is disabled following CSRF mismatch problems
"enabled": False, # can disable the use of csrf cookies completely
# domain: "example.com", # defaults to request domain, could do sso with more generic domain
# path: "/",
# httpOnly: true,
# secure: false,
# sameSite: lax,
},
# see for details: https://github.com/travisghansen/external-auth-server/blob/master/HEADERS.md
"custom_error_headers": {},
"custom_service_headers": {
"X-Token": {
"source": "access_token",
"query_engine": "jp",
"query": "$",
"query_opts": {
"single_value": True
},
}
}
}],
},
"iat": datetime.utcnow()
}Will this do.... or do you want me to fill-in some of the template options |
|
I don't see how you could hit the block of code I have in mind with that config. The only other explanation is that you have different versions of the app running. What image tag are you using for the deployment? I'm wondering if it's been running for a while and you have some containers using an old |
|
Helm chart version: external-auth-server-0.1.0 It has indeed been running for a long time.... thought the last update we made to the chart was That said, it could be that the container image has been cached in the cluster, and not been updated.... Now that i see that we are running using |
|
I'm guessing that's the root of the issue. https://github.com/travisghansen/external-auth-server/blob/master/CHANGELOG.md#0130 bullet 4 It's obviously a complex matrix of what tag you've picked, the pull policy, and the potential introduction of new nodes since that version was released |
|
I would redeploy with |
|
Give me some hours then.... |
|
Okay.... |
|
So was going through the logs after the change on Thursday. |
|
Main thing I'm trying to understand now is how i can have a valid Looking at the content of the Though i cannot quite figure out why. |
|
The only place I’ve seen that it’s possible to get that is if you use the logout features (which your tokens did not have enabled). Either that or a version mismatch from before the server-side state was implemented. |
|
Right. So i have just wiped the sessions in key-cloak, all keys stored in redis and cycled the deployment of the EAS... |
|
So far so good... Nothing in the logs yet regarding the original issue.... However, I am noticing an similar error that is still causing the EAS to crash: Now it seems that the state pointer contains the actual state ID, but that the state no longer exists when it tries to look it up in Redis. Not sure yet if its related to the wiping of Redis, though it did happen an hour after that was done. I'll give it more time to see if that is persistent or not |
|
It could be if you wiped redis. But state has a ttl on the redis entry of an hour or something….so if you take forever it won’t work in that scenario…which seems very unlikely unless someone walks away from their computer midway through the auth process. |
|
Also note state is very ephemeral. As soon as auth comes back to eas we proactively wipe it out regardless of success or failure with the op. |
|
Is this for a SPA? |
|
It is indeed a SPA, |
|
It might be that someone simply stepped away from the computer in the middle of the auth flow.... Though that is indeed odd. Though did not expect that to crash the EAS |
|
Yeah for sure we shouldn't crash, I'm pretty sure I have fix queued up for that already but will make sure it's in the next release no matter. Do you have specific logic built into the SPA to handle api failures and redirect? In the case of a SPA there may be some strange behavior due to the concurrent nature of api requests etc but I'd have to think it through more thoroughly to say for sure :( |
|
No worries... If the only remaining error is the person who decided it was time to go for a walk in the middle of the auth flow, then its very far down my priority list. So for now ill just keep on monitoring and let you know if i see anything else. For the SPA, we set the This setup actually seems to be working quite nicely, especially after fine-tuning the In case you wanna get deep into the code; import axios from 'axios'
import Vue from 'vue'
const DEFAULT_RETRIES = 1
class HTTPClient {
constructor(baseURL, retries = DEFAULT_RETRIES) {
this._retries = retries
this._axios = axios.create({
baseURL: baseURL,
headers: {
'X-Requested-With': 'XMLHttpRequest'
}
})
}
async _request(method, url, { data = null, params = null } = {}, retry = 0) {
if (retry > 0) {
console.warn(`Retry ${retry} for ${url}`)
}
try {
const response = await this._axios({
method,
url,
data,
params
})
return response.data
} catch ({ name, message, response }) {
if (retry < this._retries) {
return this._request(method, url, { data, params }, retry + 1)
} else {
this.errorCheck(response)
throw new Error(`API call failed for '${url}', with ${name}: ${message}`)
}
}
}
async get(path, params = null) {
return await this._request('GET', path, { params })
}
async post(path, data) {
return await this._request('POST', path, { data })
}
....
errorCheck(response) {
switch (response.status) {
case 401:
window.location.reload(true)
break
default:
return
}
}
} |
|
So both the SPA itself and the API are secured by eas? |
|
Yes they are. Though i can confirm that that the issue is still present, and that one user didn't even get passed the auth-flow before getting a 503, and that it does not seem to be a side effect of logging out. Still blowing up on the redirect call back to set the session cookie. The state pointer still has the state payload in its request at this point. I decrypted the incoming requests query parameter |
|
Was unable to reproduce it but since the behavior seem to follow a specific user Ithought it had something to do with the identity provider. So i created a brand new user in keycloak, singed in and tested that it worked. The user signed in, got asked to change the password. and immediately after finishing up the auth flow got a 503 back and is stuck in the same situation. I resat the password of the user and tested the user again, and surprise surprise the account and system works again. So what ever the problem is, its directly linked to this specific users machine. So ill start gathering information of the specifics here (might be a strange browser or extension). Also the flow has been tested and is working with Firefox, Chrome, Safari, Opera and Edge ....... Managed to get on a call with the user. I had 10-15 people test it as well, from every conceivable OS platform and with a all the browsers that we could get a hold of.... And we cannot reproduce it either... ~ Unsettling Though i still see plenty (once per hour or so) of exceptions of this type in the logs causing the EAS to fall over: TypeError: Cannot read properties of undefined (reading 'match')
at Object.parse (/home/eas/app/node_modules/uri-js/dist/es5/uri.all.js:874:29)
at OpenIdConnectPlugin.get_authorization_redirect_uri (/home/eas/app/src/plugin/oauth/index.js:2514:27)
at handle_auth_callback_request (/home/eas/app/src/plugin/oauth/index.js:1649:45)
at OpenIdConnectPlugin.verify (/home/eas/app/src/plugin/oauth/index.js:1924:29)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async processPipeline (/home/eas/app/src/server.js:399:13)I can categorize the error into two separate issues:
How offend it happens can be seen here: Oh actually one or two of the errors in that graph is just deprecation warnings: |
|
LOL....! I found the cause of one of the users problem..... That will most likely explain and solve the second category of issues.... |
|
No further action is needed.... i'll wait for you next release. Sorry for this long thread of silliness. I'm happy to close the issue at this point. |
|
Great! That at least explains the issue. No problem on the thread..let's leave it open for now and when I close it down I'll reference any commits that are relevant. |
Heyo!
I have a EAS installation running in a kubernetes cluster in HA-mode.
I have multiple providers/configurations hooked up to a keycloak instance for authenticating users for a SPA.
Direct sign-in using a keycloak user works.
Signing in with a SSO backed by another keycloak instance also works.
But a final SSO backed by Active directory seems to cause the EAS to crash with the following error:
I changed the log-level to silly and reproduced the error:
Surely I'm simply doing something silly here with the SSO configuration that i have overlooked. Though having trouble pin-pointing exactly what is causing it.
In addition, i think a check is missing here:
https://github.com/travisghansen/external-auth-server/blob/master/src/plugin/oauth/index.js#L1919
Either as a check for null/undefined on the
decodedStatePointer.state_idpropertyOr as a check on the resulting
statePayload....More will follow as i dive deeper...
In the meantime any advice?
The text was updated successfully, but these errors were encountered: