Propagate Nexus Operation-Timeout header #6676
Conversation
| @@ -110,6 +111,14 @@ func (ch *commandHandler) HandleScheduleCommand( | |||
| } | |||
| } | |||
|
|
|||
| if err := timestamp.ValidateProtoDuration(attrs.ScheduleToCloseTimeout); err != nil { | |||
There was a problem hiding this comment.
Not exactly related to this PR but I realized I missed this validation in #6631
| // (ScheduleToClose unset, Operation-Timeout set) -> no changes, use Operation-Timeout | ||
| header := nexus.Header(args.header) | ||
| opTimeout := args.scheduleToCloseTimeout | ||
| opTimeoutHeader, set := header["Operation-Timeout"] |
There was a problem hiding this comment.
Planning to replace these Operation-Timeout literals once the Nexus SDK is upgraded to a version with the constant added in nexus-rpc/sdk-go#24
There was a problem hiding this comment.
It's released but waiting for @justinp-tt to merge the Nexus SDK upgrade PR.
| // (ScheduleToClose unset, Operation-Timeout set) -> no changes, use Operation-Timeout | ||
| header := nexus.Header(args.header) | ||
| opTimeout := args.scheduleToCloseTimeout | ||
| opTimeoutHeader, set := header["Operation-Timeout"] |
There was a problem hiding this comment.
It's released but waiting for @justinp-tt to merge the Nexus SDK upgrade PR.
| // The following logic handles the operation ScheduleToCloseTimout parameter and Operation-Timeout header. | ||
| // The minimum of the resolved operation timeout and the configured request timeout is used for the context timeout | ||
| // when making the StartOperation request, according to the following logic: | ||
| // (ScheduleToClose set, Operation-Timeout set) -> no changes, use ScheduleToClose |
There was a problem hiding this comment.
| // (ScheduleToClose set, Operation-Timeout set) -> no changes, use ScheduleToClose | |
| // (ScheduleToClose set, Operation-Timeout set) -> Use Operation-Timeout |
There was a problem hiding this comment.
I thought we discussed that ScheduleToClose should always be used if it is set.
There was a problem hiding this comment.
I mean just the Operation-Timeout header, not the context deadline.
| // (ScheduleToClose unset, Operation-Timeout set) -> no changes, use Operation-Timeout | ||
| header := nexus.Header(args.header) | ||
| opTimeout := args.scheduleToCloseTimeout | ||
| opTimeoutHeader, set := header["Operation-Timeout"] |
There was a problem hiding this comment.
Use .Get() which is case insensitive.
| if header == nil { | ||
| header = make(nexus.Header, 1) | ||
| } | ||
| header["Operation-Timeout"] = args.scheduleToCloseTimeout.String() |
There was a problem hiding this comment.
We need to export this from the Nexus SDK and use it here and for Request-Timeout: https://github.com/nexus-rpc/sdk-go/pull/20/files#diff-98737c3c4b252474548805ea1be712acf49b5514a96e5ffdbc642c366482f282R363
There was a problem hiding this comment.
Also please use time left, not time since the operation was originally scheduled by the workflow. If this is the 10th attempt and an hour has already gone by, it should be reflected in the header.
| header = make(nexus.Header, 1) | ||
| } | ||
| header["Operation-Timeout"] = args.scheduleToCloseTimeout.String() | ||
| } else if set && args.scheduleToCloseTimeout == 0 { |
There was a problem hiding this comment.
Let's just ignore the user header and not enforce this. It's not technically supported and doesn't affect what the caller machinery scheduleToClose timeout.
| @@ -423,7 +423,18 @@ func (c *physicalTaskQueueManagerImpl) DispatchNexusTask( | |||
| request *matchingservice.DispatchNexusTaskRequest, | |||
| ) (*matchingservice.DispatchNexusTaskResponse, error) { | |||
| deadline, _ := ctx.Deadline() // If not set by user, our client will set a default. | |||
| task := newInternalNexusTask(taskId, deadline, request) | |||
| var opDeadline time.Time | |||
| opTimeoutHeader, set := request.GetRequest().GetHeader()["Operation-Timeout"] | |||
There was a problem hiding this comment.
What if header is nil here? What if the header is user provided (e.g. via curl) and is in lower case?
There was a problem hiding this comment.
Also note that this is only applicable for StartOperation requests but I don't think it hurts to manipulate on the other requests.
| opTimeout, err := time.ParseDuration(opTimeoutHeader) | ||
| if err != nil { | ||
| // Operation-Timeout header is not required so don't fail request on parsing errors. | ||
| c.logger.Warn(fmt.Sprintf("unable to parse Operation-Timeout header: %v", opTimeoutHeader), tag.Error(err)) |
There was a problem hiding this comment.
Maybe log the request ID here if one is present?
What changed?
Added support to propagate the new
Operation-Timeoutheader. If this timeout is shorter than the context timeout, the context timeout will be reduced.Why?
How did you test it?
New tests