Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement group by for single-column aggregates #144

Merged
merged 44 commits into from
Mar 1, 2025

Conversation

hallettj
Copy link
Collaborator

@hallettj hallettj commented Feb 6, 2025

Implements most of the functionality for the capability query.aggregates.group_by. There are still a couple of things to follow up on.

Counts are not implemented for group by queries yet. I'll follow up on those in ENG-1568. (Counts are implemented in #145 which can be merged after this PR is merged.)

There is a bug involving multiple references to the same relationship that should be resolved. I'll follow up in ENG-1569.

While working on this I removed the custom "count" aggregation - it is redundant, and I've been meaning to do that for a while. Users can use the standard count aggregations instead.

There is a change in here that explicitly converts aggregate result values for "average" and "sum" aggregations to the result types declared in the schema. This is necessary to avoid errors in response serialization for groups when aggregating over 128-bit decimal values. I applied the same type conversion for group and for root aggregates for consistency. This does mean there will be some loss of precision in those cases. But it also means we won't get back a JSON string in some cases, and a JSON number in others.

@hallettj hallettj self-assigned this Feb 6, 2025
Copy link
Collaborator

@codedmart codedmart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments that are not relevant to merging this. I mostly did reading and spot checking, but seems fine to me.

accumulators: accumulators_for_aggregates(&grouping.aggregates)?,
};

// TODO: ENG-1562 This implementation does not fully implement the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to do this work at a specific time or this is behind a capability?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, that's behind the query.aggregates.group_by.order capability

.collect()
}

// TODO: This function can be infallible once counts are implemented
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there follow up work we need here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've followed up on this in #145

.collect()
}

// TODO: This function can be infallible once counts are implemented
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as above?

let aggregates = serialize_aggregates(mode, path, &grouping.aggregates, doc.into())?;

// TODO: This conversion step can be removed when the aggregates map key type is
// changed from String to FieldName
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a follow up ticket for this one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Effectively yes. That change is in the latest ndc-spec, and I have a ticket for updating the spec https://linear.app/hasura/issue/ENG-1573/[mongodb]-update-to-latest-ndc-spec-revision

) -> Result<plan::GroupComparisonTarget<T>> {
let plan_target = match target {
// TODO: Do we expect the target aggregate to correspond to one of the grouping aggregates?
// Or can it be independent?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a question? Do we know the answer and can remove this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch - I found the answer but forgot to remove the comment

}
ndc::GroupOrderByTarget::Aggregate { aggregate } => {
// TODO: Do we expect the target aggregate to correspond to one of the grouping aggregates?
// Or can it be independent?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a question? Do we know the answer and can remove this?

/// The observed type may be `None` if the type of a variable use could not be inferred.
pub variable_types: VariableTypes<T::ScalarType>,

// TODO: type for unrelated collection
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a ticket here to track this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe - I don't remember where this would come up

@hallettj hallettj merged commit e466b51 into main Mar 1, 2025
1 check passed
@hallettj hallettj deleted the jessehallett/eng-1486-mongodb-implement-group-by branch March 1, 2025 22:31
hallettj added a commit that referenced this pull request Mar 3, 2025
This completes the basic functionality for group-by started in #144  by implementing all forms of count aggregations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants