Add support for custom index on hasMany / expose a list query on the secondary index created by hasMany

[bogris]

Environment information

System:
  OS: macOS 14.0
  CPU: (8) arm64 Apple M1 Pro
  Memory: 128.48 MB / 16.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 22.3.0 - ~/.nvm/versions/node/v22.3.0/bin/node
  Yarn: undefined - undefined
  npm: 10.8.1 - ~/.nvm/versions/node/v22.3.0/bin/npm
  pnpm: 8.9.0 - ~/.npm-packages/bin/pnpm
NPM Packages:
  @aws-amplify/auth-construct: Not Found
  @aws-amplify/backend: 1.0.4
  @aws-amplify/backend-auth: Not Found
  @aws-amplify/backend-cli: 1.2.2
  @aws-amplify/backend-data: Not Found
  @aws-amplify/backend-deployer: Not Found
  @aws-amplify/backend-function: Not Found
  @aws-amplify/backend-output-schemas: Not Found
  @aws-amplify/backend-output-storage: Not Found
  @aws-amplify/backend-secret: Not Found
  @aws-amplify/backend-storage: Not Found
  @aws-amplify/cli-core: Not Found
  @aws-amplify/client-config: Not Found
  @aws-amplify/deployed-backend-client: Not Found
  @aws-amplify/form-generator: Not Found
  @aws-amplify/model-generator: Not Found
  @aws-amplify/platform-core: Not Found
  @aws-amplify/plugin-types: Not Found
  @aws-amplify/sandbox: Not Found
  @aws-amplify/schema-generator: Not Found
  aws-amplify: 6.4.4
  aws-cdk: 2.151.0
  aws-cdk-lib: 2.151.0
  typescript: 5.5.3
AWS environment variables:
  AWS_STS_REGIONAL_ENDPOINTS = regional
  AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
  AWS_SDK_LOAD_CONFIG = 1
No CDK environment variables
npm notice
npm notice New patch version of npm available! 10.8.1 -> 10.8.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2
npm notice To update run: npm install -g [email protected]
npm notice
npm notice
npm notice New patch version of npm available! 10.8.1 -> 10.8.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2
npm notice To update run: npm install -g [email protected]
npm notice

Description

In this version, hasMany will create a secondary index for internal queries but:

• the secondary index dose not expose a list query in AppSync and subsequently in the client.
• we can't assign a user-created secondary index to hasMany (as in v1)

I find this important for the case where we need to query the secondary index to find children by "parentId" given that I have the parent id.

Let's assume we have this:

const schema = a.schema({
  Todo: a
    .model({
      content: a.string(),
      description: a.string(),
      comments: a.hasMany("Comment", "parentId"),
    })
    .authorization((allow) => [allow.guest()]),

  Comment: a
    .model({
      content: a.string(),
      parentId: a.string().required(),
      todo: a.belongsTo("Todo", "parentId"),
    })
    .authorization((allow) => [allow.guest()]),
});

if i need to load the todo and the comments, i can build an efficient query with selectors:

const { data: todos, errors } = await client.models.Todo.list({
    selectionSet: [
      "id",
      "content",
      "description",
      "comments.*",
    ],
  });

but i can't have a proper way to get the Comments by todoId.

The only thing I found was lazy loading the Comments by passing the todo:

const { data: comments } = await todo.comments()

This will work in POCs and small projects, but it uses a filter instead of secondary index to get data so it will scan the table each time.

If we crete a manual secondary index, we double the index storage and we also count towards the max number of indexes per table.

If we can create a secondary index and declare it for usage in hasMany, all the above would be solved.

[ykethan]

Hey,👋 thanks for raising this! I'm going to transfer this over to our API repository for better assistance 🙂.

[chrisbonifacio]

Hi @bogris 👋 thanks for raising this issue. I've marked it as a feature request for the team to consider.

[bogris]

@chrisbonifacio thanks!

I am not sure regarding the status of this issue. Can I close it?

[chrisbonifacio]

@bogris we have some other customers that have asked for this feature so I would leave it open for now

[bogris]

👌 Thanks

…

On Thu, 10 Oct 2024 at 21:14, chris ***@***.***> wrote: @bogris <https://github.com/bogris> we have some other customers that have asked for this feature so I would leave it open for now — Reply to this email directly, view it on GitHub <#2751 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACROOY5IUZO5XBNCXYVC6PTZ227XXAVCNFSM6AAAAABMG3WRL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBVG42TIMBTGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>