Amplify DataStore: many-to-many join table not syncing when using selective sync

[stephenjen]

Description

I'm experiencing an issue with Amplify DataStore in a Flutter application. The main models (Share and OperationMode) are syncing correctly, but the automatically generated join table for a many-to-many relationship (ShareOperationMode) is not syncing to the device.

Here is the schema:

type Share @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
  id: ID!
  // ... other fields ...
  operationModes: [OperationMode] @manyToMany(relationName: "ShareOperationMode")
  pOwner: String!
}

type OperationMode @model @auth(rules: [{allow: private}]) {
  id: ID!
  // ... other fields ...
  shares: [Share] @manyToMany(relationName: "ShareOperationMode")
}

Here is the syncExpressions:

final syncExpressions = [
  DataStoreSyncExpression(Share.classType, 
    () => Share.POWNER.eq(prefs.getString('userId') ?? '')
      .or(Share.POWNER.isOneOf(prefs.getStringList('friendIds') ?? []))
  ),
  DataStoreSyncExpression(OperationMode.classType, () => QueryPredicate.all),
];

Categories

• Analytics
• API (REST)
• API (GraphQL)
• Auth
• Authenticator
• DataStore
• Notifications (Push)
• Storage

Steps to Reproduce

1. Add selective sync to Share model
2. Install app
3. Wait for sync to complete. After initial syncing, I run DataStore.close() to stop so sync expression can be loaded again to load records owned by user's friends
4. Share and OperationMode sync correctly, but ShareOperationMode does not sync and is left with zero records.

Screenshots

No response

Platforms

• iOS
• Android
• Web
• macOS
• Windows
• Linux

Flutter Version

3.24.0

Amplify Flutter Version

2.4.1

Deployment Method

Amplify CLI (Gen 1)

Schema

No response

[tyllark]

Hi @stephenjen, thank you for submitting this issue. We will take a look at this issue and get back to you when we have any updates or questions.

[stephenjen]

An update - I'm able to sync ShareOperationMode items for the owner but not for the owner's friends, even through I am able to sync owner's friends' Share items, which has a sync expression of Share.POWNER.isOneOf(prefs.getStringList('friendIds') ?? []).

[stephenjen]

An update - It works properly when I clear() as opposed to stop() before restarting DataStore to reevaluate sync expressions. Please make this work for stop() too, as having to wait for a clear() is quite impractical outside of the initial sync when users first install and get the app going.

Maybe there's a better way for me. My current use case: users can find and add friends in the app, and when they do, they can also see items owned by their friends. Currently I reevaluate sync expressions to enable this functionality, but is there a way to set up my schema so I can 1) accomplish the functionality I just described; and 2) do so without users having to carry around both their friends' and non-friends' items?

[stephenjen]

Any updates?

[khatruong2009]

@stephenjen, do you get this problem for Android too or is this only an issue you're having with iOS?

[stephenjen]

I'm only developing for iOS, so can only speak to it not working in iOS

[khatruong2009]

Hi @stephenjen, isOneOf isn't in our list of supported predicates. As an alternative to that, you could building a predicate group of or predicates that checks if POWNER is equal to each item of the prefs.getStringList('friendIds')

[stephenjen]

@khatruong2009 yes, that is a custom extension that essentially creates a series of .or() predicates.

I'm not sure this is the issue as this custom extension works for the Share and other models.

The issue seems to be when the sync expressions is reevaluated for Share, the automatically generated ShareOperationMode (the many to many join model) isn't updated.

[stephenjen]

Here's the custom isOneOf extension:

extension QueryFieldX<T> on QueryField<T> {
  QueryPredicate isOneOf(List<T> items) {
    // assert(items.isNotEmpty);
    if (items.isEmpty) {
      return eq('' as T);
    }

    if (items.length == 1) {
      return eq(items[0]);
    }
    QueryPredicateGroup queryPredicate = eq(items[0]).or(eq(items[1]));
    for (var i = 2; i < items.length; i++) {
      queryPredicate = queryPredicate.or(eq(items[i]));
    }
    return queryPredicate;
  }
}

[Equartey]

Hi @stephenjen, thanks for providing that extra query. We still working to reproduce this behavior. We will let you know our findings.

[stephenjen]

Hi @Equartey , thanks for the update. Please try my schema to see if the issue can be replicated on your end. Thanks.

type Tag @model @auth(rules: [{allow: private}]) {
    id: ID!
    name: String! @index(name: "byName")
    scheduledBlocks: [ScheduledBlock] @manyToMany(relationName: "ScheduledBlockTag")
    #  trayBlocks: [TrayBlock] @manyToMany(relationName: "TrayBlockTag")
    trayBlocks: [TrayBlock] @hasMany(indexName: "TrayBlockTag")
    seriess: [Series] @manyToMany(relationName: "SeriesTag")
    metrics: [Metric] @manyToMany(relationName: "MetricTag")
    userStatsPerformanceModeMetricsAdherence: [UserStatsPerformanceModeMetricsAdherence] @hasMany(indexName: "byTag", fields: ["id"])
}

type ScheduledBlock @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [read] }]) {
    id: ID!
    title: String!
    startTime: AWSDateTime!
    endTime: AWSDateTime!
    duration: Float
    tags: [Tag] @manyToMany(relationName: "ScheduledBlockTag")
    #    activities: [Activity] @hasMany(indexName: "byScheduledBlock", fields: ["id"])
    trayBlockID: ID @index(name: "byTrayBlock", sortKeyFields: ["title"])  # problem
    trayBlock: TrayBlock @belongsTo(fields: ["trayBlockID"])  # problem
    #  seriesTrayBlockID: ID @index(name: "bySeriesTrayBlock", sortKeyFields: ["title"])
    #  seriesTrayBlock: SeriesTrayBlock @belongsTo(fields: ["seriesTrayBlockID"])
    pOwner: String
}

type Metric @model @auth(rules: [{allow: private}]) {
    id: ID!
    title: String!
    duration: Int!
    priority: Int!
    operationModeID: ID! @index(name: "byOperationMode", sortKeyFields: ["title"])
    operationMode: OperationMode! @belongsTo(fields: ["operationModeID"])
    tags: [Tag] @manyToMany(relationName: "MetricTag")
}

#  UserOperationMode
type UserOperationMode @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [read] }]) {
    id: ID!
    date: AWSDateTime!
    operationModeID: ID @index(name: "byOperationModeUserOperationMode", sortKeyFields: ["date"]) # problem
    operationMode: OperationMode @belongsTo(fields: ["operationModeID"]) # problem
    pOwner: String!
    test: String
}

type Settings @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
    id: ID!
    name: String!
    value: String!
    pOwner: String!
}

# OperationMode
type OperationMode @model @auth(rules: [{allow: private}]) {
    id: ID!
    title: String!
    shortDescription: String
    description: String!
    longDescription: String
    featured: Boolean!
    featuredPriority: Int!
    category: String!
    categoryPriority: Int!
    metric: [Metric] @hasMany(indexName: "byOperationMode", fields: ["id"])
    userOperationMode: [UserOperationMode] @hasMany(indexName: "byOperationModeUserOperationMode", fields: ["id"])  # problem
    series: [Series] @hasMany(indexName: "bySeries", fields: ["id"])
    trayBlocks: [TrayBlock] @manyToMany(relationName: "OperationModeTrayBlock")  # problem
    profileImage: String
    featuredProfileImage: String
    authorID: ID @index(name: "byAuthor", sortKeyFields: ["title"])
    author: Author @belongsTo(fields: ["authorID"])
    shares: [Share] @manyToMany(relationName: "ShareOperationMode")
    #    shareID: ID @index(name: "byOperationModeShare", sortKeyFields: ["id"])
    #    share: Share @belongsTo(fields: ["shareID"])
    activities: [Activity] @hasMany(indexName: "byOperationMode", fields: ["id"])
    stats: [UserStatsOperationMode] @hasMany(indexName: "byOperationMode", fields: ["id"]) # problem
}

type Author @model @auth(rules: [{allow: private}]) {
    id: ID!
    fname: String!
    lname: String!
    mname: String
    description: String
    profileImage: String
    operationModes: [OperationMode] @hasMany(indexName: "byAuthor", fields: ["id"])
}

type Series @model @auth(rules: [{allow: private}]) {
    id: ID!
    title: String!
    subtitle: String!
    description: String!
    longDescription: String!
    tags: [Tag] @manyToMany(relationName: "SeriesTag")
    operationModeID: ID! @index(name: "bySeries", sortKeyFields: ["title"])
    operationMode: OperationMode! @belongsTo(fields: ["operationModeID"])
    shares: [Share] @manyToMany(relationName: "ShareSeries")
    #    shareID: ID @index(name: "bySeriesShare", sortKeyFields: ["id"])
    #    share: Share @belongsTo(fields: ["shareID"])
    trayBlocks: [SeriesTrayBlock] @hasMany(indexName: "bySeriesSeriesTrayBlockA", fields: ["id"])
    activities: [Activity] @hasMany(indexName: "bySeriesActivity", fields: ["id"])
    recommendedStartTime: String
    recommendedStartTimeTime: AWSDateTime
    recommendedEndTimeTime: AWSDateTime
    featured: Boolean
    featuredPriority: Int
    duration: String
    recommendedTimeOfDay: String
    image: String
}

type TrayBlock @model @auth(rules: [{allow: private}]) {
    id: ID!
    title: String!
    description: String!
    longDescription: String
    oneWordDescription: String
    duration: Int!
    #  tags: [Tag] @manyToMany(relationName: "TrayBlockTag")
    tagID: ID! @index(name: "TrayBlockTag", sortKeyFields: ["id"])
    tag: Tag! @belongsTo(fields: ["tagID"])
    seriess: [SeriesTrayBlock] @hasMany(indexName: "byTrayBlockSeriesA", fields: ["id"])
    operationModes: [OperationMode] @manyToMany(relationName: "OperationModeTrayBlock")  # problem
    scheduledBlocks: [ScheduledBlock] @hasMany(indexName: "byTrayBlock", fields: ["id"])  # problem
}

type SeriesTrayBlock @model @auth(rules: [{allow: private}]) {
    id: ID!
    sequence: Int
    seriesID: ID! @index(name: "bySeriesSeriesTrayBlockA", sortKeyFields:["id"])
    trayBlockID: ID! @index(name: "byTrayBlockSeriesA", sortKeyFields:["id"])
    series: Series @belongsTo(fields: ["seriesID"])
    trayBlock: TrayBlock @belongsTo(fields: ["trayBlockID"])
    keyTrayBlock: Int
    #  scheduledBlocks: [ScheduledBlock] @hasMany(indexName: "bySeriesTrayBlock", fields: ["id"])
}

#type Friend @model @auth(rules: [{ allow: owner, ownerField: "pOwner"  }{ allow: private, operations: [read] }]) {
type Friend @model @auth(rules: [{ allow: private, ownerField: "pOwner"  }]) {
    id: ID!
    profileID: ID @index(name: "byProfile", sortKeyFields: ["id"]) # had to remove the ! here for activity to work
    profile: Profile @belongsTo(fields: ["profileID"]) # had to remove the ! here for activity to work
    activities: [Activity] @hasMany(indexName: "byFriend", fields: ["id"])
    approved: Boolean
    blocked: Boolean
    pOwner: String!
}
enum ActivityAction {
    STARTEDSCHEDULING
    ADDEDFRIEND
    CHANGEDOPERATIONMODE
    COMPLETIONPROGRESS
    USEDSERIES
    ACHIEVEDGOALS
    ALMOSTACHEIVEDGOALS
    UPLOADEDIMAGE
    MODIFIEDSCHEDULE
    MODIFIEDSCHEDULEMAJOR
    LIKEDACTIVITY
    LIKEDSERIES
    LIKEDBLOCK
    LIKEDACHIEVEDGOALS
    LIKEDCOMMENT
    LIKEDUPLOAD
    COMMENTEDACTIVITY
    COMMENTEDSERIES
    COMMENTEDBLOCK
    COMMENTEDACHEIVEDGOALS
    COMMENTEDALMOSTACHEIVEDGOALS
}
#type Activity @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [create, delete, read, update] }]) {
type Activity @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
    #  id: ID! @primaryKey(sortKeyFields: ["action", "pOwner", "date"])
    id: ID!
    date: AWSDateTime!
    action: ActivityAction!
    friendID: ID! @index(name: "byFriend", sortKeyFields: ["date"])
    friend: Friend! @belongsTo(fields: ["friendID"])
    operationModeID: ID @index(name: "byOperationMode", sortKeyFields: ["date"])
    operationMode: OperationMode @belongsTo(fields: ["operationModeID"])
    comments: [Comment] @hasMany(indexName: "byActivityC", fields: ["id"])
    seriesID: ID @index(name: "bySeriesActivity", sortKeyFields: ["date"])
    series: Series @belongsTo(fields: ["seriesID"])
    pOwnerProfileID: ID @index(name: "activitiesByProfile", sortKeyFields: ["date"])
    pOwnerProfile: Profile @belongsTo(fields: ["pOwnerProfileID"])
    completionPercentages: AWSJSON
    completionHours: AWSJSON
    countedSchedules: [String!]
    pOwner: String!
}

#type Like @model @auth(rules: [{ allow: owner, ownerField: "pOwner"  }{ allow: private, operations: [create, delete, read, update] }]) {
type Like @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
    id: ID!
    #    activityID: ID! @index(name: "byActivity", sortKeyFields: ["id"])
    #    activity: Activity! @belongsTo(fields: ["activityID"])
    shareID: ID! @index(name: "byShare", sortKeyFields: ["id"])
    share: Share! @belongsTo(fields: ["shareID"])
    content: String
    pOwner: String!
}

#type Comment @model @auth(rules: [{ allow: owner, ownerField: "pOwner"  }{ allow: private, operations: [create, delete, read, update] }]) {
type Comment @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
    id: ID!
    activityID: ID! @index(name: "byActivityC", sortKeyFields: ["id"])
    activity: Activity! @belongsTo(fields: ["activityID"])
    content: String!
    pOwner: String!
}
type Profile @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
    id: ID!
    userName: String
    fname: String
    lname: String
    profileImage: String
    remoteImageURL: String
    bio: String
    pOwner: String!
    friends: [Friend] @hasMany(indexName: "byProfile", fields: ["id"])
    activities: [Activity] @hasMany(indexName: "activitiesByProfile", fields: ["id"])
    shares: [Share] @hasMany(indexName: "sharesByProfile", fields: ["id"])
    onboarded: Boolean
}

type UserStatsPerformanceModeCategoriesUsed @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [read] }]) {
    id: ID!
    pOwner: String!
    lastUpdated: AWSDateTime!
    year: Int!
    category: String!
    m1: Int
    m2: Int
    m3: Int
    m4: Int
    m5: Int
    m6: Int
    m7: Int
    m8: Int
    m9: Int
    m10: Int
    m11: Int
    m12: Int
}

type UserStatsDaysScheduled @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [read] }]) {
    id: ID!
    pOwner: String!
    year: Int!
    lastUpdated: AWSDateTime!
    m1: Int
    m2: Int
    m3: Int
    m4: Int
    m5: Int
    m6: Int
    m7: Int
    m8: Int
    m9: Int
    m10: Int
    m11: Int
    m12: Int
    weeklyStats: String
}

type UserStatsOperationMode @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [read] }]) {
    id: ID!
    pOwner: String!
    operationModeID: ID @index(name: "byOperationMode", sortKeyFields: ["lastUpdated"])
    operationMode: OperationMode @belongsTo(fields: ["operationModeID"])
    lastUpdated: AWSDateTime!
    year: Int
    m1: Int
    m2: Int
    m3: Int
    m4: Int
    m5: Int
    m6: Int
    m7: Int
    m8: Int
    m9: Int
    m10: Int
    m11: Int
    m12: Int
}

type UserStatsStreaks @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [read] }]) {
    #type UserStatsStreaks @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
    id: ID!
    pOwner: String!
    year: Int!
    lastScheduledDayOverall: AWSDateTime
    m1CountOverall: Int
    m2CountOverall: Int
    m3CountOverall: Int
    m4CountOverall: Int
    m5CountOverall: Int
    m6CountOverall: Int
    m7CountOverall: Int
    m8CountOverall: Int
    m9CountOverall: Int
    m10CountOverall: Int
    m11CountOverall: Int
    m12CountOverall: Int
    m1LongestStreakOverall: Int
    m2LongestStreakOverall: Int
    m3LongestStreakOverall: Int
    m4LongestStreakOverall: Int
    m5LongestStreakOverall: Int
    m6LongestStreakOverall: Int
    m7LongestStreakOverall: Int
    m8LongestStreakOverall: Int
    m9LongestStreakOverall: Int
    m10LongestStreakOverall: Int
    m11LongestStreakOverall: Int
    m12LongestStreakOverall: Int
}

type UserStatsPerformanceModeMetricsAdherence @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [read] }]) {
    id: ID!
    pOwner: String!
    year: Int!
    tagID: ID! @index(name: "byTag", sortKeyFields: ["year"])
    tag: Tag! @belongsTo(fields: ["tagID"])
    m1MetricQuota: Int
    m1MetricScheduled: Int
    m2MetricQuota: Int
    m2MetricScheduled: Int
    m3MetricQuota: Int
    m3MetricScheduled: Int
    m4MetricQuota: Int
    m4MetricScheduled: Int
    m5MetricQuota: Int
    m5MetricScheduled: Int
    m6MetricQuota: Int
    m6MetricScheduled: Int
    m7MetricQuota: Int
    m7MetricScheduled: Int
    m8MetricQuota: Int
    m8MetricScheduled: Int
    m9MetricQuota: Int
    m9MetricScheduled: Int
    m10MetricQuota: Int
    m10MetricScheduled: Int
    m11MetricQuota: Int
    m11MetricScheduled: Int
    m12MetricQuota: Int
    m12MetricScheduled: Int
}

type Share @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }{ allow: private, operations: [read] }]) {
    #type Share @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
    id: ID!
    sharedDate: AWSDateTime!
    completionDate: AWSDateTime!
    note: String
    #    seriess: [Series] @hasMany(indexName: "bySeriesShare", fields: ["id"])
    #    operationModes: [OperationMode] @hasMany(indexName: "byOperationModeShare", fields: ["id"])
    seriess: [Series] @manyToMany(relationName: "ShareSeries")
    operationModes: [OperationMode] @manyToMany(relationName: "ShareOperationMode")
    numberOfBlocks: Int
    completionPercentages: AWSJSON
    completionHours: AWSJSON
    likes: [Like] @hasMany(indexName: "byShare", fields: ["id"])
    pOwnerProfileID: ID @index(name: "sharesByProfile", sortKeyFields: ["sharedDate"])
    pOwnerProfile: Profile @belongsTo(fields: ["pOwnerProfileID"])
    imageURL: String
    pOwner: String!
}

type FriendsList @model @auth(rules: [{ allow: owner, ownerField: "pOwner" }]) {
    id: ID!
    friendsIds: [String!]!
    pOwner: String!
}

# this is post and activity is comments
#type test @model @auth(rules: [{ allow: private, ownerField: "pOwner" }]) {
#    id: ID!
#    friendsIds: String
#    pOwner: String!
#}

[NikaHsn]

thank you for providing these details. we will look into this issue and get back to you with any updates.