Datastore: Support Multi-user data access

[dgagnon]

Description

By following the Getting started instructions and setting up the datastore Cloud Feature at this page: https://docs.amplify.aws/lib/datastore/sync/q/platform/flutter/#setup-cloud-sync

We can read this line: "DataStore's cloud synchronization uses the API category behind the scenes. Therefore, the first step is to add the API plugin."

This leads people to believing that they can use the documentation at the linked page.

In the same manner, the Setup authorization rules page (at https://docs.amplify.aws/lib/datastore/setup-auth-rules/q/platform/flutter/ ) refers a few times to the AppSync cli page (at https://docs.amplify.aws/cli/graphql/authorization-rules/#per-user--owner-based-data-access ).

On this page, a section is name "Multi-user data access". I can attest that the code in this section will not work with Datastore, I suspect due to the limitation on subscription and multi-owner models.

I tried all possible configuration (transformer v1, removing multi auth, experimental pipeline, other feature flags, as a single model, etc) and all tests were done with an empty datastore. After working 3 days on this, I am convinced there is no way to have this working in a native way.

Having a documentation that is representative of the available feature is of paramount importance to us.

Note: While this could be framed as a bug in the datastore/appsync plugins for this specific example, this is a recurring theme with these two plugins. See #1566 (comment) for technical details.

Categories

• Analytics
• API (REST)
• API (GraphQL)
• Auth
• Authenticator
• DataStore
• Notifications (Push)
• Storage

Steps to Reproduce

1. Follow the getting started instructions and create a new project
2. Add auth (default settings without social + iam as secondary auth)
3. Add api and datastore as per the documentation
4. Add a multi-owner model
5. Try to start the app.

Screenshots

Relevant Logs:

[  +45 ms] E/amplify:aws-datastore( 6140): Failure encountered while attempting to start API sync.
[        ] E/amplify:aws-datastore( 6140): DataStoreException{message=DataStore subscriptionProcessor failed to start., cause=GraphQLResponseException{message=Subscription error for Task: [GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument authors @ 'onCreateTask'', locations='null', path='null', extensions='null'}], errors=[GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument authors @ 'onCreateTask'', locations='null', path='null', extensions='null'}], recoverySuggestion=See attached list of GraphQLResponse.Error objects.}, recoverySuggestion=Check your internet.}
[        ] E/amplify:aws-datastore( 6140): 	at com.amplifyframework.datastore.syncengine.Orchestrator.lambda$startApiSync$3$com-amplifyframework-datastore-syncengine-Orchestrator(Orchestrator.java:306)
[        ] E/amplify:aws-datastore( 6140): 	at com.amplifyframework.datastore.syncengine.Orchestrator$$ExternalSyntheticLambda6.subscribe(Unknown Source:2)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletableCreate.subscribeActual(CompletableCreate.java:40)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Completable.subscribe(Completable.java:2850)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletablePeek.subscribeActual(CompletablePeek.java:51)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Completable.subscribe(Completable.java:2850)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletablePeek.subscribeActual(CompletablePeek.java:51)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Completable.subscribe(Completable.java:2850)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletablePeek.subscribeActual(CompletablePeek.java:51)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Completable.subscribe(Completable.java:2850)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletableSubscribeOn$SubscribeOnObserver.run(CompletableSubscribeOn.java:64)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:614)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
[        ] E/amplify:aws-datastore( 6140): 	at java.util.concurrent.FutureTask.run(FutureTask.java:264)
[        ] E/amplify:aws-datastore( 6140): 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307)
[        ] E/amplify:aws-datastore( 6140): 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[        ] E/amplify:aws-datastore( 6140): 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
[        ] E/amplify:aws-datastore( 6140): 	at java.lang.Thread.run(Thread.java:1012)
[        ] E/amplify:aws-datastore( 6140): Caused by: GraphQLResponseException{message=Subscription error for Task: [GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument authors @ 'onCreateTask'', locations='null', path='null', extensions='null'}], errors=[GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument authors @ 'onCreateTask'', locations='null', path='null', extensions='null'}], recoverySuggestion=See attached list of GraphQLResponse.Error objects.}

Generated code:

input ModelSubscriptionTaskFilterInput {
  content: ModelSubscriptionStringInput
  and: [ModelSubscriptionTaskFilterInput]
  or: [ModelSubscriptionTaskFilterInput]
  _deleted: ModelBooleanInput
}

type Subscription {
  onCreateTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["createTask"]) @aws_iam @aws_cognito_user_pools
  onUpdateTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["updateTask"]) @aws_iam @aws_cognito_user_pools
  onDeleteTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["deleteTask"]) @aws_iam @aws_cognito_user_pools
}

Code to replicate:

  Future<void> configureAmplify() async {
    // Add any Amplify plugins you want to use
    final authPlugin = AmplifyAuthCognito();
    // await Amplify.addPlugin(authPlugin);

    // Add the following lines to your app initialization to add the DataStore plugin
    final datastorePlugin = AmplifyDataStore(
      modelProvider: ModelProvider.instance,
      // Be sure to add the authModeStrategy
      authModeStrategy: AuthModeStrategy.multiAuth,
    );
    // await Amplify.addPlugin(datastorePlugin);

    final api = AmplifyAPI();

    // You can use addPlugins if you are going to be adding multiple plugins
    await Amplify.addPlugins([authPlugin, api, datastorePlugin]);
    // await Amplify.addPlugins([authPlugin]);

    // Once Plugins are added, configure Amplify
    // Note: Amplify can only be configured once.
    try {
      await Amplify.configure(amplifyconfig);
    } on AmplifyAlreadyConfiguredException {
      safePrint("Tried to reconfigure Amplify; this can occur when your app restarts on Android.");
    }
  }

Platforms

• iOS
• Android
• Web
• macOS
• Windows
• Linux

Flutter Version

3.10.5

Amplify Flutter Version

1.2.0

Deployment Method

Amplify CLI

Schema

type Task @model @auth(rules: [{ allow: owner, ownerField: "authors" }]) {
    content: String
    authors: [String]
}

[fjnoyp]

Hi @dgagnon thank you very much for sharing this issue, and apologies you had to spend a lot of time as a result ...

You are right, the documentation is misleading and there is no multi owner support in Datastore at the moment. We'll look to update the docs to clarify the limits of Datastore functionality.

[dgagnon]

This is a feature we are very much interested in, you can count this a feature request as well!

[dgagnon]

I have a work around that works pretty well, using VTL resolvers and adding a "sharedWith" field on models. Maintainable easily for queries and subscriptions, but a bit weird for mutations. If there is any interest, I will take the time to write it up here.

[James1R]

I have a work around that works pretty well, using VTL resolvers and adding a "sharedWith" field on models. Maintainable easily for queries and subscriptions, but a bit weird for mutations. If there is any interest, I will take the time to write it up here.

@dgagnon If you could that be great thank you.
My scenario is a chat room where chat room hosts can create a chat room and other users can be added read-only to the chat room.
Group isn't sufficient as auth rules need to be applied for each chat room. I would end up with an enournmas amount of groups.

[dgagnon]

@James1R I will do a better write up later, but here is the schema and the required VTL for it. Note that you also need a way to get the uuid of the user you wish to share to. You will need to adjust for your actual auth settings, as this affects the VTL generated. I build these by copying from the current ones I had. This is for a multi-auth config, with IAM, owner and groups.

schema.graphql:

enum Status {
    ACTIVE
    COMPLETED
    DELETED
}

enum UserStatus {
    ACTIVE
    SUSPENDED
    DELETED
}

type User @model @auth(rules: [{ allow: owner, ownerField: "owner"}, {allow: private, provider: iam}]) {
    id: ID! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    username: String! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    data: AWSJSON @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    attributes: AWSJSON @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    # https://docs.amplify.aws/lib/datastore/sync/q/platform/flutter/#advanced-use-case---query-instead-of-scan
    status: UserStatus @default(value: "ACTIVE" ) @index(name: "byStatus") @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    owner: String @index(name: "byOwner") @auth(rules: [{ allow: owner, ownerField: "owner", operations: [read, create, delete] }, { allow: private, provider: iam }])
    tasks: [Task] @hasMany(indexName: "byUser", fields: ["id"])
    sharedWith: [String] @auth(rules: [{ allow: owner, ownerField: "owner" }, { allow: private, provider: iam }])
}
type Project @model @auth(rules: [{ allow: owner, ownerField: "owner"}, {allow: private, operations: [read]}, {allow: private, provider: iam}, { allow: groups, groups: ["admin"] }]) {
    id: ID! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    title: String! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    description: String @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    data: AWSJSON @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    status: Status @default(value: "ACTIVE" ) @index(name: "byStatus") @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    dueAt: AWSDateTime! @auth(rules: [{ allow: owner, ownerField: "owner"}, { allow: private, provider: iam }, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    owner: String @index(name: "byOwner") @auth(rules: [{ allow: owner, ownerField: "owner", operations: [read, create, delete] }, { allow: private, provider: iam }, {allow: private, operations: [read]},{ allow: groups, groups: ["admin"] }])
    tasks: [Task] @hasMany(indexName: "byProject", fields: ["id"]) #https://docs.amplify.aws/cli/graphql/data-modeling/#belongs-to-relationship
}
type Task @model @auth(rules: [{ allow: owner, ownerField: "owner"}, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }]) {
    id: ID @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }])
    title: String! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }])
    description: String @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }])
    data: AWSJSON @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }])
    status: Status @default(value: "ACTIVE" ) @index(name: "byStatus") @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"] }])
    dueAt: AWSDateTime! @auth(rules: [{ allow: owner, ownerField: "owner" }, { allow: private, provider: iam }, { allow: groups, groups: ["admin"], operations: [read] }])
    # https://docs.amplify.aws/cli/graphql/authorization-rules/#multi-user-data-access
    #    owners: [String]
    # BUG: https://github.com/aws-amplify/amplify-flutter/issues/1566
    owner: String @index(name: "byOwner") @auth(rules: [{ allow: owner, ownerField: "owner", operations: [read, create, delete] }, { allow: private, provider: iam }, { allow: groups, groups: ["admin"], operations: [read] }])
    projectID: ID! @index(name: "byProject") #https://docs.amplify.aws/cli/graphql/data-modeling/#belongs-to-relationship
    project: Project! @belongsTo(fields: ["projectID"])
    sharedWith: [String] @auth(rules: [{ allow: owner, ownerField: "owner" }, { allow: private, provider: iam }, { allow: groups, groups: ["admin"], operations: [read] }])
    userID: ID @index(name: "byUser") #https://docs.amplify.aws/cli/graphql/data-modeling/#belongs-to-relationship
    user: User @belongsTo(fields: ["userID"])
}

amplify/backup/api*/resolvers/Query.getTask.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.stash.authFilter) )
    #set( $authFilter = $ctx.stash.get("authFilter") )
    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        #if( !$util.isNull($ownerClaim0) )
            $util.qr($authFilter.or.add({"sharedWith": { "contains": $ownerClaim0 }}))
        #end
    #end
    #set( $role0_0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #if( !$util.isNull($role0_0) )
        $util.qr($authFilter.or.add({"sharedWith": { "contains": $role0_0 }}))
    #end
    #set( $role0_1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($role0_1) )
        $util.qr($authFilter.or.add({"sharedWith": { "contains": $role0_1 }}))
    #end
    $util.qr($ctx.stash.put("authFilter", $authFilter))
#end
$util.toJson({})
## [End] Enable sharing. **

amplify/backup/api*/resolvers/Query.listTasks.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.stash.authFilter) )
    #set( $authFilter = $ctx.stash.get("authFilter") )
    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        #if( !$util.isNull($ownerClaim0) )
            $util.qr($authFilter.or.add({"sharedWith": { "contains": $ownerClaim0 }}))
        #end
    #end
    #set( $role0_0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #if( !$util.isNull($role0_0) )
        $util.qr($authFilter.or.add({"sharedWith": { "contains": $role0_0 }}))
    #end
    #set( $role0_1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($role0_1) )
        $util.qr($authFilter.or.add({"sharedWith": { "contains": $role0_1 }}))
    #end
    $util.qr($ctx.stash.put("authFilter", $authFilter))
#end
$util.toJson({})
## [End] Enable sharing. **

amplify/backup/api*/resolvers/Subscription.onDeleteTask.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.args.filter) )
    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $authFilter = $ctx.args.filter.or )
    #else
        #set( $authFilter = $ctx.args.filter.and[0].or )
    #end

    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        $util.qr($authFilter.add({ "sharedWith": { "contains": $currentClaim1 } }))
    #end

    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $ctx.args.filter = { "or": $authFilter } )
    #else
        #set( $ctx.args.filter.and[0] = { "or": $authFilter } )
    #end
#end
$util.toJson({})
## [End] Enable sharing. **

amplify/backup/api*/resolvers/Subscription.onUpdateTask.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.args.filter) )
    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $authFilter = $ctx.args.filter.or )
    #else
        #set( $authFilter = $ctx.args.filter.and[0].or )
    #end

    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        $util.qr($authFilter.add({ "sharedWith": { "contains": $currentClaim1 } }))
    #end

    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $ctx.args.filter = { "or": $authFilter } )
    #else
        #set( $ctx.args.filter.and[0] = { "or": $authFilter } )
    #end
#end
$util.toJson({})
## [End] Enable sharing. **

amplify/backup/api*/resolvers/Subscription.onCreateTask.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.args.filter) )
    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $authFilter = $ctx.args.filter.or )
    #else
        #set( $authFilter = $ctx.args.filter.and[0].or )
    #end

    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        $util.qr($authFilter.add({ "sharedWith": { "contains": $currentClaim1 } }))
    #end

    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $ctx.args.filter = { "or": $authFilter } )
    #else
        #set( $ctx.args.filter.and[0] = { "or": $authFilter } )
    #end
#end
$util.toJson({})
## [End] Enable sharing. **

[James1R]

Thank you for putting this up. @dgagnon

[James1R]

@fjnoyp is there any progress on this as a feature?

[osehmathias]

Also interested. 😓