v8 Client: Help define the "shape" of the API

[adrianhall]

Hey folks,

I'd like to start a discussion on the shapre of the client API. I'll start with what I am thinking - feel free to chime in as to what you think, where you see problems, and what you would like to see! Everything is on the table; from how it works to what specific names are!

First off, let's look at a model:

[OfflineDatasyncTable(Table = "tableName")]
public class MyModel : ClientTableData
{
  public string Title { get; set; }
}

The ClientTableData is an implementation of ITableData with the stuff for EF Core on the client and is optional. I'm switching from a string version to a byte[] version in this; hopefully this will allow sharing of models between client and server if that's what you want to do. You just need to implement ITableData and ensure the Id is the primary key. There are likely to be other things you need to do to set up model sharing, but it will be possible.

The attribute shows that this is a remotely synchronized table. If there is no argument, the name of the model is used. You can also use Table and Endpoint. If using Table, the table is passed through a construction for the endpoint (i.e. Table = "foo" is equivalent to Endpoint = "/tables/foo" in the default case).

We have a restricted list of types that we support - basically, primitives (int, double, decimal, string, char), DateTime, DateTimeOffset, DateOnly, TimeOnly, Guid, plus GeographyPoint (for the first time!)

Next, the DbContext is the central thing here, so how do we set that up:

public class MyDbContext : OfflineDbContext
{
  public ApplicationDbContext(DbContextOptions<MyDbContext> options) : base(options)
  {
  }

  public DbSet<MyModel> MyModels => Set<MyModel>();

  public RemoteDbSet<MyOtherModel> MyOtherModels => new RemoteDbSet<MyOtherModel>();
}

This shows the basic set up of both a local and a remote database table. The local one shows that the data is synchronized from a remote service - The DbSet is completely normal. The RemoteDbSet is similar for query, but also has direct modifications. i.e. there isn't a '.Add' or '.Update'; instead there is '.AddAsync()', '.UpdateAsync()', '.RemoveAsync()' that directly modify the remote end. To ensure equivalency, there will be the same methods as extension methods on the DbSet, but they will be equivalent of '.Add(); .SaveChangesAsync()' - this allows you to change from remote to offline (and back again) without changing the rest of your code, while preserving bulk changes to the offline database. LINQ, of course, works, but also explicit OData. So, something like the following would be supported:

var results = MyOtherModels.ODataFilter("geo.distance(Location, geography'POINT(10 20)') lt 20.0").ToListAsync();

I'll also be supporting ToObservableCollection(existingCollection) for updating an existing observable collection.

To synchronize, you will need to set up a sync context.

var dsContext = new DatasyncContext<MyDbContext>(options);

Options include everything that used to be in the DatasyncClientOptions. However, I'm going to be updating the HTTP handler so that you can use an IHttpClientFactory to create the HttpClient to use with the appropriate base URI. You can use the default one. So, reality, you could do something like the following minimally:

var options = new DatasyncContextOptions
{
  HttpClientFactory = new DefaultHttpClientFactory(myUri)
};

Some basic operations:

# Push all tables
await dsContext.PushAsync();

# Push a single table
await dsContext.PushAsync(typeof(MyModel));

# Push multiple tables
await dsContext.PushAsync(new[] { typeof(MyModel), typeof(MyModel2) });

# Pull all tables
await dsContext.PullAsync();

# Synchronize - push, then pull
await dsContext.SyncAsync();

There are the similar additions to each one. In addition, there will be a PushOptions, PullOptions, and SyncOptions to modify the operation. You'll be able to specify a pull query, for example, to only pull certain records. In addition, you'll be able to specify a dictionary of type => options for the operation, allowing you to specify both a default set of options and per-table options.

The sync context will also have eventing on it, so you will be able to get notified on updates to the database. The sync context creates the DbContext when needed (since it's meant to be short lived), so it should play nice with any EF code out there.

What else? Authentication will be done pretty much the same way - on demand with a basic authentication handler. But you will set up the pipeline when using a customer IHttpClientFactory, so you are responsible for it, really.

What have I missed? What do you think? Where are the problems?

[henda79]

Look great so far Adrian. So unlike the previous version, we get to pass in our own HttpClient, so this could be one we have built using the MAUI builder ?

[adrianhall]

Correct. There will be a default one provided which has much of the same semantics as the old one. Or, you can provide your own implementation of IHttpClientFactory which does much the same thing. We'll use the BaseUri from the HttpClient to construct the endpoints.

[fileman]

Hi,
have you planned to be case insensitive?
https://github.com/Azure/azure-mobile-apps/issues/815#issuecomment-1806613324

[adrianhall]

That’s a complex discussion.

On the server side, the TableController does its own serialization and deserialization using its own JSON serialization options. These are dependency injected through an interface, so you CAN alter them and they are independent to the rest of your application.

On the client side, I intend to switch over to System.Text.Json and provide for the same facility - a standard interface that can be provided when constructing the HTTP client for serialization and deserialization, allowing you to match what is happening on the server.

So, yes - you can be case insensitive. However, I doubt it matters since the serialization settings are linked to the table controllers, not the overall application. This is how we support co-existence with ABP.

[fileman]

On the v6 client side there are some properties hard coded, so changing case naming in serialization break some functionalities, while on server side no issue.

I can try to find the point where the error was generated if it can be useful to you.
Thanks

[adrianhall]

I understand what you are saying. The names of those will be identical (since they are encoded in the ITableData). However, you’ll be able to use serialization and deserialization to affect how they travel through the wire.

[adrianhall]

So - currently writing the "shape" (aka interface) for the remote dataset. here is what I am thinking:

public interface IReadonlyRemoteDataset<TEntity> : IQueryable<TEntity>
{
  ValueTask<TEntity> FindAsync(string id, RemoteOperationOptions options, CancellationToken token = default);
}

public interface IRemoteDataset : IReadonlyRemoteDataset
{
  ValueTask<TEntity> AddAsync(TEntity entity, RemoteOperationOptions options, CancellationToken token = default);
  ValueTask RemoveAsync(string id, byte[]? version, RemoteOperationOptions options, CancellationToken token = default);
  ValueTask<TEntity> UpdateAsync(TEntity entity, RemoteOperationOptions options, CancellationToken token = default);
}

In addition, there will be several extension methods to drop the RemoteOperationOptions (the only options right now are IncludeDeletedItems and ForceOperation). Also to FindAsync(TEntity entity), RemoveAsync(TEntity entity) so that you can refresh and remove an entity.

Also, both implementations will fully implement IQueryable (i.e. LINQ) stuff - so you'll be able to do remoteSet.Where(...).Select(...).Skip(x).Take(y).ToAsyncEnumerable() or whatever you want.

Seem reasonable?

The concrete implementation will have something like:

var remoteSet = new RemoteDataset<Entity>(relativeEndpoint, httpClientFactory);

// OR...
var remoteSet = new RemoteDataset<Entity>(options);

Where options is equivalent to the DatasyncClientOptions in the v6 codebase - i.e. it has stuff to generate the remote endpoint and HTTP client factory.

Thoughts? Do you like RemoteDataset<T> or RemoteDbSet<T> or RemoteTable<T> for this?

[fileman]

maybe RemoteDbSet is more coerent with ef naming

[henda79]

I've never been the architect type of developer, I only usually see issues after use, so I'll leave the shaping in your more than capable hands. Hopefully some others will chime in.