Azure Cosmos Index issue

[scottparise]

I just wanted to point out I ran into an issue when trying to deploy to Azure. The issue arose after upgrading to version 9.0.0. I know you changed the indexes, but there was an additional problem when running in CosmosDB.

I'm pretty sure it has to do with the line:

.SortByDescending(h => h.DatabaseVersion) (line 24 in Migrator.cs)

the fix for me was to apply an index on the DatabaseVersion in the Migration Collection. Apparently Azure Cosmos DB doesn't like using the SortByDescending on a field where there is no index. Just FYI, this all worked just fine on 8.3.3 and only came about when i upgraded to 9.x

The error i was getting was:

Exception has occurred: CLR/MongoDB.Driver.MongoCommandException
An exception of type 'MongoDB.Driver.MongoCommandException' occurred in MongoDB.Driver.Core.dll but was not handled in user code: 'Command find failed: Error=2, Details='Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 16b5a972-ebe3-4ad8-b76c-a4b22ae682e6; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 16b5a972-ebe3-4ad8-b76c-a4b22ae682e6; Reason: (Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 16b5a972-ebe3-4ad8-b76c-a4b22ae682e6; Reason: (Message: {"Errors":["The index path corresponding to the specified order-by item is excluded."]}
ActivityId: 16b5a972-ebe3-4ad8-b76c-a4b22ae682e6, Request URI: /apps/7ebc5b84-a56a-4396-80b9-52f8cbe8eff5/services/55f47b3b-bce6-46a2-9e70-cc779be77250/partitions/6d859668-08e9-4cbe-b81e-201be7326868/replicas/133675338071995097s/, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Windows/10.0.20348 cosmos-netstandard-sdk/3.18.0);););.'
at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol1.ProcessResponse(ConnectionId connectionId, CommandMessage responseMessage) at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol1.SendMessageAndProcessResponse(CommandRequestMessage message, Int32 responseTo, IConnection connection, CancellationToken cancellationToken)
at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol1.Execute(IConnection connection, CancellationToken cancellationToken) at MongoDB.Driver.Core.WireProtocol.CommandWireProtocol1.Execute(IConnection connection, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Servers.Server.ServerChannel.ExecuteProtocol[TResult](IWireProtocol1 protocol, ICoreSession session, CancellationToken cancellationToken) at MongoDB.Driver.Core.Servers.Server.ServerChannel.Command[TResult](ICoreSession session, ReadPreference readPreference, DatabaseNamespace databaseNamespace, BsonDocument command, IEnumerable1 commandPayloads, IElementNameValidator commandValidator, BsonDocument additionalOptions, Action1 postWriteAction, CommandResponseHandling responseHandling, IBsonSerializer1 resultSerializer, MessageEncoderSettings messageEncoderSettings, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.CommandOperationBase1.ExecuteProtocol(IChannelHandle channel, ICoreSessionHandle session, ReadPreference readPreference, CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.ReadCommandOperation1.ExecuteAttempt(RetryableReadContext context, Int32 attempt, Nullable1 transactionNumber, CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.RetryableReadOperationExecutor.Execute[TResult](IRetryableReadOperation1 operation, RetryableReadContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.ReadCommandOperation1.Execute(RetryableReadContext context, CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.FindOperation1.Execute(RetryableReadContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.FindOperation1.Execute(IReadBinding binding, CancellationToken cancellationToken) at MongoDB.Driver.OperationExecutor.ExecuteReadOperation[TResult](IReadBinding binding, IReadOperation1 operation, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.ExecuteReadOperation[TResult](IClientSessionHandle session, IReadOperation1 operation, ReadPreference readPreference, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.ExecuteReadOperation[TResult](IClientSessionHandle session, IReadOperation1 operation, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.FindSync[TProjection](IClientSessionHandle session, FilterDefinition1 filter, FindOptions2 options, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.<>c__DisplayClass53_01.<FindSync>b__0(IClientSessionHandle session) at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSession[TResult](Func2 func, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.FindSync[TProjection](FilterDefinition1 filter, FindOptions2 options, CancellationToken cancellationToken)
at MongoDB.Driver.FindFluent2.ToCursor(CancellationToken cancellationToken) at MongoDB.Driver.IAsyncCursorSourceExtensions.FirstOrDefault[TDocument](IAsyncCursorSource1 source, CancellationToken cancellationToken)
at MongoDB.Driver.IFindFluentExtensions.FirstOrDefault[TDocument,TProjection](IFindFluent2 find, CancellationToken cancellationToken) at AspNetCore.Identity.Mongo.Migrations.Migrator.Apply[TUser,TRole,TKey](IMongoCollection1 migrationCollection, IMongoCollection1 usersCollection, IMongoCollection1 rolesCollection)
at AspNetCore.Identity.Mongo.MongoIdentityExtensions.AddIdentityMongoDbProvider[TUser,TRole,TKey](IServiceCollection services, Action1 setupIdentityAction, Action1 setupDatabaseAction, IdentityErrorDescriber identityErrorDescriber)
at AspNetCore.Identity.Mongo.MongoIdentityExtensions.AddIdentityMongoDbProvider[TUser,TRole](IServiceCollection services, Action1 setupIdentityAction, Action1 setupDatabaseAction)

[vova3211]

huh that is interesting. Azure Cosmos DB supports MongoDB (kinda as a wrapper as I understand), but still it's not the same thing, so such limitation expected.

Anyway, there are 2 options:

1. create index on _Migrations collection
2. make client side sort

First option seems a bit overhead. Client side sort is not good overall, but in case 2-3 migration that we have it the best option.

[vova3211]

btw I'm using Azure Cosmos DB and don't have such problem. My cluster has version 6.0.0 MongoDB (created few month ago).

@scottparise I suppose your db has lower version. Could you confirm it?

[scottparise]

@vova3211 i had the same idea. I was originally on version 4. so i upgraded to version 6 which was the same as my local copy of mongodb for dev and the problems till persisted. So i then upgraded it to version 7 and it was still the same issue. The only fix was to apply the index on the _Migrations collection

[scottparise]

i guess i forgot to mention when i have run the code against a local copy of mongodb (version 6.x), there is no issue. It was only noticed when i deployed to Azure and the application failed to start. That's when i went down the rabbit hole and narrowed it down to the DatabaseVersion in the Migration Collection. Maybe Azure CosmosDB isn't following MongoDB exactly? I'm not familiar enough with mongo to know how to handle sorting on a field where there is no indexes and what is the correct course of action.

maybe as a suggestion, you can create a method which allows the developer to decide if the default index's are created? Or one that turns it off if they are on by default? Just spitballing here.

[vova3211]

hey @scottparise could you check if it's working with this pre-release version 9.0.1-rc1

P.S. I'm also worried about Projection. That could be the thing that also not supported by CosmosDB. But please let me know if you had success with new version as only you can really verify the update.