Azure
[Bug]: Odd config file requirements to get "complex" cosmosdb_nosql to work
What happened?
I have the following cosmosdb_nosql data setup with a database and 2 containers, conferences and talks:
data setup
- database
- conferences
-
[ { "endDate": "2022-06-01", "id": "072d1638-a763-49f7-850f-763eed777837", "location": "", "mainTag": "", "name": "CONFERENCE", "startDate": "2022-06-01", "url": "https://example.com/", "year": 2022, "talks": [ { "abstract": "EXAMPLE", "mainTag": "", "tags": [], "talkId": "0c8c7101-26ee-42e9-9596-32ec1d229c0b", "talkLength": 50, "talkTime": null, "title": "EXAMPLE" } ] } ]
-
- talks
-
[ { "id": "f45a36c0-4031-4779-9461-eeedf37a7b74", "mainTag": "Software Development", "title": "EXAMPLE" } ]
-
- conferences
The important thing to notice is that there are 3 models, of which 2 need to be exposed as their own entities in DAB:
- Conference (Exposable)
- Talk (Exposable)
- ConferenceTalk
- Part of
Conference, but not its own entity that needs to be exposed!
- Part of
I read the docs which tell me I need to define a graphql schema, on top of the dab-config.json, which I imagine has to look like this:
GraphQL schema
type Conference @model {
id: ID!
name: String!
location: String
year: Int!
startDate: String
endDate: String
url: String
mainTag: String
talks: [ConferenceTalk!]!
}
type ConferenceTalk {
talkId: ID!
title: String!
abstract: String
mainTag: String
talkTime: String
talkLength: Int
tags: [String]
}
type Talk @model {
id: ID!
title: String!
mainTag: String
}
Which leads me to my dab-config.json, which looks like this.
dab-config.json
{
"$schema": "https://github.com/Azure/data-api-builder/releases/download/v1.2.10/dab.draft.schema.json",
"data-source": {
"database-type": "cosmosdb_nosql",
"connection-string": "@env('COSMOSDB_CONNECTIONSTRING')",
"options": {
"database": "database",
"container": null,
"schema": "schema.gql"
}
},
"runtime": {
"rest": {
"enabled": false,
"path": "/api",
"request-body-strict": true
},
"graphql": {
"enabled": true,
"path": "/graphql",
"allow-introspection": true
},
"host": {
"cors": {
"origins": [],
"allow-credentials": false
},
"authentication": {
"provider": "StaticWebApps"
},
"mode": "development"
}
},
"entities": {
"Conference": {
"source": {
"object": "conferences"
},
"graphql": {
"enabled": true,
"type": {
"singular": "Conference",
"plural": "Conferences"
}
},
"rest": {
"enabled": false
},
"permissions": [
{
"role": "anonymous",
"actions": [
{
"action": "read"
}
]
}
]
},
"Talk": {
"source": {
"object": "talks"
},
"graphql": {
"enabled": true,
"type": {
"singular": "Talk",
"plural": "Talks"
}
},
"rest": {
"enabled": false
},
"permissions": [
{
"role": "anonymous",
"actions": [
{
"action": "read"
}
]
}
]
},
"ConferenceTalk": {
"source": {
"object": "WE_DO_NOT_EXPOSE_THIS_BUT_NEED_IT_TO_GET_DAB_TO_WORK"
},
"graphql": {
"enabled": false
},
"rest": {
"enabled": false
},
"permissions": [
{
"role": "anonymous",
"actions": [
{
"action": "read"
}
]
}
]
}
}
}
The problem
I struggled with this for a long while. In order to get both ConferenceTalk and Talk to work, I must define ConferenceTalk as an entity in dab-config.json, which is what the error also says. However, this doesn't make sense because ConferenceTalk is not its own entity and should not need to be exposed.
Expected solution
I should not need to define entities like ConferenceTalk in dab-config.json for objects that are part of main/parent entities. What if I had objects 10 layers deep? Would I need to define all of them as entities in dab-config.json? This would lead to a huge dab-config.json with filler data, just like ConferenceTalk is now.
Extra
Please improve the docs about defining graphql yourself for cosmosdb. They're very limited. I have no idea why/if I need to define @model, for example.
Version
Microsoft.DataApiBuilder 1.2.10+c7ca8db8558a63919c530e454c8f18b45d5b931c
What database are you using?
CosmosDB NoSQL
What hosting model are you using?
Local (including CLI)
Which API approach are you accessing DAB through?
GraphQL
Relevant log output
Azure.DataApiBuilder.Service.Startup[0]
Unable to complete runtime initialization. Refer to exception for error details.
Azure.DataApiBuilder.Service.Exceptions.DataApiBuilderException: The entity 'ConferenceTalk' was not found in the runtime config.
at Azure.DataApiBuilder.Core.Services.MetadataProviders.CosmosSqlMetadataProvider.AssertIfEntityIsAvailableInConfig(String entityName) in /_/src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:line 296
at Azure.DataApiBuilder.Core.Services.MetadataProviders.CosmosSqlMetadataProvider.ProcessSchema(IReadOnlyList`1 fields, Dictionary`2 schemaDocument, String currentPath, IncrementingInteger tableCounter, EntityDbPolicyCosmosModel parentEntity, HashSet`1 visitedEntities) in /_/src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:line 228
at Azure.DataApiBuilder.Core.Services.MetadataProviders.CosmosSqlMetadataProvider.ParseSchemaGraphQLFieldsForJoins() in /_/src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:line 180
at Azure.DataApiBuilder.Core.Services.MetadataProviders.CosmosSqlMetadataProvider..ctor(RuntimeConfigProvider runtimeConfigProvider, IFileSystem fileSystem) in /_/src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:line 85
at Azure.DataApiBuilder.Core.Services.MetadataProviders.MetadataProviderFactory..ctor(RuntimeConfigProvider runtimeConfigProvider, IAbstractQueryManagerFactory queryManagerFactory, ILogger`1 logger, IFileSystem fileSystem, Boolean isValidateOnly) in /_/src/Core/Services/MetadataProviders/MetadataProviderFactory.cs:line 32
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(ServiceIdentifier serviceIdentifier)
at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at Azure.DataApiBuilder.Service.Startup.PerformOnConfigChangeAsync(IApplicationBuilder app) in /_/src/Service/Startup.cs:line 620
fail: Azure.DataApiBuilder.Service.Startup[0]
Could not initialize the engine with the runtime config file: dab-config.json
info: Microsoft.Hosting.Lifetime[0]
Application is shutting down...
fail: Microsoft.Extensions.Hosting.Internal.Host[11]
Hosting failed to start
System.Threading.Tasks.TaskCanceledException: A task was canceled.
at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)
Unable to launch the Data API builder engine.
Error: Failed to start the engine.
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
I've created a reproducable repo here: https://github.com/sander1095/dab-graphql-csharp-client
where does the fix for this sit on the priority list please ? Or is there a less convoluted workaround. Thanks.
Since you're working with a NoSQL database where the schema can vary with each record, DAB is unable to predict the sub-entities present. Therefore, it is recommended to explicitly configure the entities you're interested in within the configuration. This will ensure that DAB generates the GraphQL API only for those specified entities.
Hi @sourabh1007 thank you for the technical explanation. I would however like to challenge, that from a usability point of view, it feels counter intuitive when notionally one has already provided a .gql schema defining all child class/type sub-entities relationships.
Being required to define sub-entities in config feels like an unnecessary barrier to entry for using this fabulous dab tool.
Hi @sajeetharan . Does the team have an update/priority for this issue? 😊