Web API with OData does not work properly

[nathanvj]

Describe the bug

I'm using Amazon.Lambda.AspNetCoreServer.Hosting to deploy my .NET API with OData endpoints.

The deployment is successful, but when I call an OData endpoint with one or more expressions, e.g. /v1/tickets?$top=10&$skip=100 it returns the following response at all times:

{
  "message": null
}

When I call the API locally, or after deploying it to a server, it works fine. This could be the same issue as this one.

Expected Behavior

I expect to receive the same response I get when testing the endpoint locally, which is the correct response based on the provided OData expressions, like:

• skip
• top
• orderBy
• apply
• etc...

Current Behavior

I receive the following response at all times - and NO LOGS are being published to CloudWatch:

{
  "message": null
}

Reproduction Steps

• Create .NET OData API
• Install Amazon.Lambda.AspNetCoreServer.Hosting NuGet Package
• Deploy API to AWS Lambda
• Test the OData endpoint with an OData expression.

Demo repository - see README for instructions: https://github.com/nathanvj/odata-aws-lambda-api

Possible Solution

The possible fix, mentioned in this issue does not work for me. My CustomEnableQueryAttribute looks like this - and it still happens:

public class CustomEnableQueryAttribute : EnableQueryAttribute
{
    public CustomEnableQueryAttribute()
    {
    }

    /// <summary>
    /// See https://github.com/OData/WebApi/issues/1227
    /// </summary>
    public override void OnActionExecuted(ActionExecutedContext actionExecutedContext)
    {
        actionExecutedContext.HttpContext.Response.StatusCode = 200;
        base.OnActionExecuted(actionExecutedContext);
    }

    public override void ValidateQuery(HttpRequest request, ODataQueryOptions queryOptions)
    {
        try
        {
            if (queryOptions.SelectExpand is not null)
            {
                List<string> roles = request.HttpContext.User.GetRoles();
                queryOptions.SelectExpand.Validator = new ODataExpandValidator(roles);
            }

            base.ValidateQuery(request, queryOptions);
        }
        catch (Exception e)
        {
            throw new ODataQueryException(e.Message);
        }
    }
}

Additional Information/Context

This issue is blocking me from deploying my Web API using AWS Lambda.

AWS .NET SDK and/or Package version used

Amazon.Lambda.AspNetCoreServer.Hosting 1.6.0 Microsoft.AspNetCore.OData 8.0.12

Targeted .NET Platform

.NET 6

Operating System and version

Windows 10

[nathanvj]

Update - added link to minimal reproduction of problem repository.

[ashishdhingra]

Hi @nathanvj,

Good afternoon.

Thanks for reporting the issue. Even though I have not reproduced the issue yet, looking at high level at the minimal repro, I guess you are using the wrong project template. You might want to use Lambda ASP.NET Core Web API serverless Visual Studio project template, instead of Lambda function template. The serverless template includes a CloudFormation template that includes creating resources such as API Gateway integration, etc. Whereas, Lambda function template just creates a Lambda function, which when invoked returns the response that in this case includes the message attribute.

Once you change to the correct Visual Studio template, you could use dotnet lambda deploy-serverless for deployment.

Please let me know if it works. I would then convert this issue to Q&A discussion for it to be easily accessible to other users experiencing similar issue.

Thanks, Ashish

[nathanvj]

@ashishdhingra Thanks for getting back to me! I'm trying to deploy an existing .NET Web API to AWS Lambda, so I don't really have the option to change template anymore. I watched this video (see 11:30) from Nick Chapsas, and that's why I'm using the Amazon.Lambda.AspNetCoreServer.Hosting package.

It's also worth noting that without any OData expression (such as $top=10), the API works fine.

[ashishdhingra]

Reproducible using customer's code.

STEPS:

• Created a new ASP.NET Core WebApi project named WebApiFunctionUrlTest.
• Added new file aws-lambda-tools-defaults.json with the following content:

{
  "Information": [
    "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET CLI.",
    "To learn more about the Lambda commands with the .NET CLI execute the following command at the command line in the project root directory.",
    "dotnet lambda help",
    "All the command line options for the Lambda command can be specified in this file."
  ],
  "profile": "default",
  "region": "us-east-2",
  "configuration": "release",
  "function-runtime": "dotnet6",
  "function-memory-size": 256,
  "function-timeout": 60,
  "function-handler": "WebApiFunctionUrlTest",
  "function-url-enable": true
}

• Per instructions in section ASP.NET Core minimal APIs at https://aws.amazon.com/blogs/compute/introducing-the-net-6-runtime-for-aws-lambda/,
  • Added the Amazon.Lambda.AspNetCoreServer.Hosting NuGet package to the project.
  • Added a call to AddAWSLambdaHosting in the application when the services are being defined for the application.
• Following instructions at Getting Started with ASP.NET Core OData 8 and taking customer's code as an example, configured OData. Below is the final Program.cs and .csproj file: Program.cs

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.OData;
using Microsoft.AspNetCore.OData.Query;
using Microsoft.AspNetCore.OData.Routing.Controllers;
using Microsoft.OData.Edm;
using Microsoft.OData.ModelBuilder;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRouting(options => options.LowercaseUrls = true)
    .AddControllers()
    .AddOData(options =>
        options.AddRouteComponents("api/v1", ODataEdmBuilder.GetEdmModel())
        .Select().Filter().OrderBy().Expand().Count().SetMaxTop(250)
    );

builder.Services.AddAWSLambdaHosting(LambdaEventSource.HttpApi);

var app = builder.Build();

app.UseHttpsRedirection();

app.UseRouting();
app.UseEndpoints(endpoints => endpoints.MapControllers());

app.Run();

public class TicketsController : ODataController
{
    private readonly IQueryable<Ticket> _ticketsQueryable =
        new List<Ticket>()
        {
                new (new Guid("f447e347-246b-4989-9168-90cc670267e2"), "Ticket 1", DateTime.UtcNow.Subtract(TimeSpan.FromHours(0))),
                new (new Guid("442ba2c3-45cc-4f04-9922-29095a53a92b"), "Ticket 2", DateTime.UtcNow.Subtract(TimeSpan.FromHours(1))),
                new (new Guid("edea7155-2d3d-4308-87e0-52d695bc9510"), "Ticket 3", DateTime.UtcNow.Subtract(TimeSpan.FromHours(2))),
                new (new Guid("d0d5f399-98ea-4ec5-b641-e3371be34919"), "Ticket 4", DateTime.UtcNow.Subtract(TimeSpan.FromHours(3))),
                new (new Guid("258be4be-7463-4764-8038-710f1082e7b2"), "Ticket 5", DateTime.UtcNow.Subtract(TimeSpan.FromHours(4))),
        }
        .AsQueryable();

    /// <summary>
    /// Get all tickets.
    /// </summary>
    /// <param name="options">The odata query options.</param>
    [HttpGet]
    [EnableQuery]
    public IActionResult Get(ODataQueryOptions<Ticket> options)
    {
        return Ok(_ticketsQueryable);
    }
}

public class Ticket
{
    public Ticket(Guid id, string subject, DateTime createdAt)
    {
        Id = id;
        Subject = subject;
        CreatedAt = createdAt;
    }

    public Guid Id { get; set; }
    public string Subject { get; set; }
    public DateTime CreatedAt { get; set; }
}

public static class ODataEdmBuilder
{
    public static IEdmModel GetEdmModel()
    {
        ODataConventionModelBuilder conventionModelBuilder = new();
        conventionModelBuilder.EntitySet<Ticket>("Tickets");
        conventionModelBuilder.EnableLowerCamelCase();
        return conventionModelBuilder.GetEdmModel();
    }
}

.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Amazon.Lambda.AspNetCoreServer.Hosting" Version="1.6.0" />
    <PackageReference Include="Microsoft.AspNetCore.OData" Version="8.1.0" />
  </ItemGroup>

</Project>

• Deployed to Lambda with function URL enabled using dotnet lambda deploy-function.

RESULT:

• OData query something like https://localhost:7250/api/v1/tickets?$top=1 works fine locally.
• OData query execution when deployed to Lambda environment:
  • URL https://[some-id].lambda-url.us-east-2.on.aws/api/v1/tickets (or with query string ?top=1) returns all the data.
  • URL https://[some-id].lambda-url.us-east-2.on.aws/api/v1/tickets?$top=1 returns {"message":null}, with status code 400 Bad Request.

My hunch is that the Lambda Function URLs doesn't support OData queries, sample returned outputs are documented at Invoking Lambda function URLs, which involves message structure.

Needs review with the team.

[nathanvj]

I appreciate you looking into this. It's currently blocking us from deploying our API with AWS Lambda and deploying to EC2 is lot more expensive. Is there any temporary workaround for this?

[normj]

@nathanvj this looks to be either a limitation / bug in Lambda's Function URL feature because with the $top= querystring parameter the request never even makes into the Lambda function. If I urlencode the $ like api/v1/tickets?%24top=1 then it works correctly.

Also if I use API Gateway in front of the Lambda function instead of the Lambda function URL I don't need to escape the $ and the original api/v1/tickets?$top=1 works fine. Using API Gateway will most likely still be a lot cheaper then deploying to an EC2 instance.

Here is my serverless.template I used to deploy to API Gateway via the dotnet lambda deploy-serverless command.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Transform": "AWS::Serverless-2016-10-31",
  "Description": "An AWS Serverless Application that uses the ASP.NET Core framework running in Amazon Lambda.",
  "Parameters": {},
  "Conditions": {},
  "Resources": {
    "AspNetCoreFunction": {
      "Type": "AWS::Serverless::Function",
      "Properties": {
        "Handler": "ODataAPI",
        "Runtime": "dotnet6",
        "CodeUri": "",
        "MemorySize": 256,
        "Timeout": 30,
        "Role": null,
        "Policies": [
          "AWSLambda_FullAccess"
        ],
        "Events": {
          "ProxyResource": {
            "Type": "HttpApi",
            "Properties": {
              "Path": "/{proxy+}",
              "Method": "ANY"
            }
          },
          "RootResource": {
            "Type": "HttpApi",
            "Properties": {
              "Path": "/",
              "Method": "ANY"
            }
          }
        }
      }
    }
  },
  "Outputs": {
    "ApiURL": {
      "Description": "API endpoint URL for Prod environment",
      "Value": {
        "Fn::Sub": "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/"
      }
    }
  }
}

[nathanvj]

@normj Thanks a lot for getting back to me so quickly. It's unfortunate that this is a limitation / bug.

API Gateway is definitely the better option if it works fine with that! I don't expect a huge load of traffic, so the API gateway won't make that much difference in pricing. Also appreciate the template, I'll try to set it up this way and let you know.