efcore.pg icon indicating copy to clipboard operation
efcore.pg copied to clipboard

DateTime.Now.AddDays(-1) throw Npgsql.PostgresException

Open Andrioden opened this issue 6 years ago • 28 comments
trafficstars

This always works

DateTime changedAfter = DateTime.Now.AddDays(-1);

await _context.Entity
    .Where(e => e.Updated > changedAfter)
    .ToListAsync();

My understanding is that your translater now translate a specific DateTime, which always works.

This throws exception on a certain environment

Getting an exception on linux docker container (microsoft/dotnet:2.2.2-aspnetcore-runtime) in english/utc timezone with Npgsql.EntityFrameworkCore.PostgreSQL 2.2.0, with Microsoft.AspNetCore.App 2.2.2.

Strangly enough, it dont throw an exception on my development environment which is windows 10, norwegian/utc+2 timezoned. Against the same database with the same runtime version.

await _context.Entity
    .Where(e => e.Updated > DateTime.Now.AddDays(-1))
    .ToListAsync();

My understanding is that now you are translating AddDays(-1), which fails.

Npgsql.PostgresException
22007: invalid input syntax for type interval: "−1 days"

Npgsql.PostgresException (0x80004005): 22007: invalid input syntax for type interval: "−1 days"
   at Npgsql.NpgsqlConnector.<>c__DisplayClass161_0.<<ReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlConnector.<>c__DisplayClass161_0.<<ReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming)
   at Npgsql.NpgsqlCommand.ExecuteDbDataReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.ExecuteAsync(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable`1.AsyncEnumerator.BufferlessMoveNext(DbContext _, Boolean buffer, CancellationToken cancellationToken)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable`1.AsyncEnumerator.MoveNext(CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.AsyncSelectEnumerable`2.AsyncSelectEnumerator.MoveNext(CancellationToken cancellationToken)
   at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.MoveNextCore(CancellationToken cancellationToken) in D:\a\1\s\Ix.NET\Source\System.Interactive.Async\Select.cs:line 106
   at System.Linq.AsyncEnumerable.AsyncIterator`1.MoveNext(CancellationToken cancellationToken) in D:\a\1\s\Ix.NET\Source\System.Interactive.Async\AsyncIterator.cs:line 98
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext(CancellationToken cancellationToken)
   at System.Linq.AsyncEnumerable.Aggregate_[TSource,TAccumulate,TResult](IAsyncEnumerable`1 source, TAccumulate seed, Func`3 accumulator, Func`2 resultSelector, CancellationToken cancellationToken) in D:\a\1\s\Ix.NET\Source\System.Interactive.Async\Aggregate.cs:line 120
   at Dyrket.NotifySubscriptionChangesJob.GetRecentlyChangedSubscriptionsAsync(Int32 forSupplierId) in /app/Dyrket/Jobs/Subscription/NotifySubscriptionChangesJob.cs:line 56
   at Dyrket.NotifySubscriptionChangesJob.RunAsync(IJobCancellationToken cancellationToken) in /app/Dyrket/Jobs/Subscription/NotifySubscriptionChangesJob.cs:line 34

Thanks for an otherwise very stable/reliable implementation of EF Core!

Andrioden avatar Mar 14 '19 02:03 Andrioden

@Andrioden Can you post the generated queries for each environment?

austindrenski avatar Mar 14 '19 03:03 austindrenski

@austindrenski : Ok, i will show the full code and generated queries then, to save myself a bit trouble.

Seeing the result my conclusion is that on the failing linux environment an long dash (−) is generated, and on the working windows environment an short dash (-) is generated.

Code behind all queries below

return await _context.Subscriptions
    .IgnoreQueryFilters()
    .Where(s =>
        s.Updated > DateTime.Now.AddDays(-1) &&
        s.Lines.Any(l => l.ProductVariation.SupplierId == forSupplierId)
    )
    .Include(s => s.User)
        .ThenInclude(u => u.Organization)
    .Include(s => s.Lines)
        .ThenInclude(l => l.ProductVariation)
    .ToListAsync();

Dev / Windows - Relevant parts

WHERE (s."Updated" > (NOW() + INTERVAL '-1 days'))
WHERE (s0."Updated" > (NOW() + INTERVAL '-1 days'))

Test / Linux - Relevant part

WHERE (s."Updated" > (NOW() + INTERVAL '−1 days'))

Dev / Windows - Full

-- Microsoft.EntityFrameworkCore.Database.Command:Information: Executed DbCommand (41ms) [Parameters=[@__forSupplierId_0='?' (DbType = Int32)], CommandType='Text', CommandTimeout='30']

SELECT s."Id", s."AddressId", s."Created", s."DeliverySMSNotification", s."DeliveryTypeId", s."End", s."Name", s."PaymentSystem", s."Start", s."Status", s."Updated", s."UserId", "s.User"."Id", "s.User"."AccessFailedCount", "s.User"."Birthday", "s.User"."ConcurrencyStamp", "s.User"."DeliverySMSNotification", "s.User"."Email", "s.User"."EmailConfirmed", "s.User"."FirstName", "s.User"."LastName", "s.User"."LockoutEnabled", "s.User"."LockoutEnd", "s.User"."NewsletterSubscribed", "s.User"."NormalizedEmail", "s.User"."NormalizedUserName", "s.User"."OrganizationId", "s.User"."PasswordHash", "s.User"."PhoneNumber", "s.User"."PhoneNumberConfirmed", "s.User"."ProducerId", "s.User"."SecurityStamp", "s.User"."SlackID", "s.User"."TwoFactorEnabled", "s.User"."UserName", "s.User.Organization"."Id", "s.User.Organization"."CategoryId", "s.User.Organization"."DeliveryToAnyZipCodeAllowed", "s.User.Organization"."EAccountingCustomerId", "s.User.Organization"."FreeDelivery", "s.User.Organization"."GroupedInvoicePaymentAllowed", "s.User.Organization"."InvoiceEmail", "s.User.Organization"."InvoicePaymentAllowed", "s.User.Organization"."Name", "s.User.Organization"."OrganizationNumber"
FROM "Subscription" AS s
INNER JOIN "User" AS "s.User" ON s."UserId" = "s.User"."Id"
LEFT JOIN "Organization" AS "s.User.Organization" ON "s.User"."OrganizationId" = "s.User.Organization"."Id"
WHERE (s."Updated" > (NOW() + INTERVAL '-1 days')) AND EXISTS (
    SELECT 1
    FROM "SubscriptionLine" AS l
    LEFT JOIN "ProductVariation" AS "l.ProductVariation" ON l."ProductVariationSKU" = "l.ProductVariation"."SKU"
    WHERE ("l.ProductVariation"."SupplierId" = @__forSupplierId_0) AND (s."Id" = l."SubscriptionId"))
ORDER BY s."Id"

-- Microsoft.EntityFrameworkCore.Database.Command:Information: Executed DbCommand (21ms) [Parameters=[@__forSupplierId_0='?' (DbType = Int32)], CommandType='Text', CommandTimeout='30']

SELECT "s.Lines"."Id", "s.Lines"."FridayUnits", "s.Lines"."MondayUnits", "s.Lines"."ProductVariationSKU", "s.Lines"."SaturdayUnits", "s.Lines"."SubscriptionId", "s.Lines"."SundayUnits", "s.Lines"."ThursdayUnits", "s.Lines"."TuesdayUnits", "s.Lines"."WednesdayUnits", "s.ProductVariation"."SKU", "s.ProductVariation"."BundleList", "s.ProductVariation"."CostXVAT", "s.ProductVariation"."CountryOfOriginIso2", "s.ProductVariation"."CustomerType", "s.ProductVariation"."DeliveryFriday", "s.ProductVariation"."DeliveryMonday", "s.ProductVariation"."DeliverySaturday", "s.ProductVariation"."DeliverySunday", "s.ProductVariation"."DeliveryThursday", "s.ProductVariation"."DeliveryTuesday", "s.ProductVariation"."DeliveryWednesday", "s.ProductVariation"."ImageId", "s.ProductVariation"."Ingredients", "s.ProductVariation"."NutritionalContent", "s.ProductVariation"."PackageUnit", "s.ProductVariation"."PackageUnitsCount", "s.ProductVariation"."PackageUnitsCountFromProducer", "s.ProductVariation"."PriceXVAT", "s.ProductVariation"."ProductId", "s.ProductVariation"."Published", "s.ProductVariation"."StockType", "s.ProductVariation"."SupplierId", "s.ProductVariation"."SupplierProductNumber", "s.ProductVariation"."Title", "s.ProductVariation"."UnitSize", "s.ProductVariation"."UnitSizeMeasurement", "s.ProductVariation"."VATCategoryId", "s.ProductVariation"."WarehouseCategoryId"
FROM "SubscriptionLine" AS "s.Lines"
LEFT JOIN "ProductVariation" AS "s.ProductVariation" ON "s.Lines"."ProductVariationSKU" = "s.ProductVariation"."SKU"
INNER JOIN (
    SELECT DISTINCT s0."Id"
    FROM "Subscription" AS s0
    INNER JOIN "User" AS "s.User0" ON s0."UserId" = "s.User0"."Id"
    LEFT JOIN "Organization" AS "s.User.Organization0" ON "s.User0"."OrganizationId" = "s.User.Organization0"."Id"
    WHERE (s0."Updated" > (NOW() + INTERVAL '-1 days')) AND EXISTS (
        SELECT 1
        FROM "SubscriptionLine" AS l0
        LEFT JOIN "ProductVariation" AS "l.ProductVariation0" ON l0."ProductVariationSKU" = "l.ProductVariation0"."SKU"
        WHERE ("l.ProductVariation0"."SupplierId" = @__forSupplierId_0) AND (s0."Id" = l0."SubscriptionId"))
) AS t ON "s.Lines"."SubscriptionId" = t."Id"
ORDER BY t."Id"

Test / Linux - Full

-- fail: Microsoft.EntityFrameworkCore.Database.Command[20102]
-- Failed executing DbCommand (12ms) [Parameters=[@__forSupplierId_0='?' (DbType = Int32)], CommandType='Text', CommandTimeout='30']

SELECT s."Id", s."AddressId", s."Created", s."DeliverySMSNotification", s."DeliveryTypeId", s."End", s."Name", s."PaymentSystem", s."Start", s."Status", s."Updated", s."UserId", "s.User"."Id", "s.User"."AccessFailedCount", "s.User"."Birthday", "s.User"."ConcurrencyStamp", "s.User"."DeliverySMSNotification", "s.User"."Email", "s.User"."EmailConfirmed", "s.User"."FirstName", "s.User"."LastName", "s.User"."LockoutEnabled", "s.User"."LockoutEnd", "s.User"."NewsletterSubscribed", "s.User"."NormalizedEmail", "s.User"."NormalizedUserName", "s.User"."OrganizationId", "s.User"."PasswordHash", "s.User"."PhoneNumber", "s.User"."PhoneNumberConfirmed", "s.User"."ProducerId", "s.User"."SecurityStamp", "s.User"."SlackID", "s.User"."TwoFactorEnabled", "s.User"."UserName", "s.User.Organization"."Id", "s.User.Organization"."CategoryId", "s.User.Organization"."DeliveryToAnyZipCodeAllowed", "s.User.Organization"."EAccountingCustomerId", "s.User.Organization"."FreeDelivery", "s.User.Organization"."GroupedInvoicePaymentAllowed", "s.User.Organization"."InvoiceEmail", "s.User.Organization"."InvoicePaymentAllowed", "s.User.Organization"."Name", "s.User.Organization"."OrganizationNumber"
FROM "Subscription" AS s
INNER JOIN "User" AS "s.User" ON s."UserId" = "s.User"."Id"
LEFT JOIN "Organization" AS "s.User.Organization" ON "s.User"."OrganizationId" = "s.User.Organization"."Id"
WHERE (s."Updated" > (NOW() + INTERVAL '−1 days')) AND EXISTS (
   SELECT 1
   FROM "SubscriptionLine" AS l
   LEFT JOIN "ProductVariation" AS "l.ProductVariation" ON l."ProductVariationSKU" = "l.ProductVariation"."SKU"
   WHERE ("l.ProductVariation"."SupplierId" = @__forSupplierId_0) AND (s."Id" = l."SubscriptionId"))
ORDER BY s."Id"

Andrioden avatar Mar 14 '19 05:03 Andrioden

Using docker image microsoft/dotnet:2.2.2-aspnetcore-runtime btw.

Andrioden avatar Mar 14 '19 05:03 Andrioden

Thanks for the additional information. Not sure of the specifics yet, but I imagine the line in question is here:

https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL/blob/ba141fdde35245a5c836912bd65f83126ad20826/src/EFCore.PG/Query/ExpressionTranslators/Internal/NpgsqlDateAddTranslator.cs#L89

If your suspicion is correct and the negative sign character varies by platform, then this could also affect how other literals are generated. I'll put together a local repo and see what happens.

/cc @roji

austindrenski avatar Mar 14 '19 05:03 austindrenski

On my PostgreSQL in a Docker container I got 2019-03-15 08:25:07.810692+00 when executed the following query:

SELECT NOW() + INTERVAL '−1 days';

Why so? From the documentation (emphasis mine):

interval values can be written using the following verbose syntax:

[@] quantity unit [quantity unit...] [direction]

where quantity is a number (possibly signed); unit is microsecond, millisecond, second, minute, hour, day, week, month, year, decade, century, millennium, or abbreviations or plurals of these units; direction can be ago or empty.

But there was an old bug reported here. It was fixed by Tom Lane in version 7.4.7. Looks like it came back, will report.

Anyway, there is direction which can be set to ago and this will negate the value. For example, the query below produced the correct result:

SELECT NOW() + INTERVAL '1 days ago';
-- 2019-03-13 08:25:07.810692+00

YohDeadfall avatar Mar 14 '19 08:03 YohDeadfall

@YohDeadfall That may also be happening, but I think that this specific case might just be caused by the wrong sign character:

psql (11.2 (Ubuntu 11.2-1.pgdg18.04+1))
Type "help" for help.

austindrenski=# SELECT INTERVAL '-1 days'; # using hyphen-minus (U+002D)
 interval
----------
 -1 days
(1 row)

austindrenski=# SELECT INTERVAL '−1 days'; # using minus sign (U+2212)
ERROR:  invalid input syntax for type interval: "−1 days"
LINE 1: SELECT INTERVAL '−1 days';
                        ^

austindrenski=# SELECT INTERVAL '‐1 days'; # using hyphen (U+2010)
ERROR:  invalid input syntax for type interval: "‐1 days"
LINE 1: SELECT INTERVAL '‐1 days';
                        ^

austindrenski avatar Mar 14 '19 13:03 austindrenski

If your suspicion is correct and the negative sign character varies by platform, then this could also affect how other literals are generated. I'll put together a local repo and see what happens.

This is the first time I hear of something like this... I can't really see how/why we'd be generating a different dash character - the SQL literal generation wraps the string in FormattableString.Invariant() so we shouldn't have variations on culture... Am curious to see what you'll find...

roji avatar Mar 14 '19 18:03 roji

@austindrenski Yep, it's because of the wrong character, but on my machine with the en_US locale the minus sign is completely ignored, no exception thrown. Therefore, the following thing happens:

SELECT INTERVAL '−1 days';
--  1 day

But with the hyphen-minus it's okay:

SELECT INTERVAL '−1 days';
--  -1 days

I can't really see how/why we'd be generating a different dash character - the SQL literal generation wraps the string in FormattableString.Invariant() so we shouldn't have variations on culture...

We don't use it. SqlFragmentExpression accepts a string, not a FormattableString. Therefore, a literal using the current culture is generated in the line @austindrenski mentioned.

YohDeadfall avatar Mar 15 '19 10:03 YohDeadfall

I can't really see how/why we'd be generating a different dash character - the SQL literal generation wraps the string in FormattableString.Invariant() so we shouldn't have variations on culture...

We don't use it. SqlFragmentExpression accepts a string, not a FormattableString. Therefore, a literal using the current culture is generated in the line @austindrenski mentioned.

Ah I see, I was looking at interval literal generation, whereas this is about translation of Date.AddDays(). But then shouldn't this be fixable by wrapping the SQL string with FormattableString.Invariant:

interval = new SqlFragmentExpression(FormattableString.Invariant($"INTERVAL '{amountToAdd} {datePart}'"));

roji avatar Mar 15 '19 10:03 roji

But then shouldn't this be fixable by wrapping the SQL string with FormattableString.Invariant

Possibly. But I looked into the negative sign used by .NET's Norwegian culture, and it is the same as the invariant culture. So it's not clear to me why the missing FormattableString.Invariant would matter in this case...

austindrenski avatar Mar 15 '19 13:03 austindrenski

@austindrenski Keep in mind it works on my norwegian culture dev environment, but not on the linux docker container environment which is not an norwegian culture.

Andrioden avatar Mar 17 '19 05:03 Andrioden

new SqlFragmentExpression($"INTERVAL '{amountToAdd} {datePart}'");

Interpolated strings are culture specific. EF Core codebase specifically avoids interpolated strings unless combining strings due to this. I believe right thing to do here is just use string.Format with invariant culture. Using interpolated string vs string.Format is purely coding preference and no functional impact. I would suggest checking other places in the codebase also where you using string interpolation with non-strings. For reference https://github.com/aspnet/EntityFrameworkCore/pull/5879

smitpatel avatar Mar 18 '19 19:03 smitpatel

@Andrioden we've pushed a fix to ensure using invariant culture when generating interval literals. Can you please try out Npgsql.EntityFrameworkCore.PostgreSQL 2.2.3-ci.1538+sha.58078e014 from our stable feed and confirm that the problem is gone?

roji avatar Mar 20 '19 19:03 roji

@roji @YohDeadfall After entering a tiny 30min rabbit hole of figuring out how to use the specific Nuget package from the Myget source for a docker build I still get the error.

Still works on my normal norwegian env.

Npgsql.PostgresException
22007: invalid input syntax for type interval: "−1 days"

Npgsql.PostgresException (0x80004005): 22007: invalid input syntax for type interval: "−1 days"
   at Npgsql.NpgsqlConnector.<>c__DisplayClass161_0.<<ReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlConnector.<>c__DisplayClass161_0.<<ReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming)
   at Npgsql.NpgsqlCommand.ExecuteDbDataReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.ExecuteAsync(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable`1.AsyncEnumerator.BufferlessMoveNext(DbContext _, Boolean buffer, CancellationToken cancellationToken)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable`1.AsyncEnumerator.MoveNext(CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.AsyncSelectEnumerable`2.AsyncSelectEnumerator.MoveNext(CancellationToken cancellationToken)
   at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.MoveNextCore(CancellationToken cancellationToken) in D:\a\1\s\Ix.NET\Source\System.Interactive.Async\Select.cs:line 106
   at System.Linq.AsyncEnumerable.AsyncIterator`1.MoveNext(CancellationToken cancellationToken) in D:\a\1\s\Ix.NET\Source\System.Interactive.Async\AsyncIterator.cs:line 98
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext(CancellationToken cancellationToken)
   at System.Linq.AsyncEnumerable.Aggregate_[TSource,TAccumulate,TResult](IAsyncEnumerable`1 source, TAccumulate seed, Func`3 accumulator, Func`2 resultSelector, CancellationToken cancellationToken) in D:\a\1\s\Ix.NET\Source\System.Interactive.Async\Aggregate.cs:line 120
   at Dyrket.NotifySubscriptionChangesJob.RunAsync() in /app/Dyrket/Jobs/Subscription/NotifySubscriptionChangesJob.cs:line 30

I am feeling confident i am using the specific version, seeing the docker build output Installing Npgsql.EntityFrameworkCore.PostgreSQL 2.2.3-ci.1538.

Note to future self, use following dockerfile dotnet restore step to use Myget source RUN dotnet restore ./Dyrket -s https://api.nuget.org/v3/index.json -s https://www.myget.org/F/npgsql/api/v3/index.json

Andrioden avatar Mar 21 '19 17:03 Andrioden

Also, a few more thoughts, @roji, i checked your pull request and you mention

BTW apparently the issue with the Norwegian minus sign is quite well-known: dotnet/corefx#34672

But why is this issue relevant?

  1. It fails on the non-norwegian environment
  2. The code itself input number is -1, which is not a norwegian specific minus sign? Or is it? I copied it to this unicode tester and the output is U+002D : HYPHEN-MINUS while the problematic Norwegian minus is U+2212 MINUS SIGN
  3. The code itself is stored in a Unicode (UTF-8 with signature) - Codepage 65001 saved file.
  4. The build has run (previously, not when I tested now) on a non-norwegian environment and still failed

What is it that I dont understand? How is the problem related to Norwegian? Where do my Norwegian environment impact this? In a way it makes sense that this is an Norwegian culture problem, otherwise many others must have the same problem as me, but how. O.o

Andrioden avatar Mar 21 '19 17:03 Andrioden

What is it that I dont understand?

I think you're on the same page as we are. This wasn't something we were able to test/reproduce locally, so this patch was an educated guess based on previous, similar-sounding issues that were fixed by ensuring literal generation used the invariant culture.

How is the problem related to Norwegian? Where do my Norwegian environment impact this?

Frankly, we're not sure. Culture issues have caused similar issues in the past, so it was an easy place to start looking.

Just to be really clear, we're not blaming the Norwegian culture settings. If it is a culture issue, then it's on us to wrap things in the invariant culture.

In a way it makes sense that this is an Norwegian culture problem, otherwise many others must have the same problem as me, but how. O.o

By the way, thanks for providing the additional info on your setup/codepages. We'll have to take another look on our end.

austindrenski avatar Mar 21 '19 18:03 austindrenski

@Andrioden just to be clear, even if the fix we pushed doesn't address your issue, it's still important - there are various other culture-related bugs that it prevents.

@austindrenski are you able to continue investigating this?

roji avatar Mar 21 '19 18:03 roji

@roji Yeah, I'll start on a repro in Docker tonight.

@Andrioden Could you provide a minimal version of your Dockerfile?

austindrenski avatar Mar 21 '19 19:03 austindrenski

a slightly related thing, the actual character in the source code file may also matter depending on the encoding of the file it is difficult to figure out the difference.

smitpatel avatar Mar 21 '19 19:03 smitpatel

Thanks for the suggestion @smitpatel, although this error only happens in some cases and not in others (I'd expect it to always fail if it were a source character issue...)

roji avatar Mar 21 '19 21:03 roji

I was going along the idea that if the file has bad character and server was also in same Norwegian culture then it would not treat it as bad character. But move server and code to different culture and now your server could not parse the bad character anymore. Just a wild guess in dark 🌺

smitpatel avatar Mar 21 '19 21:03 smitpatel

Anything's possible when culture issues are involved.... :(

roji avatar Mar 22 '19 09:03 roji

@austindrenski - Sorry for the slow response, we are in different time zones. Here is my docker file, its pretty minimal so I have not made any changes. It is saved in Western European (Windows) - Codepage 1252, if that matters. : P

FROM microsoft/dotnet:2.2-sdk AS build
WORKDIR /app

# copy csproj and restore as distinct layers
COPY *.sln .
COPY Dyrket/*.csproj ./Dyrket/
RUN dotnet restore ./Dyrket

# copy everything else and build app
COPY Dyrket/. ./Dyrket/
WORKDIR /app/Dyrket
RUN dotnet publish -c Release -o out

FROM microsoft/dotnet:2.2.2-aspnetcore-runtime AS runtime
WORKDIR /app
COPY --from=build /app/Dyrket/out ./
ENTRYPOINT ["dotnet", "Dyrket.dll"]

And further here is a minimal verison of the csproj file, so you should be able to duplicate my environment.

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.2</TargetFramework>
    <RuntimeFrameworkVersion>2.2.2</RuntimeFrameworkVersion>
  </PropertyGroup>
  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
    <LangVersion>7.3</LangVersion>
  </PropertyGroup>
  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <LangVersion>7.3</LangVersion>
  </PropertyGroup>
<PropertyGroup>
  <MvcRazorCompileOnPublish>false</MvcRazorCompileOnPublish>
</PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" Version="2.2.2" AllowExplicitVersion="true" />
    <PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="2.2.0" />
    <PackageReference Include="Z.EntityFramework.Plus.EFCore" Version="1.8.18" />
  </ItemGroup>
</Project>

Andrioden avatar Mar 22 '19 09:03 Andrioden

I was unable to reproduce with Docker in austindrenski/EFCore.PG-847. I don't work much with Docker though, so this could be user-error on my part.

@Andrioden Could you clone that repo and see if it triggers the error on your setup?

austindrenski avatar Mar 25 '19 01:03 austindrenski

@austindrenski : God damn. : ( Not able to reproduce it with your repo, i even did 10-20 changes to make it more similar to my environment and code and still not able to reproduce it. I also got paranoid and reran my own project with a more plainer version of the query, still get the error.

The next logical step seem to be setting up a asp.net core repo and slowly make it more and more similar to my project. I recall now and should mention I have seen this same problem with input and string to int parsing in other parts of asp.net core, without remember exactly what it was.

I realise that I should probably setup an repo and produce it more cleaner than in my full project, as I am in a pretty good position to do it, but this could take a quite a lot of time, which I am not prepared invest right now, as this is a bug I have a workaround for.

Andrioden avatar Mar 25 '19 20:03 Andrioden

Btw, I got curious and tested your fix in the context of my project

int amountToAdd = -1;

Console.WriteLine($"INTERVAL '{amountToAdd} TST'");
Console.WriteLine(FormattableString.Invariant($"INTERVAL '{amountToAdd} TST'"));
Console.WriteLine($"INTERVAL '{amountToAdd.ToString(provider: CultureInfo.InvariantCulture)} TST'");

Gives output INTERVAL '−1 TST' INTERVAL '-1 TST' INTERVAL '-1 TST'

Which means it should work? 🤔🤔🤔🤔🤔🤔 Are your sure you are applying the FormattableString.Invariant at the correct place?

Andrioden avatar Mar 25 '19 20:03 Andrioden

Not able to reproduce it with your repo, i even did 10-20 changes to make it more similar to my environment and code and still not able to reproduce it.

reran my own project with a more plainer version of the query, still get the error.

One difference I did notice was that your .csproj includes Z.EntityFramework.Plus.EFCore. I'm not sure what that project does, but if it's augmenting the EF Core pipeline, it could be related.

It might be worthwhile to start with your current project (which triggers the error), branch off and start cutting out that dependency.

austindrenski avatar Mar 26 '19 02:03 austindrenski

I tested my project without Z.EntityFramework.Plus.EFCore, still error. Also tested your repo with it, no error. So its not it. Here is all EF package installs from my build log.

Installing Npgsql.EntityFrameworkCore.PostgreSQL 2.2.0.
Installing Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore 2.2.1.
Installing Microsoft.EntityFrameworkCore.Tools 2.2.2.
Installing Microsoft.EntityFrameworkCore.Abstractions 2.2.2.
Installing Microsoft.EntityFrameworkCore.Analyzers 2.2.2.
Installing Microsoft.EntityFrameworkCore.InMemory 2.2.2.
Installing Microsoft.EntityFrameworkCore.Design 2.2.2.
Installing Microsoft.EntityFrameworkCore 2.2.2.
Installing Microsoft.EntityFrameworkCore.SqlServer 2.2.2. <- this one is wierd, dont know why, tried to search for usings, could not find one

Andrioden avatar Mar 26 '19 10:03 Andrioden

Closing for age. There have been significant changes around timestamp handling in version 6.0, it's likely that this has been resolved.

If not, please open a new issue with a minimal, runnable repro and I'll investigate.

roji avatar Mar 09 '23 10:03 roji