No flushing

[tomwimmenhove]

Steps to reproduce:

1. Create a WriteApi instance with either a FlushInterval, or occasionally flush manually
2. Write a few hundred thousand/million records using WriteApi.WriteRecord()

Expected behavior: Occasional flushing, especially when disposing the WriteApi.

Actual behavior: No flushing, constantly increasing memory usage and no flushing when the object is disposed, resulting in large amounts of data never been written to the database.

When adding a large delay (I.E. Console.ReadLine() before disposing, and not hitting enter until memory usage stabilizes :) ), all data does get correctly written.

Specifications:

• Client Version: v1.13.0
• InfluxDB Version: 2.0-rc
• Platform: .NET 4.7.2

[bednar]

Hi @tomwimmenhove,

thanks for using our client.

Could you share a piece of code how you use a client? Do you always write data into same bucket and org?

Regards

[tomwimmenhove]

I'm sorry, but I can't share my current code. If I have time, maybe I can create some snippet at a later date.

To try to work around the problem, I'm trying to split my data up in smaller chunks, and instantiate a new client/WriteApi for each chunk, and dispose it after.

The Dispose() method for my class using the InfluxDB.Client currently looks like:

       public void Dispose()
        {
            _writeApi?.Flush();
            _writeApi?.Dispose();
            _client?.Dispose();

            Console.WriteLine("HIT ENTER");
            Console.ReadLine();
        }

I've been using this to test what's happening. I've noticed that, after the "HIT ENTER" message appears, flushing hasn't actually finished. There are still queries being written to InfluxDB at this point.

Can you think of a temporary work-around to actually force my code to wait until all data has been written?

[bednar]

Sorry for the late

The WriteApi is suppose to run as a long live singleton and uses batching in background. You don't need to split your data into chunks.

To try to work around the problem, I'm trying to split my data up in smaller chunks, and instantiate a new client/WriteApi for each chunk, and dispose it after.

This sounds to me like that WriteApiAsync will be better for your use-case. The WriteApiAsync directly writes record into database without batching.

var writeApiAsync = influxDbClient.GetWriteApiAsync();

//
//
// Write by LineProtocol
//
await writeApiAsync.WriteRecordAsync("my-bucket", "my-org", WritePrecision.Ns,
                "temperature,location=north value=60.0");

//
//
// Write by Data Point
//               
var point = PointData.Measurement("temperature")
                .Tag("location", "west")
                .Field("value", 55D)
                .Timestamp(DateTime.UtcNow.AddSeconds(-10), WritePrecision.Ns);

await writeApiAsync.WritePointAsync("my-bucket", "my-org", point);

Regards

[ftkus]

Hi,

I am observing a similar issue. I am using the WriteApi to write a PointData to my database, and nothing is happening. Note that I was able to successfully write for several days (about 1 week) continuously every 5 minutes, and then suddenly nothing. Ive rebooted the machine (both writing device and DB host), but my database is still not updating.

Is there some way of getting more information (e.g. status of WriteApi)?

EDIT: system specifications:

• InfluxDB 2.0
• .NET 5
• Client v1.18

[bednar]

Hi @ftkus,

thanks for using our client.

Is there some way of getting more information (e.g. status of WriteApi)?

you can enable debug logging by: Client.SetLogLevel(LogLevel.Body);.

Could you share your debug output if you directly close writeApi via: _writeApi.Dispose();?

Regards

[ftkus]

Hi,

I actually have two projects with this issue, the other being on the .net framework 4.7.2. Thats actually the one I have access to today, so here is the output I am getting after enabling LogLevel.Body:

The unhandled exception occurs: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Extensions.ObjectPool, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.Extensions.ObjectPool, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
   at InfluxDB.Client.WriteApi.<>c.<.ctor>b__9_3(IGroupedObservable`2 grouped)
   at System.Reactive.Linq.ObservableImpl.Select`2.Selector._.OnNext(TSource value) in /_/Rx.NET/Source/src/System.Reactive/Linq/Observable/Select.cs:line 39

=== Pre-bind state information ===
LOG: DisplayName = Microsoft.Extensions.ObjectPool, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60
 (Fully-specified)
LOG: Appbase = file:///../Debug/
LOG: Initial PrivatePath = NULL
Calling assembly : InfluxDB.Client, Version=1.18.0.0, Culture=neutral, PublicKeyToken=2ffdb0e6ebde0d3f.
===
LOG: This bind starts in default load context.
LOG: Using application configuration file: ..\Debug\.exe.Config
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
LOG: Post-policy reference: Microsoft.Extensions.ObjectPool, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60
LOG: Attempting download of new URL file:///../Debug/Microsoft.Extensions.ObjectPool.DLL.
LOG: Attempting download of new URL file:///../Debug/Microsoft.Extensions.ObjectPool/Microsoft.Extensions.ObjectPool.DLL.
LOG: Attempting download of new URL file:///../Debug/Microsoft.Extensions.ObjectPool.EXE.
LOG: Attempting download of new URL file:///../Debug/Microsoft.Extensions.ObjectPool/Microsoft.Extensions.ObjectPool.EXE.

The thread 0x4a94 has exited with code 0 (0x0).
The thread 0x7598 has exited with code 0 (0x0).
Flushing batches before shutdown.
Flushing batches before shutdown.
The thread 0x2390 has exited with code 0 (0x0).
The thread 0x1a34 has exited with code 0 (0x0).
The thread 0x7ff0 has exited with code 0 (0x0).

EDIT: Both systems are actually using v1.18 of the client

[ftkus]

I've been running some tests with a coworker, and I think we found where the problem is stemming from:

• restoring our Nuget Packages seems to resolve the issue (Clear All NuGet Cache(s), followed by an Update-Package -reinstall)
• Performing a git clean -xd of our repository restores the problem. We suspect the Microsoft.Extensions.ObjectPool assembly is being removed on our repo cleanup, will look more into it and report back if we were successful in isolating the guilty party.

[michaelahojna]

The issue is no longer valid - it was fixed by migration to System.Net.Http.HttpClient - https://github.com/influxdata/influxdb-client-csharp/blob/master/CHANGELOG.md#400-2022-03-18