influxdb-client-csharp
influxdb-client-csharp copied to clipboard
Memory rising to 29Gb in one hour run
Steps to reproduce: List the minimal actions needed to reproduce the behavior.
- Singleton InfluxDBClient instance used for all writes. Single writeApi used for all calls throughout
client = new InfluxDBClient(url, Token);
var writeOptions = new WriteOptions
{
BatchSize = 5000,
FlushInterval = 500,
JitterInterval = 0,
};
converter = new DomainEntityConverter();
writeApiConv = client.GetWriteApi(writeOptions, converter);
// Below set data is called with other test client with load like:
// 200 parellel calls with each call having 500 samples, this is equal to 1lakh tags (or point data).
// Above load is called at every 1 second interval.
// Observation: Good news (happy with) writes are pretty fast in ~100ms, 200 parallel calls finishes easily.
// My main concern is memory rise.
Below details are for 10 min run memory going to ~15GB
public async Task SetData(SetDataRequest dataRequest, string bucket = bucketTagAsMeasurementName_Conv)
{
var data = new List<StaticTagAsMeasurement2>();
foreach (var sampleSet in dataRequest.SampleSets)
{
for (int sampleIndex = 0; sampleIndex < sampleSet.Samples.Count; sampleIndex++)
{
var sample = sampleSet.Samples[sampleIndex];
data.Add(new StaticTagAsMeasurement2
{
Id = Guid.Parse(sampleSet.TagId),
HashId = sampleSet.TagId.GetHashCode(),
Value = double.Parse(sample.Value.ToString()),
DataStatus = sample.DataStatus,
NodeStatus = sample.NodeStatus,
IsValid = sample.DataStatus == 0,
Time = sample.Timestamp
});
}
}
writeApiConv.WriteMeasurements(data, WritePrecision.Ns, bucket, orgName);
}
// I tried first with commented PointData approach & then with PointData.Builder approach - same memory rise with both.
DomainEntityConverter{
public PointData ConvertToPointData<T>(T entity, WritePrecision precision)
{
if (entity is StaticTagAsMeasurement2 tag2)
{
var pointBuilder = PointData.Builder.Measurement(tag2.Id.ToString());
pointBuilder = pointBuilder.Field("hashId", tag2.HashId);
pointBuilder = pointBuilder.Field("value", tag2.Value);
pointBuilder = pointBuilder.Field("isValid", tag2.IsValid);
pointBuilder = pointBuilder.Field("dataStatus", tag2.DataStatus);
pointBuilder = pointBuilder.Field("nodeStatus", tag2.NodeStatus);
pointBuilder = pointBuilder.Timestamp(new DateTime(tag2.Time, DateTimeKind.Utc), precision);
return pointBuilder.ToPointData();
// var pointData = PointData
//.Measurement(tag2.Id.ToString())
//.Field("hashId", tag2.HashId);
//.Field("value", tag2.Value)
//.Field("isValid", tag2.IsValid)
//.Field("dataStatus", tag2.DataStatus)
//.Field("nodeStatus", tag2.NodeStatus)
//.Timestamp(new DateTime(tag2.Time, DateTimeKind.Utc), precision);
// return pointData;
}
}
}
Expected behavior: The memory should be stable say around 1GB so that long runs can be achieved
Actual behavior: Memory is rising very fast
Specifications:
- Client Version:
- InfluxDB Version: v2.6
- Platform: Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz 2.59 GHz 64.0 GB (63.7 GB usable) 64-bit operating system, x64-based processor