Exception thrown for value cannot be null - parameter 'destination' - in the CsvWriter and the value.CopyTo(..) call.

[bhehe]

Describe the bug

I am implementing an output of results to a CSV file where the code is being modelled off of a pre-existing implementation; The biggest difference with this implementation is that in the pre-existing code the 'report stream' is passed into the component and then a method for 'WriteRecords' is called which in turn creates the StreamWriter/CsvWriter pair and performs the write operation for the header & the items as a single batch write via WriteRecordsAsync(..).

The initial 'report stream' was created from a File.Open call if that has any bearing.

The working code looks like this:

            using var streamWriter = new StreamWriter(reportStream, leaveOpen: true);
            using var writer = new CsvWriter(streamWriter, CultureInfo.InvariantCulture, leaveOpen: true);
            writer.WriteHeader<ReportRecord>();
            await writer.NextRecordAsync().ConfigureAwait(false);
            await writer.WriteRecordsAsync(ReportRecords.OrderBy(x => x.UserId), cancellationToken).ConfigureAwait(false);
            await writer.FlushAsync().ConfigureAwait(false);

In the new implementation, we have a similar behavior in that we open an initial stream, still using File.Open, and then we create the StreamWriter & CsvWriter; Where it differs is that we pass the CsvWriter along to the component doing the processing and then while processing each item 1-by-1 we have a method to write to the report/results output, which accepts the CsvWriter to be used along with the item to be written.

The initial call to write the headers appears to work, but when the actual item is being written I get an exception stating that the 'destination' cannot be null.

System.ArgumentNullException : Value cannot be null. (Parameter 'destination')

I've stepped into the CsvWriter code and observed it writing the header row correctly, and in that I can see the 'buffer' is present (not null) and of the expected size (default). After that, when it hits the item to actually write out, I see the code performing the value.CopyTo(..) call -and- that the buffer is indeed a null reference at that point. So somehow/somewhere the buffer is getting set to a null value while writing the actual items.

            csvWriter.WriteRecord(resultRecord);
            await csvWriter.NextRecordAsync().ConfigureAwait(false);

In trying to resolve this, I did upgrade to the latest version but I still get the same root-cause/exception returned. As this effort was during a time-crunch I've had to forego the write-as-you-go model that would have been preferable - and switched to just collecting all the items in memory and then writing them at the end which is how the prior working implementation was done. And switching back to that approach did work. So, I'm a bit stumped as to how the minor differences could result in the observed behavior.

Expected behavior Writing the items completes successfully; The buffer is a not-null value and of the expected length (per the config) while writing the individual items.

Additional Context I did try using WriteRecordsAsync(..) method and passing in a single-item array; That did not alter the outcome.

In the version I was working against (28.0.1, not the latest) I was also looking for, but not finding, an async method for writing a single record which would have been preferable to keep things consistently using async methods throughout. Thought to mention it here rather than creating a feature request - and I've not checked the latest version as I reverted back to 28.0.1 for the time being.

Thanks for any insights you can provide, though we had to punt and go with the proven/working approach for this current effort we would still like to be able to leverage this library with an approach of writing the items as they are processed.

[kirk56k]

Just thought I'd toss in my two cents here real quick. Just so you know you're not alone. I have a system that makes very heavy use of CsvHelper to write stupidly large CSV files (several gigs, unfortunately, not my choice). During one of hundreds of times my application has processed these large data sets, I ran into a very similar problem. My code is far more simplistic than yours, though, it runs in a dedicated thread and is entirely synchronous within that thread. Code looks like below:

CsvWriter csvOutput = GetCsvOutWriter(queueItem.Item1);
csvOutput.WriteRecord(queueItem.Item2);
csvOutput.NextRecord();
csvOutput.Flush();
writenIDs.Add(queueItem.Item2.id);

The CsvWriter in question is instantiated at the start of the thread where WriteHeader is called along with the first NextRecord, and then used throughout the process of monitoring and writing data from an internal memory queue. This application has been running for the better part of 6 months with the current unmodified code base, and in all that time I've only seen this exception once.

OpenFileManager Module Temporary Exception. Module Restarting...
Exception: CsvHelper.WriterException: An unexpected error occurred.
IWriter state:
   Row: 2
   Index: 0
   HeaderRecord:
2
 ---> System.ArgumentNullException: Value cannot be null.
Parameter name: destination
   at System.String.CopyTo(Int32 sourceIndex, Char[] destination, Int32 destinationIndex, Int32 count)
   at CsvHelper.CsvWriter.WriteField(String field, Boolean shouldQuote)
   at CsvHelper.CsvWriter.WriteField(String field)
   at CsvHelper.CsvWriter.WriteConvertedField(String field, Type fieldType)
   at lambda_method(Closure , LOBObjectSchemaCSV )
   at System.Action`1.Invoke(T obj)
   at CsvHelper.Expressions.RecordWriter.Write[T](T record)
   at CsvHelper.Expressions.RecordManager.Write[T](T record)
   at CsvHelper.CsvWriter.WriteRecord[T](T record)
   --- End of inner exception stack trace ---
   at CsvHelper.CsvWriter.WriteRecord[T](T record)

Unfortunately, the current iteration of this code does not do any more than spit out the stack trace as I pasted above (future versions will give a full memory dump). Also of interest is that this particular error occurred with 28.0.1 of CsvHelper. I have since updated the code base to the latest version. Also of interest is that I grabbed a copy of the source data that initially caused this exception and ran it through the same code a second time, and received no exceptions. It would seem that this is likely a HeisenBug, as such if there are any other details you might find useful, don't hesitate to ask, I'll provide what I can.

[rfink]

Same issued.

[hypdeb]

Old bug report, but in case someone ends up here googling this exception like I did, here is my experience.

I had the same issue, and in my case, I was just being silly and disposing of my writer, stream and file handle before using them... The error message is very confusing though. Maybe it's not easily doable, but if we could get an "ObjectDisposedException" instead of this, it could spare idiots like me (and I'm really talking only about myself here) a lot of time 🤣 .

You don't provide your code, but here is a sketch of what we have:

class CsvFile : IDisposable
{
    // keeps a reference to the writer, stream and file and disposes of them

    Task WriteAsync(List<Record> records)
    {
        // Writes records using the CSV Writer
    }
}

class FileProvider
{
    CsvFile GetFile()
    {
        // Open the file, create the stream and the writer. Put them on the file
    }

}

The writer was being disposed in GetFile 🤦‍♂️ before being returned as part of the File... Just wrapping them in a using statement by force of habit I guess.

Have a look and see if you're not in the same situation.