CsvHelper
CsvHelper copied to clipboard
dynamic reading speed
Dynamic reading is really slow. Look into performance.
ExpandoObject
locks every time during an add operation. As a result, dynamic reading seems to be very slow.
To solve this problem will need an implementation that does not use ExpandoObject
. (Like SqlMapper.DapperRow
class in Dapper)
I think there are two implementation plans.
- Implement inherit
Dictionary<string, object>
that implementsIDynamicMetaObjectProvider
.
- It is easy to implement, but it is not memory efficient.
- Create a class that implements
IDictionary<string, object>
andIDynamicMetaObjectProvider
specialized for CSV
- Improve memory efficiency by sharing CSV header information.
I write and test the code to improve this problem. Thanks.
Itterating with foreach over IEnumerable<dynamic>
is extremely slow and takes around 5 minutes for 50 thousand records.
@superrnovae An auxiliary library has been created for fast handling. I believe it will help. https://github.com/shibayan/CsvHelper.FastDynamic
@shibayan So this makes using dynamic
faster than using types?
@JoshClose No, it will not be faster than accessing with types because of the overhead present. However, performance should be significantly improved compared to dynamic
using ExpandoObject
.