Transformalize icon indicating copy to clipboard operation
Transformalize copied to clipboard

How to detect Header DataTypes during File Parsing

Open papyr opened this issue 5 years ago • 2 comments

in my case, the source CSV data file and header content/types change often -- basically unknown, while parsing or before parsing I need to first Discover the Data Types of the data in the cols.

I just need to load & save the data into a Sql table based on whatever data & DataTypes there are in the CSV file. Can I auto load the CSV data and datatypes into the DB automatically with ETL box. I saw your expando example but was a little lost how to achieve a high level task

For e.g high level algorithm, how would I achieve this in your lib.

// Scan first 9 rows to discover the datatypes in the CSV file cols
var etlPrasedFile = DaleNewMan.Trans.Open/parseCSV(someUnknownData.csv).ScanRows(9)
foreach (var col in etlPrasedFile )
new dataTable.addColumn(etlParsedFile.GetNextHeaderCol)
CreateNewSqlServerTable
LoadDatatoSqlTable??

thanks

papyr avatar Feb 16 '20 07:02 papyr

This doesn't look like code that uses my library.

dalenewman avatar Apr 23 '20 14:04 dalenewman

hello, yes true.

I was simply giving and example of what I want to achieve, and did not want to leave it vague, so I added some snippets.

papyr avatar Apr 23 '20 19:04 papyr