bytechef icon indicating copy to clipboard operation
bytechef copied to clipboard

[bug]: CSV imports values enclosed with enclosing caracter

Open igorbeslic opened this issue 1 year ago • 0 comments

Describe the bug

If CSV file values are enclosed with enclosing character, CSV read engine does not strip off those characters...

Steps To Reproduce

Read and print this CSV file:

"LatD", "LatM", "LatS", "NS", "LonD", "LonM", "LonS", "EW", "City", "State"
   41,    5,   59, "N",     80,   39,    0, "W", "Youngstown", OH
   42,   52,   48, "N",     97,   23,   23, "W", "Yankton", SD
   46,   35,   59, "N",    120,   30,   36, "W", "Yakima", WA
   42,   16,   12, "N",     71,   48,    0, "W", "Worcester", MA
   43,   37,   48, "N",     89,   46,   11, "W", "Wisconsin Dells", WI
   36,    5,   59, "N",     80,   15,    0, "W", "Winston-Salem", NC
   49,   52,   48, "N",     97,    9,    0, "W", "Winnipeg", MB
   39,   11,   23, "N",     78,    9,   36, "W", "Winchester", VA
   34,   14,   24, "N",     77,   55,   11, "W", "Wilmington", NC
   39,   45,    0, "N",     75,   33,    0, "W", "Wilmington", DE
   48,    9,    0, "N",    103,   37,   12, "W", "Williston", ND
   41,   15,    0, "N",     77,    0,    0, "W", "Williamsport", PA
   37,   40,   48, "N",     82,   16,   47, "W", "Williamson", WV
   33,   54,    0, "N",     98,   29,   23, "W", "Wichita Falls", TX
   37,   41,   23, "N",     97,   20,   23, "W", "Wichita", KS
   40,    4,   11, "N",     80,   43,   12, "W", "Wheeling", WV
   26,   43,   11, "N",     80,    3,    0, "W", "West Palm Beach", FL
   47,   25,   11, "N",    120,   19,   11, "W", "Wenatchee", WA
   41,   25,   11, "N",    122,   23,   23, "W", "Weed", CA
   31,   13,   11, "N",     82,   20,   59, "W", "Waycross", GA
   44,   57,   35, "N",     89,   38,   23, "W", "Wausau", WI
   42,   21,   36, "N",     87,   49,   48, "W", "Waukegan", IL
   44,   54,    0, "N",     97,    6,   36, "W", "Watertown", SD
   43,   58,   47, "N",     75,   55,   11, "W", "Watertown", NY
   42,   30,    0, "N",     92,   20,   23, "W", "Waterloo", IA
   41,   32,   59, "N",     73,    3,    0, "W", "Waterbury", CT

workflow:

{
    "label": "Read CSV and send to DB",
    "description": "Read text file lines where data is separated by comma character.",
    "inputs": [],
    "triggers": [
        {
            "label": "Manual",
            "name": "manual",
            "type": "manual/v1/manual"
        }
    ],
    "tasks": [
        {
            "label": "Filesystem",
            "name": "filesystem_1",
            "parameters": {
                "filename": "/Users/igorbeslic/dev/bytechef/mount_to_docker/cities.csv"
            },
            "type": "filesystem/v1/readFile",
            "description": "Files at local machine where ByteChef Self Hosted docker image is running. Folder is configured with environment variable.",
            "metadata": {
                "ui": {
                    "dynamicPropertyTypes": {}
                }
            }
        },
        {
            "label": "CSV File",
            "name": "csvFile_1",
            "parameters": {
                "delimiter": ",",
                "headerRow": true,
                "includeEmptyCells": false,
                "readAsString": false,
                "fileEntry": "${filesystem_1}"
            },
            "type": "csvFile/v1/read",
            "description": "Interface to physical file configured by Filesystem component.",
            "metadata": {
                "ui": {
                    "dynamicPropertyTypes": {}
                }
            }
        },
        {
            "type": "loop/v1",
            "name": "loop_1",
            "parameters": {
                "items": "${csvFile_1}",
                "iteratee": [
                    {
                        "label": "Log",
                        "name": "logEntry1",
                        "parameters": {
                            "text": "Linija: ${loop_1.item} city: ${loop_1.item['\"City\"']}"
                        },
                        "type": "logger/v1/info"
                    },
                    {
                        "label": "PostgreSQL",
                        "name": "postgresql_1",
                        "type": "postgresql/v1/insert",
                        "parameters": {
                            "schema": "public",
                            "table": "CITY",
                            "columns": [
                                "${loop_1.item['\"City\"']}",
                                "${loop_1.item['\"State\"']}"
                            ]
                        },
                        "metadata": {
                            "ui": {
                                "dynamicPropertyTypes": {
                                    "columns[0]": "STRING",
                                    "columns[1]": "STRING"
                                }
                            }
                        }
                    }
                ]
            }
        },
        {
            "label": "PostgreSQL_2",
            "name": "postgresql_2",
            "parameters": {},
            "type": "postgresql/v1/query"
        }
    ]
}

Expected behavior

CSV values should have stripped values

Version

2.3.1

OS

MacOsX

Browser

Chrome

Additional context

No response

igorbeslic avatar Jul 30 '24 22:07 igorbeslic