hello09x comments

Results 36 comments of


                                            hello09x

Error to read csv encoding utf-8 with bom and export back to parquet

I think removing the first character if it's `\ufeff` will solve this problem

Error to read csv encoding utf-8 with bom and export back to parquet

> by reading a csv encoded with UTF-8 with bom? ```python import pandas df = pandas.DataFrame({ "A": [1, 2, 3, 4, 5], "B": [1, 2, 3, 4, 5], }) df.to_csv("withbom.csv",...

Error to read csv encoding utf-8 with bom and export back to parquet

I use `utf-8-sig` because open a csv file with excel will use `gbk` default, the bom will tell excel to read with `utf-8` encoding

Error to read csv encoding utf-8 with bom and export back to parquet

> Is `df, err := imports.LoadFromCSV(ctx, fr)` correctly reading the csv? yes

Error to read csv encoding utf-8 with bom and export back to parquet

> As an experiment can you: > > 1. Iterate over []df.Series > 2. Call Rename on each series and prepend each series name with `"X"` (e.g `X年龄`) > see...

Error to read csv encoding utf-8 with bom and export back to parquet

> Okay, also call https://pkg.go.dev/strings#TrimSpace no effect

Error to read csv encoding utf-8 with bom and export back to parquet

> Also when importing CSV: > > ``` > type CSVLoadOptions struct { > // If TrimLeadingSpace is true, leading white space in a field is ignored. > // This...

Error to read csv encoding utf-8 with bom and export back to parquet

`s.Rename("X" + strings.Trim(s.Name(), "\xEF\xBB\xBF"))` works

Error to read csv encoding utf-8 with bom and export back to parquet

> I believe I have a solution. Does Python always produce `"\xEF\xBB\xBF"` when saving csv? What is the purpose of it? I'm just asking because I'm trying to figure out...