Invalid memory address error

[TBDMark]

Hey there,

Is there a column limitation to what pgfutter can import? Here's my code and resulting error message. These files have around 500 columns with a header row, but with every iteration of the file it throws an error, including after I reopen and save the .csv as UTF-8 encoding. Other, smaller files work just fine.

 ./pgfutter --user "pgfutter" \
     --pass "pgfutter" \
     --schema "model" \
     csv '/Users/user/Desktop/scwUTF.csv'

Here's the error that results. Other, much smaller .csv files import without any issue.

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x7c641]

goroutine 1 [running]:
database/sql.(*Stmt).Exec(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /goroot/src/database/sql/sql.go:1302 +0x3e1
main.NewCSVImport(0xc208035400, 0xc2080332a0, 0x8, 0xc208033420, 0x6, 0xc208108000, 0x1f1, 0x200, 0x0, 0x0, ...)
    /usr/src/pgfutter/import.go:21 +0xfd
main.importCSV(0x7fff5fbffbf9, 0x2b, 0xc208011f80, 0x6c, 0xc2080332a0, 0x8, 0xc208033420, 0x6, 0x0, 0x5511f0, ...)
    /usr/src/pgfutter/csv.go:127 +0x444
main.func·002(0xc2080ac0f0)
    /usr/src/pgfutter/pgfutter.go:141 +0x369
github.com/codegangsta/cli.Command.Run(0x3784b0, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3c7d10, 0x18, 0x0, ...)
    /gopath/src/github.com/codegangsta/cli/command.go:127 +0xffe
github.com/codegangsta/cli.(*App).Run(0xc2080a8200, 0xc20800a000, 0x9, 0x9, 0x0, 0x0)
    /gopath/src/github.com/codegangsta/cli/app.go:159 +0xd7b
main.main()
    /usr/src/pgfutter/pgfutter.go:147 +0xaca

Thanks!

[lukasmartinelli]

Thanks for the error report. I will look into this. You have a sample CSV file for me?

Otherwise I can also generate one myself.

[TBDMark]

Absolutely. Here’s a sample.

On Dec 1, 2015, at 4:42 AM, Lukas Martinelli [email protected] wrote:

Thanks for the error report. I will look into this. You have a sample CSV file for me?

Otherwise I can also generate one myself.

— Reply to this email directly or view it on GitHub https://github.com/lukasmartinelli/pgfutter/issues/6#issuecomment-160911484.

[lukasmartinelli]

http://www.postgresql.org/about/

It seems PostgreSQL has a column limit - but I will check whether that is really the problem.

Maximum Columns per Table 250 - 1600 depending on column types

[TBDMark]

Thanks - I was able to import this file manually to PostgreSQL, so I’m not sure if that’s the problem, but let me know what you find.

On Dec 2, 2015, at 6:50 AM, Lukas Martinelli [email protected] wrote:

http://www.postgresql.org/about/ http://www.postgresql.org/about/ It seems PostgreSQL has a column limit - but I will check whether that is really the problem.

Maximum Columns per Table 250 - 1600 depending on column types

— Reply to this email directly or view it on GitHub https://github.com/lukasmartinelli/pgfutter/issues/6#issuecomment-161270206.

[houshuang]

This seems to be the same bug. Minimal CSV file example:

date,time,user,method,url
2015-12-16,22:00:40,710,-,GET,display/5574/Edu5574+Home
2015-12-13,16:06:53,418,-,GET,display/5574/Edu5574+Home
2016-02-05,12:26:50,329,-,GET,display/5574/House+3+Presentation+Schedule
2016-02-05,13:09:12,760,-,GET,display/5574/Socrative+Presentation+by+Amy%2C+Bushra+and+Mark
2016-02-05,14:32:25,812,-,GET,display/5574/Geometer's+Sketchpad+Presentation
2016-02-05,15:03:17,043,-,GET,display/5574/DebateGraph%2C+Week+3
2016-02-05,16:05:48,034,-,GET,display/5574/Week+7%3A+Document+Camera
2016-02-05,16:28:15,533,-,GET,display/5574/February+5th+-+Presentation+on+Socrative.com
2016-02-03,14:00:43,990,xiangsun0205,GET,display/5574

Error (pgfutter 0.4)

~/D/P/d/wikilogs $ pgfutter --user "stian" --db "oisewiki" csv  5574sample.csv
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x7d151]

goroutine 1 [running]:
database/sql.(*Stmt).Exec(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /goroot/src/database/sql/sql.go:1302 +0x3e1
main.NewCSVImport(0x20862f4a0, 0x2085fb446, 0x6, 0x2085fb620, 0xb, 0x208672300, 0x5, 0x8, 0x0, 0x0, ...)
    /usr/src/pgfutter/import.go:21 +0xfd
main.importCSV(0x7fff5fbffc30, 0xe, 0x20867a0e0, 0x61, 0x2085fb446, 0x6, 0x2085fb620, 0xb, 0x0, 0x55c470, ...)
    /usr/src/pgfutter/csv.go:127 +0x444
main.func·003(0x208630a00)
    /usr/src/pgfutter/pgfutter.go:165 +0x369
github.com/codegangsta/cli.Command.Run(0x380590, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3d05f0, 0x18, 0x0, ...)
    /gopath/src/github.com/codegangsta/cli/command.go:174 +0x1403
github.com/codegangsta/cli.(*App).Run(0x208630780, 0x2085d4000, 0x7, 0x7, 0x0, 0x0)
    /gopath/src/github.com/codegangsta/cli/app.go:187 +0x1221
main.main()
    /usr/src/pgfutter/pgfutter.go:171 +0xba4

goroutine 5 [syscall]:
os/signal.loop()
    /goroot/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /goroot/src/os/signal/signal_unix.go:27 +0x35

[lukasmartinelli]

This seems to be the same bug. Minimal CSV file example:

Thanks for the sample. Now I can make a regression.