Error message when data section exceeds the maximum size is improved (was: Corrupt data when DB is large)

[ebclark2]

I'm running into an issue where the perl reader tells me data is corrupt when trying to read from a database file of 30 gigs I receive decode errors. Reading from a smaller, say 1gig, database works perfectly file. Are there limitations on the size of the database I can create or am I doing something wrong? I am using the same method to create both databases, which is just iterating IPs and adding a single large string value. Test code is attached.

read.pl.txt write.pl.txt

[oschwald]

The writer and most readers only support a record size of 32 currently. This means the data section will be limited to 4 GB. Ideally, the writer would throw a more meaningful error, but I think it is safe to say a 30 GB database is not currently supported.

[oschwald]

Closing this as future development is focused on the Go mmdbwriter module.