maxminddb-golang icon indicating copy to clipboard operation
maxminddb-golang copied to clipboard

Add DecodeLength and Size()

Open arp242 opened this issue 4 years ago • 4 comments

The use case for this is pretty much as in the test: I'd like to get the data section of a database file. As far as I know there's no easy way to do this currently.

I'm not super-happy with the name "DecodeLength()", but I couldn't think of anything better.

Size() is needed because otherwise we don't know if we're at the last entry.

arp242 avatar Dec 26 '20 15:12 arp242

What about an iterator over the data section instead? This could encapsulate the logic for checking the section size. It could follow a similar pattern to the Networks iterator. I think the only time DecodeLength would really be useful is if you are decoding the whole data section.

I do agree that the name DecodeLength is a bit misleading, as it has little to do with the length of the data decoded as most databases will contain many pointers in this data causing the actual length of the data to be much longer.

If this was changed to an iterator, perhaps this verification code could be updated to use it:

https://github.com/oschwald/maxminddb-golang/blob/2b2ff7ada5e406deab64fc0e729837caea5663da/verifier.go#L139-L159

oschwald avatar Dec 26 '20 15:12 oschwald

I think the only time DecodeLength would really be useful is if you are decoding the whole data section.

Yeah, probably; I was a bit surprised that it's not already in here, because as far as I know this is pretty much the easiest way to get a list of ISO-3166-2 names (so many lists on the 'net are very old, have encoding errors, drop diacritics, or just have outright bad data; sigh).

Anyway, I'll have a look and change it to an iterator; can also name it something more sensible then.

arp242 avatar Dec 26 '20 15:12 arp242

Okay, I changed it to a Data() method which returns a Data iterator, behaving pretty much the same as the Networks one.

arp242 avatar Dec 27 '20 12:12 arp242

Hey @oschwald; is there any chance you can take another look and let me know if you want any further changes? No worries if you don't have time, I can keep using my fork (also for the related PR at https://github.com/oschwald/geoip2-golang/pull/68). Thanks!

arp242 avatar Feb 25 '21 12:02 arp242