Overflow Error with some .arktribe files (xpost from arkpy-examples)

[DerekRies]

Original issue posted in DerekRies/arkpy-examples#1

I'm encountering the following error when attempting to read in .arktribe files.

OverflowError: Python int too large to convert to C long

Some have issues however, the majority (83%) of the .arktribe files I've run worked fine.

I can reproduce this issue with a script as simple as this. I have also attached some examples of .arktribe files that encounter this issue.

from arkpy import ark, utils

def main():
    file_path = 'Fails/1115390102.arktribe'
    tribe = ark.ArkTribe(file_path)

if __name__ == '__main__':
    main()

Fails.zip

Full error traceback below:

Traceback (most recent call last): File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\tribetest.py", line 9, in <module> main() File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\tribetest.py", line 6, in main tribe = ark.ArkTribe(file_path) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\ark.py", line 416, in __init__ self._load_from_stream(file_path) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\ark.py", line 433, in _load_from_stream struct = arktypes.load_struct(stream) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 28, in load_struct struct = STRUCTS.get(name, BaseStruct)(stream=stream) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 895, in __init__ self.load_and_set_next_property(stream) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 120, in load_and_set_next_property name, prop_type, value = load_property(stream) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 41, in load_property return (name, prop_type, PROPERTIES[prop_type](stream=stream)) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 219, in __init__ self.value = stream.readNullTerminatedString() File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\binary.py", line 63, in readNullTerminatedString s = self.unpack(str(length - 1) + 's', length - 1) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\binary.py", line 194, in unpack return unpack(fmt, self.readBytes(length))[0] File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\binary.py", line 22, in readBytes return self.base_stream.read(length) OverflowError: Python int too large to convert to C long

[DerekRies]

@Failoe I wasn't able to reproduce this issue as all those files work correctly for me. One thing they have in common though is the use of Unicode characters, which used to be a problem for the library. Are you on the latest version (0.1.3)?

It looks like you're using Windows, so I'm going to go ahead and say its probably an issue with my code for parsing Unicode strings on Windows. I haven't tested much of any of this on Windows so I'll have to set up a dev environment there for future testing. Might take me a couple days to find the time to do that though, so if you want to take a crack at fixing this go for it, I can always point you in the right direction.

[Failoe]

I am indeed using Windows (Server 2012 R2 specifically). I've verified that the newest version of arkpy is installed (0.1.3) and that all the modules in dependencies.txt have been installed. I'm still encountering the OverflowError: Python int too large to convert to C long error. Do you have any ideas as to steps I could take to troubleshoot this?

On Fri, Sep 23, 2016 at 4:00 PM, Derek Ries [email protected] wrote:

@Failoe https://github.com/Failoe I wasn't able to reproduce this issue as all those files work correctly for me. One thing they have in common though is the use of Unicode characters, which used to be a problem for the library. Are you on the latest version (0.1.3)?

It looks like you're using Windows, so I'm going to go ahead and say its probably an issue with my code for parsing Unicode strings on Windows. I haven't tested much of any of this on Windows so I'll have to set up a dev environment there for future testing. Might take me a couple days to find the time to do that though, so if you want to take a crack at fixing this go for it, I can always point you in the right direction.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DerekRies/arkpy/issues/20#issuecomment-249321682, or mute the thread https://github.com/notifications/unsubscribe-auth/AQVyf5VDCnlkP_Oee4_v7MLQ7v0Y6ZuAks5qtFoegaJpZM4KDK9w .

[Failoe]

As a followup to this i replaced all instances of xrange with itertool.count to try to solution in this SO article: http://stackoverflow.com/questions/22114088/overflowerror-python-int-too-large-to-convert-to-c-long

The error I received was "MemoryError" and when I printed the length variable in binary.py the value was "1953384788" which is probably my issue. Though it seems like something is wrong here unfortunately, I do not know enough about this to pinpoint it. If you have any advice I'd love to hear it.

On Fri, Oct 7, 2016 at 10:53 PM, Ryan [email protected] wrote:

I am indeed using Windows (Server 2012 R2 specifically). I've verified that the newest version of arkpy is installed (0.1.3) and that all the modules in dependencies.txt have been installed. I'm still encountering the OverflowError: Python int too large to convert to C long error. Do you have any ideas as to steps I could take to troubleshoot this?

On Fri, Sep 23, 2016 at 4:00 PM, Derek Ries [email protected] wrote:

@Failoe https://github.com/Failoe I wasn't able to reproduce this issue as all those files work correctly for me. One thing they have in common though is the use of Unicode characters, which used to be a problem for the library. Are you on the latest version (0.1.3)?

It looks like you're using Windows, so I'm going to go ahead and say its probably an issue with my code for parsing Unicode strings on Windows. I haven't tested much of any of this on Windows so I'll have to set up a dev environment there for future testing. Might take me a couple days to find the time to do that though, so if you want to take a crack at fixing this go for it, I can always point you in the right direction.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DerekRies/arkpy/issues/20#issuecomment-249321682, or mute the thread https://github.com/notifications/unsubscribe-auth/AQVyf5VDCnlkP_Oee4_v7MLQ7v0Y6ZuAks5qtFoegaJpZM4KDK9w .