ark-tools icon indicating copy to clipboard operation
ark-tools copied to clipboard

Hibernation state reading crash

Open The-Great-Sephiroth opened this issue 2 years ago • 33 comments

Since the latest patch, the tools no longer work. I have been using it for years to export data for display on a website. It appears as though the project is now abandoned. I hope that is not the case because this is a great tool for many uses. Below is the error you get no matter what you try to do.

java.lang.UnsupportedOperationException at qowyn.ark.ArkSavegame.readBinaryHibernation(ArkSavegame.java:387) at qowyn.ark.ArkSavegame.readBinary(ArkSavegame.java:172) at qowyn.ark.FileFormatBase.readBinary(FileFormatBase.java:30) at qowyn.ark.ArkSavegame.<init>(ArkSavegame.java:87) at qowyn.ark.tools.EditingCommands.exportThing(EditingCommands.java:145) at qowyn.ark.tools.App.main(App.java:125)

I have updated my JRE to the latest and no change, so I am assuming that this is a change to the save format.

The-Great-Sephiroth avatar Sep 24 '22 19:09 The-Great-Sephiroth

I get the same error when trying to read wild dino info

stolenvw avatar Sep 26 '22 12:09 stolenvw

@The-Great-Sephiroth and @stolenvw, if you would like to get it sorta working, you can comment out the lines of code: 385 to 388. I was able to create json using this. If you really want, I can create a fork.

AkitaAttribute avatar Sep 26 '22 15:09 AkitaAttribute

@The-Great-Sephiroth and @stolenvw, if you would like to get it sorta working, you can comment out the lines of code: 385 to 388. I was able to create json using this. If you really want, I can create a fork.

Hi

thank you very much for doing this :)

I've attempted to package your fork to use ark-tools.exe (although Im am not very familiar with what I am doing so may be doing something wrong) but after mvn package it I get an

Error: Could not find or load main class qowyn.ark.tools.App

can you spot what I am doing wrong please

thanks!

comical-uk avatar Oct 23 '22 11:10 comical-uk

@The-Great-Sephiroth and @stolenvw, if you would like to get it sorta working, you can comment out the lines of code: 385 to 388. I was able to create json using this. If you really want, I can create a fork.

Hi

thank you very much for doing this :)

I've attempted to package your fork to use ark-tools.exe (although Im am not very familiar with what I am doing so may be doing something wrong) but after mvn package it I get an

Error: Could not find or load main class qowyn.ark.tools.App

can you spot what I am doing wrong please

thanks!

Rather than make you struggle with downloading both of the projects, here is a packaged JAR that is working.

Here is a direct link to it in Google Drive

AkitaAttribute avatar Oct 23 '22 12:10 AkitaAttribute

The fork was updated if you are interested in the changes.

AkitaAttribute avatar Oct 23 '22 12:10 AkitaAttribute

fantastic, thank you very much!

comical-uk avatar Oct 23 '22 12:10 comical-uk

The fork was updated if you are interested in the changes.

sorry to bother you again, Ive just had the below on trying to do a m2j

Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; at qowyn.ark.ArkArchive.getString(ArkArchive.java:198) at qowyn.ark.properties.PropertyStr.(PropertyStr.java:26) at qowyn.ark.properties.PropertyRegistry.readBinary(PropertyRegistry.java:62) at qowyn.ark.GameObject.loadProperties(GameObject.java:375) at qowyn.ark.ArkSavegame.readBinaryObjectPropertiesImpl(ArkSavegame.java:340) at qowyn.ark.ArkSavegame.lambda$readBinaryObjectProperties$5(ArkSavegame.java:326) at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Unknown Source) at java.util.stream.IntPipeline$Head.forEach(Unknown Source) at qowyn.ark.ArkSavegame.readBinaryObjectProperties(ArkSavegame.java:326) at qowyn.ark.ArkSavegame.readBinary(ArkSavegame.java:169) at qowyn.ark.FileFormatBase.readBinary(FileFormatBase.java:31) at qowyn.ark.tools.ConvertingCommands.toJson(ConvertingCommands.java:89) at qowyn.ark.tools.ConvertingCommands.mapToJson(ConvertingCommands.java:24) at qowyn.ark.tools.App.main(App.java:125)

    does that mean anything to you?

comical-uk avatar Oct 24 '22 09:10 comical-uk

@comical-uk yeah. That was the issue I was fixing in the most recent change. Can I ask you to install Java 8? If not, I can modify it further.

AkitaAttribute avatar Oct 24 '22 09:10 AkitaAttribute

@comical-uk additionally, I just found that this exists yesterday while browsing Reddit: https://drive.google.com/file/d/1h1b9D1dkaGhUBTirywN0gFNZ32QjrTMU/view?usp=sharing

It's a full UI for this. It was ported to C#. I don't think you're gonna run into the same issue.

AkitaAttribute avatar Oct 24 '22 09:10 AkitaAttribute

@comical-uk additionally, I just found that this exists yesterday while browsing Reddit: https://drive.google.com/file/d/1h1b9D1dkaGhUBTirywN0gFNZ32QjrTMU/view?usp=sharing

It's a full UI for this. It was ported to C#. I don't think you're gonna run into the same issue.

thanks. I'm on Java 8u351

the ASV is a fantastic tool, I was just using the ark-tools to be able to convert it back into a .ark afterward for a few things

comical-uk avatar Oct 24 '22 10:10 comical-uk

@comical-uk yeah. That was the issue I was fixing in the most recent change. Can I ask you to install Java 8? If not, I can modify it further.

How to fix issue, I am on java 8

bomerman234 avatar Nov 11 '22 16:11 bomerman234

Rather than make you struggle with downloading both of the projects, here is a packaged JAR that is working.

Here is a direct link to it in Google Drive

didn't fix anything sadly, I'm getting even more errors with that one

BigBroza avatar Nov 13 '22 08:11 BigBroza

Rather than make you struggle with downloading both of the projects, here is a packaged JAR that is working. Here is a direct link to it in Google Drive

didn't fix anything sadly, I'm getting even more errors with that one

image

AkitaAttribute avatar Nov 13 '22 09:11 AkitaAttribute

So, for anyone who wants to get this working again, I'm willing to put in some effort to help you with it. Since it works for me, this will require some effort, and back and forth with someone who has a version that isn't working.

If you'd like to work with me to get it working, feel free to join my discord LINK and ping SATAN. Or just write something to the degree of wanting to help, since I get push notifications on my phone for that server.

AkitaAttribute avatar Nov 13 '22 10:11 AkitaAttribute

This Updated JAR I used after downloading java 8 on a fresh Windows 10 virtual machine. It should be working. I did make a few changes regarding the errors seen above.

image

AkitaAttribute avatar Nov 13 '22 10:11 AkitaAttribute

So, for anyone who wants to get this working again, I'm willing to put in some effort to help you with it. Since it works for me, this will require some effort, and back and forth with someone who has a version that isn't working.

If you'd like to work with me to get it working, feel free to join my discord LINK and ping SATAN. Or just write something to the degree of wanting to help, since I get push notifications on my phone for that server.

thank you very much for the kind offer

I'd gladly help test, I used to use it a lot both to convert from the map saves and then to convert back into the map saves

I'm going to be out for a few hours today though :)

comical-uk avatar Nov 13 '22 10:11 comical-uk

Just updated the JAR Same Link.

It now works for j2m as well. Same issues with m2j. Just some outdated library getting confused because it's overloaded. Some casting to fix it.

AkitaAttribute avatar Nov 13 '22 12:11 AkitaAttribute

image

AkitaAttribute avatar Nov 13 '22 12:11 AkitaAttribute

hey, thats seem to work fine converting them from the files that i've tested

however if I take a map save, then m2j and then j2m (without changes) then I get an error starting the server

the server starts fine normally, but not on a (fresh) map thats been m2j'd and then j2m'd

image

comical-uk avatar Nov 13 '22 16:11 comical-uk

hey, thats seem to work fine converting them from the files that i've tested

however if I take a map save, then m2j and then j2m (without changes) then I get an error starting the server

the server starts fine normally, but not on a (fresh) map thats been m2j'd and then j2m'd

image

Can you open task manager while running the game? Idk if the error message is bad, but it's indicating it's an out of memory error.

AkitaAttribute avatar Nov 13 '22 16:11 AkitaAttribute

Just updated the JAR Same Link.

It now works for j2m as well. Same issues with m2j. Just some outdated library getting confused because it's overloaded. Some casting to fix it.

thanks for the effort, I could m2j with this fix but I noticed the json is much smaller than usual. upon looking into it, it appears all creatures out of render range are missing from the json

it still helped me recover some of the data I needed so I am grateful nonetheless, god bless

BigBroza avatar Nov 13 '22 20:11 BigBroza

hey, thats seem to work fine converting them from the files that i've tested however if I take a map save, then m2j and then j2m (without changes) then I get an error starting the server the server starts fine normally, but not on a (fresh) map thats been m2j'd and then j2m'd image

Can you open task manager while running the game? Idk if the error message is bad, but it's indicating it's an out of memory error.

sorry missed this. it goes up to 7gb, then down to 3.5gb. plenty of memory left on the host when it crashes

thanks

comical-uk avatar Nov 13 '22 23:11 comical-uk

Just updated the JAR Same Link. It now works for j2m as well. Same issues with m2j. Just some outdated library getting confused because it's overloaded. Some casting to fix it.

thanks for the effort, I could m2j with this fix but I noticed the json is much smaller than usual. upon looking into it, it appears all creatures out of render range are missing from the json

it still helped me recover some of the data I needed so I am grateful nonetheless, god bless

Thanks for the reply. I will look into it, but I'm no expert when it comes to byte analysis. That's why not many people have done this, and Qowyn's stuff is still important.

I've basically commented out an error saying there is an issue with the hibernated creatures being read to get anything to work at all.

So, for now, this is probably as good as I can get it. I'll continue to poke at it now and then though.

AkitaAttribute avatar Nov 13 '22 23:11 AkitaAttribute

no worries, thanks for your efforts!

comical-uk avatar Nov 14 '22 16:11 comical-uk

I have not signed into Github in ages and just saw the notification for this. I deciphered a lot of the old Daggerfall BSA format back in the day (all binary formats) and can try to help, but it's been a decade or more since I did so.

The-Great-Sephiroth avatar Feb 17 '23 22:02 The-Great-Sephiroth

i have made a fork that fixes this completly and adds support for reading cryopot data

https://github.com/Qowyn/ark-tools/issues/47#issuecomment-1312846926 this is because it sets the hibernated objects to -1 and thats overflows so it tries to allocate a lot of memory for nothing that also explains the high file size

Alexander3a avatar Apr 08 '23 21:04 Alexander3a

@Alexander3a did you make the fork public? Can you share a link? I don't see it on your page or on the forks page of this repo.

i have made a fork that fixes this completly and adds support for reading cryopot data

AkitaAttribute avatar Apr 08 '23 22:04 AkitaAttribute

@AkitaAttribute its not a "proper" fork since i basicly merged the ark-tools and toolkit into one project so i dont have to edit them seperatly i may split it again to make a proper fork if there is enough interesst in the source but at the moment it seems pretty useless i woud be happy to give out the compiled jar file if you need it

Alexander3a avatar Apr 09 '23 17:04 Alexander3a

@Alexander3a is it published on GH? I can compile it myself, unless you would rather not make it public.

AkitaAttribute avatar Apr 09 '23 17:04 AkitaAttribute

@AkitaAttribute do your have a discord i coud add you or something?

Alexander3a avatar Apr 09 '23 18:04 Alexander3a