Starbound 1.0 support release checklist

[wizzomafizzo]

• [ ] py-starbound support for new database format (SBBF04?)
• [ ] Graphic glitches, countless ones showing up now (see Avian species for example)
• [ ] Crash when opening ship dialog, possibly to do with new character: #271
• [ ] Blueprint dialog crash if null value in list: #260
• [ ] Quests dialog completely broken, need support for new format

Less important:

• [ ] Weapon generation is completely broken (see here for some inspiration)
• [ ] Blueprint list reordering on modification: #249
• [ ] Support for more directives: #261

[wizzomafizzo]

Starfuse by @Qix- is a potential alternative to py-starbound. Will be testing soon

[wizzomafizzo]

Graphic glitches may have something to do with the background of the labels? Need to do some more testing but a lot of stuff magically fixed itself when I randomly changed the background colour

[wizzomafizzo]

@chrmoritz just fyi i think we will end up using starfuse instead of py-starbound (minus the fuse parts). qix is taking a look at the new format in nightly soon, blixt has said he has no time to work on the project at the moment

[Qix-]

Happy to look at it. Any information you have about it will definitely help, too.

[wizzomafizzo]

Shoot man I'll gladly do anything I can to help! I'm not sure what would be helpful though, happy to hear anything you think you need

[wizzomafizzo]

Don't feel like I'm dobbing you in either :P no rush on this stuff

[Qix-]

Well first off, have you tried just adding the SBBF04 string into the array that's checked in the header check for existing libs to see if 'just works'? I'm wondering how much of a change warrants the version number increase or if that's just a way to enforce certain asset files be used with certain versions. I can't imagine the devs changing a simple block format so much between versions, and SBBF03 works just fine with the SBBF02 version.

[wizzomafizzo]

I did but no dice. The new string at least for vanilla assets file is "SBAsset4". It might be as simple as making the string longer than 6 bytes but I wasn't sure if that would break things.

Btw I need to make it work with Python 3 too but I can always fork it and make the changes myself if it can't work for both like py-starbound. As far as I got all I had to do was make the module imports case sensitive and explicitly make some regular strings byte strings. That doesn't really matter for now but just a heads up

Sent from my iPhone

On 19 May 2016, at 1:51 AM, Josh Junon [email protected] wrote:

Well first off, have you tried just adding the SBBF04 string into the array that's checked in the header check for existing libs to see if 'just works'? I'm wondering how much of a change warrants the version number increase or if that's just a way to enforce certain asset files be used with certain versions. I can't imagine the devs changing a simple block format so much between versions, and SBBF03 works just fine with the SBBF02 version.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub

[Qix-]

Gotcha, I can make that a priority for you :)

[rubellyte]

The nightly SBAsset format appears to be very different from the old SBBF03 format. I spent a while trying to figure it out, and this is what I came up with:

• At the beginning of the file, magic bytes "SBAsset6"
• 8 bytes immediately following, offset of file index
• Following that, apparently unmodified files
• At offset specified in beginning of file (end of files), magic bytes "INDEX"
• Byte after INDEX indicating number of metadata entries.
• Metadata entry:
• byte containing length of key name
• key name
• Byte that probably indicates value type (0x02 for float, 0x04 for signed int, 0x05 for string, 0x06 for list, 0x07 for dict)
• Length of value (int values don't get this)
• value
• 1 to 3 bytes of file count (I cannot for the life of me figure out how this is counted, here's some data I collected for anyone who wants to try and solve it)
• File index format:
• Byte with length of path
• Complete file path
• 8 bytes containing offset of file
• 8 bytes containing length of file
• File ends without any sort of marker

I'll update this if I figure out anything else.

[Xirukai]

1.0 is out on the unstable branch. Just giving a friendly headsup, using this program tons ;)

[chrmoritz]

And the release date for stable is the 22 July. 🎉

cc @wizzomafizzo

[rubellyte]

I've got a fork that can read Cheerful player files, but it's mostly useless without something to read SBAsset4, and unfortunately I think that writing something for that is beyond my skills.

[Qix-]

Thanks @medeor413 I can play with it sometime next weekend probably and try it out with starfuse.

[wizzomafizzo]

oops well i guess people are gonna have to chill out and wait a while

@medeor413 are you saying this release changed the player files too? that's a bummer

[rubellyte]

I've already got a fork that works with 1.0 player files, and I started working on something to work with SBAsset, but I'm stuck there.

[wizzomafizzo]

@medeor413 i think the fork looks good i'd be happy to take pull requests if you like

we have to wait for a really nice person to do it :P i'm stuck too

[rubellyte]

I'll put in a PR once I get SBAsset working, since it'll be pretty useless without it.

[rubellyte]

Aaand done. Pull request here.

[chrmoritz]

As Windows nightly build of this PR is available here: https://ci.appveyor.com/project/wizzomafizzo/starcheat/build/master-b59/job/umt2qe3mxck4a9l5/artifacts (if anyone else wants to try it out without having to compile the sources).

[wizzomafizzo]

dude you rock, i'm sure you just made a lot of people happy. i will give it a shot when i get home

have you noticed much difference between this implementation and py-starbound? i just wonder if it got noticeably slower

[rubellyte]

It takes a while for me, but my loading times have always been long due to sheer size of my player files.

[wizzomafizzo]

All good. Hey I'll have to leave this one a bit longer I think, I have no internet atm and can't test anything. If you wanna make changes or have suggestions or whatever don't hold back too, I'm up for whatever

Btw would you be able to upload a screenshot of how it looks now with the UI changes you made? Interested to see them too

[wizzomafizzo]

I wonder if there is a way to speed up the player file loading too? I haven't looked at it in a while but from memory the biggest bottleneck is simply loading up the inventory icons from disk. I'm not sure if it's really avoidable

[kurisubrooks]

Here's how it currently looks. Pillow has some rendering issues, Description and Mouse slots are pending removal, and the hotbar is not currently functional, but the overall function is there.

[kurisubrooks]

I just tested loading Starcheat 0.27 (on macOS) and pending 1.0 release (on Windows 10), but to me 1.0 seems faster.

Overall I don't think there is much of a difference.

[kurisubrooks]

You can see all current outstanding issues here: https://github.com/medeor413/starcheat/issues

[wizzomafizzo]

btw i still have no internet connection at home sorry guys. @chrmoritz if you are comfortable with doing a release at some point please go ahead but i'm ok with waiting until i can test it myself

[wizzomafizzo]

hey could someone give me a run down of where this is at? not sure i'm keeping up. should we be doing a new release?

@chrmoritz could you please help with your git expertise again too i don't know how i would merge the pull request and make dev sync with master

[blixt]

Not sure if you need this anymore, but I've updated py-starbound now to support all the Starbound 1.0 file formats, including write support for SBVJ01 files that's relatively fast (a .player file that's 128 kB writes in 0.029 seconds for me):

import starbound, time
start = time.clock()
with open('SB/storage/player/916d5878483e3a40d10467dc419982c2.player', 'rb') as fh:
  vj = starbound.read_sbvj01(fh)

print('Took %s seconds to read file' % (time.clock() - start,))
print('Player name: %s' % (vj.data['identity']['name'],))

start = time.clock()
with open('SB/storage/player/916d5878483e3a40d10467dc419982c2.player.dupe', 'wb') as fh:
  starbound.write_sbvj01(fh, vj)

print('Took %s seconds to write file' % (time.clock() - start,))

You may also get better speed by using the mmap package.