The-Minecraft-Overviewer icon indicating copy to clipboard operation
The-Minecraft-Overviewer copied to clipboard

Bare-bones (Bobby) MCA files fail in the tool.

Open watchingdogs opened this issue 3 months ago • 2 comments

First of all, there is a mod named Bobby, that saves server chunks as regular MCA files, and I was trying to use those to make renders.

With using 1.21.2-rc.2, I got the following error at first, which might be relevant to #95 , as both fail at finding the spawn.

overviewer.py:557  99376 2025-10-11 18:10:06 DEBUG    Finished generating textures.
overviewer.py:563  99376 2025-10-11 18:10:06 DEBUG    Asking for regionset 'DIM0'.
world.py:171       99376 2025-10-11 18:10:06 DEBUG    You asked for 'DIM0', and I found the following candids: [<RegionSet regiondir='/src/mockworld/region'>]
world.py:2764      99376 2025-10-11 18:10:06 DEBUG    Initializing a cache with key '/src/mockworld/region'
world.py:171       99376 2025-10-11 18:10:06 DEBUG    You asked for 'DIM0', and I found the following candids: [<RegionSet regiondir='/src/mockworld/region'>]
overviewer.py:745  99376 2025-10-11 18:10:06 ERROR    An error has occurred. This may be a bug. Please let us know!
See http://docs.overviewer.org/en/latest/index.html#help

This is the error that occurred:
Traceback (most recent call last):
  File "/usr/bin/overviewer.py", line 738, in <module>
    ret = main()
          ^^^^^^
  File "/usr/bin/overviewer.py", line 613, in main
    tileSetOpts.update({"spawn": w.find_true_spawn()})  # TODO find a better way to do this
                                 ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/overviewer_core/world.py", line 221, in find_true_spawn
    chunk = regionset.get_chunk(chunkX, chunkZ)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/overviewer_core/world.py", line 2375, in get_chunk
    (blocks, data) = self._get_blockdata_v118(section, unrecognized_block_types, longarray_unpacker)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/overviewer_core/world.py", line 2162, in _get_blockdata_v118
    translated_blocks[i], translated_data[i] = self._get_block(key)
    ~~~~~~~~~~~~~~~~~^^^
OverflowError: Python integer 99999 out of bounds for uint16

To solve this I did both

def get_level_dat_data(self):
        # Return a copy
        return None

and also directly addressing the issue by:

    def _get_blockdata_v118(self, section, unrecognized_block_types, longarray_unpacker):
        block_states = section['block_states']
        palette = block_states.get('palette')
        block_states_data = block_states.get('data')

        if not block_states_data:
            # This chunk is missing its block data, skip it
            block_states_data = numpy.zeros((256,), dtype=numpy.uint16)

        # Translate each entry in the palette to a 1.2-era (block, data) int pair.
        num_palette_entries = len(palette)
        translated_blocks = numpy.zeros((num_palette_entries,), dtype=numpy.uint16)  # block IDs
        translated_data = numpy.zeros((num_palette_entries,), dtype=numpy.uint8)     # block data

        for i in range(num_palette_entries):
            key = palette[i]
            try:
                block_id, data_val = self._get_block(key)
            except KeyError:
                # Unknown mapping -> leave as air
                if unrecognized_block_types is not None:
                    try:
                        unrecognized_block_types.add(str(key))
                    except Exception:
                        pass
                continue

            # Validate ranges; anything invalid -> treat as air
            if not (0 <= block_id <= 0xFFFF and 0 <= data_val <= 0xFF):
                if unrecognized_block_types is not None:
                    try:
                        unrecognized_block_types.add(str(key))
                    except Exception:
                        pass
                # Leave zeros (air)
                continue

            translated_blocks[i] = block_id
            translated_data[i] = data_val

        # Turn the BlockStates array into a 16x16x16 numpy matrix of shorts.
        blocks = numpy.empty((4096,), dtype=numpy.uint16)
        data = numpy.empty((4096,), dtype=numpy.uint8)
        block_states = longarray_unpacker(block_states_data, 4096, num_palette_entries)
        blocks[:] = translated_blocks[block_states]
        data[:] = translated_data[block_states]

        # Turn the Data array into a 16x16x16 matrix, same as SkyLight
        blocks = blocks.reshape((16, 16, 16))
        data = data.reshape((16, 16, 16))

        return (blocks, data)

This just basically makes sure everything else that is unrecognized gets vaporized into air.

Then switched to the current main branch, built everything, and to fix one more error I copied over the chain texture file so it would stop complaining, see #95 again.

Finally I thought I was golden, then I got hit with these two errors:

textures.py:199    126711 2025-10-11 19:09:32 INFO     Found watercolor.png in '/working/The-Minecraft-Overviewer/overviewer_core/data/textures/watercolor.png'
tileset.py:1118    126711 2025-10-11 19:09:32 ERROR    Could not render chunk 25,-27 for some reason. This is likely a render primitive option error.
tileset.py:1120    126711 2025-10-11 19:09:32 ERROR    Full error was:
Traceback (most recent call last):
  File "/working/The-Minecraft-Overviewer/overviewer_core/tileset.py", line 1105, in _render_rendertile
    c_overviewer.render_loop(
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2774, in get_chunk
    retval = super(CachedRegionSet, self).get_chunk(x,z)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2582, in get_chunk
    return self._r.get_chunk(x,z)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2357, in get_chunk
    (blocks, data) = self._get_blockdata_v118(section, unrecognized_block_types, longarray_unpacker)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2215, in _get_blockdata_v118
    block_states = longarray_unpacker(block_states_data, 4096, num_palette_entries)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2163, in _packed_longarray_to_shorts_v116
    b = numpy.asarray(long_array, dtype=numpy.uint64)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer -7378697629483826927 out of bounds for uint64
textures.py:186    126733 2025-10-11 19:09:32 INFO     Starting search for watercolor.png
textures.py:196    126733 2025-10-11 19:09:32 INFO     Looking for texture in overviewer_core/data/textures
textures.py:199    126733 2025-10-11 19:09:32 INFO     Found watercolor.png in '/working/The-Minecraft-Overviewer/overviewer_core/data/textures/watercolor.png'
tileset.py:1118    126733 2025-10-11 19:09:32 ERROR    Could not render chunk 26,-26 for some reason. This is likely a render primitive option error.
tileset.py:1120    126733 2025-10-11 19:09:32 ERROR    Full error was:
Traceback (most recent call last):
  File "/working/The-Minecraft-Overviewer/overviewer_core/tileset.py", line 1105, in _render_rendertile
    c_overviewer.render_loop(
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2774, in get_chunk
    retval = super(CachedRegionSet, self).get_chunk(x,z)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2582, in get_chunk
    return self._r.get_chunk(x,z)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2357, in get_chunk
    (blocks, data) = self._get_blockdata_v118(section, unrecognized_block_types, longarray_unpacker)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/working/The-Minecraft-Overviewer/overviewer_core/world.py", line 2201, in _get_blockdata_v118
    if not (0 <= block_id <= 0xFFFF and 0 <= data_val <= 0xFF):
                                        ^^^^^^^^^^^^^^^^^^^^^
TypeError: '<=' not supported between instances of 'int' and 'str'

And that's where I am stuck. If needed I can also drop some .mca files that bobby generated to help with debugging.

watchingdogs avatar Oct 11 '25 19:10 watchingdogs

I've used a Bobby cache to render maps before (on a 1.20.4 world), and it used to "work" with some lighting glitches where some chunks don't get skylight. I seem to recall having to copy in a valid level.dat file from the world (or another world from the same version) since Bobby doesn't natively store the level.dat file. Naturally, this means things like slime overlays and spawn locations don't match up though, so it's far from an ideal situation.

If you could dump your modloader version and Bobby version too, I can probably take a look at some point but "officially" Overviewer doesn't support mods so it'll be best-effort and there are higher-priority issues with modern Minecraft to sort first.

I'd recommend falling back to running Overviewer on the main world save instead of relying on rendering from a Bobby cache for the time being.

stwalkerster avatar Oct 11 '25 20:10 stwalkerster

That's exactly what I did as well, I created a mock world with a random empty level.dat, and started the rendering that way. I got some issues, but AI came to help and rewrote some parts of the overviewer_core/world.py, if you have the time to implement some of these it would be nice. With this version I get no errors, and the rendering is done successfully. Although I get black chunks as well, that is on Bobby's end, probably some skylight issue as you were saying too.

world.py

(And if still needed I was running fabric loader v0.17.2 and v5.2.6 of Bobby.)

watchingdogs avatar Oct 12 '25 11:10 watchingdogs