Corrupted world on Windows 11

[TeksuSiK]

Describe the problem

After server restart, the server world files are corrupted. Everything works well while running pure Paper server without docker and also running same container on macOS or Linux. Recreating container doesn't help anything, after server restart I'm getting the same errors.

Container definition

version: "3.8"

services:
  mc:
    image: itzg/minecraft-server:java16
    ports:
      - "25565:25565"
    environment:
      TYPE: PAPER
      VERSION: 1.17.1
      EULA: "TRUE"
      ONLINE_MODE: "FALSE"
      COPY_CONFIG_DEST: /data
    volumes:
      - ./data/mc:/data
      - ./mc:/config

Container logs

Full server log is available here https://teksusik.pl/docker.log

[itzg]

I'm sorry to hear that happened. I'm afraid that is beyond the control of this image/container since ultimately it exec's to the requested server process. The logs provided do not show any of the container startup activity. Without those logs prior to the restart I don't know how you attempted to shutdown the server. Other external factors include the state of the attached ./data/mc host directory and any content augmented in from the ./mc host directory.

I would recommend discussing with the PaperMC team since it attempted but failed to recover the corrupted regionfile.

[TeksuSiK]

I'am restarting my server via stopping container from IntelliJ services tab, so the server is saving world, also there aren't any errors at this phase. I also probably know what's wrong. When I remove volume bindings from my docker-compose file and recreate container everything starts to work well.

[itzg]

Can this be closed now?

[TeksuSiK]

Well, I don't think this should work like this, but I don't know it's my fault or not, someone must try to reproduce it. In previous comment I meant that it only works without volume bindings and when I add them back it's again starts to serve errors.

[itzg]

I'm sorry I misunderstood your comment:

I also probably know what's wrong.

In any case, this is not a problem with the image. It's either something external or the PaperMC software itself. We can leave this issue open for others to find, but there's nothing I can do to fix it.

[itzg]

and just to make sure it wasn't a combination of those volume paths I used your compose file and tried a couple ways of restarting it in IntelliJ's service tab:

https://user-images.githubusercontent.com/988985/139678944-cfde1ee5-5832-4773-b37b-7a56e06ae8d5.mp4

[TeksuSiK]

Okay, so thanks for help. I will have to check what is wrong in my case.

[Aniello12]

Same issue but only for minecraft version 1.18.1

docker-file `version: "3"

services: mc: image: itzg/minecraft-server:latest ports: - 25565:25565 environment: EULA: "TRUE" VERSION: 1.18.1 TYPE: "BUKKIT" MEMORY: "10G" tty: true stdin_open: true restart: unless-stopped volumes: - ./server:/data

attach a directory relative to`

[itzg]

@Aniello12 or @TeksuSiK are either of you running on Windows 11? I was chatting with someone yesterday with similar file corruption type issues and narrowed down that it was Windows 11 as the primary difference.

[TeksuSiK]

Yes, I'm working on Windows 11

[MarkusKluge]

I got the same issue here in Windows 11 but even with the default version (not paper). Whenever I chose a volume path for the container it will result in corrupted chunks.

As I haven't tested this on Windows 10 I can't say if this is an issue with W11.

How to "fix" it: When not using the volume parameter (default is docker generates a anonymous docker volume for the container) or using a named docker volume it will work and I can still access data with the windows explorer at \\wsl.localhost\docker-desktop-data\version-pack-data\community\docker\volumes\ without the chunk generation issues.

[itzg]

Thanks @MarkusKluge, your finding about the anonymous and named volume confirms another pattern I was starting to see. Also, thanks for the advice on how to access docker desktop file system. I never realized it was that easy even though I knew how to access regular distro file system.

[itzg]

To help others see it, I'm going to re-open this issue but change the title to mention Windows 11. I'll plan on submitting an issue with docker desktop; however, I would need those of you with Windows 11 to possibly help guide troubleshooting from there.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Please add a comment describing the reason to keep this issue open.

[AlbertoPimpo]

After some trying on #1341 the approach of @MarkusKluge about using named volumes didn't work for me. I make it working by disabling the wsl backend and using hyper-v backend, so to me this is the only working work-around. @itzg maybe the problem is not docker but wsl itself, it can be an idea using a different distro as base distro for docker-desktop

[itzg]

@AlbertoPimpo I'm really surprised naked volumes didn't make difference.

Speaking of which, I thought I'd share this article I just saw that explains why host binding acts different than names volumes (in apparently not all cases)

https://www.docker.com/blog/file-sharing-with-docker-desktop/

Yes, it probably is a problem with WSL on Windows 11. I'm wondering if anybody has yet contacted Docker or Microsoft about this issue?

[Zedifuu]

I am experiencing this also with our server manager.

Docker Desktop on WSL2 On Windows 11, completely shreds the world on shutdown or restart :( On bare metal linux with Docker, it operates just fine :S

It's a peculiar one for sure, I wonder also if anyone has reached out to Docker or Microsoft.

[AxelBrinck]

I have this same problem on Ubuntu server. Like 30% of the chunks are misplaced. Only in 1.18.X, using any java 17 image. The error happens on all the server types, including vanilla.

I took the server to Windows 11 and only worked using named volumes.

Using named volumes could patch this issue. (Tested on W11)

[Merith-TK]

aight work around for now on windows if you 100% must have access to config files

1. in your mounts, put it in this order

• "<volume name>:/data/world"
• "./local/path/:/data"

2. start container
3. access container shell as root
4. as root run chown minecraft:minecraft /data -R
5. restart container

version: "3"

volumes:
  mc-world:
    
services:
  mc:
    image: itzg/minecraft-server:java17
    container_name: mc
    user: "1000:1000"
    stdin_open: true
    tty: true
    ports:
     - "25565:25565/tcp"
     - "25565:25565/udp" # for things that use UDP
    volumes:
      - "mc-world:/data/world" 
      - "./mc:/data"
    environment:
      PUID: "1000"
      PGID: "1000"
      TYPE: "fabric"
      VERSION: "1.18.1"
      PACKWIZ_URL: "https://gitlab.com/Merith-TK/vanillia-modpack/-/raw/server/server/pack.toml"
      EULA: "true"
      MEMORY: "3G"

the user and UID stuff is just to enforce the server running as a user and not as root, paranoid like that lol

[NotEvenANeko]

I disable the 'Use the WSL 2 based engine' option in Docker Desktop settings to use Hyper-V as backend, and it just works fine.

I think it's a bug of WSL 2.

[quiquelhappy]

has anyone found a solution to this project that doesnt require disabling WSL? This may be related? https://github.com/microsoft/WSL/issues/7977

What I'm seeing whenever a chunk is generated/loaded:

[13:22:17 ERROR]: Chunk [25, 10] header is truncated: expected 368640 but read 0

[13:22:17 ERROR]: [com.destroystokyo.paper.io.PaperFileIOThread] Failed to read chunk data for task: Task for world: 'world' at 26,9 poi: true, hash: 1394880741

java.lang.IllegalArgumentException: Negative position

at sun.nio.ch.FileChannelImpl.read(Unknown Source) ~[?:?]

at net.minecraft.world.level.chunk.storage.RegionFile.getChunkDataInputStream(RegionFile.java:604) ~[paper-1.18.2.jar:git-Paper-375]

at net.minecraft.world.level.chunk.storage.RegionFileStorage.read(RegionFileStorage.java:195) ~[?:?]

at net.minecraft.world.level.chunk.storage.RegionFileStorage.read(RegionFileStorage.java:187) ~[?:?]

at net.minecraft.world.entity.ai.village.poi.PoiManager.a(PoiManager.java:450) ~[?:?]

at net.minecraft.server.level.ServerLevel$1.readData(ServerLevel.java:327) ~[?:?]

at com.destroystokyo.paper.io.PaperFileIOThread$ChunkDataTask.run(PaperFileIOThread.java:508) ~[paper-1.18.2.jar:git-Paper-375]

at com.destroystokyo.paper.io.QueueExecutorThread.pollTasks(QueueExecutorThread.java:118) ~[paper-1.18.2.jar:git-Paper-375]

at com.destroystokyo.paper.io.QueueExecutorThread.run(QueueExecutorThread.java:51) ~[paper-1.18.2.jar:git-Paper-375]

[13:22:17 ERROR]: [com.destroystokyo.paper.io.PaperFileIOThread] Could not load chunk for task: Chunk task: class:com.destroystokyo.paper.io.chunk.ChunkLoadTask, for world 'world', (26,9), hashcode:1052169024, priority: -1, file IO thread has dumped the relevant exception above

here is a video showing the startup progress and the in-game experience: https://youtu.be/Sdo74YhdN6g

Here are all the logs for that same video: https://gist.github.com/quiquelhappy/a332d4a9eef5b46e6b81c544dd985651

[itzg]

@quiquelhappy there's workarounds mentioned in the comments above

[quiquelhappy]

@quiquelhappy there's workarounds mentioned in the comments above

Thanks, somehow I completely ignored one of the answers 🤔.

Hoping this gets fixed soon, seems like a WSL bug. Thanks for replying and for keeping up the repo! Have a nice day.

[itzg]

Luckily my PC doesn't have compatible hardware to run Windows 11 😉

Agreed, I'm surprised they haven't resolved this issue by now since it seems like quite a fundamental issue.

[langestefan]

@itzg not sure if this is still relevant, but I found that if I interacted with the docker commandline directly from my Ubuntu WSL2 machine I did not get any corrupted data. If I stopped a container using the windows command line I suddenly got issues.

[itzg]

I'm not sure if it is (I'm avoiding upgrading to Windows 11 because of things like this 😀), but thanks for the info.

[langestefan]

I think I got it. It's because the volumes are stored in the windows filesystem rather than the linux filesystem. When using a WSL2 backend with files stored in the linux filesystem there are no issues. Ps. windows 11 is awesome!

[itzg]

Thanks @langestefan for confirmation on both parts 😀

[quiquelhappy]

I think I got it. It's because the volumes are stored in the windows filesystem rather than the linux filesystem. When using a WSL2 backend with files stored in the linux filesystem there are no issues. Ps. windows 11 is awesome!

Pretty sure this was the case. For some reason, every time I installed Docker Desktop on my Windows computer, with a single WSL OS installed, it never automatically enabled Docker support for such OS. My recommendation to anyone having this issue is to use Docker from a WSL OS directly.

If you are new to Docker on Windows 11:

• install Docker Desktop
• if you get a prompt saying WSL isn't enabled on your system, you can enable it on the most recent versions of Windows 11 by looking for the 'WSL 2' app on the Microsoft Store; if that doesn't work, follow the guide linked by the prompt on the Docker Desktop installation
• install any distro of your liking, for example Debian or Ubuntu; you can find these on the Microsoft Store
• after installing and configuring your distro (simply search for the distro name and open it as you would do with any other application), go to Docker Desktop > Settings > Resources > WSL Integration, and enable the integration for the distro you just installed
• when running commands, you can open your distro, or open the newly released Terminal app on Windows 11, making sure you select a docker-enabled distro when opening a tab.
• you can find your files by going to /mnt/c/. If you want to be able to access the files from windows directly, make sure to use a volume pointing to a folder within /mnt/c/

Small note if you are transitioning to Windows 11 and are worried about your privacy (@itzg ✌️). There is no straight forward way of creating a local account while installing Windows 11 anymore, you'll have to enter [email protected] as your email when the setup asks you to login with your Microsoft Account, you should get an error and then be allowed to create a local account. When installing apps from the Microsoft Store, if you get a prompt asking you to login, just close it, the apps should install nevertheless. Happy holidays everyone!🎅

Verified this is working fine:

https://user-images.githubusercontent.com/12534357/209533668-6ea46ebf-fdf8-48d0-90e4-89d2ba0e941d.mp4

[BiliBC034]

I've tried to do what @langestefan, @quiquelhappy and @Merith-TK said, but does not work.

@itzg not sure if this is still relevant, but I found that if I interacted with the docker commandline directly from my Ubuntu WSL2 machine I did not get any corrupted data. If I stopped a container using the windows command line I suddenly got issues.

Pretty sure this was the case. For some reason, every time I installed Docker Desktop on my Windows computer, with a single WSL OS installed, it never automatically enabled Docker support for such OS. My recommendation to anyone having this issue is to use Docker from a WSL OS directly...

aight work around for now on windows if you 100% must have access to config files

1. in your mounts, put it in this order

• "<volume name>:/data/world"
• "./local/path/:/data"

2. start container
3. access container shell as root
4. as root run chown minecraft:minecraft /data -R
5. restart container ...

I can not disable WSL and use Hyper-V as backend.

I've tried to mount the container in a named volume and worked, but I can not access there easily from win11. Is there any way to mount a named volume in a specific volume of win11 or to do a reference of the named volume into win11?

What I have now is this, but does not work:

version: "3"

services:
  mc:
    image: itzg/minecraft-server:latest
    container_name: mc-Final-Paradox
    ports:
      - 25565:25565
    expose:
      - 25565
    environment:
      EULA: "TRUE"
      DIFFICULTY: 'hard'
      VERSION: 1.16.5
      WHITE_LIST: "TRUE"
      GAMEMODE: survival
      MEMORY: '8G'
    tty: true
    stdin_open: true
    restart: unless-stopped
    volumes:
      - FP-data:/data
volumes:
  FP-data:
    driver: local
    driver_opts:
       o: bind
       type: none
       device: C:\Users\user\Desktop\minecraft_server\mincreft-data

If I leave it like this, works well, but I want to access the volume from win11:

...
volumes:
      - FP-data:/data
volumes:
  FP-data: