Add verbose logging to update script

[weihao]

root@fc446ebcfefc:/app# python3 update.py config.json
stat "craft_bundle" is not supported by server version 2586 (required: 2681)
stat "craft_spyglass" is not supported by server version 2586 (required: 2681)        
stat "kill_goat" is not supported by server version 2586 (required: 2705)
stat "kill_axolotl" is not supported by server version 2586 (required: 2687)
stat "mine_amethyst" is not supported by server version 2586 (required: 2681)
stat "mine_copper_ore" is not supported by server version 2586 (required: 2681)       
stat "mine_pointed_dripstone" is not supported by server version 2586 (required: 2683)
stat "place_candle" is not supported by server version 2586 (required: 2681)
stat "place_lightning_rod" is not supported by server version 2586 (required: 2681)   
update finished
root@fc446ebcfefc:/app# ls data/playerlist/
root@fc446ebcfefc:/app# cat data/players.json 
{}root@fc446ebcfefc:/app# 

The update script is finished but the data is empty. Not sure what went wrong, could you add more logging when executing? Print out variables like reading path or processing file name would help

[pdinklag]

Hm, this is a bit strange indeed. The script must have found player data, otherwise there would be no server version. Did the first update you ran also look like this? All players must have been sorted out for some reason (minimum playtime? last online too long ago?).

So I totally agree that more troubleshooting info would be needed here.

[weihao]

More debugging log

172.18.0.1 - - [23/Oct/2021 17:46:27] code 404, message File not found
172.18.0.1 - - [23/Oct/2021 17:46:27] "GET /data/playerlist/active1.json.gz HTTP/1.1" 404 -
172.18.0.1 - - [23/Oct/2021 17:46:30] "GET / HTTP/1.1" 304 -
172.18.0.1 - - [23/Oct/2021 17:46:31] code 404, message File not found
172.18.0.1 - - [23/Oct/2021 17:46:31] "GET /data/server-icon.png HTTP/1.1" 404 -
172.18.0.1 - - [23/Oct/2021 17:46:31] code 404, message File not found
172.18.0.1 - - [23/Oct/2021 17:46:31] "GET /data/server-icon.png HTTP/1.1" 404 -
172.18.0.1 - - [23/Oct/2021 17:46:31] "GET /data/summary.json.gz HTTP/1.1" 200 -
172.18.0.1 - - [23/Oct/2021 17:46:31] "GET /localization/en.json HTTP/1.1" 200 -
172.18.0.1 - - [23/Oct/2021 17:46:31] code 404, message File not found
172.18.0.1 - - [23/Oct/2021 17:46:31] "GET /data/playerlist/active1.json.gz HTTP/1.1" 404 -
172.18.0.1 - - [23/Oct/2021 17:46:39] code 404, message File not found
172.18.0.1 - - [23/Oct/2021 17:46:39] "GET /data/playerlist/active1.json.gz HTTP/1.1" 404 -

[pdinklag]

Well these are from the browser, and the 404s are not surprising as no player data has been written.

Have you tried to look into why no players are considered? As said, it may be that either not a single player has played long enough (do you have a minPlaytime set in the config?), and/or the last time any player has been online is too long ago (check inactiveDays in the config).

[weihao]

yes, there are stats in /stats because I used /save-all in game and I have checked that there are json files in the directory.

The server was tested in 1.16.5

[pdinklag]

The server was tested in 1.16.5

That's fine, anything 1.13 and later is supported by MinecraftStats.

yes, there are stats in /stats because I used /save-all in game and I have checked that there are json files in the directory.

OK, so I guess that there should be at least one player to be considered active, but it doesn't answer whether you have maybe a minimum playing time configured in the MinecraftStats config.json and that's preventing any player from being tracked? What does your config.json look like?

[weihao]

I am using default config.json

{
  "configVersion": 1,
  "database": "data",
  "server": {
    "sources": [
      {
        "path": "/data",
        "worldName": "world"
      }
    ],
    "customName": false
  },
  "client": {
    "defaultLanguage": "en",
    "playersPerPage": 100,
    "playerCacheUUIDPrefix": 2,
    "showLastOnline": true
  },
  "players": {
    "profileUpdateInterval": 3,
    "updateInactive": false,
    "inactiveDays": 7,
    "minPlaytime": 60,
    "excludeBanned": true,
    "excludeOps": false,
    "excludeUUIDs": []
  },
  "crown": {
    "gold": 4,
    "silver": 2,
    "bronze": 1
  },
  "events": []
}

[pdinklag]

Then maybe try setting the minPlaytime to zero, maybe that's what's preventing any players from being considered?

Independent of that of course, the original issue to add verbose output will be implemented in the future to make these things easier.

[weihao]

I solved the problems below and finally got the app to run. again, adding logs/errors in python script would help with debugging.

1. The sidecar container did not have proper permission to access the stats folder
2. The sidecar container did not have proper permission to write to the output destination.
3. The python script lost the scope and accessed an invalid path.

Keeping this issue open.

[pdinklag]

I solved the problems below and finally got the app to run. again, adding logs/errors in python script would help with debugging.

1. The sidecar container did not have proper permission to access the stats folder

This can't really be true, because the Python script found your server version (or rather latest version of any player). This means that it must have successfully read the player JSON files.

2. The sidecar container did not have proper permission to write to the output destination.

Well, since Python did not throw any exceptions, this means that the sidecar container tricked Python into believing that all the file operations were successful. If this is the case, there is no way for an application to figure this out and thus a verbose mode would not have helped here. This is rather an issue with the container software.

3. The python script lost the scope and accessed an invalid path.

I'm not sure what exactly you mean by this, but again, if the container simply reroutes things to a different path and tricks the contained application into believing that everything's fine, then that's not something you would be able to detect by the application's output.

Keeping this issue open.

I still agree of course that a verbose mode may help in some occasions or to rule out certain causes, so yes, let's keep this open.

[pdinklag]

The Java port will be somewhat more verbose when errors occur. If they do or don't help in a certain situation should be decided when concrete issues turn up.