porisius
GetTrainStation segfault on Linux
As originally reported by Fezalion on Discord, a request to /getTrainStation with at least one train station built appears to reliably cause a crash on a Linux dedicated server. This is presumably a result of the changes made in 1.1.0 as part of fixing #101.
See the Discord conversation for more details, though when I reproduced this the error I received was as follows.
[...]
Unhandled Exception: SIGSEGV: invalid attempt to read memory at address 0x0000000000000010
[2024.10.27-22.19.08:868][159]LogCore: Fatal error!
0x000014d5b252bc51 libFactoryServer-CoreUObject-Linux-Shipping.so!UObjectBase::IsValidLowLevel() const [C:/BuildAgent/work/b731a33f2a691e17/UE4/Engine/Source/./Runtime/CoreUObject/Private/UObject/UObjectBase.cpp:324]
0x000014d53c598ff2 libFactoryServer-FicsitRemoteMonitoring-Linux-Shipping.so!UFRM_Trains::getTrainStation(UObject*) [A:/workspace/SatisfactoryModLoader/Mods/FicsitRemoteMonitoring/Source/FicsitRemoteMonitoring/Private/FRM_Trains.cpp:167]
0x000014d53c5a62dd libFactoryServer-FicsitRemoteMonitoring-Linux-Shipping.so!AFicsitRemoteMonitoring::getTrainStation(UObject*) [A:/workspace/SatisfactoryModLoader/Mods/FicsitRemoteMonitoring/Source/ThirdParty/uWebSockets/../../FicsitRemoteMonitoring/Public/FicsitRemoteMonitoring.h:458]
[...]
The steps I used were as follows.
- Launch a fresh server using wolveix/satisfactory-server on a physical Linux server.
- Fezalion's logs indicate the game was running under WSL.
- Install SML+FRM via SMM.
- Create a new game with most of the advanced game settings enabled.
- Make a GET to $server:8080/getTrainStation and confirm it returns an empty array.
- Build a train station. It doesn't need to be connected to anything.
- Make a GET to $server:8080/getTrainStation. The server crashes immediately, though the client might take a few seconds to realise.
@derpierre65 noted that there is a similar issue #83.
While it's definitely plausible that report is related to this one, there are a few differences.
- That user was presumably running 1.0.10, before the changes in #101.
- They seem to indicate that FRM worked fine for some time before the crash.
- There aren't any getTrainStation symbols in the stack trace.
So if I had to guess it sounds more like a crash for some underlying issue that just happened to occur during a call to getTrainStation for that user, though I am not certain of that.
Built the dev SML and deployed that to my server. Same behaviour as with the version in SMM. Expected, but just wanted to rule it out.
Built the branch that I linked https://github.com/porisius/FicsitRemoteMonitoring/issues/101#issuecomment-2425647118 for Linux and ran that (after switching the server back to the SMM version of SML). It works.
[
{
"Name": "Skövde",
"ClassName": "FGTrainStationIdentifier",
"location": {
"x": -58641.01671660268,
"y": 260820.33240582858,
"z": -3778.3621509763266,
"rotation": 90
},
"PowerConsumption": 0.10000000149011612,
"TransferRate": 0,
"InflowRate": 0,
"OutflowRate": 0,
"CargoInventory": [
{
"Name": "Freight Platform",
"ClassName": "Build_TrainDockingStation_C",
"location": {
"x": -57041,
"y": 260820.30000000002,
"z": -3778.4,
"rotation": 90
},
"PowerConsumption": 0.10000000149011612,
"TransferRate": 0,
"InflowRate": 0,
"OutflowRate": 0,
"LoadingMode": "Loading",
"LoadingStatus": "Idle",
"DockingStatus": "Waiting To Start",
"Inventory": []
}
],
"CircuitID": 0.10000000149011612,
"features": {
"properties": {
"name": "Skövde",
"type": "Train Station"
},
"geometry": {
"coordinates": {
"x": -58641.01671660268,
"y": 260820.33240582858,
"z": -3778.3621509763266
},
"type": "Point"
}
}
}
]
There are a few differences between the code in my branch and what made it in to FRM 1.1.0. Most notably, I didn't use IsValidLowLevel. I wonder if we're running into a difference in behaviour between clang/llvm and whatever compiler gets used for Windows?
Possibly the lines at https://github.com/porisius/FicsitRemoteMonitoring/blob/b2020e181b7431f33d48a2b516deff437044b6cb/Source/FicsitRemoteMonitoring/Private/FRM_Trains.cpp#L170
and
https://github.com/porisius/FicsitRemoteMonitoring/blob/b2020e181b7431f33d48a2b516deff437044b6cb/Source/FicsitRemoteMonitoring/Private/FRM_Trains.cpp#L204
were intended to use && rather than || so that the IsValidLowLevel test isn't run if TrainPlatformConnetion is falsy (e.g. null) and that the fact that it works on Windows is just an odd quirk of the compiler.
It is still crashing with version 1.1.1
[2024.10.29-09.18.02:903][552]LogCore: Fatal error!
0x000078ef5552bc51 libFactoryServer-CoreUObject-Linux-Shipping.so!UObjectBase::IsValidLowLevel() const [C:/BuildAgent/work/b731a33f2a691e17/UE4/Engine/Source/./Runtime/CoreUObject/Private/UObject/UObjectBase.cpp:324]
0x000078ef274c6466 libFactoryServer-FicsitRemoteMonitoring-Linux-Shipping.so!UFRM_Trains::getTrainStation(UObject*) [A:/workspace/SatisfactoryModLoader/Mods/FicsitRemoteMonitoring/Source/FicsitRemoteMonitoring/Private/FRM_Trains.cpp:282]
0x000078ef274d22dd libFactoryServer-FicsitRemoteMonitoring-Linux-Shipping.so!AFicsitRemoteMonitoring::getTrainStation(UObject*) [A:/workspace/SatisfactoryModLoader/Mods/FicsitRemoteMonitoring/Source/ThirdParty/uWebSockets/../../FicsitRemoteMonitoring/Public/FicsitRemoteMonitoring.h:458]
0x000078ef27477ffe libFactoryServer-FicsitRemoteMonitoring-Linux-Shipping.so!AFicsitRemoteMonitoring::execgetTrainStation(UObject*, FFrame&, void*) [A:/workspace/SatisfactoryModLoader/Mods/FicsitRemoteMonitoring/Intermediate/Build/Linux/FactoryServer/Inc/FicsitRemoteMonitoring/UHT/FicsitRemoteMonitoring.gen.cpp:269]
0x000078ef552d4767 libFactoryServer-CoreUObject-Linux-Shipping.so!UFunction::Invoke(UObject*, FFrame&, void*) [C:/BuildAgent/work/b731a33f2a691e17/UE4/Engine/Source/./Runtime/CoreUObject/Private/UObject/Class.cpp:6665]
0x000078ef554fe99a libFactoryServer-CoreUObject-Linux-Shipping.so!UObject::ProcessEvent(UFunction*, void*) [C:/BuildAgent/work/b731a33f2a691e17/UE4/Engine/Source/./Runtime/CoreUObject/Private/UObject/ScriptCore.cpp:2159]
0x000078ef274774bc libFactoryServer-FicsitRemoteMonitoring-Linux-Shipping.so!FAPICallback_DelegateWrapper(TScriptDelegate<FNotThreadSafeDelegateMode> const&, UObject const*) [A:/workspace/SatisfactoryModLoader/Mods/FicsitRemoteMonitoring/Intermediate/Build/Linux/FactoryServer/Inc/FicsitRemoteMonitoring/UHT/FicsitRemoteMonitoring.gen.cpp:90]
0x000078ef2747af72 libFactoryServer-FicsitRemoteMonitoring-Linux-Shipping.so!AFicsitRemoteMonitoring::CallEndpoint(UObject*, FString, bool&) [A:/workspace/SatisfactoryModLoader/Mods/FicsitRemoteMonitoring/Source/FicsitRemoteMonitoring/Private/FicsitRemoteMonitoring.cpp:637]
0x000078ef2747b5da libFactoryServer-FicsitRemoteMonitoring-Linux-Shipping.so!AFicsitRemoteMonitoring::HandleEndpoint(UObject*, FString, bool&) [A:/workspace/SatisfactoryModLoader/Mods/FicsitRemoteMonitoring/Source/FicsitRemoteMonitoring/Private/FicsitRemoteMonitoring.cpp:671]
Confirmed that I'm seeing the same thing in 1.1.1. Looks like there's two issues here:
- There's some more code similar to the lines changed in #130, though not quite the same. This time with no is-null test and IsValid rather than IsValidLowLevel https://github.com/porisius/FicsitRemoteMonitoring/blob/39ff003b5db84e795de6be01808227ad192133b9/Source/FicsitRemoteMonitoring/Private/FRM_Trains.cpp#L281
- As well as a difference in compiled code between Win/Linux there's also seemingly a difference between dev/release builds.
I'm recompiling 09ecf57376a6397d9adb614724a6ae9c5202a6fb as a release now to confirm whether the latter is the case or if something odd happened in 1.1.1 specifically. Once I know that I'll hopefully know how to correctly test this.