Garbage collection lag

[TeamDman]

https://discord.com/channels/967118679370264627/1430414686884466780/1430414686884466780

__stdcall OP — 12:37 AM

On my ATM10 server I have a friend, he uses SFM to automate everything. And recently the server becomes shuttering, after some investigation i found the root cause to be jvm gc lag spikes. I created a heapdump and it revealed that there are a lot of objects related to SFM eating up much of the heap memory. I'm confused because count of SFM labels he and I had created should never go above like ~200. But there are so many in the memory. Need some help for this, thanks!

[TeamDman]

I did /test runall and let them idle for a bit afterwards, I am seeing many more objects than I would expect.

My hypothesis is that the programs are being reconstructed in some capacity every tick, which is not what should happen. There's like 120 tests and even if each program in the test had 10 IO statements that should only be 1200 LabelAccess objects but there's 2000+ so something funky going on

[TeamDman]

https://github.com/TeamDman/SuperFactoryManager/blob/ec03f35d83f010323d8439940f7f8d3c8a2f34e5/src/main/java/ca/teamdman/sfml/ast/ForgetStatement.java#L26-L35

Looks like the way I implemented Forget statements is that a new LabelAccess is getting created, which makes sense but in practice this can be optimized

Found this using the memory inspector in IntelliJ

I wasn't able to get "Show referring objects" to work

However, even more useful was using "Enable tracking for new instances" which showed a stack trace after resuming the game for a bit and pausing again

[TeamDman]

The way FORGET is implemented is such that existing matching input statements are replaced with input statements that do not include the forgotten labels.

INPUT FROM a,b,c
OUTPUT TO d
FORGET b,c -- nobody specifies stuff here I bet lol
OUTPUT TO e

For things like predictive execution using ctrl+space on input and output statements, SFM tracks the new input statement that is created during the forget's execution.

We can't determine the new INPUT statement at program build because

INPUT FROM a,b
IF c has GT 1 stick THEN
  FORGET b
END
OUTPUT TO d

has as many possible states as there are code flows in the program, which can grow quite large.

The real problem is that the new InputStatement objects are not being garbage collected, leaving the LabelAccess objects dangling.

This is because the tracking is using strong references.

https://github.com/TeamDman/SuperFactoryManager/blob/ec03f35d83f010323d8439940f7f8d3c8a2f34e5/src/main/java/ca/teamdman/sfml/ast/ForgetStatement.java#L36

https://github.com/TeamDman/SuperFactoryManager/blob/37784c8d4a60b4c1d97697b63049d482f7c44ca6/src/main/java/ca/teamdman/sfml/ast/ASTBuilder.java#L34-L39

https://github.com/TeamDman/SuperFactoryManager/blob/37784c8d4a60b4c1d97697b63049d482f7c44ca6/src/main/java/ca/teamdman/sfml/ast/ASTBuilder.java#L18

Switching this to use a weak reference should fix it.

[TeamDman]

before

---

after fixed

[TeamDman]

This is available in the new release of SFM 4.27.0