dwarfcorp
dwarfcorp copied to clipboard
Blecki
Tracking: Weekly Performance Profiling
I figure I'll make a habit out of logging some profiling data each week until it's not as generally widespread an issue. Bugs will still be logged for performance hits that have clearly just one cause. When general performance is up to scratch, this bug will be closed.
Below are my general PC specs. Let me know if you need any further detail there. Only thing added to this is a wireless mouse, which I'll make a separate compat bug for, if it ever becomes an issue.
If there's ever any specific situation you want me to test out, just request it here. Right now organic play is enough to expose issues.
System Information: System Info.txt
4/16/18
This was started from a save that's about an hour and a half of game play. Main performance issue noticeable FPS drop. #583 still occurs. By the time of the attached save, lag was drastic. Like a frame update of once every few seconds or so.
Save: Names's Wake_304_131683895098910402.zip
World: Names's Wake.zip
Profiler info: https://drive.google.com/open?id=1q8aFQEy1qnzl3nqeFG9vVyuLnvUJjLC_
9e581f9d
Relevant Steam Threads:
https://steamcommunity.com/app/252390/discussions/0/1483232961051514339/
https://steamcommunity.com/app/252390/discussions/0/1499000547490926002/
https://steamcommunity.com/app/252390/discussions/0/1483232961039553632/
For reference, other performance tracking can be found via these issues: Windows - #711 Mac - #710 Linux - #712 DwarfCorp's Weekly Tracking - #651
FYI, I think .vspx files can't be opened by another user since we're missing the folder structure of your project. Could you export a csv file for us instead? Also the other performance tracker yo u used before worked for me.
Here ya go: https://drive.google.com/open?id=1nPamBI0ksU6EiUqEsGjy67HxGA5SF38h
Okay, thank you very much for these data. So the low hanging fruit seems to be:
- Designation drawing is too slow. This makes large fields of guard/mine designations slow to draw. I have just fixed this in
master. - The octree is too slow. Large numbers of resource entities will slow down the game. A fix for this is currently in progress.
Sounds good. Would you rather I wait for the second fix to be pushed before I do the weekly performance profiling or nah?
I would like a test to see if my changes to drawing made some stuff faster for you.
4/23/18
Performance improved overall, but is still an issue. An hour in to a game, the most noticeable difference is playing at normal speed doesn't produce much lag. Playing at 2x and 3x makes the lag an issue again. The lag at these speeds has changed somewhat. It seems to go up and down, reaching some points of low lag, and even some with zero lag, which is an improvement from constant lag. However, the majority of play time in 2x and 3x is, at its worse, near a stand still.
Sampling Info: DwarfCorp180423(1) Weekly_CallerCalleeSummary.zip
World: Glittertrackeria.zip
Save: Glittertrackeria_638_131689927459290222.zip
cf2d71db
I can try. For some reason the FPS read out from F10 doesn't look like it's working, but here is a general description.
I'm defining lag as any FPS drop that would be considered an obstacle to the player experience. The worst it's gotten is hard pauses that last a couple of seconds or more, so my FPS is flatlining there. It's going up and down a lot, but tends to on average be somewhere where the game is still moving, just pretty slowly. I'd guess 10-15 FPS. When the player does not perform any actions and doesn't move the camera, the FPS will return to about normal, though still fluctuate somewhat.
The worst it's gotten is hard pauses that last a couple of seconds or more,
Wow, that's totally unacceptable. Are these events associated with any particular action you're taking?
I'd guess 10-15 FPS
That's also unacceptable. We'll continue working on bottlenecks to make this better.
I'll also try getting some numbers from a more developed game some time to day, which is when the performance hits are at their worst.
Culling is definitely a big issue as expected. With a new game doing nothing it's somewhat noticeable, but I assigned a few tasks and as soon as I began moving camera I got as low as 0 FPS.
cf2d71db
Testing my previous save on newest master branch 40d844c8. Oh man. Thought I had given you a different save. This isn't even it at its worse, honestly.
Anyway, there is some noticeable improvement to game performance at 1x speed. The first ten minutes of the save felt almost entirely smooth and without visually noticeable lag. After this, the performance at 1x does lessen though. Moving camera is still a main trigger to lag and getting as low as 0 FPS when aggravating it.
2x:
2x I'm still getting up and down performance. When not even moving the camera, I'm getting a high of around 25 and a low of 0.
3x:
Given where it used to be, this actually feels like a huge improvement. At least while not moving the camera. As you can see it still spends a good amount of time in the red area, averaging somewhere in the 10-5 FPS, with lows of 0 and momentary highs of near 20. Still, compared to the never ending near flatlining of FPS that was happening on my PC during 3x before, it's notably better.
After some examination of this save, we now believe most of the performance issues are from the interaction between the path planner and the octree. We're investigating a solution to this problem.
5/1/18
@mklingen Won't have time to do profiling this week, but I have a good save for you. I suggest you open it up, crank it up to 3x, order a few different tasks, and try to play for at least five to ten minutes (performance still tends to get worse over time). It's still dipping to 0 FPS on my machine, though less frequently than previous builds.
Overall, performance is way better though. The fixes reaalllly improved things, and I think the players are going to start noticing pretty soon, if they haven't already.
Save: Horn Realm_785_131696252859277966.zip
World: Horn Realm.zip
ac0e675
managed to load this save. A couple of things are going on. One, you have an absolutely huge number of bear traps. These are rendered kind of inefficiently at the moment (they are "Simple Sprites"), so there might be some gains for us there with batching. You're also rendering a huge number of mining designations. I'd like to say "the player should know not to make the whole world a mining designation" but we could probably render those more efficiently as well.
Most recent changes seem to have fixed the multisecond pauses for me in this save, so that's good.
5/9/18
Here is some sampling. In general, performance is now basically passable for an hour into game, though we are still getting drops to 0, so I hesitate to say that we're at stable. Still, for the first time it feels like at 3x is not struggling and visually looks normal around 70-80% of the time (around 9-20 FPS).
Here's what seemed to slow things down to noticeable lag:
- Exiting the trade menu
- Battling war parties
- Deselecting guard tiles
- Unpausing the game
- Though much less, the camera still sometimes aggravates things.
- Hiring new dwarves
Sampling Info 1: DwarfCorp180509(1)_CallTreeSummary1.zip
Sampling Info 2: DwarfCorp180509(2)_CallerCalleeSummary.zip
Didn't get saves for these, cus forgot + the quickest game exit is quit. For some reason game isn't autosaving, so no failsafe there. Will investigate this as a separate issue.
126d7ff8
https://steamcommunity.com/app/252390/discussions/2/1696043263497502599/?tscn=1525998190
The game now lags so bad it is unplayable. Will wait for further updates.
5/14/18
1st attempt at profiling, ran into lots of freezes. Not low FPS, but a straight up freeze. Some occurred from loading, but also go a freeze from a large dig task selection.
30 Min profiling - freezes.zip
5fee6751
Since #720 got re-opened, figured I'd profile it. I'm getting 1 FPS for pretty much the entire game.
Profiling: DwarfCorp180605_CallerCalleeSummary.zip
Save: Aberrant Realm_223_131726963744840677.zip
World: Aberrant Realm.zip
3eea47e7
Since #720 got re-opened, figured I'd profile it. I'm getting 1 FPS for pretty much the entire game. Wtf that's completely unacceptable. This happens on a new game without any interaction?
Yup.
- New game.
- Select large dig designation
- Observe performance hit including the (not responding) printout
7/3/18
Hello again. So, I'm getting low performance after half an hour or so of play. Low FPS (as low as 1FPS) /Framerate is the main symptom as usual, but I am noticing a few other issues.
When I start a game, the terrain is taking longer to render than usual, and you usually catch it updating itself piece by piece. This also happens when you press E and snap to terrain.
Generating a very large advanced world causes my computer to briefly overheat, and it takes its time, about four minutes. (Side note: Sometimes the loading hourglass anim doesn't animate) Used to generate a very large map in about a minute.
Moving the camera is slowing down FPS. 3x is slowing it down too.
Large dig designations. I performed one while paused that caused the game to freeze and switch to the (not responding) status for ten seconds or so.
Anyways, here's some profiling data: DwarfCorp180703_FunctionSummary.zip
Save/world: Aboundingland_497_131751279578550528.zip
06c9d3dd
Very very important information: did you select the whole world as mining designations again? I haven't had time to look at the save, but obviously things are going to go really slowly when you do that. The profiling data is all stuff related to handling mining designations. If you recall, doing that used to crash the game, then it caused miserable performance on my machine, and now it's just barely fast enough to get 10-20 FPS on my machine. That's a feature we continue to work on, but it's getting to the point where it's about as optimized as it can be now.
Just opened your save. There is definitely a huge slowdown here. Investigating.
Nvm, in debug mode there was definitely a slowdown, but in release mode my machine can still handle this save at 60 FPS.
Analysis:
Save starts at 60 FPS, and after a few minutes declines to 1 FPS. Checking the profiling data suggests that the main culprit is the 300,000 mining tasks you've created. Problem was multifaceted but the biggest bottleneck appears to be https://github.com/CompletelyFairGames/dwarfcorp/issues/788
Remedy:
Break up checking for mining task validity/completion into chunks of 2048 tasks at a time. Cache whether a mining task is valid (i.e not completely surrounded by other voxels) and only update when a voxel is destroyed.
There is still a little bit of a slowdown on my machine, but it's as optimized as I can get it for now. Ultimately, creating 300,000 mining designations is going to make things really difficult for the engine, as anything involving selecting the best task, checking if a task is complete, or similar is going to take a long time. We might have to think about alternate ways of representing mining tasks to make this less of an issue.
I thought we had discussed outlining large areas of mining instead drawing all the individual voxel designations?
I moved them into the chunk geometry effectively making that optimization pointless.
On Mon, Jul 9, 2018, 2:47 PM com1clyf3 [email protected] wrote:
I thought we had discussed outlining large areas of mining instead drawing all the individual voxel designations?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/CompletelyFairGames/dwarfcorp/issues/651#issuecomment-403581796, or mute the thread https://github.com/notifications/unsubscribe-auth/ABPKrNUAhJlPhIHFty7FiwwMpDId9BB3ks5uE6VXgaJpZM4TXWA6 .
Loaded AboundingLand save in newest build 27b0c862.
Release mode with task list + moving camera, I see a notable improvement. 02-06 FPS, never dipping below 2 FPS. Not perfect, but better.
Same save on newest build on debug mode with 3x time, tasks lists + moving camera, I'm getting 02 FPS for the most part. 3-4 FPS on 1x. 28-32% CPU and 490-530 MB RAM usage.
So some subtle to mild improvement on perf.