DFHack
Feature Request: set viewport size, ascii mode performance
The Problem
The old Dwarf Fortress used to let you run it at 80x25 characters. While far from perfect, this used to be a way to keep FPS in check on older machines.
The new Dwarf Fortress past version 50 has a reworked UI that is more accessible to new users, but it won't let you set the resolution lower than 114x46 characters. It's hard-coded, and the likely reason for this design choice seems to be that the NEW interface doesn't scale that well to low resolution.
In other words if you like ascii mode now you got two options - high resolution so the UI displays right but very low FPS, or low resolution for better FPS but then the UI doesn't scale well as it wasn't really designed for such low resolution (it still works but it's a bit awkward to navigate).
Proposed solution
Setting the resolution of the viewport only (new proposed dfhack tool). This way both the UI would show correctly, and the game would run at higher FPS. No minimum viewport size limit (just nonzero), hypothetically letting it run on older PCs or to adjust when FPS gets too low due to a lot going on on the screen.
Simulated example of what it would look like (before, after):
The idea is reducing the play area like you used to be able to do in doom for a higher FPS. DF used to have something like this in that the old UI used to cover half the screen, effectively reducing the viewport and decreasing the computational load.
The viewport size would be adjusted independently of the overall resolution. Changing the viewport size could be made into a keyboard shortcut like the zooming.
The smallest embark you can make, which is 48x48 char or 1x1 areas accomplishes something similar to what I'm describing but I think it's because it forces the viewport to be small. Pretty sure that if you had a bigger embark but with the same viewport as 1x1 area forces - it would run just as fast.
Of course ideally you'd want multiple threads in the game itself, or a faster CPU core, or avoiding high FPS situations, but this is a proposed in-between solution for when those options aren't available or have been exhausted.
Implementation
I'm relatively new to dfhack development. I don't know yet if the proposed tool is technically feasible at all, or if maybe someone implemented something like this already, or would be willing to. I'm trying to see if the API has any variables that are relevant to this. Help is appreciated.
Have you benchmarked to make sure the interface size is a bottleneck, on modern-ish machines?
I've often been shocked at what actually slows down code, versus what I expected would be the slowdown areas.
I'd suggest uncapping FPS in prefs/init.txt, and experimenting with both your reported minimum size 114x46 and with a 8x8 font at fullscreen
1920x1080 with an 8x8 font would be 240x135 characters, that's only 6x the character count, which I would expect to be trivial nowadays.
But as I say, I've often been absolutely boggled at where bottlenecks actually show up.
Pretty sure. @SilasD
You can try it yourself. Here's the exact steps to reproduce the issue:
- Make sure
[FPS:YES]and[USE_CLASSIC_ASCII:YES]are set inprefs/init.txt - Load an embark that is big enough to still cover the screen when zoomed out.
- Take note of the number in brackets on the cyan rectangle where it says FPS at the bottom of the screen.
- Zoom out a bunch of times by pressing
](this effectively increases the viewport dimensions) - Watch the "graphical FPS" at the bottom (number in brackets) tank compared to previous.
- It takes a good few seconds for the new number to adjust as it's showing averages.
- Now zoom all the way in and notice the difference in "graphical FPS", especially if uncapped.
- If this isn't impacting game play on your setup that's ok but it's the same idea
There's other bottlenecks but this is a big one for me. This may not be impacting the graphical FPS on a modern machine as much to notice. A way to simulate an older machine is via a virtual machine or with cpulimit.
On my machine zooming out makes the game almost unplayable except for 48x48 char embarks (forces 48x48 viewport) or for when zooming all the way in (forces 114x46 character viewport). In both cases I get about 20 or 30 graphical FPS which makes it alright but it drops with more things going on. This is on a new embark with nothing too heavy graphically going on just yet, and on the surface (underground it runs much better). The workaround solution that I'm proposing is intended for old PCs mainly. Idea is resizing the viewport INDEPENDENT of resolution or zoom level. It should help.
The forum post I linked has a bunch of other things in it, this is the relevant part from Toady commenting on it:
I think Toady must have made a typo as the actual hardcoded minimum x char count in v52 seems to be 114, not 116, or maybe that changed since or something.
My main point is that the "play area" is not the same as the overall resolution. Before v50 you used to be able to really shrink the "play area" aka "viewport" effectively by having things open from the side and keeping them open. You can no longer do that since v50. I understand that Toady likely prioritises the steam release and user-friendliness of the interface over performance on older machines.
If support for higher performance on older machines is a concern then reducing the "play area" as such should be a relatively straightforward way to increase the graphical FPS (number shown in brackets) - as it does that already - while maintaining a usable new UI. This doesn't seem to be on Toady's todo list so I was hoping to write myself a dfhack script to accomplish this, or to perhaps find one that does this - if this is doable.
It's an approach used in older video games a lot - only a part of the available screen is rendered. This way there's less to compute of what needs to be shown of what constantly updates, and this way the overall performance increases.
This approach should work especially well for df because unlike shooter games you don't need to take in all that much with your peripheral vision and if something important is happening the alerts let you know. The area your eyes focus on in practice tends to be pretty small.
Already done some searches on this issue (reducing "viewport" or "play area" in df independent of zoom or resolution) but haven't yet found all that much of anything relevant.
Years ago I wrote a few example lua scripts for dfhack so I roughly know how things work though I didn't get too deep into it and forgot a lot since, and some things changed. I roughly know how to use gdb. I don't know if changing the viewport size is something that's been mapped from the df executable, because if it wasn't then this may not be accomplishable yet via dfhack. I'm trying to find anything relevant in the API and df-structures. If anyone wishes to point me in the right direction meanwhile it's appreciated.
I just performed the experiment recommended here. Using my currently "just messing around" fort (5x5 embark with 345 dwarves) in "classic" graphics mode, I get basically the same frame rate no matter what zoom I use: between 20 and 22 FPS.
I captured 10 second vTune hotspot profiles with both zoom all the way in and all the way out. I will note that the time spent in nvd3dumx.dll (the userspace half of the drivers for my NVidia GTX 1650) is only 0.015s zoomed all the way in, while it's 0.294s in the zoomed all the way out. Thus, in the zoomed-in, indeed, user-mode GPU overhead is only 0.15% of CPU time while in zoomed-out it's 2.94%. This isn't terribly surprising, given that at the deeper zoom the renderer has to blit many more textures over to the GPU, and that can't be done instantly. Still even at the deepest allowed zoom the GPU driver time is using less than 3% of wallclock time.
Similarly, time in SDL2.dll went from 0.002s to 0.053s.
Another thing I noticed is that in the fully-zoomed-out configuration, there is 1.023 s spent in NtWaitForSingleObject, called from two places: one in the NVidia user mode driver's internal thread and the other from SDL_Delay which is SDL's internal function for waiting because the application told it to, called from DF's render thread. The NVidia thread is waiting because it ran out of work, and the SDL thread is waiting because it ran out of work before DF's simulation thread had a new "gframe" ready to render. In the fully-zoomed-in version, the time spent in NtWaitForSingleObject is only 0.281 s, which, ironically, this means that it's actually rendering faster when zoomed out.... which is the exact opposite of what the OP claims it's doing.
There is an explanation for all of these observations, but it requires a fairly extensive discussion of how modern GPUs work, and what SDL is doing under the hood to make that as efficient as possible. I don't feel like making this issue discussion an introduction into modern GPU hardware acceleration technology and the software APIs used to implement this.
The most likely explanation for the behavior the OP is seeing is due to using an very low quality graphics driver on their system, quite possibly even a wholly unaccelerated software renderer, that doesn't leverage any of the hardware acceleration or texmap caching that SDL can perform with even quite old hardware-accelerated GPUs. If you choose to run software that is designed to make use of hardware acceleration on a machine that has none (or which has it but which is not using it due to a misconfigured operational environment), you should entirely expect poor performance. Even 15 year old systems almost always have hardware acceleration that SDL can leverage; even old Intel integrated GPUs offer at least some functionality here. The problem here is almost certainly that the environment is blocking SDL from accessing those features, for some reason I can't begin to comprehend, which is forcing SDL to fall back onto an unaccelerated software renderer that is CPU blitting each texmap onto the render surface one at a time instead of asking the GPU to do it for it. Needless to say, the latter is orders of magnitude faster.
I think you'll get more mileage out of figuring out what you've got misconfigured in your environment than you will out of trying to muck around with DF internally. In any case, any attempt to alter the behavior of DF's renderer to render less than the full screen would probably require a code detour within the renderer implementation, which is something we've decided is out of scope for DFHack in order to implement a feature (as opposed to a bug fix), due to the long-term maintenance load imposed by code detours, and as such we are not likely to support it or put much effort into attempting to facilitate it.
Wellp, that seems fairly definitive, when a senior developer says it's both very hard and unsustainable.
I did do my own benchmarking tests, which I'll copy below. I didn't do as throrough as job as ab9rf did.
With 100 dwarves working on the surface, my benchmarks showed 22 to 51 FPS depending on the zoom level (default-font) or the particular custom font used.
I think that's reasonable performance.
That's on tile sizes that I would play myself. I also tested outliers that I don't consider to be playable.
Here's my notes, if you care.
Windows 10. AMD Ryzen 5 2600. Radeon RX 580 Series.
That's a seven or eight year old machine, BTW.
Make a new game directory.
Extract df_52_02_win_s.zip
Extract dfhack-52.02-r2-Windows-64bit.zip
mkdir prefs
touch prefs\portable.txt
copy data\init\* prefs\
copy prefs\init_default.txt prefs\init.txt
copy prefs\d_init_default.txt prefs\d_init.txt
Edit prefs\init.txt:
Set [FPS:YES]
Set [FPS_CAP:1000]
Set [G_FPS_CAP:1000]
Set [USE_CLASSIC_ASCII:YES] (already set.)
Download and save to data\art:
https://dwarffortresswiki.org/images/f/fa/Taffer_10x10.png
https://dwarffortresswiki.org/images/2/29/Kren_13x13.png
https://dwarffortresswiki.org/images/e/ee/Guybrush_square_16x16.png
https://dwarffortresswiki.org/images/7/76/Taffer_20x20.png
Don't change fonts yet.
Start the game, check that it's behaving properly.
At title screen, FPS caps at 1000, G_FPS hovers at 389.
Continue Active Game is enabled.
Check Settings / Game to verify that it's in portable mode. (It is.)
Check Continue Active Game, it is showing a fort-mode game with three saves (Shrug.)
Create new world, Smaller, 100 years. No mods. Xah Ashi, in The Age of the Roc.
Keep world and return to main menu.
Note that region2 is in the local save directory. (Shrug again.)
Run: startdwarf 100
Start new game in existing world, region2, Fortress.
Embark, 6x6, choose a flat area, Play now!
Create a large food stockpile.
Mark a *large* area for plant gathering.
Mark some trees for cutting.
Queue up some workshops to be built when wood is available.
Save and return to title menu.
Get a baseline:
Reload the game. Use the default zoom level.
Put the viewport over the plant-gathering zone.
Unpause the game. Look at `em go! Wait for FPS/GFPS to stabilize.
GFPS (the number in parenthesis) stablized at ~34 with most units on-screen.
Reload the game. Zoom out as far as possible. Unplayably far.
Center the embark in the viewport.
Unpause the game. Wait for FPS/GFPS to stabilize.
Literally use a magnifying glass to read GFPS.
GFPS stabilized at 9 to 10.
`die` and restart DF. Reload the game. Zoom out 10 notches with `]`
Zoomed out so far this is essentially unplayable.
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 15 to 16.
`die` and restart DF. Reload the game. Zoom out 5 notches with `]`
I wouldn't play this zoom level either.
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 22.
`die` and restart DF. Reload the game. Zoom out 2 notches with `]`
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 26-27.
`die` and restart DF. Reload the game. Don't zoom in or out.
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 34-35. That matches the first baseline measurement.
`die` and restart DF. Reload the game. Zoom in 2 notches with `[`
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 42 ± 2.
`die` and restart DF. Reload the game. Zoom in 4 notches with `[`
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 51 ± 2.
`die` and restart DF. Reload the game. Zoom in 6 notches (the max) with `[`
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 62 ± 2.
Edit prefs\init.txt. Mutter to self about how DF handles settings files.
Set [INTERFACE_SCALING_TO_DESIRED_GRID:NO]
Set [FULLFONT:Taffer_10x10.png]
Calculate that there should be 1920/10*1080/10 = 20736 screen tiles.
`die` and restart DF. Reload the game. Don't zoom in or out.
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 22 ± 1.
Set [FULLFONT:Kren_13x13.png]
Calculate that there should be 1920/x*1080/x = 12212 screen tiles.
`die` and restart DF. Reload the game. Don't zoom in or out.
Note that something is wrong with the zoom.
Fix it by pressing F11 twice to change to/from windows mode.
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 36 ± 1.
Set [FULLFONT:Guybrush_square_16x16.png]
Calculate that there should be 1920/x*1080/x = 8100 screen tiles.
`die` and restart DF. Reload the game. Don't zoom in or out.
Note that the colors get badly messed up and FPS is unreadable.
Abort this test.
Set [FULLFONT:MRC_square_16x16.png]
Calculate that there should be 1920/x*1080/x = 8100 screen tiles.
`die` and restart DF. Reload the game. Don't zoom in or out.
Note that something is wrong with the zoom.
Fix it by pressing F11 twice to change to/from windows mode.
Note that part of the DF interface is missing; it doesn't fit.
I wouldn't play this; it's too zoomed-in.
Unpause the game. Wait for FPS/GFPS to stabilize.
GFPS stabilized at 51 ± 2.
Set [FULLFONT:Taffer_20x20.png]
Calculate that there should be 1920/x*1080/x = 8100 screen tiles.
`die` and restart DF. Reload the game. Don't zoom in or out.
Note that something is wrong with the zoom.
Fix it by pressing F11 twice to change to/from windows mode.
Note that part of the DF interface is missing; it doesn't fit.
Note that the bottom row of buttons cannot be pressed.
The game is literally unplayable with a 20x20 font.
GFPS stabilized at 52 ± 3.
OK. I changed the default [PRINT_MODE:STANDARD] to [PRINT_MODE:2D] and my graphical FPS signifficantly increased, from the 7 previously to 30 now on the normal zoom, wow! That's already as-is a huge performance increase for me.
For some reason changing this setting escaped my attention completely, even though I vaguely remember it from years ago. I think I used to have it on "SHADER" back then but that changed since. Now, with the "2D" setting the graphical FPS although seems still tied to viewport size in chars somewhat - it's way way less so than with the "STANDARD" setting on my setup, meaning, I get 30 graphical FPS even if I zoom all the way in and if I zoom out to "normal" it's 30 graphical FPS, if I zoom out further it's 20 etc so there's some correlation, but it's not as stark as with the "STANDARD" setting.
Now what you were describing makes sense. Reducing the viewport was helping but it's because of the specific PRINT_MODE setting which overall is sub-optimal on my setup.
The whole idea behind the proposed solution was that smaller viewport = higher performance, but it no longer to seems to be case with a different PRINT_MODE setting, a setting that performs better overall. This was unexpected. I need to rethink this completely if I want to troubleshoot performance yet further, although this alone makes now the game playable for me already.
Long term probably I'd be better off finding some kind of "profiler" (or equivalent) if wanted to troubleshoot the performance further, like the pie chart in minecraft, as so far the tests I was doing were rough, but that's beyond the scope of this issue ticket I guess.
Closing the issue as the proposed solution in the first post no longer seems to me like the way to go for increasing performance further. Thanks for all the comments. Hopefully this helps someone else as well.
Thanks but some things still don't add up for me, even though I get much better overall graphical FPS now with PRINT_MODE:2D instead of PRINT_MODE:STANDARD.
Testing this on 1920x1080, no zooming, paused, on surface post embark. (240x90 characters, default 8x12 pixel font)
I get ridiculously high FPS on a 1x1 embark in comparison. Half that value on a 6x6 embark with normal centered view.
However, if I force the view into a corner on a 6x6 embark by repeatedly pressing w and a until it can go no further - the graphical FPS increases to almost same value as on a 1x1 embark.
My hypothesis: the graphical FPS depends on the effective viewport size in characters and not (unless it limits it) the size of the embark/overall pixel resolution of the screen/the character resolution of the screen, and I should get the same high graphical FPS as on a tiny embark on a large embark if I found a way to shrink the viewport to the same size.
The old dwarf fortress versions let you make the viewport really tiny if you wanted, it helped with graphical FPS.
I'm interested in seeing if it's possible to do somehow in v52 with dfhack. I'll post the result if anything works.
If those of you who are interested can think of relevant specific values in df-structures please let me know. I've found some mentions of "viewport" in it but so far I haven't got it to work yet. Meanwhile I'll do some tests. Thanks in advance.
I think you should use gui/gm-editor to explore df.global.enabler, df.global.enabler.renderer, df.global.gps, df.global.map_renderer. In particular, there are viewport structures in df.global.gps.
I've only glanced at these; I know they exist but not what anything does.
Expect lockups from tweaking things. Lockups may be possible from even viewing certain fields.
Certain fields will be defined as a single uint8_t when they are actually arrays. In Lua you have to use ref:_displace(index[,step]) or maybe df._displace(obj,index[,step]) to see each cell of the array. I don't actually know how to do this, but there's a few examples in the source code.
Also, be aware that there is likely a lot of unused structures in there, left over from the old 0.47 interface.
df.global.gps.main_map_port may be relevant, or it may not.
After reading the additional details, my conclusion is that the OP has out of date, incorrect, or misinstalled drivers for their RX580. The described behavior fits precisely with SDL selecting its software renderer because none of the hardware accelerated renderers were able to initialized. This shouldn't be the case for a properly installed and configured RX580.
Another possibility is that there's a second lower-capability GPU installed and SDL is selecting it instead of the RX580. I have seen this happen: my daughter has a gaming laptop with an NVidia RTX 4000 series GPU, but it also has a secondary integrated Intel GPU, and until she fixed the configuration Minecraft would use the potato Intel GPU instead of the much more capable RTX 4000, resulting in extremely poor video performance. Given the statement that switching from STANDARD (which, IIRC, means "use the software renderer") to 2D (which means "use whatever hardware acceleration is available") provides some improvement tends to make me think this is most likely what is happening.
Again, I reiterate that the solution here isn't to blinker DF, but instead to fix the configuration problems in the OP's environment. An RX580 is easily performant enough to be able to provide enough hardware rendering assistance to avoid the problems the OP is reporting, which to me indicates that the OP has an environmental issue that prevents the RX580 from being used.
@ab9rf
It's Nvidia GeForce GT 730 with AMD FX-4350. Archlinux, Nouveau driver. Acceleration as such works in other video games.
Noveau allows recklocking via /sys/kernel/debug/dri/0/pstate Tested it before and it signifficantly improves performance in some other video games that use 3D acceleration. In df tried reclocking with either PRINT_MODE setting value mentioned, no difference. There is no second GPU.
I'd rater not test this with nvidia-39xx-dkms driver (requires rebuilding on each kernel update, softlocked lowres VT, obscure config, other things already work as expected) but can if sure that it's relevant.
Given the statement that switching from
STANDARD(which, IIRC, means "use the software renderer") to2D(which means "use whatever hardware acceleration is available")
This is from init_default.txt in v52:
If set to SOFTWARE or 2D, this will force the game to use software rendering; if set to AUTO, STANDARD or anything else, SDL2 will try to auto-select based on the system the game is being run on.
[PRINT_MODE:AUTO]
Continuing,
fix the configuration problems in the OP's environment
Would have to know what specific steps you propose as a fix. As far as I can tell everything is working as expected in the environment as such.
@SilasD
That's very helpful, thank you.
I don't mind the lockups. Can just restart it. Thanks for the Lua tip.
I've done some tests since, if interested.
This shrinks the "viewport" though also makes the rest of the map not scrollable. Graphic FPS doubles:
[DFHack]# lua df.global.world.map.x_count=48;df.global.world.map.y_count=48
This allows for setting the char resolution much lower than the hardcoded 114x46 minimum limit. Unfortunately it resizes the UI as well and doesn't seem to impact Graphical FPS.
[DFHack]# lua df.global.init.display.grid_x=90;df.global.init.display.grid_y=50
It could be that it's called something different in df-structures. Another idea is that if I know that the "viewport" is 48 characters then maybe I could look for anything that has the value 48 or 47, then if I know it's 96 now, can check which of previously found changes to 96 or 95.
Are you running DF in Proton or natively?
I am not in a position to provide specific advice for Linux systems as I don't use Linux as a desktop, but my understanding is that virtually all NVidia drivers for Linux suck to some degree or another. It's highly likely that, for whatever reason, you're getting the SDL2 software renderer, because, again, for whatever reason, SDL2 isn't able to interoperate with the hardware resources that are available to it. Troubleshooting that is beyond my knowledge, but ultimately the issue is that if you want hardware acceleration, you need a driver that interoperates with the graphics system of the application you're running, and by the sounds of it you don't have one, or you're running the application in an environment that blocks the application from accessing the hardware directly.
Installed it via the dwarffortress Archlinux package. No Proton.
The setup is an artificial potato because the official binary driver which fully uses the hardware is no longer supported by Nvidia and the FOSS Nouveau driver that I use doesn't yet have all hardware features implemented.
for whatever reason, you're getting the SDL2 software renderer for whatever reason, SDL2 isn't able to interoperate with the hardware resources that are available to it
That's the reason.
Troubleshooting that is beyond my knowledge
If there's a better approach I'd need to know the specifics of what to do different. Just installed 390xx and linux-lst 6.12 to be sure, some people made patches for new kernels, didn't see any nvidia or nouveau modules in lsmod on reboot, startx says "no screens found", so uninstalled it.
The solution proposal in OP is what I see as the realistic workaround meanwhile. I'm closer to having it work.
My personal advice would be to downgrade your kernel to a version that can run a GPU driver that supports your hardware. That, or figure out how to adapt the last known good GPU driver to work with a more modern kernel.