source-sdk-2013 icon indicating copy to clipboard operation
source-sdk-2013 copied to clipboard

[Request] New Source SDK Update should be on its own AppID instead of being an update to Source SDK Base 2013 MP.

Open Wolfcl0ck opened this issue 10 months ago • 21 comments

TL;DR The new Source SDK update breaks every single mod created for Source SDK Base 2013 MP from before today, it's not feasible to expect users to know how to change branches, and it's extremely tedious to have to switch branches repeatedly for anyone who wants to play an MP mod released before the update, so there should be a new SDK Base on Steam instead.

I think I can speak for everyone who has ever released a mod on SP branch when I say that end-users do not understand how to change branches on Steam. It does not matter how many video tutorials and text tutorials and warnings and highlighted text you put in, some users - possibly even the majority of users - expect everything to work as-is right out of the box and will not read install information beyond "drag and drop into steamapps/sourcemods." (even with extremely detailed information, I still get around 25% of users either failing to set Source SDK Base 2013 SP to "Upcoming" or drag-and-dropping "Steamapps/Sourcemods/[ZipFileName]/Mod" instead of "Steamapps/Sourcemods/Mod.")

Secondly, Wanting to play any mod that was created on Source SDK Base 2013 MP from the years 2014 to 2024 will mean that users will have to switch from the current branch to "previous." Even if the user knows how to do this, it's a lot of extra time and extra downloading to switch between branches, and for all mods that aren't planning to be released onto Steam directly from here onwards, it means that players are going to need to be swapping actively between branches if they want to play older mods. I understand that it's likely that many mods aren't going to be in this category - the majority of MP base mods that are active these days are Team Fortress 2 mods anyway - but disregarding any prior mod - especially mods that were completed but are no longer actively maintained - is essentially just disregarding the last decade of modding history for the source engine.

The solution to this problem is rather simple, as it is a solution that had already been done 4 times already with SDK Bases 2006, 2007, 2013 Singleplayer, and 2013 Multiplayer: Simply creating a new 2025 SDK Base on its own AppID.

The arguments for doing this heavily outweigh the arguments against it. Doing this would require players to use more space on their drives for both Source SDK Base 2013 Multiplayer and Source SDK Base 2025, yes, but without it, users have to check dates or just guess if the mod they want to play was pre or post update, and will have to wait for a reinstallation of the SDK Base on a different branch every time the mod that they want to play is from before or after said update. Furthermore, at this time, the first release of the SDK update still has a lot of bugs with it. Missing files have led the updated SDK to have compile errors upon installation. Currently, all popular mods that were running on MP base are broken as a result of the update, either leading to crashes on launch, or being generally unstable. The SDK update released a matter of hours ago, and the Team Fortress 2 Classic Subreddit is already being filled with posts by users saying that the mod no longer works, exemplifying the problem here.

I want to finish this post off by saying that this is not a criticism of the update itself. All of the issues in the update are going to be fixed in due time and I think that this is a huge step forward for the modding community as a whole. However, just because this is good for the future of mod development on Source 1 in general does not mean that the mods made for the past decade should be buried. If the Engine differences between Half-Life 2 Episode One and Half-Life 2 Episode Two were large enough to warrant a new SDK base despite only being 1 year apart, then surely the changes done to the SDK today warrant a new SDK Base as well. Not sure what the best way to end this post is, so I'll just finish off by saying that I hope my reasons stated were reasonable enough to warrant this change. Thank you.

Wolfcl0ck avatar Feb 18 '25 23:02 Wolfcl0ck

Addendum: To clarify, from the major changes in the update, I assumed that the issues that are crashing/breaking older mods wouldn't be fixable due to the swap from 32 bit to 64 bit. However, if my assumption here is not correct, then by all means, the better solution here is just to fix the crashes so that switching branches is unnecessary. My only major concern here is forcing players to change branches on mods that are not updated.

Wolfcl0ck avatar Feb 18 '25 23:02 Wolfcl0ck

Even if some of the crashes are fixed, there should still be a separate SDK base made. Some mods over the years have had to rely on directly hooking engine bytecode in order to accomplish certain things, and those are almost definitely going to break no matter what fixes are made to the updated SDK. Other mods might just be dependent on specific quirks of the engine which changed over the 12 years since 2013 in ways that aren't actually bugs, but break older mods none the less. A separate base app would ensure that those old mods can still be played out of the box on the engine version they require, instead of needing to switch branches.

vrad-exe avatar Feb 19 '25 00:02 vrad-exe

Glad someone else is bringing this up so I don't have to make my own post 😸

An entirely new AppID would be best for everyone, I believe. I've already spun up 5-6 sourcemods that no longer work at all because of the upate.

blackletum avatar Feb 19 '25 01:02 blackletum

Can you send some examples of broken mods?

As long as they aren't doing anything naughty like hooking/patching engine functions at runtime, everything should work, the ABI should be identical (sourcetest is never recompiled, eg.)

misyltoad avatar Feb 19 '25 05:02 misyltoad

if the interfaces didn't change it should be fine /shrug

eepycats avatar Feb 19 '25 06:02 eepycats

...I've already spun up 5-6 sourcemods that no longer work at all because of the update.

@blackletum The previous2021 branch can be used to launch mods that haven't been updated or they do engine patching.

ktwrd avatar Feb 19 '25 10:02 ktwrd

Can you send some examples of broken mods?

As long as they aren't doing anything naughty like hooking/patching engine functions at runtime, everything should work, the ABI should be identical (sourcetest is never recompiled, eg.)

As mentioned in https://github.com/ValveSoftware/source-sdk-2013/issues/611#issuecomment-2668262889, IVEngineClient and IBaseClientDLL have changed, causing issues such as #611 when the game is in Windowed mode

Image

azzyr avatar Feb 19 '25 10:02 azzyr

Appending to interfaces is completely fine, unless there is multiple of inheritance.

This is why sourcetest still works.

misyltoad avatar Feb 19 '25 12:02 misyltoad

Hmm, I guess IVEngineClient inherits from something else, (013) bleh... I'll look into fixing this up for existing 32-bit mods.

Surprised sourcetest didnt catch that.

misyltoad avatar Feb 19 '25 12:02 misyltoad

Can you send some examples of broken mods?

As long as they aren't doing anything naughty like hooking/patching engine functions at runtime, everything should work, the ABI should be identical (sourcetest is never recompiled, eg.)

Went and tested a handful of mp mods. Usually I'd want compensation for QA - I spent 5 years getting an MS for this - but I'll do a lil charity today.

System Info:

OS: Windows 10 Home 64 bit, build 19045.5247 CPU: AMD Ryzen 9 3900X 12-Core Processor GPU: NVIDIA GeForce RTX 2070 GPU Drivers: 32.0.15.7216 Hardware Driver GeForce Game Ready Driver and Nvidia Studio Driver Version 572.16

Steam Info:

Steam Beta Branch: Steam Beta Update Steam Version: 1739497777 Steam Client Build Date: Thu, Feb 13 12:31 PM UTC -08:00 Steam Web Build Date: Thu, Feb 13 5:41 PM UTC -08:00 Steam API Version: SteamClient021

Source SDK Base Info:

Version: Source SDK Base 2013 Multiplayer Branch: public Update: 4655552166532639865? (unsure, I verified tool files before running these tests) Tool Location: C Drive, Samsung SSD 860 EVO 1TB

Mod: Team Fortress 2 Classic

Mod Version: 2.2.2, released February 1st 2025 Mod Download: https://tf2classic.com/download Install Location: F drive, WDC WD10EZEX-00BN5A0 HDD Method of Launch: 1st time using dedicated TF2C Launcher, second time using Source SDK Base 2013 Multiplayer hl2.exe shortcut (NOT hl2_win64.exe) Launch Parameters: -game (mod location) -windowed -noborder -h 1440 -w 2560 -novid -toconsole Result: Instant crash while attempting to load the main menu with the following popup: Engine Error - Failed to apply client patches! Crash Dump: crash_hl2.exe_20250219130933_1.dmp

Mod: Zombie Master: Reborn

Mod Version: Beta 6, released June 12th, 2021 Mod Download: https://www.moddb.com/mods/zombie-master-reborn Install Location: C drive, Samsung SSD 860 EVO 1TB, followed by F drive for a second test Method of Launch: Source SDK Base 2013 Multiplayer hl2.exe shortcut (NOT hl2_win64.exe) Launch Parameters: -game (mod location) -windowed -noborder -h 1440 -w 2560 -novid -toconsole Result: Mod works as expected, but the console is being flooded with attempts to record a demo every single tick with "record (null): invalid filename." Crash Dump: N/A

Mod: Open Fortress

Mod Version: Patch 21, released July 21st, 2024 Mod Download: https://openfortress.fun/download Install Location: F drive, WDC WD10EZEX-00BN5A0 HDD Method of Launch: 1st time using dedicated TF2C Launcher, second time using Source SDK Base 2013 Multiplayer hl2.exe shortcut (NOT hl2_win64.exe) Launch Parameters: -game (mod location) -windowed -noborder -h 1440 -w 2560 -novid -toconsole Result: Instant crash while attempting to load the main menu with the following popup: Engine Error - Failed to apply client patches! Crash Dump: crash_hl2.exe_20250219135917_1.dmp

Mod: Fortress Obscura

Mod Version: Unknown, github pull from February 4th, 2024 Mod Download: https://github.com/ChargingTurnip/Fortress_Obscura Install Location: F drive, WDC WD10EZEX-00BN5A0 HDD Method of Launch: Source SDK Base 2013 Multiplayer hl2.exe shortcut (NOT hl2_win64.exe) Launch Parameters: -game (mod location) -windowed -noborder -h 1440 -w 2560 -novid -toconsole Result: Mod works as expected, but the console is being flooded with attempts to record a demo every single tick with "record (null): invalid filename." Crash Dump: N/A

Personal Concluding thoughts

It looks like you were correct about the engine function hooking and I owe you an apology for incorrectly assuming that every 32 bit mod was broken by the update. My assumptions were based off of a game of telephone being played in the community about what was and wasn't in the update, so I apologize.

With that being said, I still think that this update needs to be on its own AppID rather than an update to the existing base. There are more changes in this update than there are between 2007 base and 2013 base, and the runtime engine function hooks in question weren't nefarious, they were fixes for issues that went unfixed for the past decade with the base. The bytecode hacks in TF2C and OF, for instance, were done to circumvent the arbitrary mat_picmip lock that was put into all versions... to fix a crash on OSX. It doesn't seem fair to decide that every single mod that did that for the past decade is out of luck and that players will have to swap branches to play them if said mods aren't still actively being developed. (Also the demo recording issue is happening for most if not all MP mods, but I assume you're already aware of and are working on fixing that)

crash_hl2.exe_20250219135917_1.dmp crash_hl2.exe_20250219130933_1.dmp

Wolfcl0ck avatar Feb 19 '25 20:02 Wolfcl0ck

Looks like w0lf was doing the same thing as me!

...I've already spun up 5-6 sourcemods that no longer work at all because of the update.

@blackletum The previous2021 branch can be used to launch mods that haven't been updated or they do engine patching.

That is true, but I dont want to have to switch it around every single time I play an old sourcemod, then have to switch it back to play new ones. That's simply not a solution to this issue.

Can you send some examples of broken mods?

As long as they aren't doing anything naughty like hooking/patching engine functions at runtime, everything should work, the ABI should be identical (sourcetest is never recompiled, eg.)

Mods I have tested that have issues:

Beta Fortress – Launches, but character models and weapons are completely black. Other random items like the resupply locker are multi-colored. Also seems to be crash-happy which it wasn't previously. Server browser/console are both completely black.

Call of Duty Source – Main menu items are all missing their original colors. Server browser/console are both completely black. Starting a map crashes the game.

Cyber Diver – "X10 ERROR -1" pops up when launched, taking up the entire screen.

Day of Conquest – Launches, but crashes immediately after starting a map.

Exterminatus – Launches, but crashes immediately after starting a map.

Fortress Connected – Crashes before hitting the main menu.

Fortress Zombies – Server browser/console are both completely black. Crashes after starting a map.

Human Error Coop – Beta 1.0.6 – Crashes when starting a map.

Half-Life 2: Capture the Flag 2.1 – As far as I can tell it works fine, but certain sounds like footsteps are really broken.

Lambda Fortress: Extended – Launches, but all TF2-related sound files such as shooting weapons and voice responses are missing. Ambient sounds and music are also missing.

Open Fortress – Crashes on launch.

Open Fortress ZOMBIES – Crashes when you try to load a map.

OPEN2 – All menu buttons are black, console/server browser are completely black. Crashes immediately after map load.

Pre-Fortress 2 – Crashes after launching.

Pre-Fortress 2 Classic – Launches, but character models and weapons are completely black. Other random items like the resupply locker are multi-colored. Server browser/console are both completely black.

Team Fortress 2 Classic – Crashes immediately after launching.

Team Fortress 2: Gold Rush – Crashes immediately after launching.

Team Fortress 2 Vintage – Crashes immediately after launching.

Touhou Fortress 2 – Launches, but character models and weapons are completely black. Other random items like the resupply locker are multi-colored. Server browser/console are both completely black.

Out of all the Source SDK 2013 MP sourcemods I tested, only one, an abandoned game called "GOONSQUAD", actually worked properly, best I could tell.

A (very) small handful of these are still currently in development and will be patched in the future, I'm sure, but the overwhelming majority are old or abandoned projects. Projects like Exterminatus, for instance, has a small but dedicated fanbase even all these years later.

I think it would be an incredible detriment to the sourcemodding community to not make the updated SDK have it's own AppID.

blackletum avatar Feb 19 '25 21:02 blackletum

now if im not mistaken tf2c and open fortress both do engine hooks and patches which as misyltoad said wouldnt work and tbh was never supported

sylveonsylvia avatar Feb 19 '25 22:02 sylveonsylvia

Honestly, even if it is just engine hooks that break it, I still think the new version of the engine should be a separate app. Even if engine hooking is unsupported, it was done under the reasonable assumption that SDK bases are versions of the engine that never get major updates, which is really the reason they were made in the first place - that's why you can still run old 2006 engine mods today without much hassle. I don't think it's really fair to suddenly update the existing SDK base used by mods and then blame those mods for breaking, when they specifically used the build of the engine that's supposed to not get major updates.

Either way, I don't really see why making a new "SDK Base 2025" app or whatever seems to be such an issue, does valve have some internal policy against that or something?

vrad-exe avatar Feb 19 '25 22:02 vrad-exe

now if im not mistaken tf2c and open fortress both do engine hooks and patches which as misyltoad said wouldnt work and tbh was never supported

The function of a system is what it does, not what it was intended to do. If a mod released in, for example, 2016 with an engine hook to fix an error in the SDK base's engine, the fault lies on the SDK base for having the error, not on the mod for fixing it during runtime. Consequently, if the SDK base is updated later and said update breaks the mod that had said fix, the fault now lies on the SDK both for being broken and requiring a non-conventional fix in the first place and for breaking the mod that needed to fix it manually.

Arguing that using engine hooks in a mod is "against the rules" and therefor means it's okay for said mods to break is a logical fallacy that fails to take the reasons why they were used in the first place into account. The SDK was left to rot for an entire decade. Modders couldn't even let players change their playermodels without kitbashing a base out of 2 different SDKs (2013 SP/MP as the base and the Alien Swarm SDK for its GameUI code.)

Again, the SDK receiving attention again is a very good thing, I'm not making an argument here that it's not. I also want to clarify that I understand that there is no intended malice here. There's no argument here that this update should not have happened, nor is there an argument here that it should be reverted outright due to mods breaking from it. What we're saying is that it should be placed on its own AppID so that older mods aren't made unplayable as a result of it. A lot of mods have done this so circumvent arbitrary restrictions and bugs caused on the engine-side of the SDK like mat_picmip and cl_playermodel.

The worst thing that happens if the new update is placed onto a dedicated "Source SDK Base 2025" AppID is that the already existing mods that would work without any changes to them are left behind on Source SDK Base 2013 MP, which doesn't matter anyway because they could be patched by anyone simply by updating the "SteamAppId" field in the gameinfo. The worst thing that happens if the new update is not placed onto a dedicated "Source SDK Base 2025" AppID is that the mods that are affected break and remain broken permanently without players manually reverting the SDK Base back to previous2021.

One more thing, I want it to be made extremely clear to Misyl that I fully understand you are doing everything in your power here. I am in no way blaming you for any of this and I suspect that you were already aware that this would be an issue before the update ever dropped. If you had full reigns on this stuff, I imagine you probably would have done this in the first place. The point of the arguments I'm making here are moreso to give you something to show to whomever you'd need to show that "hey, users are reporting these issues and giving reasons for this solution" in hopes that you can be granted the green light for it.

Wolfcl0ck avatar Feb 19 '25 23:02 Wolfcl0ck

Arguing that using engine hooks in a mod is "against the rules"

i never said this and i wasnt implying that either. my point is this was kinda expected, i think the best way to solve this would be to create a new archived version of the 2013 sdk as its own app id so that its minimal work to get old mods working again (a simple gameinfo edit like you have to do with pre steampipe era mods anyways)

if we wanna talk about things being against the rules though most of the mods mentioned are based on the leaked 2008 era tf2 code and should be rebased on the official tf2 code release ASAP anyways

sylveonsylvia avatar Feb 19 '25 23:02 sylveonsylvia

if we wanna talk about things being against the rules though most of the mods mentioned are based on the leaked 2008 era tf2 code and should be rebased on the official tf2 code release ASAP anyways

Agreed. Apologies for sounding accusatory.

Wolfcl0ck avatar Feb 20 '25 00:02 Wolfcl0ck

Looks like w0lf was doing the same thing as me!

...I've already spun up 5-6 sourcemods that no longer work at all because of the update.

@blackletum The previous2021 branch can be used to launch mods that haven't been updated or they do engine patching.

That is true, but I dont want to have to switch it around every single time I play an old sourcemod, then have to switch it back to play new ones. That's simply not a solution to this issue.

@blackletum Some of those mods are using sappho's engine patches, which myself and a couple of other people in the Open Fortress & TF2C dev team are working to fix it.

@misyltoad There are some other issues I've ran into while fixing those engine patches and that INetChannel.GetRemoteAddress() fails when loading into a map in a local server via the console (specifically with map itemtest). I'll most likely encounter more when I continue working on OF later today.

ktwrd avatar Feb 20 '25 01:02 ktwrd

If a mod released in, for example, 2016 with an engine hook to fix an error in the SDK base's engine, the fault lies on the SDK base for having the error, not on the mod for fixing it during runtime.

Its the engines fault for having the bug but its practically impossible to keep existing hooks in mods working when updating the engine.

Even just slight changes in the toolchain or a given function will result in things being shuffled around and changed which will break anything from hardcoded offsets to function patterns.

For something like the sappho's sdk13-gigalib this could be worked around by turning it into a dynamic library that can be independently updated of any respective game mode, but that still requires it to stay ABI and API compatible long term

Jan200101 avatar Feb 21 '25 07:02 Jan200101

Its the engines fault for having the bug but its practically impossible to keep existing hooks in mods working when updating the engine.

Well yes, that's why this update needs to be put on its own SDK Base instead.

Wolfcl0ck avatar Feb 21 '25 15:02 Wolfcl0ck

Even just slight changes in the toolchain or a given function will result in things being shuffled around and changed which will break anything from hardcoded offsets to function patterns.

Inane comments do not advance the issue

borkfomb avatar Feb 21 '25 17:02 borkfomb

There's also the fact that mods that "play nice" and don't hook or patch anything at runtime get broken anyway by the update because of the high DPI changes done to GameUI that have proven to be breaking when they were first pushed to TF2 last year (case in point: https://github.com/ValveSoftware/Source-1-Games/issues/6429)

TorutheRedFox avatar Feb 27 '25 17:02 TorutheRedFox

There's also the fact that mods that "play nice" and don't hook or patch anything at runtime get broken anyway by the update because of the high DPI changes done to GameUI that have proven to be breaking when they were first pushed to TF2 last year (case in point: ValveSoftware/Source-1-Games#6429)

Even when disabling all hooks (and patches!) and making sure that all types are properly defined, mods don't work well when porting anything over. Especially since I've been wrestling with the engine for the past few days just to get VGUI stuff working when porting a mod to 64bit.

ktwrd avatar Feb 28 '25 06:02 ktwrd

Has there been any consideration for this yet?

blackletum avatar Oct 07 '25 03:10 blackletum