wg
wg copied to clipboard
rust-gamedev
Rust on Consoles
Hi there,
I'd like to help get Rust onto Consoles more. I have two devkits from the major platform holders (I'm trying to be a little subtle here to avoid breaking any NDAs) and have signed an NDA with the third.
I have tried at work to get others onto using Rust in gamedev, but the hardstop, for the kinds of games I work in, is console support. If you've ever wondered why Godot seems to have so little traction, a lot of it is a lack of console support. Every conversation about Godot, and, unfortunately, Rust, starts with "wow I love this" and ends with "pity I can't easily get it to run on consoles."
I think we would all agree, however, that Rust is excellent for gamedev, and it would be great to code less in C++. However, to even get to that point, we need to build some infrastructure. I think this would be in the purview of the WG and would love to discuss that with some of you. I imagine this would take the form of a private repository managed by the WG which requires an invite, predicated on demonstration of an NDA signed, to join. I would be very willing to help with that. It would be easier to talk about this over voice and in a less written way of course.
Getting that infrastructure down first allows us to tackle the actual code issues necessary for console support.
Let me know what you guys things
I think, for now, all we can practically do is voice these wishes / ideas on the dev forums as those are places that filter ppl by NDA agreements already.
+1 on using the pre-existing first-party-sanctioned means to discuss. (Sorry for speaking with all the charm of a wet blanket! I would love to see progress here too!)
Another concern is that even if every technical issue were solved the title still has to pass cert. You might need a waiver to use non-sanctioned toolchains and Iām not sure how keen first parties will be to grant that.
There is still lots of space for tools and backend services for AAA. I also feel that rust has to prove itself on PC and mobile before first parties will get behind rust in a meaningful way.
Maybe we should try to kick off a conversation on one of them. We may need to coordinate which one, discord private messages might be the best place for that.
We could start with the platforms where the tech-approach seems promissing.
Hint-hint:
- http://llvm.org/devmtg/2013-11/slides/Robinson-PS4Toolchain.pdf
- https://www.youtube.com/watch?v=9_7exO60EA8
- windows-pc š
The reality is that gamedev & reasearchy things don't blend well together due to the usually hellish delivery constraints that gamedevs have. So maybe a studio that loves Rust can start something more serious in this space (@repi ? š).
But as I said, let's take this discussion to the dev forums.
Rust console support has not yet been a priority for us here at Embark, been focusing on desktops (windows, Mac, Linux) first. But I think it will be later during this year so likely will do some more experiments with it, but not ship anything with it this year.
We a tracking issue on our page: https://github.com/EmbarkStudios/rust-ecosystem/issues/18
But yes given the nature of the consoles and the NDAs, work on them can't be done in public. But if we do start on it we would want to do it as a private github repo with the platform companies managing access
I'd definitely support this, if we can figure out a way to navigate the maze of NDAs the console APIs are wrapped up in. From what I understand, getting Xbox One support should be the easiest out of all of the consoles, since they support UWP, but I haven't looked into that too much so I'm not 100% certain on that.
The easiest way to get going is (probably) to just shove the whole game into a staticlib, and then you'd have the platform specific stuff be the binary (wouldn't even have to be in rust) that links to the static lib, boots necessary services, and calls game_do_frame(args) every frame.
This way, each platform can have whatever platform specific top level quirks, and the game doesn't really have to know much about it. You'd pass in a few function pointers for the "services" that the game will need to have provided for it (allocator, vulkan instance, how to read a file, etc etc) and then the inside of the game can otherwise be pretty agnostic to the platform in question.
It's not the most ideal perhaps, but every console SDK can doubtless link to a C static lib, so to start just make Rust pretend to be a C static lib to the outside world.
Hey, with that model that @Lokathor just mentioned, what if we then created a public crate that would define the standardized interface for communicating to the console. That crate would allow you to register callbacks for the console APIs that your game needs and it would provide the extern C functions that would be called from the console SDK.
The crate could be public because all it does is define a C interface for use with the console SDK and a Rust interface used to hook those C calls to your game's Rust code. Then we could, in the open, collaborate on a console agnostic API.
Obviously there are console-specific APIs that you might need to interface with, so maybe those would need to be separate private crates?
Anyway, at that point, behind the "NDA walls" we could develop the glue for each console that links the console SDK to our interface crate's C API.
You can't quite do that with functions because the thing with the functions to call would be your own game, not some dependency of the game.
However, in terms of defining a common set of input types that can be used to form the "interchange" between outer OS shell and inner game lib, that's pretty doable.
(Bonus: if you have this "game as a lib" setup working you can generally hotswap the game during development with not much difficulty)
You can't quite do that with functions because the thing with the functions to call would be your own game, not some dependency of the game.
Ah, OK. Maybe we could make a crate with the standard types, like you said, and also include macros to generate the extern C functions for you? Maybe writing those isn't a big deal and it wouldn't be worth it, but it might help it feel a little more Rusty if you could just set_whatever_callback!(my_normal_rust_fn); or something like that in your lib.rs.
I would suggest plain and obvious function declaration, with no macros.
Don't complicate things just for the sake of it being "rusty feeling".
As much as I am pro rust-on-consoles, discussing it here just isn't practical at this stage.
I know it's anoying but proposing things here leads to strange situations where ppl who know about how console SDKs look like and are capable of can't say anything, while people who didn't sign NDAs are left to speculate without gettting any feedback if any of the sugestions are practical or not (or even if they address an actual issue).
For those of us who have NDAs signed, the forums are already there to facilitate more practical next steps. So, for these reasons I am semi-tempted to suggest parking this public issue for now.
That whole situation is a bummer, but unfortunately I think that's sound reasoning. It probably isn't best to discuss here.
Oh, well. Their consoles their rules I guess.
@AlexEne I agree, with a caveat: XBox One supports UWP, and UWP is a public API.
I wonder how LuaJIT got console support. I do know that its console support is 100% open-source. Is it because LuaJIT just uses essentially C standard library functions?
Ideally, we'd have an organization like "RustOnThisParticularConsole", which would have all the repositories in private, would only accept members who signed the NDA, and allow these Rust community members to collaborate on the ports. Otherwise, Embark does one thing, other team does another thing, single developers re-discover all those things and probably waste a lot of time. Did anybody try to confirm with platform holders if such organization/private repos structure is acceptable?
Ideally, we'd have an organization like "RustOnThisParticularConsole", which would have all the repositories in private, would only accept members who signed the NDA, and allow these Rust community members to collaborate on the ports. Otherwise, Embark does one thing, other team does another thing, single developers re-discover all those things and probably waste a lot of time. Did anybody try to confirm with platform holders if such organization/private repos structure is acceptable?
Is UWP an option on XBox?
https://github.com/rust-lang/rust/pull/88991 ("Add Nintendo Switch as tier 3 target")
rust-lang/rust#88991 ("Add Nintendo Switch as tier 3 target")
I feel like I need to find more info there. That PR is uhhh it feels kinda sketchy as hell. I think I need to have an offline conversation with them to find out what they intend haha
Ideally, we'd have an organization like "RustOnThisParticularConsole", which would have all the repositories in private, would only accept members who signed the NDA, and allow these Rust community members to collaborate on the ports. Otherwise, Embark does one thing, other team does another thing, single developers re-discover all those things and probably waste a lot of time. Did anybody try to confirm with platform holders if such organization/private repos structure is acceptable?
I spoke to Ryan C. Gordon, the primary maintainer of SDL, which as you might know, has a Switch port in a private Github repo. I asked them how they came to that arrangement, and they said that once they'd done the work of making the port, they spoke to their developer relations contact at Nintendo and asked if the username/password for the repo could be posted on the forums. They agreed to that. Perhaps we could do something similar for adding the Switch as a target for Rust?
We can also create such a repo via the nda'd forums, though we'd want to talk to them before we post any links to it in the forums.
I'm slowly starting studio "Eukaryote Interactive" focused on porting rustc and needed libraries for major consoles but costs of signing NDAs are high, but first we have to discuss technical side:
- Xbox is not using UWP anymore but instead, uses win32 + gdkx which makes it easiest target to port (except that, devkit requires working game prototype or finished one)
- PlayStation is mostly easy to port, rustc have to use Sony's LLVM fork and then you just copy-paste FreeBSD std implementation and add some console specific libs
- Switch uses OS written from scratch which makes it harder to port std
- Libraries to port:
alloc,std,raw-window-handle,wgpu,winitand so on + console specific bindings (wgpu could be upstreamed by using nintendo's public vulkan extensions)
About the costs... let's start from that, after signing NDAs you cannot be a part of any other company (mostly tech ones like some studio, sony, nintendo, microsoft, apple etc.). Not every devkit is free or "easy" to acquire, switch one costs ~$500, xbox requires game prototype or already finished project to be approved and PS5 requires only a company just like others. Costs of maintaining studio in my country (Poland) are ~800ā¬/mo without office place.
Here is the beginning of the real problem, signing NDAs forces to quit your job which slowly makes it even harder to maintain and that I'm currently unable to hire people in order to help with ports of compiler, at least 10 different libraries, prototype etc. which may take much longer. (thus that may require some fundings š¢). If anyone is willing to help, you cannot be employed by any tech company, possibly have own studio or I will talk to console vendors about signing NDAs with them.
EDIT: I forgot to add, code related to any console cannot be hosted as private repo on any public hosting because that NDA violation in some ways. It must be hosted independently on private host without direct access for outsiders (like gitlab/gitea/sourcehut instance, perforce and plastic-scm)
@HeavyRain266 Is this approach an option?
Yes, you need to wrap your game in C (tested on PS5) to sign binaries with Sony's clang and deploy to the store
See this gist: https://gist.github.com/HeavyRain266/824593ae4a961f1c1cf712e1cfc1ffb7
@HeavyRain266 Is this approach an option?
Yes, you need to wrap your game in C (tested on PS5) to sign binaries with Sony's clang and deploy to the store
See this gist: https://gist.github.com/HeavyRain266/824593ae4a961f1c1cf712e1cfc1ffb7
What I was referring to, and what I believe @zicklag intended, is to have an abstraction layer that is independent of any particular console, and which is developed in the open. Specific implementations of this abstraction layer would be behind an NDA.
What I was referring to, and what I believe @zicklag intended, is to have an abstraction layer that is independent of any particular console, and which is developed in the open. Specific implementations of this abstraction layer would be behind an NDA.
Ohh, got it!
Well thats not really possible, there is no need for abstraction layer. What you really and only need is, port of allocator, standard library and then generated bindings used to start porting Bevy Engine by creating console specific backends in libraries such as raw-window-handle, winit, naga and wgpu. Where wgpu port for switch is easiest because you possibly can upstream some public Vulkan extensions which are pre-existing in ash.
All what I meant, is that. None of changes other than possibly Switch extensions in wgpu can be upstreamed as public abstraction layer described above. WGSL to PSSL translation in naga is questionable because specification isn't public and WGSL spec doesn't cover core parts such as mesh shaders and WebGPU doesn't support raytracing just yet.
Ah, I see. Essentially @HeavyRain266, you're saying that we already have platform abstractions in the form of wgpu, winit, etc. All we need is people to make backends compatible with the platform abstractions we already use.
That makes sense to me.