[DEMO: DO NOT MERGE] ugfx: Microcode powered 3D&2D graphics

[snacchus]

This PR mainly introduces a new system called "ugfx", which is short for "Microcode graphics".

As the name suggests, this system is capable of rendering graphics utilizing the RSP. The API and functionality are heavily inspired by libultra, but the implementation is written from scratch. I originally based this on the rudimentary graphics microcode that is part of n64chain, but over time it has evolved into something completely different.

These are the changes made:

• Add include/ugfx.h and src/ugfx.c, which contain the entire API.
• Add src/rsp_ugfx.S, which contains the microcode. For better readability, it includes quite a few additional source files from the src/ugfx directory.
• Add a new example ROM ugfxtest, which showcases a lot of ugfx's functionality. The source code also contains some explanatory comments on the API.
• Add some new functions to display.h, which proved very useful in the example ROM.
• Add a new macro PhysicalAddr, which is needed in some places for specifying DMA addresses for the RSP.
• Modify some RSP assembly macros in rsp.inc. Since most vector Load/store operations shift the offset by the operand size, this is now reflected in the generated instructions. This means that the offset is always specified in bytes, which resembles the original RSP language by nintendo more closely. This will most likely break existing microcode.
• Add some more useful defines in rsp.inc.
• Fix an issue in n64.mk, where RSP assembler invocations were missing some command line flags.

Some notes on this PR:

• I am aware of the recently introduced RSP microcode library and coding guidelines, but since I have started writing the ucode quite a while ago, I have not been able to adapt the code accordingly or make use of the new library. This can be addressed in the future of course.
• As mentioned above, the design of this system is "heavily inspired by libultra". This means a lot of the API is quite similar, but some changes and even a couple of improvements have been made. For some of the trickier parts of the microcode, I resorted to inspecting a disassembled version of the official microcode (which I did not disassemble myself), which acted as a valuable reference implementation for me. I never copied&pasted entire code segments from it, though.
• I have not added any doxygen comments to the API yet. Partly due to lack of time, but also because the API may be subject to change in response to feedback on this PR.
• And finally: The API & implementation are far from perfect and tons of things still need to be improved and optimized. But I just want to get this code out there and gather some feedback since I feel like its current state is fairly presentable 😄

[snacchus]

I just realized that I didn't test the final version of the example ROM on hardware before submitting the PR 🤦 Anyway, I just tested it and I'm getting a floating point exception in ugfx_matrix_from_column_major, probably in the macro expansion of float_to_fixed. ~~Apparently the multiplication is generating an "inexact operation" exception. I looked this up and apparently most floating point operations will generate this exception, and it can be safely ignored. Maybe some flags need to be set in the FPU control register to ignore this exception? I will investigate this further when I have the time.~~ EDIT: Looks like I got the specific exception type wrong. It's actually an "Unimplemented Operation Exception" which in this case is triggered by one of the operands being a denormalized number.

The question is: How should this be handled? The VR4300 Manual says "How to use the unimplemented operation exception is arbitrarily determined by the system", which to me sounds like "it doesn't matter". I looked at how n64chain handles this for reference. Their exception handler seems to ignore all floating point exceptions. Should Libdragon do the same? If so, I could implement an appriopriate check in inthandler.S.

[rasky]

Hi @snacchus thank you very much for the submission! It looks great On first sight. I'll make a first round of review over the weekend.

[snacchus]

I managed to make the example ROM work on hardware. I couldn't verify this in detail, but the floating point exception must have been caused by the perspective matrix not being initialized properly.

If you start the example ROM, you should see a blue rotating cube with some texture on it. I tested it with an Everdrive x7 on a PAL console. There seems to be a slight problem with screen tearing, though.

[rasky]

I managed to make the example ROM work on hardware. I couldn't verify this in detail, but the floating point exception must have been caused by the perspective matrix not being initialized properly.

Thanks, and agreed. Floating point exceptions have helped me find several hairy bugs in the code. It's true they were turned off in the original toolchain, but I think we're doing a much better job at making a easy-to-use toolchain here, including all asserts for invalid arguments that would simply crash the console otherwise. So I think we'd rather keep those exceptions on.

[snacchus]

@rasky Thank you very much for the review! I already pushed some fixes for the minor issues, and replied to all of your other comments. I would say I generally agree with your suggestions.

Regarding the copyright concerns: I was afraid this would be an issue, and I'm not going to deny that the ucode I wrote is quite similar to Fast3D in some places. The DMA function for example is kind of an oversight of mine, but luckily it can just be replaced by the one in rsp_dma.inc.

I suspect the other areas of concern are mainly the output_flush function in output.S and various parts of draw_triangle.S. Admittedly, I think I was quite lazy while implementing output_flush and still don't fully understand what happens there, so that one's on me. We can probably find a better solution together.

As for the triangle drawing function: There are some intricate details about assembling the triangle commands that I just couldn't work out without looking at Fast3D. I'm not sure how to implement something equivalent without just making it look very similar. I hope we can work this out together as well.

Please correct me if I'm missing other parts that were of particular concern. I think generally it would be a good idea to discuss the ucode in closer detail, which maybe GitHub isn't even the best place for. (Maybe now is the time for me to finally join the Discord).

As I commented before, I really like your idea of a more generic, modular microcode system. This got me thinking, and perhaps some of the copyright concerns would disappear if we came up with a different rendering pipeline altogether, more akin to OpenGL for example? I guess I limited myself to replicating what libultra does while implementing this system, which was a mistake in hindsight. I gained a lot of practice in writing ucode though and would be willing to reimplement it as a consequence of big design changes. What do you think?

[rasky]

As for the triangle drawing function: There are some intricate details about assembling the triangle commands that I just couldn't work out without looking at Fast3D. I'm not sure how to implement something equivalent without just making it look very similar. I hope we can work this out together as well.

In these cases, I think we can work together. I haven't seen original microcode, so I think I can come up with a different solution.

Please correct me if I'm missing other parts that were of particular concern.

I will ask. I think in general also adapting it to the libdragon register style (see also my ucode) would make it look different.

I think generally it would be a good idea to discuss the ucode in closer detail, which maybe GitHub isn't even the best place for. (Maybe now is the time for me to finally join the Discord).

If you join the n64brew discord, we can discuss there, sure.

As I commented before, I really like your idea of a more generic, modular microcode system. This got me thinking, and perhaps some of the copyright concerns would disappear if we came up with a different rendering pipeline altogether, more akin to OpenGL for example? I guess I limited myself to replicating what libultra does while implementing this system, which was a mistake in hindsight. I gained a lot of practice in writing ucode though and would be willing to reimplement it as a consequence of big design changes. What do you think?

Yes. I don't want to sound conceited, but I think we can do much better than the original microcode. We have more years of experience in what works on the console and what doesn't, more tools at disposal (eg: I use my own RSP debugger which is very handy), and no commercial deadlines to meet. So I would start from a fresh approach and see where we get with it.

[rasky]

@snacchus I think I've screwed up the rebase. Would you please rebase it on master? I'd love to keep this up to date every once in awhile

[snacchus]

@rasky Done!

[rasky]

We can close it now. The unstable branch features a more complete 3D solution based on the OpenGL API.