KallistiOS
Drop KOS symbols export list
Instead of having a list of KOS symbols to export when the INIT_EXPORT init flag is set, we defer the responsability of providing the exports list to the applications themselves.
This is the best solution as the list of symbols was completely arbitrary, and would give the false impression of a stable ABI.
Instead of having a list of KOS symbols to export when the
INIT_EXPORTinit flag is set, we defer the responsability of providing the exports list to the applications themselves.This is the best solution as the list of symbols was completely arbitrary, and would give the false impression of a stable ABI.
I generally agree, but in order to take away what was already there, you need to give something in return. For example, an autogenerator from headers or something like that. Otherwise, it turns out that you now need to manually go and fill in all these functions anew each time for everyone. Although in fact, most of the functions were written there, I tried to correct this list whenever possible.
All you need to do is copy the contents of KOS master's exports.txt (and/or kernel/arch/dreamcast/export-*.txt) into your own exports file.
All you need to do is copy the contents of KOS master's
exports.txt(and/orkernel/arch/dreamcast/export-*.txt) into your own exports file.
You removed exports files from KOS, that the main issue for future usage. Making it difficult for other users to enter. It's not just me.
I don't think having an auto-generator is a good idea. Because then you would export every single KOS function, which means that every single KOS function (along with the data it uses etc.) ends up in the executable even if you don't actually need to call any of them, wasting lots of RAM.
Having to "manually go an fill these functions" is what we've been doing all along and it's been a pain to maintain. I'm just switching the responsibility for doing so from KOS to applications.
I don't think having an auto-generator is a good idea. Because then you would export every single KOS function, which means that every single KOS function (along with the data it uses etc.) ends up in the executable even if you don't actually need to call any of them, wasting lots of RAM.
Having to "manually go an fill these functions" is what we've been doing all along and it's been a pain to maintain. I'm just switching the responsibility for doing so from KOS to applications.
I don't think having an auto-generator is a good idea. Because then you would export every single KOS function, which means that every single KOS function (along with the data it uses etc.) ends up in the executable even if you don't actually need to call any of them, wasting lots of RAM.
Having to "manually go an fill these functions" is what we've been doing all along and it's been a pain to maintain. I'm just switching the responsibility for doing so from KOS to applications.
The user will choose whether he needs an auto-generator or not. Not everyone has a shortage of RAM in their projects. And someone, like me for example, needs to export all functions.
Well, maybe you're right, it's not up to me to decide. I don’t really understand why this needs to be touched at all now, since there is no alternative solution that won’t further aggravate this problem with exports. Moreover, for those who do not use the INIT_EXPORT flag, this will not add memory in any way.
Yea, I'd have to agree with SWAT that without some other sort of mechanism to replace the functionality there's not a sufficient reason to drop it while it's actively being used. In an alternate situation where there was no dreamshell, I'd say this is fine as it is clunky and there's got to be a better way to go about it. But it's not even this, but the fact that we throw everything into public headers with good documentation even when it's internal AND don't break the API without good reason and consideration for compatibility that give the false impression of a stable API.
I think as we move towards having a more clearly API layers, there should be a point where an autogenerator could pull from <kos.h> to properly generate a public API but we've still got quite a bit of refactoring and cleanup to get to that point.
Was there some concern of RAM usage with the current system? Because if there's still parts that don't get dropped even with it gated by the INIT flag, then we should certainly take some action on that.
Yea, I'd have to agree with SWAT that without some other sort of mechanism to replace the functionality there's not a sufficient reason to drop it while it's actively being used.
What mechanism is there to replace exactly? Right now the exports list is edited manually, it's not like we have it automatized. Literally the only thing that would change, is going from SWAT having to PR changes to the export list according to what he uses in DreamShell, to him updating the same exports list in his DreamShell repo.
In an alternate situation where there was no dreamshell, I'd say this is fine as it is clunky and there's got to be a better way to go about it. But it's not even this, but the fact that we throw everything into public headers with good documentation even when it's internal AND don't break the API without good reason and consideration for compatibility that give the false impression of a stable API.
I think as we move towards having a more clearly API layers, there should be a point where an autogenerator could pull from
<kos.h>to properly generate a public API but we've still got quite a bit of refactoring and cleanup to get to that point.
As I wrote above, if you pull everything from <kos.h> to the exports list, you'd greatly increase RAM usage, so I don't think it's a good idea.
Was there some concern of RAM usage with the current system? Because if there's still parts that don't get dropped even with it gated by the INIT flag, then we should certainly take some action on that.
If all the API functions (arch-specific and arch-agnostic) are listed in the exports list, then you generate a huge table containing pointers to these functions. As a result all these functions aren't garbage-collected and end up in the executable even if they never get called. The init flags don't change anything there.
Moving the responsibility of providing the exports list to the application also means that the application developer can choose to only list the symbols it needs and their dependencies, instead of pulling megabytes of stuff it doesn't need.
Yea, I'd have to agree with SWAT that without some other sort of mechanism to replace the functionality there's not a sufficient reason to drop it while it's actively being used.
What mechanism is there to replace exactly? Right now the exports list is edited manually, it's not like we have it automatized. Literally the only thing that would change, is going from SWAT having to PR changes to the export list according to what he uses in DreamShell, to him updating the same exports list in his DreamShell repo.
In an alternate situation where there was no dreamshell, I'd say this is fine as it is clunky and there's got to be a better way to go about it. But it's not even this, but the fact that we throw everything into public headers with good documentation even when it's internal AND don't break the API without good reason and consideration for compatibility that give the false impression of a stable API. I think as we move towards having a more clearly API layers, there should be a point where an autogenerator could pull from
<kos.h>to properly generate a public API but we've still got quite a bit of refactoring and cleanup to get to that point.As I wrote above, if you pull everything from
<kos.h>to the exports list, you'd greatly increase RAM usage, so I don't think it's a good idea.Was there some concern of RAM usage with the current system? Because if there's still parts that don't get dropped even with it gated by the INIT flag, then we should certainly take some action on that.
If all the API functions (arch-specific and arch-agnostic) are listed in the exports list, then you generate a huge table containing pointers to these functions. As a result all these functions aren't garbage-collected and end up in the executable even if they never get called. The init flags don't change anything there.
Moving the responsibility of providing the exports list to the application also means that the application developer can choose to only list the symbols it needs and their dependencies, instead of pulling megabytes of stuff it doesn't need.
The main problem is that if a user starts compiling something, they'll get a ton of errors about missing this and that, and they'll have to spend a lot of time writing all of them into exports. Not an easy start. And then they'll have to keep them up to date with every update. Does he really need that? And again, not everyone has memory issues. Some people prefer an automatic parser to avoid the hassle, and it has enough memory for the table and the entire kernel. Generally, memory issues only occur in heavy games; in all other cases, there's usually enough memory. I have about 2200 more exports in DreamShell, in addition to those in KOS and it works great, and I have plenty of memory. So I think there should be a choice: automatically parse everything or nothing and leave the choice to the developer.
The main problem is that if a user starts compiling something, they'll get a ton of errors about missing this and that, and they'll have to spend a lot of time writing all of them into exports. Not an easy start. And then they'll have to keep them up to date with every update. Does he really need that? And again, not everyone has memory issues. Some people prefer an automatic parser to avoid the hassle, and it has enough memory for the table and the entire kernel. Generally, memory issues only occur in heavy games; in all other cases, there's usually enough memory. I have about 2200 more exports in DreamShell, in addition to those in KOS and it works great, and I have plenty of memory. So I think there should be a choice: automatically parse everything or nothing and leave the choice to the developer.
I think both @DC-SWAT and @pcercuei have valid concerns here. Given that we already have a toggle in environ.sh for position-independent code for this scenario, perhaps we create a flag there to enable/disable the exports? Since position-independent code is disabled by default, the exports would be too?
The main problem is that if a user starts compiling something, they'll get a ton of errors about missing this and that, and they'll have to spend a lot of time writing all of them into exports. Not an easy start. And then they'll have to keep them up to date with every update. Does he really need that? And again, not everyone has memory issues. Some people prefer an automatic parser to avoid the hassle, and it has enough memory for the table and the entire kernel. Generally, memory issues only occur in heavy games; in all other cases, there's usually enough memory. I have about 2200 more exports in DreamShell, in addition to those in KOS and it works great, and I have plenty of memory. So I think there should be a choice: automatically parse everything or nothing and leave the choice to the developer.
I think both @DC-SWAT and @pcercuei have valid concerns here. Given that we already have a toggle in
environ.shfor position-independent code for this scenario, perhaps we create a flag there to enable/disable the exports? Since position-independent code is disabled by default, the exports would be too?
It seems to me that autogeneration should be enabled by default, so that it doesn't cause problems for beginners, and advanced users can consciously turn it off themselves. Although if it takes a lot of time, then of course it would be better to disable it by default. Linking of exports is only enabled if you add the flag INIT_EXPORT, that's by default is not used.