glm icon indicating copy to clipboard operation
glm copied to clipboard

Multiplatform?

Open Dominaezzz opened this issue 6 years ago • 60 comments

I want to use this library in a multiplatform application. Since the bulk of this library is written in pure Kotlin, going multiplatform would be fairly trivial. The only possible issue I can think of is that JitPack may not work with it, breaking current users. Would need to setup build for all platforms to publish to Bintray or Maven central.

Dominaezzz avatar Jan 26 '19 16:01 Dominaezzz

With pleasure, however JitPack is enough for me at the moment

I know Bintray and Maven require going throw so much pain that makes me giving up.

But if you want to try, you are very welcome.. I sent you an invite :)

elect86 avatar Jan 29 '19 18:01 elect86

Ah sweet, I'll give it a try. I also just noticed it has a dependency on kotlin-unsigned, which will have to go multi-platform too. Unless you're thinking of migrating to kotlin's unsigned types.

Dominaezzz avatar Jan 29 '19 18:01 Dominaezzz

There is another problem you have to solve first. I think the way we handle native memory is dependent on lwjgl or at least dependent on java.nio. I don't think there is a multiplatform version of any of those. I think it should be possible to move all those dependencies to the jvm version, but I'm not sure how much work this is actually.

Wasabi375 avatar Jan 30 '19 11:01 Wasabi375

Well, I just did a search for ByteBuffer and it looks like there are a couple options. Migrate them to use kotlinx.io.core.IoBuffer which is multi-platform (except for android native), abstract away the JVM specific bits into a separate jvm "extension" library or only implement the java.nio related parts in the JVM source set.

How much do you care about breaking changes?

Dominaezzz avatar Jan 30 '19 11:01 Dominaezzz

I personally don't care. If we go multiplatform IoBuffer seems like the best choice for the future even though it's still experimental. As for breaking changes, I don't think they will be any problem.

Wasabi375 avatar Jan 30 '19 18:01 Wasabi375

Oh man, I lost the reply mail..

anyway @Dominaezzz, the limitation with the stdlib unsigned is that they dont extend Number because inline classes cant extend classes, I explicitely asked to change Number to interface, but my request was turned down because it didnt seem like an issue for them

Extending Number is cool because you can pass it as generic and then converting to the type you want.

We may discuss if this is actually useful also for unsigned.. maybe we could switch to stdlib unsigned and make unsigned vectors somehow special compared to the others.

Regarding breaking changes, I personally repute that if the change makes sense, then we should just go for it

Although I'd pay attention for any regression performance-wise

elect86 avatar Feb 05 '19 14:02 elect86

any news, @Dominaezzz ?

elect86 avatar Dec 10 '19 14:12 elect86

I made a branch with some changes. Have you looked at it?

Dominaezzz avatar Dec 10 '19 15:12 Dominaezzz

The current code is a bit of a tangle of interfaces, which is quite bothersome to make multiplatform. Would be easier to wait for io-2 or make the library more kotlin "friendly" (less like the C api). The former being easier.

Dominaezzz avatar Dec 10 '19 15:12 Dominaezzz

I made a branch with some changes. Have you looked at it?

Yep, they are only confined to gradle or?

The current code is a bit of a tangle of interfaces, which is quite bothersome to make multiplatform. Would be easier to wait for io-2 or make the library more kotlin "friendly" (less like the C api). The former being easier.

I'm totally available to change the current structure if it makes sense.

By io-2 do you mean kotlin.io? is there already an ETA?

elect86 avatar Mar 06 '20 12:03 elect86

Yes, the new revamped kotlinx-io. It should be released sometime around Kotlin 1.4 release.

Oh, maybe I didn't push. Structural changes would be required, it's been a while so I don't remember what exactly needed to change. I just remember pulling interfaces into common code required pulling so many other things at once.

Also, dependencies would also need to be made multiplatform. unsigned won't work well with Kotlin native, there would be a fair bit of copying involved.

It's a bit of work.

Dominaezzz avatar Mar 06 '20 15:03 Dominaezzz

Yes, the new revamped kotlinx-io. It should be released sometime around Kotlin 1.4 release.

Where is possible to read something about that?

Oh, maybe I didn't push. Structural changes would be required, it's been a while so I don't remember what exactly needed to change. I just remember pulling interfaces into common code required pulling so many other things at once.

What would you do then? Top functions inside the glm package?

Also, dependencies would also need to be made multiplatform. unsigned won't work well with Kotlin native, there would be a fair bit of copying involved.

Problem with the stdlib unsigned is that they do not extend the Number abstract class..

We could keep unsigned a jvm-only dep

elect86 avatar Mar 06 '20 18:03 elect86

Where is possible to read something about that?

The most official thing I can point to is the Kotlin 1.4 blog. Other places would be hand wavy estimates from Kotlin devs in Slack.

What would you do then? Top functions inside the glm package?

You would need to make a lot of the member functions into extension functions or top-level functions. This would make the library not very nice to use from Java. You could start moving the smaller interfaces into common code, I'm not very familiar with the library so I wouldn't know exactly where to start.

Problem with the stdlib unsigned is that they do not extend the Number abstract class.

I understand the need for this but using Number means boxing very often. Also, I guess there isn't much Kotlin devs can do about it, it's a Java api. :man_shrugging:

Dominaezzz avatar Mar 06 '20 21:03 Dominaezzz

You would need to make a lot of the member functions into extension functions or top-level functions. This would make the library not very nice to use from Java.

No other ways?

Otherwise we can theoretical have the top level functions as the real multiplatform core and the jvm interfaces would just point to those (maybe inlined)

You could start moving the smaller interfaces into common code, I'm not very familiar with the library so I wouldn't know exactly where to start.

Ok, let's start with the projection matrix functions, if you can prepare the structure and point me to the right location I'll move a couple of those

I understand the need for this but using Number means boxing very often. Also, I guess there isn't much Kotlin devs can do about it, it's a Java api. 🤷‍♂

Yeah, but the convenience is huge. Otherwise we shall write a lot of boilerplate code (or auto-generate)

elect86 avatar Mar 09 '20 09:03 elect86

Hi @Dominaezzz, still interested in this? From my side there is plenty of availability

I've been notified of some rising interest into multiplatform libraries for images and fonts (@Sylvyrfysh).

I'd love to merge all the spare efforts in an unique one.

Also, dependencies would also need to be made multiplatform. unsigned won't work well with Kotlin native, there would be a fair bit of copying involved.

It's a bit of work.

We can skip unsigned for the moment and postpone any decision/evaluation for later

You would need to make a lot of the member functions into extension functions or top-level functions. This would make the library not very nice to use from Java.

What is preferred among the two?

We can always add an additional layer/wrapper explicitely for Java to make it play nicely with it

You could start moving the smaller interfaces into common code, I'm not very familiar with the library so I wouldn't know exactly where to start.

We can start by the matrixClipSpace extension with functions to create ortho and perspective projection matrices, they are first methods used in the simplest scenarios.

What do you think?

elect86 avatar May 14 '20 08:05 elect86

What is preferred among the two?

I think extension functions. (Might have to be on a case by case basis).

We can start by the matrixClipSpace extension with functions to create ortho and perspective projection matrices, they are first methods used in the simplest scenarios.

We could start there, after pulling out Mat4 into common.

still interested in this?

I am but I'm not available to do the work on it.

Dominaezzz avatar May 14 '20 15:05 Dominaezzz

I think extension functions. (Might have to be on a case by case basis).

Ok, then extension functions be

We could start there, after pulling out Mat4 into common.

Fine, but I have no idea where to start from

I am but I'm not available to do the work on it.

It's ok, I can do the part about writing the mathematical code (methods, classes and so on), but I have no experience in native/MP, that's your knowledge I guess

Could you take care of the project structure?

I just deleted everything in the multiplatform branch, created a dummy Mat4 and a dummy ortho

elect86 avatar May 14 '20 18:05 elect86

What backends do you want, @Dominaezzz ? The code I've started working with is aimed at everything, do you also want everything or is a subset what you want?

Sylvyrfysh avatar May 14 '20 23:05 Sylvyrfysh

Everything really, I don't expect this to require anything platform specific.

Dominaezzz avatar May 14 '20 23:05 Dominaezzz

I gave it a refresh, @Dominaezzz

I started with the gradle build file, moved to kotlin script, followed the tutorial on the website removed the glm-test module and all external dependencies except lwjgl ones

I have some problems understanding how to distinguish common code from the jvm specific one and how to set that in gradle

if you could help on that, it'd be great

elect86 avatar May 25 '20 12:05 elect86

Reading through the thread and seeing what I can do to help. Some thoughts:

  • Is this library intended to be a Kotlin-only port of GLM? Given that the repository is under kotlin-graphics I would guess so, but I also read a lot above about using this library from Java, which makes the multiplatform implementation more difficult, if not impractical. What's your goal?
  • With unsigned types built into the stdlib for a while now, why are we bothering with kotlin-unsigned?
  • With SIMD support for native targets since 1.3.70, a split is necessary between native and non-native definitions to get the best performance. Possibly even a separate native implementation just for SIMD, in case there are native platforms that don't support SIMD

nlbuescher avatar May 25 '20 20:05 nlbuescher

  • the initial scope was that. But nothing stops us from extending it and adding new functionalities, like I myself did

We can forget about Java for the moment. If you need the methods as extension function, this is no problem at all.

However I can see a solution to that. We can define all our methods as extension functions as Dominic suggested and then I'll add an additional wrapper exclusively for the jvm, to make it play nice with Java (at least the core functions, already today there are stuff, like custom vector swizzle, which is basically kotlin-only).

My final goal is to have a library which helps with 3d real time graphics. Having something resembling the standard de-facto library on C/C++ world is a huge adavantage.

  • the problem with the stdlib unsigned is that they do not extend the abstract class Number (which is a technical obstacles for inline classes), I tried to ask Jetbrains for a switch and make it as an interface, but I was told there were not enough usage cases for that. Hopefully they will change their mind if we ask it all together.

The advantage of an unsigned extending Number, is that, of course, one can have a generic method accepting it and then you convert it internally to what you need there.

But again, this is not a strict requirement

  • I'm all for it. SIMD for all the supported targets, native and jvm (it's coming with panama, vectorIntrinsics)

I'll send you an invitation :)

elect86 avatar May 26 '20 06:05 elect86

  • the initial scope was that. But nothing stops us from extending it and adding new functionalities, like I myself did

We can forget about Java for the moment. If you need the methods as extension function, this is no problem at all.

However I can see a solution to that. We can define all our methods as extension functions as Dominic suggested and then I'll add an additional wrapper exclusively for the jvm, to make it play nice with Java (at least the core functions, already today there are stuff, like custom vector swizzle, which is basically kotlin-only).

That's a good approach, I think

My final goal is to have a library which helps with 3d real time graphics. Having something resembling the standard de-facto library on C/C++ world is a huge adavantage.

Definitely.

  • the problem with the stdlib unsigned is that they do not extend the abstract class Number (which is a technical obstacles for inline classes), I tried to ask Jetbrains for a switch and make it as an interface, but I was told there were not enough usage cases for that. Hopefully they will change their mind if we ask it all together.

The advantage of an unsigned extending Number, is that, of course, one can have a generic method accepting it and then you convert it internally to what you need there.

But again, this is not a strict requirement

As Dominic mentioned above, using generics actually causes automatic boxing (ie, the storage type of Vec3 and Mat4 will end up being Object on the JVM, and possibly elsewhere). In order to avoid that we'd actually have to define the various vector and matrix types separately, and depending on how many there are, it might be best to do that through code generation using something like kotlinpoet. That would eliminate the need for the Number extend.

I'll take a look what I can do about the Multiplatform build

nlbuescher avatar May 26 '20 14:05 nlbuescher

As Dominic mentioned above, using generics actually causes automatic boxing (ie, the storage type of Vec3 and Mat4 will end up being Object on the JVM, and possibly elsewhere). In order to avoid that we'd actually have to define the various vector and matrix types separately, and depending on how many there are, it might be best to do that through code generation using something like kotlinpoet. That would eliminate the need for the Number extend.

Yep, that would do eliminate it

However it might be worth starting small with a manual writing of some classes and see how it plays on different platform before trying to scale up and use kotlinpoet (are you familiar with it though?)

elect86 avatar May 26 '20 16:05 elect86

definitely. yeah I've used Kotlin poet before with the kgl implementation, but I have to admit it's a pain to use codegen in general, but it's also a pain to have to write everything manually.

nlbuescher avatar May 26 '20 16:05 nlbuescher

Yeah, which platform(s) in particular are you interested in?

elect86 avatar May 26 '20 16:05 elect86

I'm personally working with Kotlin/Native (Linux, macOS, and Windows), but with common code, it's trivial to run on any platform.

nlbuescher avatar May 26 '20 16:05 nlbuescher

@elect86 what was the reason for choosing glm_ as a package name instead of simply glm?

nlbuescher avatar May 26 '20 19:05 nlbuescher

Clashes with the glm object

elect86 avatar May 26 '20 19:05 elect86

I'm having difficulty getting a multiplatform configuration to compile because of java module shenanigans, probably caused by lwjgl. What exactly is the purpose of the lwjgl dependency?

EDIT: To be clear, the compiler is crashing for reasons unknown.

nlbuescher avatar May 26 '20 20:05 nlbuescher