[Announcement] [Gdscript] Godot 4.x Client Work in Progress

[Goutte]

Hello !

I started experimenting with a Godot client (GDScript), and albeit there's not much to show right now I wanted to write something here in case anyone else is interested in joining forces. I looked here for the keywords Godot and GDScript.

I'll link a branch here soon.

Rationale

Being able to quickly draw a pure GDScript client for any OAS would be amazing for some online games, even though Godot has its own network engine, because in HTML5 exports only the HTTP client is available (for now). Anyway, there's plenty of uses for HTTP connections to a server in games (auth, leaderboards, news, etc.).

I don't know if GDScript will have all the features we need, we'll see along the way.

Y U NO C ?

Having to compile Godot itself and distribute it to all collaborators is no small task. And I'd rather not have Mono if I can help it.

Design

Some ideas, constraints inherent to Godot, etc. Nothing is frozen yet.

Indent using Tabs

Godot uses tabs by default, although it is configurable.

Java code should be indented like the other generators.

Try not to pollute global space (with class_name) ?

Godot does not have namespaces. (for now)

What should we add in the global namespace (via class_name, or perhaps singleton nodes) ?

If we do decide to use class_name (and pollute global space) for models and api, which I suspect we'll need to get nice type defs, we need to set up for all class names a prefix that will act as a namespace.

We can rather easily do this for models with --model-name-prefix, just need to make sure our users know about it. I found --api-name-suffix but no prefix, although it is used in toApiName(). Probably because of lack of support across targets ? Should I add my own --api-name-prefix CLI option or wait ?

snake_case vs camelCase

I'd love some configuration but I won't bother with it until at least I have a working client.

Static typing

Godot does not allow nullable types (we're waiting for them).

This means we either skip types, or try to figure out how to replace null by the appropriate pseudo-null value for the given type. ("" for String, etc.) But it becomes awkward for int values, where 0 might not be a sensible pseudo-null.

Async

In HTML5 exports, only async is available. Since it is my personal target, I'll favor it.

Godot 4 has Callables and lambda defs, but no Promises yet. I think the best method signature for our generated async API method endpoints would be to return a Promise. But if we start waiting on the promises of Godot… ;)

Therefore, for the first major version of the generator, we'll use Callables in that fashion, unless you have a better idea :

func createUser(
	# … ← generated params from the endpoint schema
	on_success: Callable,  # func(result: User)
	on_failure: Callable  # func(error: ApiError)
):
	pass  # ← method implementation

---

State of Things

I managed to bootstrap the gdscript target, compile and generate the petstore sample with dummy mustache templates. Now we need to fill the mustache templates with working code, one feature at a time.

• [x] Models template basic bones
• [ ] Apis template basic bones
• [x] Basic Java CodeGen child
• [ ] Enums
• [ ] Demo & Test
• [ ] Unit Tests (I usually prefer TDD, but I don't understand yet how these work)
• [x] HTTPClient async constraint
• [ ] HTTPClient reuse through Apis
• [ ] Override endpoints for easier template overriding (perhaps via template partials?)

I'm looking at the other language targets and throwing darts. Wish me luck !

Burning questions

1. Is there somewhere a docker image (or whatever) for a server with a working petstore I can use for my demo during dev ?

   Yes → https://github.com/OpenAPITools/openapi-generator/wiki/Integration-Tests (thanks @wing328)

2. Should I use handlebars straight away or keep using mustache, as mildly recommended ? I'm used to (and fond of) jinja2 and Twig (Pebble, for Java).

   Moved to handlebars because the truthy value of mustache includes "" and "null" and that does not play well with {{#example}}.

[wing328]

Should I use handlebars straight away or keep using mustache, as mildly recommended ? I'm used to (and fond of) jinja2 and Twig (Pebble, for Java).

It's up to you and I think the community gets used to mustache.

[wing328]

Is there somewhere a docker image (or whatever) for a server with a working petstore I can use for my demo during dev ?

Would this help? https://github.com/OpenAPITools/openapi-generator/wiki/Integration-Tests

[Goutte]

The backbone is here and as I'm getting slowly more proficient in OAS code gen, it's time to delve into tests.

[Goutte]

The doc states:

A simple test script/app to test Petstore (e.g. test.php) to serve as a starting point for developers to easily play with the auto-generated SDK

I can't find any test.php in the source, and I'm not sure where in the repo to put my own test file. (same question for the python util to extract reserved words).

Also, for integration testing, I think I can make a single (or more later as needed) GDScript test file and run it from the CLI, and return an error code > 0 if any test failed.

I can wrap the call(s) into a shell script for convenience. Where should I put such files ?

[Goutte]

godot4 --headless samples/client/petstore/gdscript

:tada: I have a test-suite in the form of a Godot project in samples, that I can run in --headless mode and return a meaningful exit code.

Now, the serious business can finally start…

[Goutte]

I'm glad to announce that the integration test can create a user, log in, and create a pet. It's very rough around the edges, but it works.

I'm stuck, though

I'm now using that test to do some refactoring and exploration into namespaces.

I can rather easily prefix and suffix the Api and Models with their already-defined parameters. I'm not doing anything, it just works®.

Yet I have some core files (base class, error) that I want to be able to customize the same way, using for example coreNamePrefix in the YAML config:

generatorName: gdscript
outputDir: samples/client/petstore/gdscript
inputSpec: modules/openapi-generator/src/test/resources/3_0/petstore.yaml
templateDir: modules/openapi-generator/src/main/resources/gdscript
additionalProperties:
  hideGenerationTimestamp: "true"
  apiNamePrefix: Demo
  modelNamePrefix: Demo
  coreNamePrefix: Demo

The property coreNamePrefix shows up in the templates, even if not defined in GdscriptClientCodegenOptionsProvider, but even if defined there I can't figure out how to access it from GdscriptClientCodegen's constructor or processOpts().

additionalProperties.containsKey(CORE_NAME_PREFIX) yields false

I want to change the filename when defining my SupportedFile like so :

supportingFiles.add(new SupportingFile("ApiBee.handlebars", "core", toCoreFilename("ApiBee") + ".gd"));

The method toCoreFilename tries to find the coreNamePrefix additional property in additionalProperties and fails. I'm guessing I'm not looking in the right place, or at the right time, or perhaps I overlooked the adequate override.

This probably ties into GdscriptClientCodegenOptionsProvider whose purpose I do not understand anymore, as I thought was used for precisely such custom parameters like coreNamePrefix.

Any lights on these would be appreciated. @wing328 ?

[Goutte]

It was too baffling ; I should have noticed my confusion earlier, but it's only after re-reading my message above that I thought of the wrong assumption I was making.

public static final String CORE_NAME_PREFIX = "coreNamePréfix";

When I wrote this line, I checked against presence in the template, and io and behold the variable was there, so I went merrily on my way.

Textbook confirmation bias.

Got stuck way too long on a sneaky diacritic, but at least I learned a lot about the internals of the generator along the way :fox_face: !

[Goutte]

I'm going to take some time and consume a real-world OAS with this, before I carry on and freeze things for MR. Might take a few months. I'm also not convinced by the Callable approach, since it yields pretty ugly code on usage. (see the demo)

~~I'm also not comfortable with the feature set configuration, if you have any tips, or any other specific gen I should mirror, I'm all ears.~~

[wing328]

@Goutte thanks for your work on this. Can you please PM via Slack me when you've time?

https://join.slack.com/t/openapi-generator/shared_invite/zt-12jxxd7p2-XUeQM~4pzsU9x~eGLQqX2g

[Frontrider]

At least Godot 3 has a style guide, that is based on how the c++ code is formatted. I don't think it changed for 4, would be the best to follow this.

https://www.reddit.com/r/godot/comments/yngda3/gdstyle_naming_convention_and_code_order_cheat/ https://www.gdquest.com/docs/guidelines/best-practices/godot-gdscript/

[iamarkdev]

Any updates on this @Goutte ?

[saatsazov]

Hello, I found a bug with generating default values for strings. More details: https://github.com/Goutte/openapi-generator/pull/1

The problem:

[wing328]

@Goutte can you please submit a PR so that we can incorporate this new generator into the official repo so that more Godot users can benefit from this and submit PRs to further enhance it?

[saatsazov]

@wing328 Shall we implement more features or we can merge current WIP? I started using this generator for my pet project. So I may found more problem in future. And may address them.

[saatsazov]

Actually there is one more bug, which I haven't fix for now. Just modified generated code. Godot 4 was under active development. Now they seem to freeze changes. (At least they mark some unstable api which they are going to modify in future releases)

The method connect_to_host now accept 3 parameters. Generator currently have 4 parameters here: https://github.com/saatsazov/openapi-generator/blob/feat-gdscript/modules/openapi-generator/src/main/resources/gdscript/ApiBee.handlebars#L86

And this is a latest documentation: https://docs.godotengine.org/en/stable/classes/class_httpclient.html#class-httpclient-method-connect-to-host

Basically as I understood they merge ssl_enabled and verify_host to single tls_options

[Goutte]

Oops ; something unexpected is happening, and I'm not receiving github's notification emails anymore. Sorry for not responding

Thanks @saatsazov for the bug finds ! ~~Do you have a branch with your fixs that I should rebase/merge ?~~ (done, thanks!)

I'll look into wrapping this up for a PR now, it will help me procrastinate the writing of a CV. :roll_eyes:

[Goutte]

@saatsazov I've added support for TLS and some underscores to method names, to mark them as protected. It's here : https://github.com/Goutte/openapi-generator/pull/2

It's a breaking change ; there might be more before the MR, since I frown at the current API. (even the bee prefix is bad, since :bee: may be a legit model)

[saatsazov]

Hello @Goutte,

No worries about breaking changes. My project is not on a stage to worry about that. But thanks for worried mentioning breaking changes.

I actually started doing some quick and dirty fixes in my fork. One of the biggest problem I encountered is connected with connection. When you start doing request. the whole game is very laggy. So what I did is started a separate thread. I suppose it should be some better way to do so.

I tracked problem to this loop https://github.com/Goutte/openapi-generator/blob/13d3c5c79d1f2a4bf4a464a00a59d1b9d68f1793/modules/openapi-generator/src/main/resources/gdscript/ApiBee.handlebars#L100

The delay seems to block main thread. So as a quick solution I added a thread.

As you mentioned in a comments here https://github.com/Goutte/openapi-generator/blob/13d3c5c79d1f2a4bf4a464a00a59d1b9d68f1793/modules/openapi-generator/src/main/resources/gdscript/ApiBee.handlebars#L64

there is no such thing as idle frame. So we should think how to approach this. I may provide an example project later if you want.

[Goutte]

Good point !

Whatever the await, it's probably not a good idea to do network on the main thread, right ? We can provide a func xxxx_threaded(…) -> Thread that wraps the api call and returns a Thread, perhaps ?

---

Inline objects hurdle

TLS works, but now I need to support inline objects like these:

  /authentication_token:
    post:
      operationId: login_check_post
      tags:
        - 'Login Check'
      responses:
        200:
          description: 'User token created'
          content:
            application/json:
              schema:
                type: object
                properties:
                  token: { readOnly: true, type: string, nullable: false }
                required:
                  - token
      summary: 'Creates a user token.'
      requestBody:
        description: 'The login data'
        content:
          application/json:
            schema:
              type: object
              properties:
                username:
                  type: string
                  nullable: false
                password:
                  type: string
                  nullable: false
              required:
                - username
                - password
        required: true
    parameters: []

The login_check_post API method will require a GoasLoginCheckPostRequest object.

I looked at how typescript did it in modules/openapi-generator/src/main/resources/typescript/types/ObjectParamAPI.mustache, but Godot works best with one class per file (things get pretty weird & buggy for inline public classes).

[saatsazov]

At the moment I can't say how would be better to work with async. So we can address this issues later when we have more details.

Speaking of inline object I use this generator for PHP project and this is how it looks like. It generate a bunch of of classes per each inline request and response. This may mitigate the problem you mention one class per file.

[Goutte]

I've added rudimentary support for inline objects (request/response, mainly).

Still not sure what to put in the await… I'm also considering adding the Config to the Api's init() instead of injecting bee_config. Godot recommends not using init params, but that's only for Resources, if I remember correctly. (and Api is a Reference)

[Goutte]

Breaking

• replaced bee for bzz, less collision chance
• Apis now receive config and client via constructor

@saatsazov let's break as much as we can before MR, if you have concerns or refacto ideas, go full EAFP while nothing is frozen !

Testing

I remember managing to set up a test project with a custom MainLoop or something, for automated integration testing, but it's complicated to set up as it requires Godot to run, and docker for the petstore server. Unit tests don't feel as relevant, though.

[Frontrider]

Breaking

* replaced `bee` for `bzz`, less collision chance

* Apis now receive `config` and `client` via constructor

@saatsazov let's break as much as we can before MR, if you have concerns or refacto ideas, go full EAFP while nothing is frozen !

Testing

I remember managing to set up a test project with a custom MainLoop or something, for automated integration testing, but it's complicated to set up as it requires Godot to run, and docker for the petstore server. Unit tests don't feel as relevant, though.

Gdunit+docker should be fine imo.

[Goutte]

Gdunit+docker should be fine imo.

0. Thanks for caring ;)
1. I meant unit tests in java ; I've looked at what other targets did, and we can have some but they will provide little value
2. I usually favor GUT over GdUnit for my games, but it's not a strong preference ; do you have reason to believe GdUnit is more appropriate in our case ?

I'll look today into refactoring the test project (in samples/client/petstore/gdscript) to put the generated client into an addon instead of the root, and adding a proper assertion engine.

[Frontrider]

I usually favor GUT over GdUnit for my games, but it's not a strong preference ; do you have reason to believe GdUnit is more appropriate in our case ?

I like the assertions more, to me they are more readable. Nothing else.

I meant unit tests in java ; I've looked at what other targets did, and we can have some but they will provide little value

Executing the generated code in godot as a functional test would have value imo. That can also be launched from java if needed.

[Goutte]

Executing the generated code in godot as a functional test would have value

Definitely ! So far we can register, authenticate and CRUD a monkey on the petstore. I tried using the new echo server, first, but there's a collision with the word color, so I'm using ye olde petstore for now.

The command line is still a bit complex, but it passes (provided the dockerized petstore is running):

godot --debug --headless --path samples/client/petstore/gdscript --script addons/gut/gut_cmdln.gd Yes, it's GUT, sorry about that ; started hacking around before you answered ; we can have both, or switch

Breaking soon

I'm currently wrapping the object sent back into the on_success callback into an ApiResponse which will hold the response HTTP details (code, headers, etc.) as well as the deserialized data. (if the client detects a model and manages to deserialize it)

Even if we don't provide the full response analysis for now, changing the callback signature now will help us implement these features later on without breaking.

[Frontrider]

Yes, it's GUT, sorry about that ; started hacking around before you answered ; we can have both, or switch

I don't feel anyhow about it. I'll most likely just be a consumer anyways. :sweat_smile:

[Goutte]

Now I remember why I had not added a test engine but made a minimalist (disgusting) one. Oh well… The review of this target is going to be more painful, but I guess it's worth it ?

---

@saatsazov Does it still lag a lot when you set polling_interval_ms = 0 in your ApiConfig instance ?

So far I'm using the client in menus, but in a few months I will start using it during heavy 3D rendering as well, so I will inspect the async/threading issue more closely then.

I'll let this sit until then (I fixed my email notification issue), perhaps I will fix/implement the response's character set detection, but that should not be a breaking change.

Latest draft has been merged in feat-gdscript.

[Goutte]

Also, this warrants interest : https://github.com/D2klaas/Godot-4-HTTPManager

[jchu231]

I am interested in helping test out this generator. I am trying to utilize feat-gdscript via @openapitools/openapi-generator-cli, but am having trouble enabling your generator with it, any guidance would be awesome!

And thanks @Goutte for your work on this generator!