[FEATURE] UI: introduce javascript hydration.

[thibsy]

Hi @Amstutz and @klees,

Since it's been a while, I will quickly recap for you and the general public as well:

This project has come to life upon stumbling over an issue with the client-side rendering process for ILIAS\UI\Component\Input\Field\HasDynamicInputs.

While migrating the file input to an ES6 module, we have realised that we cannot run additional JavaScript code provided by a template (component). This is a problem, because some components need client-side initialisation to fully implement their functionality. This is caused by our lookup process being unidirectional, meaning we can find a components HTML structure by searching the DOM after its HTMLElement.id, but we cannot search JavaScript code by looking up this ID somewhere.

To address this issue, I have come up the idea of introducing the concept of (re)hydration to the ILIAS UI framework. The main purpose of this concept is to keep application state persistent between a frontend and a backend. However, this PR currently only aims to implement the fundamentals needed to "hydrate" a component on the client, which is essentially how components are initialised. This mechanism has allowed a different sort of coupling between a components HTML and JavaScript code, so when hydrating, components can theoretically reuse the same JavaScript code.

When implementing this mechanism I tried to be minimally invasive, but as you can see, many unit-tests are currently failing. This is because each JavaScriptBindables HTML structure has changed, because we are now centrally adjusting the output of such components and need to register some metadata as data-* attributes.

This PR is still a work-in-progress, because I have run into a lot of issues while trying to implement this. Not all of them are properly solved right now. However, before we continue to tackle some more specific issues, I believe it makes sense to evaluate this concept and the first implementation of the hydration process again.

Now lets dive into the concrete changes:

• AbstractComponentRenderer::renderComponent(): At first, I thought we could fully centralise the "dehydration" process, the server-side counterpart to client-side hydration, which adds the necessary data-* attributes. Therefore, I implemented the Renderer::render() method directly in the abstract component renderer and delegated the actual rendering to a new abstract method. I later noticed that we cannot fully centralise the dehydration process (yet), so this change isn't actually necessary right now - but it will be in the future. Since this is a neat improvement anyways I kept this in already.
• AbstractComponentRenderer::dehydrateComponent(): This method, as already mentioned, is the counterpart for our client-side hydration. At the moment, this method only modifies the given JavaScriptBindable component so the client-side initialisation is added. Note that dehydration is actually the process of converting a components state and inject it into the output, so the client can pick-up where the server has left off. Right now this is not the case, but since we want to embrace this concept in the future, I already named this function accordingly.
• $id_binders: As you can see in the method above, I added a id-binder argument, which can be utilised by renderers so they can provide a function which sets the HTMLs ID. For some components, this currently contains the entire rendering process, for others this just sets the ID variable. This argument is merely for backwards compatibility. It can be dropped in the future, once every single component has decoupled their JavaScript code from the HTML elements ID.
• New templates: For some components I needed to add some additional templates to make them compliant with the new id-binder mechanism. I also needed to wrap all legacy components in an additional div, because we cannot know for sure if the content will consist of only one top-level HTML element. This will most likely lead to some issues I have not yet discovered, but IMO the component should not be used anyways.
• tpl.javascript_bindable.html: I added a template which is used to ship a JavaScriptBindable components JavaScript code alongside its HTML. I realised however, as long as we allow consumers to provide custom JavaScript code, we will most likely need to distinguish between asynchronous and synchronous rendering. This is due to the fact that the JavaScript code shipped in a <script> tag will be executed immediately once the DOM is parsed, but the facilities required to register hydration functions is not yet loaded. This can be omitted by registering all or specified resources in the HTML <head> already, but I would rather like to think about the way we ship JavaScript code again. Do you have any opinions on this? At the moment, I will parse the JavaScript into an if-else, where it's either immediately added to the registry, or on load.
• il.UI.core.hydrateComponents(): This function is highly inspired by Reacts hydrateRoot() function. It can be used to hydrate components itself and/or contained inside of an HTMLElement. This flexibility allows to gradually initialise our client in the future, maybe according to e.g. the island architecture, where components priority is determined by most actively used areas. An important note to add here, is that we need to think about organising our client-side initialisation process. I noticed when implementing this, that many components did not work, because they relied on other components being initialised first. Prime example for this is the mainbar: as long as the mainbar items are not initialised, initialising the mainbar will lead to an empty view because items register themselves in the mainbar's construct. I solved this issue temporarily by ordering the hydration registry by the order of which components are rendered on the server. This problem is caused because in JavaScript, the DOM is searched downwards, whereas our components are rendered the other way around, due to the recursive nature of our rendering process. Maybe this solution is already it, but maybe we can organise this in a more sophisticated manner.
• il.UI.core.AsyncRenderer: I have replaced the il.UI.core.replaceContent() function and its usages by a new asynchronous renderer. This had to be done, since asynchronously rendered components will need to be hydrated using hydrateComponents() now.

Implementing these changes has led to the following issues:

• Mainbar "more" button: For some reason, and for the life of me I could not see why, the "more" button in the mainbar is always being displayed - even if all entries are visible.
• Popovers: Their example page looks extremely broken, but this appears to be a problem which is not related to this PR. Regardless, the component cannot be hydrated properly because there is no HTML being rendered for this component. I did not look into this because the library which is used for this has not been recommended for ILIAS 10, and its probably not really necessary too.
• There is something wrong with the footer, again, I could not see why. I believe this has probably something to do with CSS breaking due to adding more <script> tags, but this is just an assumption.
• Probably many more issues I could not yet find, because I only looked at our UI examples.

Before we continue, I would like to hear your feedback on the current state of this project. If you like, we can talk about this in a Hangout-session on Discord.

Also note, the initial issue with dynamically rendered components is not yet fixed, but this serves as a baseline to address this issue now.

Kind regards, @thibsy

[klees]

Hi @thibsy,

for documentation purpose: We discussed the changes in the PR and found some iterations we could do before implementing the big change. These would be valuable even without the big change. The general idea is: Get rid of some old complexity before adding new complexity. These changes are;

• Improve AbstractComponentRenderer along the lines presented here.
• Improve the input rendering to simplify the complicated handling of html ids. We should try to get rid of wrapInFormContext, maybe by using ilTemplate behaviour for wrapping instead of php functions.
• Change the interface of withAdditionalOnLoadCode. Instead of demanding a php closure that takes some html id, demand a string that contains a js-function over some html element. Make the ui framework pass that element to the function.

I like the general direction of this proposal. I think it would be valuable to make some changes to our code that simplify the change we are ultimately targeting. Thanks for working on this, feel free to contact me anytime for further discussions.

Kind regards!