Send object with functions via uibuilder.onChange()

[TobiasKorn]

Current Behavior

I send in NodeRed the following to the UIB node:

return {
    payload: function() {console.log("Test")},
    _socketId: msg._socketId
}

Using the uiChange node

uibuilder.onChange('msg', function(msg) {
	console.log(msg)
}

the payload property is not shown in the browser console.

Expected Behavior

The problem is that i want to be able to send functional components from NodeRed to UI Builder, when i for example use ChartJS. The option and plugins object of ChartJS can contain callback functions.

Steps To Reproduce

Test example flow below

Example Node-RED flow

[
    {
        "id": "d2e69f2bb17aea8b",
        "type": "uibuilder",
        "z": "5b8cb9d23554e7f9",
        "name": "",
        "topic": "",
        "url": "uib",
        "fwdInMessages": false,
        "allowScripts": false,
        "allowStyles": false,
        "copyIndex": true,
        "templateFolder": "iife-blank-client",
        "extTemplate": "",
        "showfolder": false,
        "reload": false,
        "sourceFolder": "src",
        "deployedVersion": "6.6.0",
        "showMsgUib": false,
        "title": "",
        "descr": "",
        "x": 1150,
        "y": 180,
        "wires": [
            [],
            [
                "e28a0bee3a80de11"
            ]
        ]
    },
    {
        "id": "e28a0bee3a80de11",
        "type": "function",
        "z": "5b8cb9d23554e7f9",
        "name": "Send",
        "func": "return {\n    payload: function() {console.log(\"Test\")},\n    _socketId: msg._socketId\n}",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 990,
        "y": 180,
        "wires": [
            [
                "d2e69f2bb17aea8b"
            ]
        ]
    }
]

Environment

• UIBUILDER version: 6.8.2
• Node-RED version: v3.0.2
• Node.js version: v16.18.0
• npm version: 8.19.2
• Platform/OS: Windows_NT 10.0.20348 x64 LE
• Browser: Edge

How is Node-RED installed? (globally/locally? admin/user?): globally

How/where is UIBUILDER installed (palette-manager/globally/locally? Which folder?) palette-manager

Have you changed any of the uibuilder settings in your settings.js file? no

[TotallyInformation]

Hi Tobias, sorry about the delayed response.

As I understand it, you are trying to send a JavaScript FUNCTION from Node-RED over the network to your front-end?

That is not permitted directly. The connection uses Socket.IO and has to stringify all content to transmit it. In doing so of course, any code is actually lost.

However, I have allowed for this in UIBUILDER. You can use the low-code UI features for this.

You should be able to use the load method which is documented here: https://totallyinformation.github.io/node-red-contrib-uibuilder/#/client-docs/config-driven-ui?id=method-load

In Node-RED, convert your function to text as in this simplistic example for a function node:

function fred() {
    var temp = context.get("last_update"); //create a node variable
    var current = new Date();  //capture the current date
    msg.payload = Number(current) - temp;  //set the payload to the time date of the last update
    context.set("last_update",current);  //save the node variable
}

msg._ui = {
    "method": "load",
    // Add script tag to HTML converting the text to JavaScript
    "txtScripts": [
        // Will be able to do `fred()` in the browser dev console.
        fred.toString(),
        // But of course, we can execute immediately as well.
        "fred()"
    ]
}

return msg

Sending that msg to your uibuilder node will send it to the front-end and the function will be available to your front-end code in the browser.

Please do note the warnings in the documentation though. And make sure you only send trusted code.

[TobiasKorn]

Thank you for your answer and clarifying this issue. To get around the problem of sending FUNCTIONS from Node-Red to the front-end I am packing all objects containing FUNCTIONS into a JS file, which is loaded, when a new client connects to NodeRed.

Then by sending an identifying string, e.g. "getFunc1" I select the object in the JS file, including the FUNCTIONS.

let myObj
if (myString == "getFunc1") {
    myObj = {
        myFunc1: () => console.log("Test")
    }
}

[TotallyInformation]

OK, let me see if I understand.

The code is in a static file that you load into index.html. Then you are trying to send effectively a "remote control" from Node-RED to the client to execute a specific function from that file?

If it were me, I would slightly re-engineer the js file to load a single object (or class if you prefer) to the global namespace:

window['myFunctions'] = {
  function fn1() {
    // ....
  },
  // ....
}

Then you can simply send the name of the function to execute in the msg payload and have this in your front-end code:

uibuilder.onChange('msg', (msg) => {
  if (msg.topic === 'runCode') {
    try {
      window['myFunctions'][msg.payload]()
    } catch (e) {
      console.error (`Code could not be run: ${e.message}`, e)
    }
  }
}

If I've misunderstood, perhaps you can share a more comprehensive example?

This method limits the amount of clutter in the global context to help avoid any name clashes.

[TobiasKorn]

You understood my post correctly. Maybe it is not clear why I should want to do something like this. So I will describe the reason a little bit.

I want to create dynamic content, by sending objects from NodeRed to the client. For example the object:

{
  { chart: { type: "line", options: "Indicators" } },
  { columns1_1: [[
        { datepicker: { label: "Date start" } },
        { dropdown: { label: "Time format", options: { 0: "Time", 1: "Date & Time", 2: "Date" } } },
      ], [
        { datepicker: { label: "Date end" } },
        { switch: { label: "Hide zero values" } },
      ]]
   }
}

will result in a page setup like on the image (here displayed with some data in the chart):

The charts that I create are based on chartjs (v.2.9.4). So I need to pass options to the chart when creating it.

chart[id].ctx = new Chart(htmlElem.getContext('2d'), {
	type: props.type,
	data: props.data,
	plugins: ChartPlugins(props.plugins),
	options: ChartOptions(props.options, props.optionsAdd || {})
});

Options can contain functional components e.g. callback-functions. So my first attempt was to pass the options directly from NodeRed to the client dynamically. Instead of this I use now a function ChartOptions (also ChartPlugins ) which selects the right options on base of the keyword send by NodeRed. These funcitons are stored in a static JS file, which is load into index.html

var ChartOptions = function (key, optionsAdd = {}) {
       let optionsKey = {}
	switch(key) {
		case "Indicators":
			optionsKey = {
				aspectRatio: 1.5,
				legend: { display: false },
				scales: {
					yAxes: [
						{
							ticks: {
								fontColor: '#333F50',
								callback: function (value, index, ticks) {
									return new Intl.NumberFormat('de-DE').format(value);
								}
							}
						}
					]
				}
			}
			break
                case "otherCases":
                        break
		default:
	}
	return Func.deepMerge(optionsKey, optionsAdd)
}

[TotallyInformation]

OK, if I've got this right, I think you might be stuck in the wrong rut here. Because you seem more than happy writing some JavaScript code. And in that case, you should focus on writing that code in the front-end and not trying to send it from Node-RED. You don't really NEED to send code from Node-RED and you are really pretty much there with functions to create charts so all you need in your index.js file is an incoming msg handler that either creates a chart for you or updates the chart with new data. Both of these events can be a simple JavaScript function with lots of parameters.

const charts = {}

uibuilder.onTopic('recreateChart', (msg) => {
   doChartCreate(msg)
})
uibuilder.onTopic('updChartData', (msg) => {
   doChartUpdate(msg)
})

function doChartCreate(msg) {
  const chartParent = $(msg.chartParentEl) // returns an element or undefined
  if (!chartParent) {
    console.error('Oops, the chosen element does not exist')
    return
  }

  charts[msg.chartId] = new Chart(chartParent, {
	type: msg.type,
	data: msg.data,
	plugins: ChartPlugins(msg.plugins),
	options: ChartOptions(msg.options, msg.optionsAdd || {})
   });
}

Add the appropriate configuration data to the msg however you want. This is just a partial example of course and untested. Obviously, you can put all of your chart functions in your own .js library file if you like.

If you really want to get fancy - or, more importantly want to simplify the use of this approach greatly, lets talk about how to create a custom web component which would let you have some fairly simple HTML that automatically creates an instance of a chart, passes some parameters via HTML attributes and could automatically create topic listeners for updates. See some examples here: https://github.com/TotallyInformation/web-components

---

As an alternative, you could build your own node that worked with uibuilder, there is an example here: https://github.com/TotallyInformation/nr-uibuilder-uplot - though obviously that might be seen as a "little" more complex. 😊

[TobiasKorn]

Yes, I don't have problems writing JS code. The functions doChartCreate and updChartData are already part of my project, just with a different name.

My current approach is to keep one Client application for multiple NodeRed projects. I don't want to change index.html or index.js afterwards, only pass different content from NR to the application. And if I would apply this approach strictly I would need to pass chart options (including functional components) from NR to the client as well. So I am now making an acception to my approach by defining options in an static JS file, which is part of the client application (see function ChartOptions)

Anyway, thank you for your tips. Maybe I will have the time to look at them in deepth.

[TotallyInformation]

My current approach is to keep one Client application for multiple NodeRed projects. I don't want to change index.html or index.js afterwards,

Right, sensible. That would be another good reason for making it into a custom web component. But even without that, I'd move that code to its own package on GitHub and install it from there. You can make it a standard JavaScript package and install using uibuilder's library manager tab (which simply uses npm under the skin).

When installed using the library manager, the library becomes available to all instances of uibuilder on your Node-RED instance. If you have multiple Node-RED instances, you can install to each of them. If you need to, you can publish the library to npmjs to make it a little easier still to install but the library manager supports public and private repo's from npmjs, GitHub, local installs - indeed, anything that the npm package manager supports.

BTW, you could start with the plain library in your package and enhance to a custom component later on in the same package if/when time permitted. Your component would use the same library of course. A simple build step would integrate them nicely.

[TobiasKorn]

Sounds like a good idea, even though I lack of experience how to do this. Anyway this would make only sense for me, if I can also pass chart options (with functional components) from NR to the client, which is currently not possible.

[TotallyInformation]

Anyway this would make only sense for me, if I can also pass chart options (with functional components) from NR to the client, which is currently not possible.

Not really, since you could simply embed the fixed settings in your initial index.js and then enhance with data from Node-RED as needed.

To be clear, this is a very common web application approach and merges common approaches from "traditional" web development with the strengths of Node-RED (its strong communications features). This was the original approach of uibuilder when it was first starting out. It is only more recently that uibuilder gained a load of features to work in other ways from within Node-RED.

Your fixed setup is in your fixed code (index.html and index.js) - in your case, using an external single-file js library as well for convenience and repeatability. With dynamic data sent as available or needed from Node-RED but processed in the browser (index.js) using your library.

You already seem to have all the components ready, it seems to me to need just a slight mind-shift.

[TobiasKorn]

As far as I understood, I should create a public library on Github, which I include in all my NR projects, containing the core features of my UIBuilder application (create/update dynamic HTML page based on an object structure received from NR).

In every NR project individual extra data, like chart options (containing functional components, which cannot be send from NR to the client) are part of the index.js.

If this is correct, can CSS and HTML files be also part of the library? HTML contains some basic stuff (navigation, toolbar, container for notifications, see below), which is static content for my application.

My current index.html:

<!doctype html>
<html lang="en">
    <head>
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <link rel="icon" href="../uibuilder/images/node-blue.ico">
        <title>Machine overview</title>
        <meta name="description" content="Node-RED uibuilder - Blank template">
    
        <!-- CSS -->
        <link type="text/css" rel="stylesheet" href="./index.css" media="all">
        
        <!-- #region Supporting Scripts. These MUST be in the right order. Note no leading / -->
        <script defer src="../uibuilder/uibuilder.iife.min.js"></script>
        <script defer src="./js/dependencies/gauge.min.js"></script>
	<script defer src="./js/dependencies/raphael-min.js"></script>
	<script defer src="./js/dependencies/justgage.min.js"></script>
        <script defer src="./js/dependencies/Chart.min.2.9.4.js"></script>
	<script defer src="./js/dependencies/chartjs-plugin-datalabels.min.js"></script>
	<script defer src="./js/funclib.js"></script>
	<script defer src="./js/chartconfig.js"></script>
        <script defer src="./js/index.js"></script>
        <!-- #endregion -->
    </head>
    
    <body class="uib">
        <div id="toolbar"> <!-- Absolute positioned Flexbox, content space-between, item-align center-->
            <div class="toolbarLeft"> <!-- Left side -->
                <div id="burgerMenu"></div>
                <h1 id = "heading"></h1>
            </div>
            <div class="toolbarRight"> <!-- Right side -->
		<select id = "machineSelect" name="Machine"></select>
		<div class="langChange" id = "DE"></div>
                <div class="langChange" id = "EN"></div>
		<span class="mat-icon" id = "loginOpen">account_circle</span>
                <div id = "logo"></div>
            </div>
        </div>
        
        <div id="fader"></div>
        <div id="sidenav"></div>
	<div id="login"></div>

	<div id="dialogbox">
		<div id="dialogboxhead"></div>
		<div id="dialogboxbody"></div>
		<button id="dialogboxConfirm">&#10004;</button>
		<button id="dialogboxClose">&#x2715;</button>
	</div>
				
	<main class="flex"></main>

    </body>
</html>

[TotallyInformation]

As far as I understood

Yes, that isn't the only way but it is a pretty good one.

can CSS and HTML files be also part of the library

Absolutely, yes.

The CSS is easy since it is just another resource that uibuilder will make available for you. Check the default index.css file. You will see that, by default, it loads the uibuilder brand css resource for you. You can keep that and add your own or you can ditch it and just use your own.

Things get a bit more complex for the HTML though. That's because HTML doesn't allow quite the same flexibility to load other HTML as does CSS and JavaScript. This is one of the reasons that uibuilder has the concept of Templates for the base setup.

I think that here you have some choices. To some extent, which you make use of is likely to depend on how many instances you expect to end up with and how often you might be looking to update them.

1. Instead of (or as well as) a generic javascript/css library, you could create your own uibuilder external template. This would let you standardise your HTML and use it as a template direct from GitHub when creating a new uibuilder instance. See https://totallyinformation.github.io/node-red-contrib-uibuilder/#/creating-templates for details.

2. You could use JavaScript to load the standard parts of you HTML early in the load.

   I can't say I'm a fan of this approach. It might result in some visible changes as the page loads and might end up being a little fragile.

#1 is the best approach. However, it does not include an easy update mechanism automatically. To partly get around that, you should look to connect each uibuilder instance's root folder (which is what the template creates) back to your GitHub template repo. Not sure how familiar you are with GIT and GitHub so we might need to talk through the best steps on how to do that.

Of course, if you aren't expecting to update that core very often and you didn't have too many instances, you could skip the update processing and simply re-apply the external template as needed.

Hopefully this diagram might help explain

To update from the external repos, the best thing is to include a script somewhere to run git commands.

The library js/css:

cd ~/.node-red/uibuilder
npm update

The template (assuming your node's url is test:

cd  ~/.node-red/uibuilder/test
# This only needs doing once
git remote add upstream https://github.com/myGHacct/myTemplateRepo.git
# This updates the local folders to the latest remote
git fetch upstream

Of course, if the number of instances is manageable, you can do this all within Node-RED manually by re-applying the external template and using the library manager to update your library.

[TobiasKorn]

Thanks for your detailed description. I think if I would use method #2 then I would write code like this in my JS library:

window.addEventListener('load', () => {
    let body = document.getElementsByTagName("body")[0]
    body.innerHTML = `<div id="toolbar"> ... <main class="flex"></main>`
});

Or maybe use not the window.load but document.load instead, to be more early in the page setup.

For the other stuff I will need more time to get familiar with, but I think I have now a roadmap what I can do, to make my project better maintainable. Since I NR is not my only project, I will deal with this later, and would maybe ask you if I have further questions.

[TotallyInformation]

Hi, thanks to you raising this, I revisited ways to enable scripts to be included when passing HTML via the low-code features.

I'm happy to report that I learned something I never new about HTML/JavaScript - that being DOM Ranges. I'd searched for ways to do this long and hard previously and the previous best was not pretty and didn't really work for a general case.

However, I have now unlocked the magic formula and the next release of uibuilder will contain the update and will let you include scripts in HTML you send via any of the no-/low-code nodes such as uib-element, uib-tag, etc.

Of course, you should note that the script will auto-run every time you send the code to the front-end so make sure you wrap it in a flag to prevent it running multiple times if you need to.

If you need a short-term work-around for your own use, you could send some HTML as a string in a standard msg (you can use the standard template tag to make that easier if you need to), then in your front-end code, something like this should work:

uibuilder.onChange('msg', (msg) => {
  if (msg.topic !== 'iamascript') return

  // Create doc frag and apply html string - I'm assuming the string is wrapped in <script>...</script> here
  // But you could, of course, add that here.
  const tempFrag = document.createRange().createContextualFragment(msg.payload)

  // Element used to attach script - best to use something you can empty - I assume a div with id of myscript
  const el = document.getElementById('myscript')

  // Remove content of el and replace with tempFrag
  const elRange = document.createRange()
  elRange.selectNodeContents(el)
  elRange.deleteContents()
  el.append(tempFrag)

  // Any script tag in the msg.payload will be automatically executed.
  // `msg.payload = "<script>console.log('HELLO!')</script>"
})

(NB: I've not actually tried this script out yet).

[TobiasKorn]

Interesting approach to send HTML including scripts, which are executed after receiving them. However, I can't see how this could help me, when I want to send an object, containing functional components. As far as I understand this method replaces all JS scripts contained in the #myscript element, and makes it possible to execute them again after the initial execution of the onload event (el.append(tempFrag)).

[github-actions[bot]]

Stale issue message