Incorrect data in the response

[akalagov]

I'm trying to get data from a UniFi OS UCK G2 Plus 3.0.13, but I'm getting something unreadable in response. However, an attempt to obtain the same data through the browser is completed successfully. Please tell me what could be the reason?

Unifi-request /proxy/network/api/s/default/stat/health: { "payload": "\u001F�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000�T�n�0\u0010���O�D\u0010\u0017�\\ޚ<�}�\"UQUE\u0011��d�\u0016\f�f�jſwlX�a�*�V+�s<���\u0019��\u0006-Hz$]NR����B\u0018��HT�Q\u0007��Fj��$\u000E�}��\n:�2��=(MRo��fz�/��A]�J�3�'\u000E�ΰ��#�\u0010���մ�T@�$M�עi5\u0014K\\�Jl*\u0003x\u000B�7RB���\u0016dQ�-ƣ�:�r�퀪V��\u001F����5�\u0012%�8v\u0003���\u001F�!�[�a\u0010\u0007\\�xN�<aIеP;�\n\u0018sO?\t�<Q\u0003������A�z��m\u001EJQ�n|/xmq�\u0005\u0011�W\u00163/f�d�ja.wC��M\n4-X��i$�%͆�>����ɮksLe\u001A#o{�=$k�\"�7\\�Z�\u0011]q�����3R�]�\";��_Y�[\u0010��dj\u0003�0�B\f���V�V\u0003\u000F\u0019�15\u0018\n1\u0016��4%\\�2�2J\u0013�\u001B���l����w��t�\u001F2��b�\u0015y2�\u0017]0\fd}�\u001F�r�o��Q\u001DE1�84]c,\tB�����zl\n��9*�Aαj\u0001\n���E÷�\u0002r�TB鮗S�-h{j��\\�����V�.�\u001Es}\u001E��騷��_g;L�(\nW�͸\u001F&��\rkf�[\u000F��M�\u0017�;�\u001B\rVx\u0006r�@�{���܃�gM��\u001D�t�\u000BB�GF+�\u001D�7s�)U"���>\u0017�ϢR0>��\u0001��2m�\u0005\u0000\u0000", "inputMsg": { "_msgid": "48683126e9fad11e" }, "_msgid": "2b2b0e7bb6e54201" }`

Same, via browser: { "meta": { "rc": "ok" }, "data": [ { "subsystem": "wlan", "num_user": 51, "num_guest": 0, "num_iot": 0, "tx_bytes-r": 151409, "rx_bytes-r": 21562, "status": "ok", "num_ap": 9, "num_adopted": 9, "num_disabled": 0, "num_disconnected": 0, "num_pending": 0 }, { "subsystem": "wan", "num_gw": 1, "num_adopted": 1, "num_disconnected": 0, "num_pending": 0, "status": "ok", "wan_ip": "xxx.xxx.xxx.xxx", "gateways": [ "xxx.xxx.xxx.xxx" ], "netmask": "255.255.255.192", "nameservers": [ "127.0.0.1" ], "num_sta": 103, "tx_bytes-r": 156492, "rx_bytes-r": 34743, "gw_mac": "b4:fb:e4:d5:cc:7a", "gw_name": "USG", "gw_system-stats": { "cpu": "0", "mem": "6", "temps": { "Board (CPU)": "45 C", "Board (PHY)": "46 C", "CPU": "64 C", "PHY": "73 C" }, "uptime": "6355684" }, "gw_version": "4.4.56.5449062", "uptime_stats": { "WAN": { "latency_average": 69 } } }, { "subsystem": "www", "status": "ok", "tx_bytes-r": 156492, "rx_bytes-r": 34743, "latency": 69, "uptime": 78912, "drops": 23, "xput_up": 0.0, "xput_down": 0.0, "speedtest_status": "Idle", "speedtest_lastrun": 0, "speedtest_ping": 0, "gw_mac": "b4:fb:e4:d5:cc:7a" }, { "subsystem": "lan", "lan_ip": "192.168.70.1", "status": "ok", "num_user": 52, "num_guest": 0, "num_iot": 0, "tx_bytes-r": 3333290, "rx_bytes-r": 3840264, "num_sw": 10, "num_adopted": 10, "num_disconnected": 0, "num_pending": 0 }, { "subsystem": "vpn", "status": "ok", "remote_user_enabled": true, "remote_user_num_active": 0, "remote_user_num_inactive": 0, "remote_user_rx_bytes": 0, "remote_user_tx_bytes": 0, "remote_user_rx_packets": 0, "remote_user_tx_packets": 0, "site_to_site_enabled": false } ] }

[crxporter]

Hello! This is odd - it certainly shouldn't look like that.

I just tried to reproduce and I couldn't get that kind of output no matter what I pick on the UniFi HTTP node. Let's make sure you are configured properly - to access the endpoint you listed, I would send an input to a node like this:

And my output is just like what you've noted you can see when accessing on the web browser.

Can you verify that you are on the latest plugin version 0.8.0 and show how you are set up? Screenshots of the nodes would be fine.

Also - if you haven't yet, try restarting nodered to clear any old connections... we are still working on closing things better.

[akalagov]

I use settings similar to yours, the plugin is loaded from https://flows.nodered.org/node/node-red-contrib-unifi-os every time Node-RED is restarted (I use Node-RED as an addon for Home Assistant). I have a suspicion that there is no proper handling of unicode. Maybe in Node-RED some additional package related to this should be installed?

[crxporter]

You have an interesting theory on the unicode error - it does indeed appear to be something like that. I don't know ~~many~~ any words using Cyrillic but I renamed my UDM to some random Cyrillic characters and... no error.

What is your Cloud Key language set to? Looking at your browser copy of the JSON though shows all Latin characters... But maybe there's something odd happening there. Can you set to English and test again? I'd set mine to Kazakh but I don't think I could recover...

[akalagov]

I use English on the controller, but once Russian was used. I also looked at the response headers in the browser and saw that the response comes with content-type: application/json;charset=UTF-8

By the way, when using the unifi-web-socket node, the data comes in normal form. I also tried the node-red-contrib-unifi node, when using it, the answers also come in normal form.

[crxporter]

Thanks for checking those things.

Time to call in @marcus-j-davies and @Shaquu to see if they have some more ideas.

[akalagov]

Thank you for your responsiveness!

Just in case, I'll also point out that I have Russia installed in the country settings (although I use equipment in Kazakhstan) - this is due to the supported ranges and Wi-Fi power limits. I could change this setting, but I don't know which Latin-based country supports the same Wi-Fi ranges...

[marcus-j-davies]

\u001F�\b = 0x1F 0x8B

aka The Gzip Signature (I believe)

you can verify this.... open up your browser network log, and check the content-encoding or transfer-encoding header for this resource, it may say GZip

if it does, then:

• Either the HA version of Node RED cant handle GZip
• We have never needed to decode Gip responses until now.

I'll be very surprised if the Axios lib doesn't handle GZip internally (but could be the case, therefore we need to decode it our self)

In a nutshell, the response is a byte buffer, that when GZip decoded, is your Json response

Of course, i could be completely wrong here 😅 but that signature looks awfully familiar

[marcus-j-davies]

You could test it further, set the response type to buffer and use some GZip node (I'm sure there is one about) to see if it can decode the value)

But I'm pretty sure this is a GZip response

[akalagov]

This is very strange, but after various attempts to change the type of response to the unifi-request node, writing output to a flow variable (file), rebooting Node-RED and other manipulations, I suddenly began to receive normal JSON on the output. And you're right, I get content-encoding: gzip in the browser.

[marcus-j-davies]

@akalagov

I'm going to put this down to a freak moment! - as I cant reproduce it. I'll close for now.

[aw-jansen]

Experiencing the same issue.

Started having this issue when upgrading from version 0.6.0.

This is what the output looks like:

As suggested in this thread I used a gzip node to "decode" the message, followed by a json node and this gives me usable output similar to as it was before the upgrade. I have no problem using it like this but it does feel like something is wrong.

[marcus-j-davies]

Im going to re-open this - as it seems to be more occurring.

We will implement a routine on the response and check the gzip signature, and if found, we will decode it internally before passing it to user land

[marcus-j-davies]

We no longer return packed payloads The is an upstream issue.