Upgrade to 5.0.2 - settings-omnichannel only gives me an empty grey screen

[tom666-debug]

Omnichannel feature cannot be activated or configured as the according settings cannot be opened. When I try to open the Omnichannel setting page RC just gives me blank grey screen. The remaining other setting items are working as expected.

Steps to reproduce:

1. Go to Administration - Settings
2. Click on Omnichannel

Expected behavior:

Omnichannel Settings page will be opened

Actual behavior:

RC only gives me blank grey screen

Server Setup Information:

• Version of Rocket.Chat Server: 5.0.2
• Operating System: Debian 10.7
• Deployment Method: Docker
• Number of Running Instances: 1
• DB Replicaset Oplog: ?
• NodeJS Version: v14.19.3
• MongoDB Version: 5.0.6 / wiredTiger (oplog Enabled)

Client Setup Information

• Desktop App or Browser Version: Both - Desktop App and Browser version
• Operating System: Windows 11

Relevant logs:

rc Docker container: _rocketchat_rocketchat_1_logs.txt

[sleeplessnight2]

this is happening since 5.0.0 at my installation...

[hibobmaster]

Same here.

[tom666-debug]

Still broken with version 5.0.3

[hibobmaster]

@sampaiodiego

[sampaiodiego]

@KevLehman @murtaza98

[murtaza98]

Hi @tom666-debug @sleeplessnight2 @hibobmaster Can you guys please share some console logs from the browser when you get the grey screen? Because I just checked to access the settings within a few env running the 5.0.2 version & they all seem to be loading the settings page without any issue

[hibobmaster]

@murtaza98 Thanks for your reply, here is the error log.

Error: Minified React error #130; visit https://reactjs.org/docs/error-decoder.html?invariant=130&args[]=undefined&args[]= for the full message or use the non-minified dev environment for full errors and additional helpful warnings.

[tom666-debug]

here you go:

[urangsabah]

also experiencing similar issue with our omnichannel rollout

Server Setup Information:

Version of Rocket.Chat Server: 5.0.3 Operating System: Ubuntu 20.04.2 LTS Deployment Method: Docker Number of Running Instances: 1 DB Replicaset Oplog: (default) NodeJS Version: v14.19.3 MongoDB Version: 4.4.12 / wiredTiger (oplog Enabled)

Client Setup Information

Desktop App or Browser Version: Chrome 104.0.5112.81, Firefox 103.0.2 (x64) Operating System: Windows 10

Additional context

Last known working version was rocketchat 4.8.1 on mongodb 4.0.x - upgraded to current config on 2022-08-14 01:00:00 GMT+0

Also broken on Android App > Admin Panel > Settings > "Omnichannel" > Click Button >> "white screen"

Relevant logs:

Chrome Dev Console

31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:13 fuselage: 0.31.16
31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1558 MessageType.render is deprecated. Use MessageType.message instead. livechat_webrtc_video_call
registerType @ 31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1558
31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124 Error: Minified React error #130; visit https://reactjs.org/docs/error-decoder.html?invariant=130&args[]=undefined&args[]= for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at vu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:107534)
    at d (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:53373)
    at g (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:54643)
    at 31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:56704
    at ui (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:67706)
    at Rs (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:113409)
    at nu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98840)
    at tu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98768)
    at eu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98631)
    at Gs (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:95485)
Ti @ 31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124
31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124 Uncaught Error: Minified React error #130; visit https://reactjs.org/docs/error-decoder.html?invariant=130&args[]=undefined&args[]= for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at vu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:107534)
    at d (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:53373)
    at g (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:54643)
    at 31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:56704
    at ui (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:67706)
    at Rs (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:113409)
    at nu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98840)
    at tu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98768)
    at eu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98631)
    at Gs (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:95485)

Firefox Dev Console

similar log: 1660528602260.log - url redacted

[tom666-debug]

Still broken with version 5.0.4

[tom666-debug]

@murtaza98 Just wondering why the status of this ticket is still "need more info"?! What additional info can we provide?

[PlusaN]

Description:

Omnichannel settings don't work.

Steps to reproduce:

1. Go to Administration - Settings
2. Click on Omnichannel

Expected behavior:

Omnichannel Settings page will be opened

Actual behavior:

RC only gives me blank grey screen

Server Setup Information:

• Version of Rocket.Chat Server: 5.0.4
• Operating System: Debian 10.12
• Deployment Method: tar
• Number of Running Instances: 1
• DB Replicaset Oplog: -
• NodeJS Version: 14.19.3
• MongoDB Version: 4.4.16

Client Setup Information

• Desktop App or Browser Version: Rocket.Chat.Electron 3.8.8, Firefox 104.0, Chromium 104.0.5112.81
• Operating System: Windows 10

Additional context

Last known working version was rocketchat 4.8.2

Relevant logs:

chromium.log

[tom666-debug]

Version 5.0.5 still broken ... ridiculous

[localshot]

Acknowledged. 5.0.5 - grey screen in Omnichannel settings

[KevLehman]

Hey! Small update We have been trying to reproduce this issue for quite a while, with multiple envs running versions from 4.8 to 5.1 (candidate) and issue doesn't appear.

A few questions:

1. Do this happen only with Omnichannel settings, or does it happen with other sections too?
2. Does the same happen on incognito windows? Different browsers? After clearing cache/local storage?
3. What was the last version that was working for you?

We understand this issue prevents you from using that part of the product. As workaround, you can update settings individually by using the REST API. We'll keep trying to find why this is happening.

[urangsabah]

@KevLehman

greetings - not OP - but to anwer your questions since everybody here is reporting the same browser blank screen (and console logs) on this. i should note that if we leave the blank screen open, it will eventually hang that specific tab/browser session and make it unresponsive

re: your questions

1. Do this happen only with Omnichannel settings, or does it happen with other sections too? ONLY ON OMNICHANNEL SETTINGS, that is Administration > Settings > "Omnichannel" > Open
2. Does the same happen on incognito windows? Different browsers? After clearing cache/local storage? YES, we have tested this on Firefox, Chrome, Edge, on a variety of OSes (Win10, Win11, Chromebook, Ubuntu). Same error on incognito/private modes, with cleared storage.
3. Last known working version is rocketchat 4.8.1, mongodb 4.2 (see upgrade timeline below)

timeline

For clarity, we planned an upgrade of our docker-based rocketchat instance and had a planned roll-out of omnichannel - which has since stalled due to this bug (all steps below occurred on same day mid-August over 3 hours):

1. rocketchat version was initially 4.8.1 - at this point Administration > Settings > "Omnichannel" was working
2. mongodb upgraded from 4.0 mmapv1 to 4.0 wiredtiger then 4.2 wiredtiger
3. the following omnichannel settings were changed via the web gui: omnichannel > enable omnichannel > routing algorithm: manual omnichannel > business hours > (set all to 24x7 - all days, 12am-12pm, but was not enabled) omnichannel > time for livechat abandoment > 3 hours
4. omnichannel was tested at this point (on rc 4.8.1, mongo 4.2 wiredtiger), no issues were noted
5. rocketchat upgraded to 5.0.3
6. mongodb upgraded from 4.2 to 4.4
7. it was at this point the blank screen on the web gui for Administration > Settings > "Omnichannel" was observed

additional notes

• unfortunately for (5), we don't have a complete list of the initial modifications to the administration settings, but these were the most signficant settings that we recall, since we intended to tweak and confirm them after the final rc upgrade to 5.x
• our omnichannel had no integrations configured (i.e. no facebook, etc)
• original report is here

further investigation via API?

noted on the REST API, and was considering this as a workaround - in fact, if you need us to check specific omnichannel settings we are using, let us know which keys to dump via API for further investigation (we are not running livechat in production yet)

[tom666-debug]

Hi,

As requested:

Do this happen only with Omnichannel settings, or does it happen with other sections too? As stated in my initial post, only Omnichannel seems to be affected

Does the same happen on incognito windows? Different browsers? After clearing cache/local storage? Different browsers (Chrome (incl. incognito), Edge, FireFox) - with cleared cache and even on a fresh installation - allways the same result

What was the last version that was working for you? RC 4.8.4 - Mongo 5.0.6

[imahbub]

Same! I vote!

[tom666-debug]

Version 5.1.0 still broken - of course. It's becoming embarrassing.

[fabiosurrage]

I have exactly the same problem. Working omnichannel settings page with version 4.8.2. Empty page with version 5.0.5 and 5.1.1.

But I have one test instance with version 5.0.5 that works fine, then I am searching for some proxy problem or diference between instalations ...

[tom666-debug]

Confirmed, version 5.1.1 still broken.

I know it is really tricky to analyse an issue that you can't hardly reproduce. But no feedback or further questions at all regarding to this issue? Wow! For me it just feels like they dont care at all - pretty sad.

[billynoah]

This is affecting us as well.

[billynoah]

Hey! Small update We have been trying to reproduce this issue for quite a while, with multiple envs running versions from 4.8 to 5.1 (candidate) and issue doesn't appear.

A few questions:

1. Do this happen _only_ with Omnichannel settings, or does it happen with other sections too?

2. Does the same happen on incognito windows? Different browsers? After clearing cache/local storage?

3. What was the last version that was working for you?

We understand this issue prevents you from using that part of the product. As workaround, you can update settings individually by using the REST API. We'll keep trying to find why this is happening.

Hi @KevLehman - Since you're having trouble reproducing the issue, I could provide access to our instance (where Omnichannel settings are broken) if someone from your team wants to use it as a case example. Would that help?

[fabiosurrage]

I got this erro in log when accessing the page /admin/settings/Omnichannel

{"level":50,"time":"2022-09-12T12:58:20.713Z","pid":1,"hostname":"rocketchat-server-deployment-6475c587d5-z2bww","name":"System","msg":"Exception while invoking method autoTranslate.getSupportedLanguages 'Auto-Translate is disabled [error-autotranslate-disabled]'"}

[dudanogueira]

{"level":50,"time":"2022-09-12T12:58:20.713Z","pid":1,"hostname":"rocketchat-server-deployment-6475c587d5-z2bww","name":"System","msg":"Exception while invoking method autoTranslate.getSupportedLanguages 'Auto-Translate is disabled [error-autotranslate-disabled]'"}

not sure this error is related, as I also see this error with a working instance, FWIW

[KevLehman]

Hey! We have a possible fix. Thanks to @fabiosurrage that provided an env with the issue happening 😬 (and also thanks to @billynoah for offering help too!). We were able to reproduce it locally and investigate it.

The actual reason of the issue is still unclear. Possibly, a really old DB migration created a setting (an EE setting) that shouldn't be there. This invalid setting was causing issues, as it didn't have all the data we expect from a setting and therefore, broke the UI. This seems to have only happened to envs that have been fully community always, without any touch of EE features (cause that would've fixed the setting automatically).

The problem:

UI was receiving an invalid setting

    {
        "_id": "Canned_Responses_Enable",
        "_updatedAt": "2021-10-26T10:59:20.368Z",
        "group": "Omnichannel"
    },

As this setting doesn't have all the data required, UI broke (here's a valid setting for reference)

    {
        "_id": "Livechat_Allow_collect_and_store_HTTP_header_informations",
        "_updatedAt": "2021-11-12T09:54:14.872Z",
        "autocomplete": true,
        "blocked": false,
        "createdAt": "2021-11-12T09:54:14.872Z",
        "group": "Omnichannel",
        "hidden": false,
        "i18nDescription": "Allow_collect_and_store_HTTP_header_informations_description",
        "i18nLabel": "Allow_collect_and_store_HTTP_header_informations",
        "packageValue": false,
        "public": true,
        "sorter": 8000,
        "ts": "2021-11-12T09:54:14.872Z",
        "type": "boolean",
        "value": false,
        "valueSource": "packageValue",
        "secret": false,
        "section": "GDPR",
        "enterprise": false,
        "requiredOnWizard": false,
        "enableQuery": "{\"_id\":\"Livechat_enabled\",\"value\":true}",
        "env": false
    },

The solution

Remove the offending setting from the DB. Sadly, this will involve some DB touching (I'll work on doing a proper code fix for a following release, so consider this as a "workaround")

// Remove setting with ID canned_responses_enable from rocketchat_settings collection
// Note: driver syntax may vary
db.collection('rocketchat_settings').deleteOne({ _id: 'Canned_Responses_Enable' }) // From a MongoDB shell

Then, clear browser data (Go to Dev tools => Application => Storage => Clear data) and refresh.

That should work 😬

Thanks for your patience, just pls have us a little bit more of patience while we develop the actual fix (for cases where you don't have DB access or other special cases too)

[billynoah]

@KevLehman - thanks that worked!

[urangsabah]

@KevLehman - the workaround works - thank you!

here's what we did:

rs0:PRIMARY> db.rocketchat_settings.find({ _id: 'Canned_Responses_Enable'  })
{ "_id" : "Canned_Responses_Enable", "_updatedAt" : ISODate("2021-12-20T03:42:28.590Z"), "group" : "Omnichannel" }

rs0:PRIMARY> db.rocketchat_settings.deleteOne({ _id: 'Canned_Responses_Enable'  })
{ "acknowledged" : true, "deletedCount" : 1 }

then...

• Clear browser cache/storage
  • Chrome > Dev Tools > Application > Storage > Clear site data
  • Firefox > Lock Icon (URL bar) > Clear cookies and site data...
• Login again and check on Administration > Settings > Omnichannel > Open

---

only happened to envs that have been fully community always

...is right - we've been running on community since 3.x back in 2019, and progressively upgraded on community till 5.x recently

[fabiosurrage]

Thanks a lot ! Worked!

[privnote42]

Hello, is there an official solution to this in the meantime?

the workaround does not work for me.

At the command db.collection('rocketchat_settings').deleteOne({ _id: 'Canned_Responses_Enable' })

I get an error message

uncaught exception: TypeError: db.collection is not a function : @(shell):1:1

and when I use the commands from @urangsabah then I get no output on the first command and the following on the second command: { "acknowledged" : true, "deletedCount" : 0 }

deleting the cache of the page did not help any more than the private mode of the browser.

I am using rocketchat 5.3.1 and mongodb 4.4.18