Can I use Shiny.addBinaryMessageHandler?

[daeyoonlee]

System details

Browser Version: Chrome 101.0.4951.54

Output of sessionInfo():

R version 4.1.2 (2021-11-01)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=ko_KR.UTF8        LC_NUMERIC=C               LC_TIME=ko_KR.UTF8         LC_COLLATE=ko_KR.UTF8      LC_MONETARY=ko_KR.UTF8    
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] magrittr_2.0.3 shiny_1.7.1   

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.8.3             lubridate_1.8.0          lattice_0.20-44          png_0.1-7                shinyWidgets_0.6.2      
 [6] assertthat_0.2.1         digest_0.6.29            utf8_1.2.2               shinybusy_0.2.2          mime_0.12               
[11] R6_2.5.1                 reactR_0.4.4             pillar_1.7.0             rlang_1.0.2              fontawesome_0.2.2       
[16] jquerylib_0.1.4          R.oo_1.24.0              R.utils_2.11.0           Matrix_1.3-4             reticulate_1.22         
[21] shinyjs_2.0.0            dbx_0.2.8                reactable_0.2.3.9000     configr_0.3.5            readr_2.1.1             
[26] stringr_1.4.0            htmlwidgets_1.5.4        bit_4.0.4                compiler_4.1.2           httpuv_1.6.5            
[31] RMariaDB_1.2.0           pkgconfig_2.0.3          htmltools_0.5.2          tidyselect_1.1.2         tibble_3.1.6            
[36] fansi_1.0.2              crayon_1.5.0             dplyr_1.0.8              tzdb_0.2.0               withr_2.5.0             
[41] later_1.3.0              R.methodsS3_1.8.1        grid_4.1.2               jsonlite_1.8.0           xtable_1.8-4            
[46] lifecycle_1.0.1          DBI_1.1.1                cli_3.2.0                stringi_1.7.6            cachem_1.0.6            
[51] promises_1.2.0.1         ini_0.3.1                bslib_0.3.1              ellipsis_0.3.2           vctrs_0.3.8             
[56] generics_0.1.2           tools_4.1.2              bit64_4.0.5              glue_1.6.2               purrr_0.3.4             
[61] shinytoastr_2.1.1        crosstalk_1.2.0          hms_1.1.1                fastmap_1.1.0            RcppTOML_0.1.7          
[66] yaml_2.3.5               sodium_1.2.0             shinydashboardPlus_2.0.3 shinydashboard_0.7.2     sass_0.4.1 

Example application or steps to reproduce the problem

Any Shiny App 

Describe the problem in detail

Thank you for maintaining Shiny.

There is session$sendBinaryMessage() on the server, but Shiny.addBinaryMessageHandler does not exist on the javascript. It's on the shiny official site, but it doesn't show up in the client.

What am I missing? In order to send large json to the client, I need a way to send gzip-compressed binary.

[philibe]

Without reproducible example or at minimal an extract of code it will be difficult to answer.

Have you read those pages ?

• https://shiny.rstudio.com/articles/js-events.html
• https://shiny.rstudio.com/articles/communicating-with-js.html
• https://shiny.rstudio.com/articles/js-send-message.html

Philippe

PS: I am a simple R developer.

[daeyoonlee]

Thank you for your attention. This is a minimal example. can not find Shiny.addBinaryMessageHandler.

library(shiny)

runApp(
  list(ui = fluidPage(
    tags$head(
      tags$script('
        Shiny.addBinaryMessageHandler("testmessage", function(message) {
          alert("function is running properly");
        });
      ')
    ),
    
    titlePanel("sendBinaryMessage example"),
    fluidRow(
      column(4, wellPanel(
        sliderInput("controller", "Controller:", min = 1, max = 20, value = 15),

      ))
    )
  ),
  server = function(input, output, session){
    observeEvent(
      eventExpr = input$controller,
      handlerExpr = {
        session$sendBinaryMessage(type = 'testmessage', message = as.raw(1))
      }
    )
  })
)

[philibe]

Try with Shiny.addCustomMessageHandler(), it seems to work.

It is maybe an error in the documentation . I see in this repository code only Shiny.addCustomMessageHandler() and not Shiny.addBinaryMessageHandler(). (same when console.log(Shiny) (ping @wch :) )

Links I've found where I found Shiny.addCustomMessageHandler() and not Shiny.addBinaryMessageHandler() :

• https://github.com/rstudio/shiny/blob/78d77ce3733d947b292614a626e04ca5167aa445/NEWS.md#L978
• => https://github.com/rstudio/shiny/pull/1316
• => https://github.com/rstudio/shiny/pull/1316/files
• => srcjs/shinyapp.js

[daeyoonlee]

Thanks for the advice. I solved it by compressing and encoding with addCustomMessageHandler. If the json size is several hundred megabytes, it is more than 50 times faster than pure json transper.

Server.R
data <- data %>% memCompress(type = "gzip") # compress
data <- data %>% base64enc::base64encode() # encode
session$sendCustomMessage(type = "send_data_to_client", message = data)

javascript
  Shiny.addCustomMessageHandler('send_data_to_client',
    function(data) {
      
      var decode = atob(data); // decode
      var decompress = pako.ungzip(decode, { to: 'string' }); // decompress
      
      // do something
    
    }
  );

[jcheng5]

Thanks for the report! I can't get this to work properly either. I'll see about getting that fixed.

I have a branch on httpuv that implements websocket compression natively--I wonder if that would give you even better performance? If you'd be willing to try it out, I'd love to hear what the results are. remotes::install_github("rstudio/httpuv@joe/feature/permessage-deflate")

Can I ask what kind of app you're building that sends so much data over the websocket? Just curious.

[jcheng5]

~Ah, spoke too soon? That httpuv branch might be in a broken state right now.~ Never mind, it does seem to work, I was fooled by an unrelated problem on my machine

[daeyoonlee]

@jcheng5 Thanks for the reply.

I am making a google sheet-like web-excel app by combining shiny and spreadjs. https://www.grapecity.com/spreadjs/demos/ Unfortunately spreadjs is a paid program. Among the open sources, I couldn't find anything perfectly compatible with Excel. (xspreadsheet.. luckysheet)

When I tested with pako.js before, the performance difference between deflate and gzip did not seem to be large. Crucially, I couldn't figure out how to compress/decompress deflate in R.

[JapieVanTonder]

Thanks for the useful thread! I managed to get the following working on my end, which may help someone in future.

The file to download is located under ./www/. The file is read as binary, sent across the websocket to the front-end, where the binary 'stream' is decoded in JS, and wrapped in a Blob before being downloaded.

(I couldn't use shiny::downloadHandler() for other reasons...)

library(shiny)

runApp(
  list(ui = fluidPage(
    tags$head(
      tags$script(
      "
        // handle binary data stream
        function s2ab(s) {
          var buf = new ArrayBuffer(s.length);
          var view = new Uint8Array(buf);
          for (var i=0; i!=s.length; ++i) view[i] = s.charCodeAt(i);
          return buf;
        }
        
        // download handler
        function downloadFile(contentURL, fileName) {
          var element = document.createElement('a');
          element.setAttribute('href', contentURL);
          element.setAttribute('download', fileName);
          document.body.appendChild(element);
          element.click();
          document.body.removeChild(element);
        }
      
        // download file on actionButton click
        $(document).ready(function() {
          Shiny.addCustomMessageHandler(
          'send_data_to_client',
          function(d) {
            var blobUrl = window.URL.createObjectURL(new Blob([s2ab(atob(d.bin))], {type: \"application/octet-stream\"}));
            downloadFile(blobUrl, d.name)
          });
        });
      ")
    ),
    
    titlePanel("sendBinaryMessage example"),
    fluidRow(
      column(4, wellPanel(
        actionButton("controller", "Test"),
        
      ))
    )
  ),
  
  
  server = function(input, output, session){
    
    observeEvent(
      input$controller,
      {
        file_path <- "./www/test.xlsx"
        data <- list(
          name = "test.xlsx",
          bin = readBin(file_path, what=raw(), n=file.info(file_path)$size)
        )
        session$sendCustomMessage(type = "send_data_to_client", message = data)
      }
    )
    
  })
)