Error: ruby: trying to pop an empty stack in rule

[jrieken]

• create a ruby model
• paste the code below
• 💥 see the error below

some text - ruby
some text - #-------------------------------------------------------------------------
some text - # Copyright (c) Microsoft. All rights reserved.
some text - #--------------------------------------------------------------------------
some text - 
some text - module Azure
  module Blob
    class Blob

      def initialize
        @properties = {}
        @metadata = {}
        yield self if block_given?
      end

      attr_accessor :name
      attr_accessor :snapshot
      attr_accessor :properties
      attr_accessor :metadata
    end
  end
end

Error: ruby: trying to pop an empty stack in rule: tokenizer.root: ^(\s*)([a-z_]\w*[!?=]?)
    at Object.throwError (http://localhost:8080/vscode/out/vs/editor/common/modes/monarch/monarchCommon.js:57:15)
    at MonarchLexer.tokenize (http://localhost:8080/vscode/out/vs/editor/common/modes/monarch/monarchLexer.js:219:43)
    at MonarchLexer.tokenize (http://localhost:8080/vscode/out/vs/editor/common/modes/monarch/monarchLexer.js:287:41)
    at MonarchLexer.tokenize (http://localhost:8080/vscode/out/vs/editor/common/modes/monarch/monarchLexer.js:272:29)
    at _loop_1 (http://localhost:8080/vscode/out/vs/editor/common/modes/supports/tokenizationSupport.js:122:43)
    at TokenizationSupport._myTokenize (http://localhost:8080/vscode/out/vs/editor/common/modes/supports/tokenizationSupport.js:178:31)
    at TokenizationSupport.tokenize (http://localhost:8080/vscode/out/vs/editor/common/modes/supports/tokenizationSupport.js:46:29)
    at Model.TextModelWithTokens._updateTokensUntilLine (http://localhost:8080/vscode/out/vs/editor/common/model/textModelWithTokens.js:460:60)
    at Model.TextModelWithTokens._revalidateTokensNow (http://localhost:8080/vscode/out/vs/editor/common/model/textModelWithTokens.js:420:22)
    at http://localhost:8080/vscode/out/vs/editor/common/model/textModelWithTokens.js:373:27

---

Playground link: https://microsoft.github.io/monaco-editor/playground.html?source=v0.36.0-dev.20230224#XQAAAAKsAwAAAAAAAABBqQkHQ5NjdMjwa-jY7SIQ9S7DNlzs5W-mwj0fe1ZCDRFc9ws9XQE0SJE1jc2VKxhaLFIw9vEWSxW3yscw4l5XSjQNeA7FQlbpOv2DA3EMY8CYQ3CFqhMFQnNbwxh6oNnQu4zdOiVyZGIB-vxwfpQbbeL25j50ItdHT-fQwzmNrusFBv0v9iXe5FWKDdzF8qMWVilBieAxIj4YzEs2HvBWFa6oHrPfbFjcfukVP7Y67jQn7LR_78a0e5lCiJ4f0jt-ZfrCbQVBQx7p-rHH-01ebDeFknzHKwwPo1YSMk7mUwuhixaQ3Jxn2h39O8zbDkTDADlUx5vni1MWpdhr0ycH36ylOun0HyTXX2WiY9-fUjcEm_4jYcBQG0Mek-78kaAHK--3TjOSfxeSxy13tgflHn1b5iO--GLIKgJ6YiUsPWGq-hEpK3wWTHasBaWjhmZACQjceGdbzyzIa9_Ri01WzmgBwuAojyHSNACyOtGUrpuUpCy4R10K3aBBTYjBfgANlYcM1yYvaw5RMxNmI9PJBp_rKMUXikHmHMWDQggW3AtRM34-_6k-Zu8dMGCywbowkcnsX_PElAHJvBSib_iCWLzTvp_dSrwy73f6Ymr_gavT2Q

[nic-hartley]

Anyone mind if I look at this? It looks interesting. (@alexandrudima @jrieken)

[alexdima]

PR welcome, although beware this is complicated.

[XDfield]

same here. I use monaco.editor.colorize to colorize some ruby code. It raises the error:

TypeError: Cannot read property 'name' of null
MonarchTokenizer../node_modules/monaco-editor/esm/vs/editor/standalone/common/monarch/monarchLexer.js.MonarchTokenizer._myTokenize
node_modules/monaco-editor/esm/vs/editor/standalone/common/monarch/monarchLexer.js:539
  536 | }
  537 | else if (action.next === '@pop') {
  538 |     if (stack.depth <= 1) {
> 539 |         monarchCommon.throwError(this._lexer, 'trying to pop an empty stack in rule: ' + rule.name);
  540 |     }
  541 |     else {
  542 |         stack = stack.pop();

[bbogason]

I am getting a similar error for this ruby file (which looks legit to me)

# frozen_string_literal: true

class BaseModelConfiguration
  attr_reader :table_properties
  attr_reader :signatures
  attr_reader :links
  attr_reader :form_fields
  attr_reader :entity
  attr_reader :model_name
  attr_reader :edited_tabs
  attr_reader :api_hash
  attr_reader :cur_page

  def initialize
    @table_properties = []
    @signatures = []
    @links = []
    @form_fields = []
    @entity = nil
    @model_name = nil
    @edited_tabs = nil
    @cur_page = nil
    @api_hash = {}
    construct
  end

  public def construct; end

  def transform_params
    out_params = ''
    form_fields.each_with_index do |attr, index|
      backurl_cond = attr[:caption][:class] != 'backurl'
      entity_file_cond = attr[:block][:name] != 'field_for_model/file'
      hidden_field = !attr[:block][:content][:permit_hidden]
      view_text = (attr[:block][:name] == 'field_for_model/input' && attr[:block][:view] == 'text')

      sep_cond = (index != 0) && entity_file_cond && hidden_field && backurl_cond && !view_text && out_params != ''
      sep = if sep_cond
              ', '
            else
              ''
            end
      out_params += "#{sep}:#{attr[:block][:content][:value]}" if entity_file_cond && hidden_field && backurl_cond && !view_text
    end
    "params.require(:#{@model_name}).permit(#{out_params})"
  end
end

Error
ruby: trying to pop an empty stack in rule: (unknown)

Error: ruby: trying to pop an empty stack in rule: (unknown)
at Object.createError (http://localhost:6006/vendors~main.iframe.bundle.js:324065:12)
at MonarchTokenizer._myTokenize (http://localhost:6006/vendors~main.iframe.bundle.js:325256:104)
at MonarchTokenizer._tokenize (http://localhost:6006/vendors~main.iframe.bundle.js:325050:25)
at MonarchTokenizer.tokenize2 (http://localhost:6006/vendors~main.iframe.bundle.js:325042:33)
at safeTokenize (http://localhost:6006/vendors~main.iframe.bundle.js:262809:37)
at TextModelTokenization._updateTokensUntilLine (http://localhost:6006/vendors~main.iframe.bundle.js:262733:23)
at TextModelTokenization._tokenizeOneInvalidLine (http://localhost:6006/vendors~main.iframe.bundle.js:262719:14)
at TextModelTokenization._revalidateTokensNow (http://localhost:6006/vendors~main.iframe.bundle.js:262670:40)

[hediet]

Still reproduces:

https://microsoft.github.io/monaco-editor/playground.html?source=v0.36.0-dev.20230224#XQAAAAKsAwAAAAAAAABBqQkHQ5NjdMjwa-jY7SIQ9S7DNlzs5W-mwj0fe1ZCDRFc9ws9XQE0SJE1jc2VKxhaLFIw9vEWSxW3yscw4l5XSjQNeA7FQlbpOv2DA3EMY8CYQ3CFqhMFQnNbwxh6oNnQu4zdOiVyZGIB-vxwfpQbbeL25j50ItdHT-fQwzmNrusFBv0v9iXe5FWKDdzF8qMWVilBieAxIj4YzEs2HvBWFa6oHrPfbFjcfukVP7Y67jQn7LR_78a0e5lCiJ4f0jt-ZfrCbQVBQx7p-rHH-01ebDeFknzHKwwPo1YSMk7mUwuhixaQ3Jxn2h39O8zbDkTDADlUx5vni1MWpdhr0ycH36ylOun0HyTXX2WiY9-fUjcEm_4jYcBQG0Mek-78kaAHK--3TjOSfxeSxy13tgflHn1b5iO--GLIKgJ6YiUsPWGq-hEpK3wWTHasBaWjhmZACQjceGdbzyzIa9_Ri01WzmgBwuAojyHSNACyOtGUrpuUpCy4R10K3aBBTYjBfgANlYcM1yYvaw5RMxNmI9PJBp_rKMUXikHmHMWDQggW3AtRM34-_6k-Zu8dMGCywbowkcnsX_PElAHJvBSib_iCWLzTvp_dSrwy73f6Ymr_gavT2Q

[onassis-turing]

any workarounds for this? this issue is still present

[OfekShilon]

Maybe I'm missing something? The original example isn't valid ruby - the text at the first lines isn't commented and there's an extra end at the closing line. Anyway here's a smaller, valid ruby example demonstrating the same symptom: https://microsoft.github.io/monaco-editor/playground.html?source=v0.36.0-dev.20230224#XQAAAALsAQAAAAAAAABBqQkHQ5NjdMjwa-jY7SIQ9S7DNlzs5W-mwj0fe1ZCDRFc9ws9XQE0SJE1jc2VKxhaLFIw9vEWSxW3yscw0N99wfdL6lWdZifSWhc9gCAnQP7m4UuZTiIrIS9pAX04f3NkJ4AQau3w4ptki3gIWzH16T25difg-USIlgIuHWaQW4UjFBxI-jBU9CCZvEu13dcLUzxHa2RS-cZPn3SUFjJ7GMRNa5TRRLvbbknTUUPnGnxar4EkxtMcz3UWzcPw7gwPrXA2ETDLZD6pzubSr6d8RLpF8vTYQgsoyeOHr_Z66mbCKiQdHphYNw6-4DDZJowvAFS07YRNc_AYYry6Pgx71Q5ZWjji0oU8xenjhZCQgdFNCCnXZI2PY5EFiP22l-TE_riTe3Rsjg442seSPP4HgMSS-Ahxk08IovLRCBPy0kgz3jYjovcFvf9JTPIA

It seems the problem has to do with consecutive ends. In this particular example the last end line isn't tokenized properly.

[OfekShilon]

@alexdima ?