Console-like output without new tab with mixed content [idea]

[Spiritdude]

I simplified the web layout, and added an #output DIV, which now contains all the status messages. In index.js I added stdout(s) and stderr(s) which adds the messages there, adds a scrollbar if required and keeps bottom seen (autoscroll). It behaves like a read-only terminal output (the trailing '_' just indicates the end of the output).

Perhaps later it can become an interactive console, so we can enter commands as well, which either are processed in the browser or sent to the device (this comes closer to LUA WebIDE at https://github.com/jumjum123/ESP8266WebIDE which seems abandoned).

Depending on the MIME of the returned content from the httpserver we can determine how the content is rendered, e.g. text/html fully replaces entire content of DIV #output, if text/plain we add it content with <pre> + s + </pre> or so. So the dofile("httpserver-header.lc") call we hide and make it more approachable, keeping some notion of stdout/stderr/web from the server side to the webgui knows how to render.

At the end some apps like this should be possible:

printw.select('stdout,console')  -- the default
printw("hello world")

and when executed via the terminal (without the browser) it outputs via the terminal; when executed within the WebIDE and its httpserver it would return text/plain and X-WEBIDE-SELECT: stdout so it outputs nicely in the WebIDE. One code-base but behaves as hoped or expected depending in which context (serial or webide).

printw.select('web')
printw("<h1>hello world</h1>")

would render html content direct in #output DIV (replace all previous content).

Nice to have would be to override print() itself but not sure how this can be done.

• console: the NodeMCU serial console
• stdout: the normal WebIDE output
• stderr: the slightly more prominent WebIDE output
• web: full html content

[HHHartmann]

There is a feature in the NodeMCU Lua to redirect input/output to Lua code which allows a simple console redirecting which can be used to redirect print as well.

See here for documentation and an example.

Unfortunately it cannot (yet) redirect stderr. But there will propably be a workaround for that soon: softuart example

As for the printw you suggest: Adding configuration for importance of output could also be done. printw.level("debug") -- or trace/ ... Then use printw.debug("debug message") or printw("message", "debug")

[Spiritdude]

Where would I add the new LUA code of printw/print redirects?

[HHHartmann]

Maybe it would even be possible to redirect the print statement. Maybe something like

oldprint = print

function print (...)
    printResult = ""
    for i,v in ipairs(arg) do
        printResult = printResult .. tostring(v) .. "\t"
    end
    printResult = printResult .. "\n"

    if doprint then
        oldprint(printResult) 
    else 
        webprint(printResult) 
    end 
end

Just checked with pil on how to do variable number of params in Lua ;-)

It would have to be placed in some script which is executed best before starting httpserver.

Or if only redirecting is used you could start it as websocket script, restoring to serial output when the connection is lost. Note however that the esp can only take 5 simultaneous connections.

[Spiritdude]

I used node.output() in system-info2.lua I wrote (more compact and readable), I defined the _webout(s) as node.output(_webout,0) but it screws up internal state of the httpserver at least, I couldn't save changes, filelist got slow, and overall experience became highly unstable (PANICS). Even though the _webout() function was local as part of return (connection, req, args) ... it seemed to live on. So, I'm not experienced enough to determine the core of the problem here, so perhaps I postpone this part for now.

[HHHartmann]

Not exactly sure what you did. Maybe seeing the code would help. If you pass the _webout to node.output it does live on (and everything it uses) unless you reset it calling node.output(nil,0) which will drop the reference to it.

[Spiritdude]

function _webout(s)
    connection:send(s)
end
node.output(_webout,0); 

I added node.output(nil,0) before I the end, but it made no difference. It seems print is used elsewhere (likely in httpserver) and I might have to ensure _webout(...) and not just (s) single variable - I will check this further.

[Update]: I did _webout(...) but the same problem,

  function _print(...)
    local r = ""
    for i,v in ipairs(arg) do
      r = r .. tostring(v) .. "\t"
    end
    r = r .. "\n"
    connection:send(r) 
  end

node.output() redirect within .lua as processed in httpserver screws up internals:

PANIC: unprotected error in call to Lua API (attempt to yield across metamethod/C-call boundary)

when I use _print() direct (without node.output(_print,0)) it works fine.

Btw, why is dofile('httpserver-header.lc')(connection,200,'html'), why not just call a function, why reading a file? How efficient is dofile() with .lc?

[HHHartmann]

ok about the metamethod error I am not sure but I assume that it is because httpserver calls its client functions in a coroutine. the connection:send collects output until it is large enough to make sense to be sent. This is done by a content switch. Then a print of httpserver is hit and causes the error, or somewhere around the context switch(yield)

Using dofile is to save ram which is exxtremely tight on nodemcu (only around 45Kbyte) So the prpogram part does not stay in memory all the time. (Nope, no joke that's the kind of things you need to do)

There is a section in the docu about development of nodemcu which contains some more information.

[Spiritdude]

I started to read more thoroughly the nodemcu docs (I glanced over it before), here a funny passage in "Techniques for Reducing RAM and SPIFFS footprint" > "How do I minimise the footprint of an application?":

• If you are trying to implement a user-interface or HTTP webserver in your ESP8266 then you are really abusing its intended purpose. When it comes to scoping your ESP8266 applications, the adage Keep It Simple Stupid truly applies.

While working on nodemcu-webide to suit my needs I got a sense how fragile it is. I began to write some entries now in the WIKI here to describe my experience (often reseting the device to get a defined state) before/after uploading and testing my changes.

I give it a few more days, I have the slight impression that I might end up developing LUA more on the host, and use nodemcu-uploader or nodemcu-tool with a fine-tuned Makefile to upload to the device as the file editing within the httpserver/webide seems not reliable enough, and also eats up RAM.

@HHHartmann how does your setup look like, how to do develop apps/code on ESP8266? (If you prefer another way of communication, e.g. email or a chat let me know)

[HHHartmann]

Personally I feel no pressure to reduce SPIFFS footprint. 4 MB seems to be quite a lot.

In the readme of the httpserver it says as an answer to the intended purpose you mentioned above:

Let the abuse begin.

My setup is using ESPlorer to upload files and monitor the console. I use notepad++ to edit (at least supports highlighting of Lua code) I enhanced the Upload sample code included in httpserver to have access to all files as opposed to only the files in http/ and added a download button to download all filetypes including *.lua files. I recently tried ESP Cut, but it does not seem to work. Then I stumbled across this project and was hoping to be able to use it ...

I feel that we should not misuse the bugtracking system anymore but also be visible to the community and @Godzil .

[Spiritdude]

@HHHartmann contact me spiritdude AT gmail.com as I like to discuss general WebIDE ideas (with httpserver or alternatives), and/or @Godzil is welcome too :-)