[WIP] New Vue Web UI

[Tofandel]

This pull request is a work in progress and will in term create a new user interface using Vue replacing the old jquery lib and templates logic and dividing the size of the html build by 2 I am estimating a 150Kb image non gziped, 40Kb gziped, will depend a lot on build flags for integrations, I guess we'll see (In comparison the current image is 213Kb, 58Kb gziped)

Components:

• [x] Menu
• [x] Form
• [x] Input
• [x] Repeater for templates
• [x] Button
• [x] Create components for each template
• [x] Separate each tab into it's own component for readability
• [x] App - The container
• [x] Grid (Row and Col) - to replace pure-u

Organization

• [X] Move debug to status
• [X] Move wifi to general
• [X] Move NTP to Admin
• [X] Move schedules to colors and switches templates
• [X] Change menu order

Javascript

• [ ] Rewrite all functions into methods
• [x] Change all name="**" to vue data

Css / styles

• [x] Import and purify+minify (will remove unused selectors on build) previous vendor css
• [x] Simplify button classes (bootstrap like btn-success)
• [x] How about a new favicon?
• [x] It's 2020 so remove all browser dependent's css (-webkit, -ms, -moz etc)

Build process

• [x] Add new conditional for each integrations
• [x] Configure the build process (Currently 40Kb Gziped without the most part rewritten)
• [ ] Implement a npm build process on PIO compilation
• [x] Create env flags and conditions for different builds

HTML

• [ ] Review all the code and remove what's not useful anymore
• [x] Simplify the Pure markup which takes too much place

General

• [x] Add more options in UI instead of build flags

Core

• [x] Refactor websocket output into modules
• [x] Implement the new/moved options

Tests

• [ ] Create jest unit tests for components
• [ ] Create jest unit tests for websocket
• [ ] Integrate testing in travis

PWA

• [ ] Create a new build mode "PWA" that will be the default, it will generate a full PWA app that can be hosted on espurna.io for all users or locally as well, and the device will not need a web interface since the PWA will be accessible offline reducing the build size considerably
• [ ] Use localStorage to store password and list of devices and authenticate automatically

[tonilopezmr]

I :) I'm learning web development for embedded applications and I'm testing preact and just curious about Vue.

Can you explain in the PR message which will your strategy for build the project, also it is important to test the website, have different components and in the build create a single HTML with all styles and JS. How are you going to manage that?

After some research using LitElement and preact, preact is way smaller, with the same application LitElement is 21Kb size and Preact 4Kb. I used Webpack to inline the css and JS code in a single HTML

Thanks for contributing to that awesome website!!

[Tofandel]

Build will be made by webpack so with a bit of configuration it will handle the minification of html/js/css and inlining them into one html which then will need to be gziped, it by default already packs all components/scripts into one app.js and css into one app.css so not much left really

Never heard of preact but vue makes a lot of sense for single page UIs it's really easy to understand what's happening and having worked with react the component logic and writing is simpler (meaning more intuitive for the community to support and more lightweight as well)

https://vuejs.org/v2/guide/comparison.html#Preact-and-Other-React-Like-Libraries

4kb is indeed a very small framework size, I wonder if they include all important features of react

[tonilopezmr]

Yes, Vue was another alternative I studied but I've never used it before (I used react) and it was easy for me to create a small application to test preact, it includes the important features.

Thank for your explanation 🎉 🎉 , I hope I can test Vue as well to see the comparison.

[mcspr]

Configure the build process Implement a npm build process on PIO compilation

Some previous attempts were here #970 and #1680 , but the way it was done is by updating PlatformIO SCons filelists. Recent platformio versions make it so that each environment has it's own scons file list, so if I were to add web files to the track list there it would always rebuild the UI when the firmware is built. It can also not do that and just call npm-build and hope it did something (but the question is now, how it would know about which modules it wants)

Configuration should be easy enough to finally implement, but all the versions so far used to run gcc preprocessor and manually parse the result via regexp. I wonder if that could be avoided somehow and still show an expected build config.

As you mentioned in the other issue, backend updates can go straight to the dev optionally / updating existing webui too.

[Tofandel]

Little update: around 50% of the work is completed

[Tofandel]

Adding #1898 #1899 #1931 in the WIP queue A server is not needed, the data can be mocked with jest in unit tests or directly passed to the vue instance

See https://www.npmjs.com/package/jest-websocket-mock

[mcspr]

Create a new build mode "PWA" that will be the default, it will generate a full PWA app that can be hosted on espurna.io for all users or locally as well, and the device will not need a web interface since the PWA will be accessible offline reducing the build size considerably

:+1:

[Tofandel]

Anybody wants to make a nice svg icon for Espurna? Would be nice for the PWA and the favicon

This is how the UI currently looks

[tonilopezmr]

Create a new build mode "PWA" that will be the default, it will generate a full PWA app that can be hosted on espurna.io for all users or locally as well, and the device will not need a web interface since the PWA will be accessible offline reducing the build size considerably

How can you deal with the device locally using PWA? You need to write the IP or something like that? I mean, how are you going to handle that? with a login selecting the IP?

Great work! I'd help to add things when it finishes

[Tofandel]

@tonilopezmr Originally yes, but I'm writing a clever way to discover local devices via the PWA, the app will include a new header: "Powered-by: espurna, version" on a new unauthenticated endpoint

I can call a webRTC service to get the local IP address of the device and then ping all the local ips on the new endpoint exposing only their names without authentication and list them

The user can then choose from the list and authenticate himself when the device is selected

The scripts also detects ip change so that when you connect to an unconfigured device it will automatically connect to it

If the device is outdated, it will instead of proposing a connection, propose to upgrade by sending the .bin file

I will also add an option later on to remember each device auth individually via localstorage or similar

~~This will however require all devices to be SSL enabled, so there might be some work required on that before we can make the PWA a default for users~~

Edit: After some checking, a PWA doesn't necessarily need SSL (contrary to what I read), meaning no problem to connect to non SSL websockets, so PWA is a go :confetti_ball:

[mcspr]

Anybody wants to make a nice svg icon for Espurna? Would be nice for the PWA and the favicon

This is how the UI currently looks image

From recent memory, @xoseperez used this https://twitter.com/espurna_io/status/1172545472742993920 as a graphics for twitter account & stickers. So it is either this one or more high-res current one?

[tonilopezmr]

Can I propose some new logo for a new webpage?

Here is my proposition based on the current one, what do you think? @xoseperez

[Tofandel]

@tonilopezmr Nice, I think we'll stay with the triangle, the round resembles the messenger icon too much :p Do you have an svg file of this?

Could you try with this color #479fd6?

[tonilopezmr]

haha !! That's right, I don't use facebook messenger :P

With the new color you suggested:

What do you think about this new spark rotation? Spark and the triangle have parallel lines. 👀

@Tofandel let me know what do you want in SVG, and when @xoseperez and @mcspr decide about what to do with these logos, I can share more svg's !

By the way: If you have any font preference for the ESPURNA name in the logo, let me know and I can create vertical and horizontal logos with the logo + espurna name.

[Tofandel]

@tonilopezmr I prefer the first triangle without the rotation, so you can share an svg of this one, for the fonts, better use websafe fonts like sans-serif, Arial or Helvetica

[tonilopezmr]

Here is the triangle without rotation, you can change the colors of the spark: fill="#FFF" and the triangle fill="#479FD6"

<svg width="314" height="274" xmlns="http://www.w3.org/2000/svg"><g fill="none" fill-rule="evenodd"><path d="M142.470398,9.29075342 C145.007501,3.80554959 150.559334,0 157,0 C162.276925,0 166.957198,2.55456904 169.871611,6.49449779 L169.878436,6.49432499 L169.905835,6.54093835 C170.54666,7.41379758 171.101067,8.35405427 171.556447,9.34909967 L311.349361,247.1767 C313.024484,249.706016 314,252.739086 314,256 C314,261.491018 311.233941,266.335988 307.019193,269.217539 C307.01146,269.233639 307.003967,269.24842 306.996715,269.261882 C306.501106,270.181799 303.571264,271.482348 298.207188,273.163528 C125.158239,273.659028 31.4239887,273.927513 17.0044381,273.968984 C16.6722676,273.989559 16.337353,274 16,274 C15.6943218,274 15.3906458,273.991428 15.0891991,273.974511 C15.0557934,273.97461 15.0388355,273.974661 15.0383253,273.974665 C15.0337382,273.974704 15.0209092,273.972899 14.9998386,273.969248 C6.62918507,273.452972 0,266.500651 0,258 C0,254.485714 1.13300007,251.236055 3.05364923,248.596373 L142.470398,9.29075342 Z" fill-opacity=".969" fill="#479FD6"/><path d="M92.4038518,179.458661 C87.6396803,178.694444 84,174.564946 84,169.585354 C84,167.267315 84.78871,165.133489 86.1124116,163.437594 C86.9749307,161.236318 89.2761341,158.851482 93.0160219,156.283085 C116.103709,140.427438 143.567307,123.470158 152.210043,118.049116 C163.060152,111.243529 180.224216,101.56049 203.702234,89 C198.356849,95.111686 190.407299,103.052795 179.853583,112.823327 C169.299868,122.593858 153.02397,137.080445 131.02589,156.283085 L206.242482,156.604776 C206.243996,156.60761 206.245509,156.610443 206.247022,156.613277 C206.495594,156.594772 206.746697,156.585354 207,156.585354 C212.522847,156.585354 217,161.062507 217,166.585354 C217,169.298781 215.919283,171.759793 214.164801,173.561437 C214.163691,173.572256 214.162351,173.582704 214.160782,173.592781 C213.900224,175.265948 210.554329,178.029136 210.412911,178.144485 C198.867208,187.561875 185.452608,197.045166 179.853583,201.124744 C157.882644,217.133272 142.00794,226.230791 136.270846,230.139256 C127.749732,235.94437 114.474758,243.840366 96.4459235,253.827246 C112.511758,237.246684 124.599765,225.004058 132.709944,217.099368 C140.820124,209.194678 153.510403,197.286831 170.780781,181.375827 C134.988268,181.058701 114.605265,180.844374 109.631774,180.732847 C105.575529,180.641889 100.881325,180.681646 96.6848067,180.26461 C96.0521615,180.201739 94.6294797,179.943615 92.4167613,179.490238 L92.4038518,179.458661 Z" fill="#FFF"/></g></svg>

[Tofandel]

Super compressed svg (without the triangle and with the primary color) <svg width="256" height="256" viewbox="0 0 256 256" xmlns="http://www.w3.org/2000/svg"><path transform="matrix(.91 0 0 .91 -20 .23)" d="m81 154c-17-3-18-22-3.6-33l155-100-101 101 113 1.4c19 1.5 13 23-3.4 33l-159 104 106-103z" fill="#ff952f"/></svg>

[Tofandel]

Favicon is lit like this :)

[tonilopezmr]

@Tofandel How did you compress the svg ?? :O The favicon can be an svg?

I love it 👏 👏 👏

[Tofandel]

@tonilopezmr A favicon can be an svg but it's not well supported (safari, firefox, opera only, https://caniuse.com/#feat=link-icon-svg), I can still have a 32x32 or 16x16 .ico generated in the non pwa app

I compressed it by reducing the curved lines needed (only 2 curves the rest are lines) and then in inkscape when exporting I set the decimal number to 2 instead of 6, final optimization by hand (remove the group and useless html attributes)

[Tofandel]

70% rolled, depending on the weather it will be finished in 2 or 3 weeks

[tonilopezmr]

When the web finished I'd like to help testing it, now I have more knowledge about web development what do you think about forgetting about unit testing and add more acceptance tests doing screenshots of the website depending on some states.

Screenshot testing lets you create super easy tests taking screenshots of the webpage with the desired state

[Tofandel]

@tonilopezmr You can integrate both, but it's always a good idea to have unit tests for functions/components to avoid edge cases that you wouldn't really get in acceptance testing and there is also things you cannot check that way (eg: make sure the data received and sent is the correct format)

Screenshots tests are more of a general appearance check rather than a true feature test, I would call it a testing aid (it may point out a new issue very fast) but doesn't replace unit testing

Also while having 10 screenshots to check is fine, having 100 screenshots to check on each release would be prone to human error and some bugs will definitely be missed, so I would be inclined to do a maximum of unit tests and just a few screenshot as acceptance testing aid (1 per tab for example, to make sure the layout is correct and no field is missing or that the design is broken)

It would however be a great tool to check for css issues in different browsers (Right now I'm only checking in chrome but it would be nice to have safari, mozilla, edge and opera)

And to check responsive design (phone, tablet, laptop)

[tonilopezmr]

I agree with you, it was only a solution to create a test easily than unit test if people don't do it. It's better to have a screenshots test than nothing. I think the best way is the scenario you said. 👍

[mcspr]

I might need some help getting the public API built though (To output device name, description, board name and espurna version on a public endpoint)

Some time ago there was this example of local discovery app. I simply iterates local subnet sequentially and tries to open up /discovery with some public data: https://github.com/KimNyholm/espurna-device-locator/blob/master/src/services/ESPurnaDeviceClass.js Maybe device itself could help with discovery a bit, through mDNS or raw UDP multicasts since it is not restricted by the browser stack

BTW, how one would go about adding a module? Effectively App.vue hides loaders manually, I would need to type out everything like that? Or is there some way to have "sources" / directories with components that are loaded dynamically? e.g., if user does not want to change original sources at all to allow easy updates between versions

[Tofandel]

BTW, how one would go about adding a module? Effectively App.vue hides loaders manually, I would need to type out everything like that? Or is there some way to have "sources" / directories with components that are loaded dynamically? e.g., if user does not want to change original sources at all to allow easy updates between versions

I'm not sure I understand, are you talking about a user creating a module? Yes right now it needs to be added to App.vue, anyway creating a module requires a bit of knowledge of vue so I don't think it's a huge problem to have to add the import manually

A dynamic loader that would import all files would be simpler but would not allow conditional import and would result in a max bundle size all the time

Though possible https://stackoverflow.com/questions/29722270/is-it-possible-to-import-modules-from-all-files-in-a-directory-using-a-wildcard

After there can be another directory for external components, where we wildcard and add them to the menu automatically (pretty simple and takes 5 lines of code)

For the PWA

Iterating on the subnet is what I'm doing for now (with auto detection of the subnet) But does the /discovery endpoint exists? And is it public?

[mcspr]

What I meant are these modules, if I correctly understood the loading process:

            <!-- #if process.env.VUE_APP_RFBRIDGE === 'true' -->
            <template v-if="data.modules.rfb" #rfb>
                <Rfb v-bind="data"/>
            </template>
            <!-- #endif -->

As previously mentioned, webpack conditional loader will strip this from the resulting bundle when env VUE_APP_RFBRIDGE != "true". Thus I was wondering if that could be generated somehow, akin to concatenation of code/espurna/*.ino for the firmware itself but with some conditional loading still present e.g. <filename> / <module-name> is only added when VUE_APP_<uppercase-filename-or-module>="true"

---

=> /discover endpoint is without any auth

[Tofandel]

Okay I see, that would require a code generator of some sort or a custom loader

I don't know of any webpack plugin that would allow this but it may be possible to edit webpack conditional loader to get a code preprocessor basically, but that's not a priority, being 5 line per module that can be copy pasted

The normal solution would be to use a dynamic import but that wouldn't work in a single compressed file like currently

=> /discover endpoint is without any auth

Wow, just tried it, I just need to add the description in that, that's amazing, gonna save a lot of work

[mcspr]

No problem. Looking at https://github.com/caiogondim/webpack-conditional-loader/blob/master/src/index.js, transformation is pretty straightforward anyway (so I guess hiding things like import Rfb ... would work too)

[mcspr]

Kind-of a ping with questions:

• anything I can improve from ws / api side of things?
• does schema / arrays still make sense or is there some way better to stream data to the UI? Keeping in mind RAM requirements previously discussed
• from semi-recent gitter comments, Xose had idea about generating "schema-like" descriptions of sensors to generate UI elements. Considering some options for this, would making the device dump array like {group, cfgkey, type, default} (e.g. {wifi, ssid, CfgType::List, ""}) make sense for such a thing? I am still thinking of how this would even work without eating RAM, but I wonder...

[Tofandel]

@mcspr Quite busy so don't have time to roll it up for beta testing yet

There is a few settings I added on the interface that don't exist yet

Under relays:

• dblDl (int, time in ms time in between two clicks to retain a double click, replaces a general setting to a relay basis, prev setting btnDelay, flag BUTTON_DBLCLICK_DELAY)

• lngDl (int, long click delay in ms, replaces a general FLAG on a relay basis, prev flag BUTTON_LNGCLICK_DELAY)

• lngLngDl (int, long long click delay in ms, replaces a general FLAG on a relay basis, prev flag BUTTON_LNGLNGCLICK_DELAY)

• sndAllEvts (boolean, to send all mqtt events or not, replaces a general FLAG on a relay basis, only if module mqtt, prev flag BUTTON_MQTT_SEND_ALL_EVENTS)

I think that's all I added, they are in the schema sent already but not used yet

If you could implement those options on the .ino side that'd be amazing (on this PR or on dev up to you)

I agree that the sensors could use a little refactoring but I am not too familiar with it, maybe first let's roll this out and then think on how to refactor the sensor to minimize feature change in this PR

[mcspr]

idk if you are testing something, but what is up with newline removal of debug? why is it needed?

[Tofandel]

Just experimenting

[mcspr]

Under relays:

*buttons? as: btnDblDl <-> BUTTON_DBLCLICK_DELAY (btw this already was btnDelay) btnLngDl <-> BUTTON_LNGCLICK_DELAY btnLngLngDl <-> BUTTON_LNGLNGCLICK_DELAY btnSndAllEvts <-> BUTTON_MQTT_SEND_ALL_EVENTS

All of those are indexed, i.e. per button?

[mcspr]

While you are looking at HA module, check out https://github.com/xoseperez/espurna/compare/dev...mcspr:ha/ext-structs

Idk if this is still relevant, but the idea is to put string-string container on top and generate either yaml of json from that. c++ is weird though, since we probably need std::unordered_map for hashmap instead of std::map, b/c it uses rbtree + binary search comparisons (and idk how well that works huh)