flowfuse
flowfuse copied to clipboard
FlowFuse
Onboarding: FlowFuse Self-Hosted
Description
May be some overlap with #3839
This Epic is to capture hurdles and areas for improvement for the onboarding process for FlowFuse Self-Hosted.
Review
As part of this exercise, we will be conducting reviews of the full installation and setup of FlowFuse in self-hosted environments.
For each scenario it is important to have a blank slate, install FlowFuse and setup FlowFuse with the relevant settings such that it is possible to run an instance of Node-RED in FlowFuse.
### Scenarios
- [ ] https://github.com/FlowFuse/flowfuse/issues/4571
- [ ] https://github.com/FlowFuse/flowfuse/issues/4573
- [ ] https://github.com/FlowFuse/flowfuse/issues/4555
Any friction, concerns and areas for improvement should then be documented in the following tasks lists which are broken into two sections:
Installation
The process of retrieving a copy of FlowFuse and running in a particular environment. Ideally, this would be a single line install in some capacity. This can also cover improvements for onboarding resources like documentation/videos.
### Installation
- [ ] https://github.com/FlowFuse/flowfuse/issues/4570
- [ ] https://github.com/FlowFuse/flowfuse/issues/4574
- [ ] https://github.com/FlowFuse/flowfuse/issues/4572
- [ ] https://github.com/FlowFuse/customer/issues/249
- [ ] Remove "flowforge" mentions from our installation docs
- [ ] https://github.com/FlowFuse/flowfuse/issues/4631
- [ ] https://github.com/FlowFuse/flowfuse/issues/4632
- [ ] https://github.com/FlowFuse/flowfuse/issues/4633
- [ ] https://github.com/FlowFuse/flowfuse/issues/4634
- [ ] https://github.com/FlowFuse/flowfuse/issues/4635
- [ ] https://github.com/FlowFuse/flowfuse/issues/4636
Setup & Configuration
Having an available instance of FlowFuse, this is a review/assessment of the setup steps required to then having a running instance of Node-RED.
### Setup & Configuration
- [ ] https://github.com/FlowFuse/flowfuse/issues/4638
- [ ] Idea: for quick-start purposes allow to run pre-configured app and present login page instead of setup
Which customers would this be available to
Self-Hosted
Lots of issues with wildcard DNS. Larger orgs do not like the vulnerability it poses and it can take a month or more to get approval just for DNS, delaying evaluation.
Customers/prospects that had an issue:
https://app-eu1.hubspot.com/contacts/26586079/record/0-2/7845572820 https://app-eu1.hubspot.com/contacts/26586079/record/0-2/9080239048 https://app-eu1.hubspot.com/contacts/26586079/record/0-2/11792021958
The wildcard DNS requirement is a fundamental design point of the Docker & Kubernetes implementation, it can not be removed.
On Kubernetes if the user is prepared to use External DNS then the system can create each DNS entry individually, but this requires granting K8s the ability to dynamically update the DNS which is likely to be even more unwanted than a wildcard entry.
One pain point for both docker/k8s is the need to create the config files needed to run the platform. We should provide interactive scripts to generate the files. Have added items for both to the tasks above.
We have a script from the AWS/Digtial Ocean build to build the config file for docker, we should be able to reuse that
https://github.com/FlowFuse/digital-ocean-droplet/blob/main/files/opt/flowforge/first-login.sh
How technical are we seeing our customers that install FF locally? Is there first question here that a user is going to know the answer to: "What operating system are you running?"
For manufacturing clients, are they even going to know k8s/docker as assets? We're recently seen the challenges of getting FF running in Windows given the requirement for Docker Desktop too?
Regarding the guided script you mentioned @hardillb - would that be guiding users through https://flowfuse.com/docs/install/docker/#configuring-flowfuse?
Yes, it does the search/replace for the domain and email settings in the flowforge.yml and the docker-compose.yml files
Chatting with @ppawlowski about what questions user's are likely to have:
- Are you installing for evaluation or for production environment?
- Are you deploying into a Cloud environment (e.g. AWS) or dedicated hardware (e.g. factory floor)
- If on Dedicated Hardware - What operating System?
What other considerations do they need to have when deciding with of the four paths (Docket, k8s, Do, Docker on AWS) that a user would need to know?
@joepavitt Any updates on this issue? I'd love to understand what's being done to have a bigger install base for FlowFuse
@robmarcer is going to provide a flow chart that details the general customer workflow we get, which roles we're talking to, and the path we find for installation/configuration of FF
@ppawlowski as part of your exercise here, could you produce a simple checklist of things that a user needs to actually setup/install, in order to have an instance of FlowFuse running please? e.g. DNS Records, SMTP Server for e-mail notifications, TLS configuration
Chatting with Piotr - "Quickstart Guide" option on the install page, "Docker Compose, 2/3 commands and you are done"
e.g. https://doc.traefik.io/traefik/getting-started/quick-start/
Just having a call with @ppawlowski - just under 50% of traffic to "Install Self Hosted FlowFuse" then click the button "setup an alternative to DNS".
I know this has been brought up before, but it's the single biggest point of friction when setting up FF locally. Just want to make sure we have documented the why on this. @hardillb can you give a quick TLDR please, I know we've discussed this before, just want to document here.
Also, general update on flow here:
- 40 people to the page
- 25 people clicked the "copy" for "Step 2: Download Compose File"
- 12 clicked "copy" for "Step 3: Start the Application"
- 9 people clicked to get more info on the setup in the "guide"
The DNS wildcard entry is required so that each instance can has it's own hostname (on a single shared sub domain) without having to manually set up a DNS entry for the instance at the time it is created. It basically *.ff.example.com points to the IP address of the Docker host machine where * means anything
Each Node-RED instance requires it's own hostname (or what the browser considers to be, e.g. different port numbers for localfs) for the following reasons:
- It provides the security boundary between instances (e.g. one instance can have basic http auth and another have FF User auth and not clash). This explicitly not for the Editor (why we can do path based proxying for the remote device editor)
- Keeps web applications hosted in the instances sandboxed from each other.
- The reverse proxying to expose the containers from inside docker uses the hostname to determine which container to route to
The goal is for self-hosted customers to have the easiest path possible from clicking to download FlowFuse to deploying a Node-RED instance. Along with https://github.com/FlowFuse/flowfuse/issues/5950, these serve as focus areas for the next release: improving the self-hosted setup experience.
We will review the different paths a user can take and ensure that there are either technical solutions or conveniently accessible education solutions provided to the customer.
@ppawlowski Please validate if the wildcard DNS requirement applies outside of Docker/Kubernetes. What about Windows?
The work for this issue will be to discover the smoothest path(s) possible to SH installation.
@ppawlowski Please validate if the wildcard DNS requirement applies outside of Docker/Kubernetes. What about Windows?
It applies everywhere.
@gstout52 The only "native" install on Windows is LocalFS and that is not a practical solution for a production deployment.
Docker on Windows comes with extra licensing costs from Docker, and while you can add Windows nodes to a Kubernetes cluster it's a specialised use case, and we do not build any Windows native containers.
In short Windows is just not a practical platform to deploy FlowFuse on (running a device agent on Windows perfectly fine, but not the full stack).
@dgatti0213 I would appreciate your thoughts on this thread in light of https://github.com/FlowFuse/flowfuse/issues/6214
To be crystal clear here wildcard DNS (or by granting the FlowFuse environment the ability to create it's own DNS records) is a fundamental requirement of the product.
There is no way round it
To be crystal clear here wildcard DNS (or by granting the FlowFuse environment the ability to create it's own DNS records) is a fundamental requirement of the product.
There is no way round it
Isn't wildcard DNS only required for creating Hosted instances of Node-Red within the Flow-Fuse platform?
Isn't wildcard DNS only required for creating Hosted instances of Node-Red within the Flow-Fuse platform?
That is the core premise of the product....
But it is also used to provide hostnames for a number of other features like MQTT broker and NPM registry.