No Licence Information

[nrbw]

I'm missing licence information and/or source code. What have you planned ?

[gjthompson1]

General Update

I'm going to use this issue to address some of the other issues and make some general statements about this project.

TLDR:

The desktop app is no longer supported. Going forward this project is a cloud native app that can be accessed here:

https://app.gluedata.io/

Open source vs Closed source

• I had always planned to be able to make money from this project.
• The original plan was to use github for releasing the software and issue / bug tracking / feedback.
• Open sourcing is always an option but at this time I have no intentions of doing that.
• I'm not against open sourcing aspects of the project. I could see open sourcing the editor server. However being able to open source certain parts and leave closed source other parts would require more resources than I currently have.

Desktop vs Cloud

• My original plan with the project was to create a desktop app that would allow interacting with local files and create a more native experience in general.
• After testing desktop out with different users and exploring other features that I would like to add I decided that desktop doesn't make sense as an initial MVP. Here are some reasons
  • I wanted to allow scheduling which wouldn't really work well if the users machine was off.
  • I wanted to allow larger jobs e.g. remotely running the graph e.g. Run this graph on 64GB of memory.
  • Supporting different OS requires more complexity for each feature slowing development down.
  • Testing on different OS was a time suck. Having to test Mac, Windows, Linux etc
  • Running on desktop requires users to bring their own compute which they don't always have leading to a bad experience.
  • Licencing was a pain. Windows required an EV code signing certificate which is ~$1,000 for 2 years. Apple is $100. The process is a pain. Cloud costs $$ too but I hope to offset that with paying customers.
  • Allowing users to install dependencies wasn't possible with the PyInstaller setup that Glue was using.
  • Users in general didn't want to download something and often weren't able to download something due to their company policy.
• With focusing on cloud native Glue can be delivered to the most people with the least effort.
• For now I wont be updating the desktop app. You are free to keep using it but won't be as feature rich as the web app.

New Features / Web app refactor

• 🆕 Editor sessions: The graph editor process now runs on a machine in the cloud. In what is called an editor_session. Glue supports different editor instance types of different sizes e.g. 4GB, 8GB, 16GB and more. Glue provides a certain quantity of free editor session minutes.
• 🆕 Scheduling: Glue now supports scheduling. Graphs can be scheduled to run. This is a paid feature. You can check out scheduling even on the free tier but wont be able to create a schedule unless you are a paid user.
  • Environment variables are supported here.
• 🆕 Data Resources: The input output has been generalized a little through the use of data_resource this is a reusable connection type. You define the general configuration of the connection and then within the editor can specify more detailed configurations e.g. general: PostgreSQL connection credentials, specific: your specific sql query.
• 🆕 Runs: This is a run history of your graph and contains meta information about the run e.g. status, run time, editor session instance type etc. It also contains output files etc. So that this information can be accessed later. Runs can be created manually, within the editor session or as part of a schedule.
  • Environment variables are supported here.
• 🆕 Google Sheets: The nodes are mostly all the same but I added support for google sheets.
• ❌ Local Files: For now you can't just drag and drop a local file. You first have to upload the file. The process is slightly slower for smaller files and can be much slower for large files. It depend on your connection speed. In the future (if supporting desktop again) Glue could support both options.
• 🆕 User management: As you would expect with most web apps the usual, login, user profile, account profile, users CRUD, billing are supported.
• 🛠️ Road map: This has now been updated to reflect feedback / the new cloud native approach. The main focus for the next stretch will be i) fixing bugs / streamlining what already exists (API documentation). ii) Adding support for more data resource types, e.g. General API I/O, Python I/O, Oracle DB, Microsoft SQL Server, Email I/O. One long term goal is to support user defined nodes. Due to this large refactor new feature development has been slow but with this new infrastructure in place new feature development will hopefully speed up again.

You can test out the new app here: https://app.gluedata.io/