make-sense icon indicating copy to clipboard operation
make-sense copied to clipboard

saving my continuous progress so i won't have to start over.

Open DR2495 opened this issue 2 years ago • 14 comments

saving my continuous progress so I won't have to start over.

DR2495 avatar Jul 23 '21 16:07 DR2495

I suggest make-sense should add the concept of projects. We can click new project to start our work and save for next time.

iamfaith avatar Aug 13 '21 03:08 iamfaith

Hi 👋 , @DR2495 & @iamfaith! We could do that, but as long as make-sense will not have backend, files related to that project would need to be downloaded to your machine, for example in .zip format. And could be reuploaded next time. Do you feel it is a valuable option?

I am also very sorry for the late reply. Work factors have meant that I have had no time to develop MS.

SkalskiP avatar Aug 27 '21 06:08 SkalskiP

Hi @SkalskiP , I found something interesting here : Paint.js

rasyidf avatar Sep 16 '21 08:09 rasyidf

Hello, once again @rasyidf 👋. That is a true, very interesting project. But... what is the relation of paint.js with that issue? 🧐

SkalskiP avatar Sep 16 '21 18:09 SkalskiP

I guess I missed the details before,

If we want to fully utilize the PWA capability to just install the webapp locally, then integrate file handler.

Web.dev file handling

we, as a user can

  • start project
  • do stuff
  • save as myproject.mks (for example)
  • close the app

on our desktop,

  • click myproject.mks.
  • resume works.

rasyidf avatar Sep 17 '21 19:09 rasyidf

@rasyidf I was thinking about it. We could do something similar. But to make it work similarly - load a whole project from a single file, we would need that file to be quite big. We would need to use some kind of zip structure and hide all images inside along with labels and metadata required.

SkalskiP avatar Sep 20 '21 10:09 SkalskiP

I see, so basically you want the project behave like open Document format. read here => Wikipedia Open Doc.

make a zipped folder, with Metadata and every assets.

rasyidf avatar Sep 21 '21 14:09 rasyidf

Yes but instead of using XML we would use JSON to store labels and metadata. We are already saving zip in TypeScript so I do not think it would be an issue. One thing that is worrying is the size of the saved file. It can be easily a few hundred MB. So for some users with slower machines it can be an issue :/

SkalskiP avatar Sep 21 '21 19:09 SkalskiP

If the make-sense is installed offline. how about making it folder based project with something like .sense/ subfolder with it's workspaced settings. it's opinionated but can be easily predicted by program

something like this : project.json

{
    "name": "pet-behaviour",
    "version": "2.3.0",
    "description": "Pet Behaviour Recognition",
    "keywords": ["veterinary", "cats", "pets", "dogs", "YOLOv5"],
    "author": "John Doe",
    "engine": {
      "model": "YOLOv5",
      "weight" : "yolov5.weights"
    },
    "paths": {
      "images": [
        "images/*"
      ],
      "annotations": [
        "labels/*"
      ],
      "masks": [
        "masks/*"
      ],
      "code": [
        "src/*"
      ]
    }
  }

then, we can make the folder structured as described, then load it and the images will be lazy loaded, after every setup finalized.

rasyidf avatar Sep 23 '21 15:09 rasyidf

@rasyidf that's an interesting suggestion, but I really wouldn't like to have a separate solution for people using make-sense online and offline. We have around 6000 people visiting https://www.makesense.ai/ every week. And I think that we should build new features mostly for those people.

SkalskiP avatar Sep 24 '21 10:09 SkalskiP

I believe we could use either indexedDB (similar to MySQL) or https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage We could store the pending annotations by item name/hash into indexedDB and that would persist by the user side without the need for any storage on our servers (this being particularly important in our client-side system)

AxelAli avatar Nov 08 '21 14:11 AxelAli

Hello, @AxelAli 👋 ! Many thanks for your interest in our project and for your ideas. However, both of them have some drawbacks that will probably stop us from using them.

  • DB - To minimize the cost MakeSense is a purely frontend-based solution. Consequently, the proposed solution must work without a backend. The DB would have to be hosted somewhere which would result in a cost to the project.
  • localStorage - It will only allow us to store labels information. We would like to store the entire project, including photos.

SkalskiP avatar Nov 08 '21 15:11 SkalskiP

Hi @SkalskiP , IndexedDB lives in the user's browser, no installs required. Here you can see an example demo (Try adding a note, changing the URL to Google or something, and open the same link in another tab), notes will persist. Data isn't stored anywhere, and it's persisted even without internet Regarding images , we can either do something like this, or store them in base64 and then drop them in indexedDB https://hacks.mozilla.org/2012/02/storing-images-and-files-in-indexeddb/

AxelAli avatar Nov 08 '21 15:11 AxelAli

Hi @AxelAli! I have to admit that these are very interesting ideas. 💪 I'll have to look into that. I'm curious to see how efficient this solution is. We have to remember that projects have very often 500 or 1000 images. But thank you very much for this idea. I am also sorry that I reacted so negatively. I was simply not aware of the existence of this feature. 🙏

Let me do the research.

SkalskiP avatar Nov 10 '21 08:11 SkalskiP