Spoolman icon indicating copy to clipboard operation
Spoolman copied to clipboard

Centralized database of Vendors and Filaments

Open mlee12382 opened this issue 1 year ago • 14 comments

I would be really nice to have a centralized database of Vendors and Filaments that users can pull from and submit new entries to in order to streamline database creation. If the Spoolman container can contact a centralized database to access a community built set of Vendors and Filaments, then we could all benefit from each other's contributions. If a Vendor or Filament doesn't exist in the database we can then submit a request to add it so that others can use it. Once said Vendors or Filaments are on our local databases we can still edit them if needed for our own needs (ie price differences or other tweaks). Thoughts? Would this be possible?

mlee12382 avatar Oct 09 '23 00:10 mlee12382

That would be indeed great. We could team up @Donkie to create uniform database/application. Actually there are several out in the wild, but most of them don't have an API or are pretty much outdated. That seems to be a good project which shouldn't take too long (technically).

The only thing I'm aware of, is the quality of the crowdsourced data. There have to be some (experienced) people to validate that data and collect the garbage.

Disane87 avatar Oct 13 '23 12:10 Disane87

Yeah I've been debating whether to host a website to provide this information or just bake it into the source of Spoolman, and then we'd update it through PRs

Donkie avatar Oct 13 '23 12:10 Donkie

Yeah I've been debating whether to host a website to provide this information or just bake it into the source of Spoolman, and then we'd update it through PRs

I would host it, but on github pages with data in json files. Actually I've created a new project (with angular/tailwind etc.). I would not bake it into spoolman, since it should be an independent project. But you could rely on that.

Shoud I invite you to the project?

Disane87 avatar Oct 13 '23 12:10 Disane87

I agree, centrally hosted would be better for more people, especially when you introduce things like my install script for the end users. There are a lot of people who would likely contribute but have no clue about how to do a pull request or interact with github even beyond following copy / paste instructions.

As far as managing it and keeping an eye on bad / spammy entries, we should set up a way to report an entry so we can manage it via peer review. Users should be able to report bad profiles and / or submit missing information (ie a filament wasn't initially submitted with an empty spool weight).

It may be beneficial to split the database into at least 3 separate files that interact with each other. One each for Vendors, Filaments and Spools with only the former 2 being centrally hosted. If we go the json route, which seems like it would be easier to maintain on the github side, we could even split it down to where each new vendor get a file with their various filaments in it. It might also be beneficial to add an element for spool type since some manufacturers like PolyMaker offer the same filaments on both plastic and cardboard spools. Thus requiring 2 seemingly identical entries which might confuse users.

mlee12382 avatar Oct 13 '23 13:10 mlee12382

I agree, centrally hosted would be better for more people, especially when you introduce things like my install script for the end users. There are a lot of people who would likely contribute but have no clue about how to do a pull request or interact with github even beyond following copy / paste instructions.

I guess that we will break down the effort/knowlegde of github to add new vendors/spools etc. since GitHub lately enabled forms for new types of issues. With that, we could ask for the needed fields and get the json of that filled in form easily. With that we could enrich the mentioned json files.

As far as managing it and keeping an eye on bad / spammy entries, we should set up a way to report an entry so we can manage it via peer review. Users should be able to report bad profiles and / or submit missing information (ie a filament wasn't initially submitted with an empty spool weight).

We could reealize that with that GitHub form issue templates too.

One each for Vendors, Filaments and Spools with only the former 2 being centrally hosted. I wouldn't implement spools on that. The global database should only have vendors, menufacturers and filaments (with materials and some other meta data). The spools you actually bought should be handled by Spoolman, since they are the actual product you''ve bought and the global database should not handle (spooldata vary from the general filament, like lot numbers etc. you can't know in the global data).

I will invite you both to the new repo where we can discuss this in the context of that project, apart from Spoolman.

Disane87 avatar Oct 13 '23 13:10 Disane87

Joined, I may not be much help with the coding side of things, though I'm still learning. I've been using ChatGPT to help me learn scripting, and I'm picking up a lot, but I'm still limited on my knowledge of the more advanced programming stuff.

I was thinking it might be nice to be able to do the submission of new vendors and spools through the Spoolman interface if that's possible. And have an interface to check the central database when a user is adding a new entry to their own inventory. Not sure how difficult that would be to implement.

mlee12382 avatar Oct 13 '23 13:10 mlee12382

I was thinking it might be nice to be able to do the submission of new vendors and spools through the Spoolman interface if that's possible. And have an interface to check the central database when a user is adding a new entry to their own inventory. Not sure how difficult that would be to implement.

Yeah, that could be very handy, I even thought about this too.

Disane87 avatar Oct 13 '23 14:10 Disane87

No “Spool” database. But, in my opinion, a good first point of contact for hex color codes for filaments

https://vorondb.com/filament

Possibly color suggestions could be included depending on the given name(?)

Greetings

livinhack avatar Oct 15 '23 18:10 livinhack

im also intrested in this and would be happy to help. probably cant help much with adding spools because i only realy use a few brands but if i can help with the code let me know

TomW1605 avatar Oct 17 '23 15:10 TomW1605

Thanks @TomW1605 for getting in touch with us. I'm currently building up a project in the background. I will invite you, so you can track the progress 🙂

Disane87 avatar Oct 17 '23 15:10 Disane87

im also intrested in this and would be happy to help. probably cant help much with adding spools because i only realy use a few brands but if i can help with the code let me know

Are you by chance the same Thomas White that made the usb to can bridge guide?

mlee12382 avatar Oct 17 '23 15:10 mlee12382

Thanks @TomW1605 for getting in touch with us. I'm currently building up a project in the background. I will invite you, so you can track the progress 🙂

thanks, ill take a look

im also intrested in this and would be happy to help. probably cant help much with adding spools because i only realy use a few brands but if i can help with the code let me know

Are you by chance the same Thomas White that made the usb to can bridge guide?

no sorry, i havnt done any work with can bus. i want to but cant afford to right now

TomW1605 avatar Oct 17 '23 15:10 TomW1605

I'm almost done implementing this. I have started on my own database in https://github.com/Donkie/SpoolmanDB . Please help me populate this repo with data from vendors, I have started with adding some esun and polymaker filaments so the format should be clear.

Contribute by directly editing/adding files and submitting a PR. In the future there could perhaps be some proper form to add/edit stuff but this will do for now.

image

Donkie avatar May 12 '24 20:05 Donkie

I've started collecting known spool weight/filament databases here: https://github.com/Donkie/SpoolmanDB/issues/13 with the intention of verifying and adding them to SpoolmanDB.

I'm pretty sure I've missed some, so contributions are welcome!

StuSerious avatar May 16 '24 10:05 StuSerious