Filament manager

[mateuszbrzezinski]

module name: filament manager

Component to manage filaments and track filament usage.

Signed-off-by: Mateusz Brzezinski [email protected]

[mateuszbrzezinski]

Endpoints examples:

List all spools

GET http://octopi.local:7125/filament/spool/list

List materials

GET http://octopi.local:7125/filament/materials

Add Spool

POST http://octopi.local:7125/filament/spool Content-Type: application/json

{ "name": "test1", "color_name": "red", "color_code": "#FF0000", "vendor": "eSUN", "total_weight": 1000, "used_weight": 0, "diameter": 1.75, "density": 1.24, "material": "PLA", "cost": 50.0 }

Update Spool

POST http://octopi.local:7125/filament/spool Content-Type: application/json

{ "id": "00000C", "active": true, "used_weight": 123, "diameter": 1.75 }

Get spool details

GET http://octopi.local:7125/filament/spool?id=000003

Delete spool

DELETE http://octopi.local:7125/filament/spool?id=00000E

Get spool set as active

GET http://octopi.local:7125/filament/active_spool

Set active spool

POST http://octopi.local:7125/filament/active_spool Content-Type: application/json

{ "id": "000000" }

[jordanruthe]

I don't have enough time to put all my thoughts out at this moment. By here are a few thoughts:

What about filament expended between jobs? It doesn't seem like this is tracking that and ideally a filament manager will track all actions, not just during a job.

It would be ideal to have a field for starting_weight or something where we could track the weight of the filament+spool so if I weigh it later I know exactly how much is left of filament.

I think having the ability to mark a spool as spent rather than having to delete it would also be nice. So you can track overall filament usage.

When I have time to do a better review I'm make some more comments.

[mateuszbrzezinski]

What about filament expended between jobs? It doesn't seem like this is tracking that and ideally a filament manager will track all actions, not just during a job.

@jordanruthe Not sure what you mean. Do you mean like manual filament extrude? Hmm not sure how to track that? If so these are rather small consumption, within the measurement error.

It would be ideal to have a field for starting_weight or something where we could track the weight of the filament+spool so if I weigh it later I know exactly how much is left of filament.

I can add spool weight field.

I think having the ability to mark a spool as spent rather than having to delete it would also be nice. So you can track overall filament usage.

Delete option is rather for different purpose, just to delete spool. There is already active field, but may have sense to rename it to spent

When I have time to do a better review I'm make some more comments.

Thanks

[OSHW-Rico]

I like the idea with spool weight. In my opinion we should not account for every possible input weight configuration. Because it gets complicated if you want to manually update the weight, because you removed some filament or just entered the wrong values at the beginning. Take a look at the Octoprint Spool Manager, in my Opinion it is overkill how many input options there are. Some are calculated based on others, but still can be set manually. At also it isn't working as expected.

I think a spool creation should consist of selecting a Filament from a Database nothing more ideally. But if the filament is not in a database, i should be not more than Total Weight + Assumed Filament on Spool. Because that are the two things I can measure/know. Yes refill is a thing, where you could weight the filament on it's own, but you could also weight the spool before and after the refill is put on. But that's what I mean with it gets complicated with many input options.

[OSHW-Rico]

I think having the ability to mark a spool as spent rather than having to delete it would also be nice. So you can track overall filament usage.

Delete option is rather for different purpose, just to delete spool. There is already active field, but may have sense to rename it to spent

You could create and endpoint to get only spools with more than x meters left. So you don't have to load all spools into the front-end to filter than for spool which can be activated. But it is still possible to list all spools if desired.

[mateuszbrzezinski]

You could create and endpoint to get only spools with more than x meters left. So you don't have to load all spools into the front-end to filter than for spool which can be activated. But it is still possible to list all spools if desired.

That might be good idea, actually when filament usage exceeds filament total weight, then it can be auto marked as active=false, and will add parameter show_inactive with default value false to spools list ednpoint

[Arksine]

I left a few comments after an initial review. I'd like to get Jordan's feedback and make some decisions on what direction we want this to go before proceeding with a more detailed review. I think feedback from other client developers would be useful as well. Some things I would like to hammer out:

• Does spool_manager better describe this module's functionality than filament_manager?
• What fields can be calculated in the request? For example, used weight can be calculated from diameter, density and used length, so there is no need to store its value.
• What is the best way to store static spool data? Manufacturer, total weight, filament_diameter, density, etc. We could add a config file that contains spool templates and allow users to specify custom templates in moonraker.conf. The spool entries in the database could just hold a reference to the correct template. I think @OSHW-Rico makes a good point that we want adding a spool to be as painless as possible.
• I'm okay with keeping a "history" of spools, however I think we need to set a sane limit. Truthfully the same should done for the history module. Long term it is possible we could run into issues if we dont.

When working locally my commits look a lot like yours. However it is a good idea to squash them and create proper commit messages (as outlined in contributing.md) in a pull request, as it makes reviewing the code easier. Changes to any other modules (such as history.py) should be done in a separate commit.

[mateuszbrzezinski]

@Arksine

• Does spool_manager better describe this module's functionality than filament_manager?

yes, that makes sense, fixed that

• What fields can be calculated in the request? For example, used weight can be calculated from diameter, density and used length, so there is no need to store its value.

fixed that

• What is the best way to store static spool data? Manufacturer, total weight, filament_diameter, density, etc. We could add a config file that contains spool templates and allow users to specify custom templates in moonraker.conf. The spool entries in the database could just hold a reference to the correct template. I think @OSHW-Rico makes a good point that we want adding a spool to be as painless as possible.

not sure if that is good idea, what if you want to add new spool to db, but is already used a bit, or sometimes partial spools are available for purchase, so I rather would not do that on configuration level. We can make the default value for total_weight to 1000g, diameter to 1.75, and let user to use that default values or edit that, that should be enough. Also adding new spool for new manufacturer, I don't feel like it should involve editing configuration file. Although filament materials, that might be good idea to put it in configuration, that is rather finite/static set, rarely modified, but even then I would use that just as template, default value, but rather to let to edit that and store that as value parameter than just reference to configured material. Similar for manufacturers, that is something dynamic, so when you add first filament, you have empty completion list, you need to add manufacturer by yourself, later we can provide available manufacturers in dropdown list in UI collecting existing manufacturers from existing spools, or add spool with new manufacturer, that would be more flexible.

• I'm okay with keeping a "history" of spools, however I think we need to set a sane limit. Truthfully the same should done for the history module. Long term it is possible we could run into issues if we dont.

that is to consideration, but you would rather not add 1000000 spools, rather like 20-100

When working locally my commits look a lot like yours. However it is a good idea to squash them and create proper commit messages (as outlined in contributing.md) in a pull request, as it makes reviewing the code easier. Changes to any other modules (such as history.py) should be done in a separate commit.

Definitely would squash commits on merging PR to master, but also squashed existing commits

For now I left changes to history module here, until we sort everything, otherwise that PR at least for now would be withiout context

Happy to get some comments from @jordanruthe

[Arksine]

not sure if that is good idea, what if you want to add new spool to db, but is already used a bit, or sometimes partial spools are available for purchase, so I rather would not do that on configuration level. We can make the default value for total_weight to 1000g, diameter to 1.75, and let user to use that default values or edit that, that should be enough. Also adding new spool for new manufacturer, I don't feel like it should involve editing configuration file. Although filament materials, that might be good idea to put it in configuration, that is rather finite/static set, rarely modified, but even then I would use that just as template, default value, but rather to let to edit that and store that as value parameter than just reference to configured material. Similar for manufacturers, that is something dynamic, so when you add first filament, you have empty completion list, you need to add manufacturer by yourself, later we can provide available manufacturers in dropdown list in UI collecting existing manufacturers from existing spools, or add spool with new manufacturer, that would be more flexible.

I was thinking that specific manufacturers may produce materials with slightly different properties. For example, maybe Prusa PLA has a different density than ESun PLA. This information is static and could be templated through a config file. That said, I'm okay with just providing generic templates and allowing the user to add custom values.

that is to consideration, but you would rather not add 1000000 spools, rather like 20-100

I'm thinking that we max out on 1000 spool entries. I pushed a recent commit that sets a 10000 job limit for the history module. When (if?) the namespace reaches 10k entries it will pop off the oldest entry and add a new one.

[mateuszbrzezinski]

I was thinking that specific manufacturers may produce materials with slightly different properties. For example, maybe Prusa PLA has a different density than ESun PLA. This information is static and could be templated through a config file. That said, I'm okay with just providing generic templates and allowing the user to add custom values.

Could you elaborate a bit more, how would you want to implement that? Can it be like it is now, predefined filaments in code, but export it to configuration file, then the filament name, and details are stored in spool data Or you mean to create separate db namespace to store filaments, with dedicated endpoint to add/edit/delete?

I'm thinking that we max out on 1000 spool entries. I pushed a recent commit that sets a 10000 job limit for the history module. When (if?) the namespace reaches 10k entries it will pop off the oldest entry and add a new one.

That is a neat idea, in case of spools will be a bit more complex, as I guess we should pop off the oldest (last_used) inactive spool. But anyway that won't be a problem.

EDIT: eventually I took different approach, as spools are managed by user, so just in case limit is reached, endpoint will return error, so user has to delete spool by himself.

[marcandre83]

Thank you so much for developing this! If there is any possibility to, please add the option to use the filament database across several klipper instances in a local network.

[mateuszbrzezinski]

Thank you so much for developing this! If there is any possibility to, please add the option to use the filament database across several klipper instances in a local network.

that will be rather much more complicated, and I guess rather in another release, as currently Moonraker has only local db Maybe @Arksine has some better view of that?

[OSHW-Rico]

Regarding the database for default values for spools. I thought more about a public github repo with all sort of default data to choose from. Everybody can do a PR with new Spool Values, Filamenttypes, Densitys, Netto/Brutto Weight of the spools etc. Maybe you could add your own private Repo via the Config file.

This could solve the MultiPrinter MultiClient issue and generate a big database (outside of Moonraker) which could even evolve into a bigger filament database as than just the data used by moonraker.

[Arksine]

Thank you so much for developing this! If there is any possibility to, please add the option to use the filament database across several klipper instances in a local network.

Moonraker isn't aware of other instances on the network. Clients could possibly sync spools across instances, however this can get complex as they must deal with new instances added, some instances not having the spool manager configured, some instances not reachable.

Regarding the database for default values for spools. I thought more about a public github repo with all sort of default data to choose from. Everybody can do a PR with new Spool Values, Filamenttypes, Densitys, Netto/Brutto Weight of the spools etc. Maybe you could add your own private Repo via the Config file.

Interesting idea, however I'm weary of having Moonraker rely on remote service. Some users have connection issues, it has already caused headaches with the update manager. In my view it would be better for clients to query such a service directly, they could then add these templates to the local instance when a user selects them.

With regard the current state of the module, I have a few comments:

• It only needs to emit an event when a new spool is selected. I don't think spool_added and spool_update events are necessary.

• We should remove the dependency on history. Instead it should subscribe to toolhead.position in a server:klippy_ready event handler. This subscription can be used to initialize a last_epos attribute, which it can use to calculate the difference in epos and accumulate filament used on the active spool. Unlike the other toolhead positions we shouldn't need to worry about its position being reset. We can save the active spool to the database when it is either deselected or in an on_exit() method in the SpoolManager class. If a component has defined on_exit() then moonraker will call it before it closes the database on shutdown/restart.

• For the configured templates, I was thinking about something like the following:

  [spool_manager template prusament_pla]
  material: PLA
  vendor: Prusa
  density:  1.24
  diameter: 1.75
  total_weight: 970
  spool_weight: (whatever it is)
  cost: 24.99
  

  A spool entry can store a reference to the template, the cost can be overridden. I don't think the spool entries need to store active, color_code, or comment, I'd prefer to streamline those entries and not store extraneous data. Of course an alternative is to have clients add this information to the database. If we go that route, we would need to store the templates separate from the spool entries.

• We will need client developers to chime in on the API. They can give specific guidance as to what they are looking for.

[Drachenkaetzchen]

• We will need client developers to chime in on the API.

I'm playing around with an RDM6300 RFID reader and tags with the intention to track which spool is inserted. My use cases are:

• Verify that the inserted spool matches the slicer's preset
• Check if the job about to be printed has enough filament left

I think all that should be possible once this PR is merged, however, a few things I noticed:

• The ID returned when adding a spool should be a GUID, and/or the ID should be settable by the user. This makes migration or merging of the database easier if a user decides to do so.

• I agree that the database should store as few fields as possible. The fields material, vendor and cost, color etc are not used for any calculation, and often they are not filled at all. Even though we developers wish that all fields are properly used, even I as a user am often lazy and might put "PLA eSun Green" in the name instead of the respective fields.

• However, I think there should be a way to store these values if needed. Not sure if that would work with the current database system in use, but a simple metadata field with JSON values could to the trick here. If that doesn't work, it's up to the person implementing additional features to create a separate database with all additional fields needed.

[RomRider]

I'm thinking about trying to implement a central spool management system and wondering what would the right approach be (I'm not yet familiar with the source code of moonraker).

1. Idea 1:

   • Moonraker provides a config section to integrate with the spool management system, say [spool_manager] with api key, url etc... It could also run locally if required.
   • Moonraker would provide api endpoints to set the active spool/get the active spool
   • Moonraker would push consumption to the spool manager at certain point in time (Events?) on the selected active spool
   • The interface (fluidd, mainsail, ...) would have to integrate with the spool manager atop of moonraker directly or as an alternative moonraker could also proxy all the requests/api calls to specific endpoints to the spool manager (I think proxy is better)
   • If spool manager is configured, the print wouldn't start if the user doesn't provide a spool in the api call/if there is no active spool

2. Idea 2:

   • Moonraker doesn't integrate with anything but the spool manager would integrate with each moonraker instances
   • the spool manager would subscribe to the toolhead position in each case
   • the spool manager would provide all the apis endpoints but would need to understand the concept of printers and map a spool with a printer
   • There wouldn't be a way to "block" the print from starting without a spool selected
   • The interface would have to intergrate directly with the spool manager

For both options either spool manager implements also frontend to manage the spools, or we let that implementation be done in fluidd/mainsail

What do you think? I think idea 1 is better in terms of architecture, I like better that the spool manager hasn't any knowledge about the printer system thus it could also be used with other systems in the future.

[Tahx]

Any updates on this?

[hifihedgehog]

Ditto. Any updates on this?

[narodo]

@mateuszbrzezinski, are you still working on this? Is there anything I can help with? Anything else we can do to get this PR merged?

[SnorreSelmer]

I would very much like to see this project finished as well. A spool-manager is a very handy feature to have!

BTW, if people have several machines running Moonraker, maybe have a way for the system to have a server/client relationship? This way you can set an IP or machine-name on each client for where the data is stored. Then you have a centralized location for all spool information. No need to sync data, easier to backup.

[feniarus]

Would it be a Problem if i redo the whole thing and make a new PR so that it can get merged?? Is that ok? I am also working on the UI Part for Mainsail.

[locki-cz]

Would it be a Problem if i redo the whole thing and make a new PR so that it can get merged?? Is that ok? I am also working on the UI Part for Mainsail.

YES please! It looks like @mateuszbrzezinski not working on this, and this missing to many peaople which moved from octoprint to Mainsail :)

If you need some betatesters let me know!

[RPhilbo]

Would it be a Problem if i redo the whole thing and make a new PR so that it can get merged?? Is that ok? I am also working on the UI Part for Mainsail.

Would appreciate that a lot, Iam waiting for this from the minute I read about it :-)

Count me in as a tester and beer supporter :) ( Because Iam a noob in programming )

[Hesi-Re]

I’m really interested in this plugin. This will make me change from Octoprint to Mainsail definitely

[locki-cz]

Can we expect it soon or later? Is there any chance it will work in Mainsail like some plugin?

[locki-cz]

Any update?

[campoyyano]

Hi everyone, any update on this ?? Is a very nice feature to have available for the klipper ecosystem

[locki-cz]

Unfortunatly still not merged to moonraker, not know where is problem ;((

[Tareku99]

It would be very nice to see this happen. What are the merge conflicts ?

[Digityle]

I too am interested in this project, would love to contribute. Have emailed @mateuszbrzezinski but no response, know very little python but am learning for this very project. I have a hx711 filament weight scale and RFID as @Drachenkaetzchen mentioned, managed through home assistant and would like very much to see this usable in Klipper. @Feniarus any updates? Thx all...