[16.0][ADD] queue_job_chunk: add module

[clementmbr]

Adds the notion of queue job chunks, essentially a queue job with some metadata. Useful to store the JSON received by an API request before their treatment in Odoo.

cc @florian-dacosta @kevinkhao @bealdav @sebastienbeau @rvalyi

[clementmbr]

@bealdav following your suggestion I moved the PR https://github.com/OCA/sale-channel/pull/20 to this repo. @rvalyi I squashed the commit and cleaned history

[rvalyi]

Adds the notion of queue job chunks, essentially a queue job with some metadata. Useful to store the JSON received by an API request before their treatment in Odoo.

cc @florian-dacosta @kevinkhao @bealdav @sebastienbeau @rvalyi

Hello, please pardon me if I'm wrong, but isn't it a bit the same idea of the split method that was merged recently in the queue_job module https://github.com/OCA/queue/pull/658 ? If not, how does it differ?
In fact @paradoxxxzero even did a comment suggesting it was a bit similar in https://github.com/OCA/queue/pull/658#issue-2332785213

cc @sebastienbeau @simahawk @clementmbr

[clementmbr]

@rvalyi

Hello, please pardon me if I'm wrong, but isn't it a bit the same idea of the split method that was merged recently in the queue_job module #658 ? If not, how does it differ?

From what I understand the main difference with @paradoxxxzero 's split() method is that the chunk allows a more in depth control of the incoming data, with the fields like data_str, state_info and the stack_trace in case of errors, which is vital when importing Sale Orders like in sale_import_base.

So it looks like different use cases.

[florian-dacosta]

Yes, I think the goal here is not to use queue job directly because it is a very technical object. I guess it is a bit in the same spirit than https://github.com/OCA/server-tools/tree/16.0/attachment_queue

We want to use queue job and all the machinery to run stuff automatically and all, but we want to have a separate object which allow to :

• Have a dedicated menu, allowing to filter stuff and to make it more understandable to user than a queue.job
• Avoid the deletion of the data after one month
• Do not overcomplicate queue job object with new fields specific to one use case (as the payload received here between other stuff
• Possibility to edit the payload in case of mistake
• ...

Now, @rvalyi has a point... if it is not used to split jobs...why is it named queue_job_chunk? It does not really make sense. Also, do we really want something so generic without other use cases presently ? Probably not.

I think this should be fully refactored doing the following :

• Rename it sale_import_payload or something similar/more specific to its real current use case.
• Rename the model accordingly
• Move the menu to the "Sale App", probably with a new sale group
• Change reference field by many2one toward sale.channel and remove obsolete field (like model_name, record_id, processor...
• Make it more userfriendly (Show create_date, change order to display more recent first, add default filters (on create date, on payload, on error, maybe more ?),
• Log changes on payload, maybe error also to have some history if the payload is retried...
• Log a message on sale order when created from sale_import_payload so we can easily find which was the payload (and easily be sure that the sale order was created from a payload)
• Move it back to sale_channel
• Make a migraton script. Since the module will change name, I guess we can put a pre_init_hook, which will check if the old module was installed and do the migration if it is the case.
• ... Maybe I forgot some stuff, but it can be improved more later

Nice to have but can wait I guess

• Implement a configurable system to purge it, maybe with a default retention time of 1 year for instance

[github-actions[bot]]

There hasn't been any activity on this pull request in the past 4 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days. If you want this PR to never become stale, please ask a PSC member to apply the "no stale" label.