OCA
Automatic partner detection on bank statement lines
I would like to discuss partner detection on bank statement lines when importing bank statements via file import (OFX, Camt, ...) AND by online import (Ponto, Qonto, ...).
The code that handle partner detection:
-
in account_statement_import, there is native support for automatic partner detection using account number, cf https://github.com/OCA/bank-statement-import/blob/14.0/account_statement_import/wizard/account_statement_import.py#L281 If you want to add other detection methods, you have to inherit _complete_stmts_vals() from account.statement.import, cf https://github.com/OCA/bank-statement-import/blob/14.0/account_statement_import/wizard/account_statement_import.py#L289
-
in account_statement_import_online (which depends on account_statement_import), there is native support for automatic partner detection using account number, cf https://github.com/OCA/bank-statement-import/blob/14.0/account_statement_import_online/models/online_bank_statement_provider.py#L366 If you want to add other detection methods, you have to inherit _get_statement_filtered_lines from online.bank.statement.provider cf https://github.com/OCA/bank-statement-import/blob/14.0/account_statement_import_online/models/online_bank_statement_provider.py#L256 (I didn't try it yet, but I don't see a better way).
I know that banks in Belgium (and maybe other countries) give the account_number of the party that send a wire transfer on your bank account, so the match using account_number works well and is very reliable. In France, we never have such information ; the only way we can "try" to detect a partner is by using the bank statement line label.
Partner detection methods other than account_number:
-
in the official "account" module, there is an object "account.reconcile.model.partner.mapping" https://github.com/odoo/odoo/blob/14.0/addons/account/models/account_reconcile_model.py#L13 which is designed to detect a partner using a subset of the bank statement line label. It seems that this code is not used anywhere in the "account" module nor in the OCA module account_reconciliation_widget https://github.com/OCA/account-reconcile/tree/14.0/account_reconciliation_widget
-
I maintain a community module account_statement_completion_label_simple (and account_statement_completion_label_simple_sale) in this Akretion repo https://github.com/akretion/bank-statement-reconcile-simple I use this module in all my Odoo deployments in France. It adds partner detection using 4 methods: partner name, a subset of bank statement line label, invoice number and sale order number. The 4 methods can be enabled/disabled on a per-journal basis. You can easily make the system learn new subsets of bank statement line labels from a new bank statement. It is a small & simple module. This module is optimized for speed. Currently, this module works with file import, but not yet with account_statement_import_online. This module is not in OCA at the moment, but I could submit it to OCA.
-
There are other initiatives such as this OCA PR to add a module account_bank_statement_import_guess_partner https://github.com/OCA/bank-statement-import/pull/456 to detect partner based on invoice number or sale order number. This module has only 2 of the 4 methods proposed by account_statement_completion_label_simple.
-
There are other OCA modules that do complex partner/invoice matching, usually used in ecommerce deployments with high volumes of transactions. This is out-of-scope here.
I think we really need to improve this situation. The minimum is to be able to add partner detection method by inheriting a single method that would be common for file import and online import.
Maybe we should also consider to have more partner detection methods in the base OCA modules for bank statement import. If yes, I don't think we should use "account.reconcile.model.partner.mapping" (see point "1" above): Odoo SA removed the bank reconciliation interface in Odoo v13, so they could remove the model "account.reconcile.model" at any time (it is still present in the master branch... but we never know what can happen in the future). My module "account_statement_completion_label_simple" seems like a good example of simplicity and good feature coverage.
My proposals:
Proposal 1 : Add a module account_statement_import_base.
The 2 existing modules account_statement_import and account_statement_import_online would depend on it. In v16, we could rename account_statement_import to account_statement_import_file. The module account_statement_import_online should not depend any more on account_statement_import: for that, we should move from account_statement_import to account_statement_import_base all the required code to remove this dependency. In this "base" module, we would implement a method for partner detection on statement lines. This method would be used by the 2 modules (file import and online import).
Proposal 1+ : Implement in account_statement_import_base partner detection based on partner name, subset of statement line label and invoice/sale.order number
This is an "extension" of proposal 1. In account_statement_import_base we would implement partner matching using partner name, statement line label, invoice number and sale order number, in addition to account number (the only method in the current implementation). For partner matching based on statement line labels, we have to decide if we use "account.reconcile.model" or the datamodel currently used by "account_statement_completion_label_simple". We could also have a feature to auto-validate some statement lines with a specific counter-part. For example, autovalidate statement line that contain the label "MyBank monthly fee" with the bank fee expense account. Again, we have to decide which is the best datamodel for that.
My opinion: I'd like to implement "Proposal 1" ASAP for v14. We use the online import on several deployments and we really need to have the autodetection of partners with the same system as for file import. For Proposal 1+, I don't have a strong opinion. I think it should be a standard feature of OCA, but, OCA history has shown that it was very difficult to have a simple solution on reconciliation because many people want many different things, in particular in high-volume e-commerce scenarios, and then we end-up with very complex modules that we don't want to maintain/use when we don't have a high-volume project. So I'm happy with my small/easy module account_statement_completion_label_simple which I maintain outside OCA and I can continue to do so in the coming years. I think Proposal 1+ could be implemented for v16.
Please give your opinion on my analysis and my proposals.
@alexey-pelykh @NL66278 you are listed as the contributors of account_statement_import_online, so your opinion on this would be very appreciated.
@alexis-via thanks for raising this up. I'm super happy that the module brings value to you. In principle, I agree with both parts of the proposal, thus you have my support. Small note on 1+ - we should consider moving advanced matching to a separate module just to make maintenance easier.
Hi ! I think we should keep using account.reconcile.model as far as it exist, it provide good reconciliation process for small and medium companies.
You should check v16 had a big rewrite on reconciliation (even the CE part) https://github.com/odoo/odoo/blob/080d41a34119191d20b3e07f72f6aeec677ee1a8/addons/account/models/account_bank_statement.py#L1027 It contains code for partner lookup in all statement texts.
So in the end I'm not sure if it the responsibility of the import statement part or the reconciliation part. This may not be true with mass reconcile ?
@alexis-via partner detection is indeed a missing feature from functional perspective if you import bank statement lines by one of the described methods.
@alexis-via I very much agree with your analysis and proposals. If you add a PR to the repo to add your module https://github.com/akretion/bank-statement-reconcile-simple it would be much appreciated. My module https://github.com/OCA/bank-statement-import/pull/456 is working quite well in production, but as your module is more complete, we can go for that.
Also I am bothered for some time by the duplication of functionality in the online and non-online imports. Having a common base module for both would be a great step. I also would contribute to some refactoring of the online modules, to have some large methods split op and make them easier to override in inheriting modules.
@alexis-via I do see some functionality in the guess_partner one that does not overlap with your modules, like also checking on the invoice partner, and having some extra effort to use the right partner, so maybe we should try to combine these functionalities. Also it would be nice to have some more explanation how the matching with labels actually works.