LedgerSMB icon indicating copy to clipboard operation
LedgerSMB copied to clipboard

OFX import cannot handle version 1 of the OFX standard.

Open auspex opened this issue 6 months ago • 8 comments

Version

1.12

What browsers are you seeing the problem on?

This problem isn't browser related

What happened?

  • Go to Cash / Reconciliation
  • Create a new Reconciliation report
  • Select all the outstanding transactions
  • in Bank transactions upload select an OFX file
  • in Upload file format choose OFX
  • click on Update
  • I get the error Can't call method "transactions" on an undefined value at lib/LedgerSMB/Reconciliation/Parser/OFX.pm line 31.
Image

What should have happened?

I'm not really sure what should happen as I've never successfully imported an OFX file, but it should at least correctly parse the OFX file.

It's a long time since I used to import OFX files to Quicken and GnuCash, but iirc the banks often play fast and loose with the OFX format. However, this bank is by some metrics in the top ten largest banks in the world, and LSMB should be able to handle it.

The first thing that comes to my mind is that the file is not well-formatted XML, though I don't think I've ever encountered an OFX file that was. In any case, there are ten lines of text before the <OFX> tag:

OFXHEADER:100
DATA:OFXSGML
VERSION:102
SECURITY:TYPE1
ENCODING:USASCII
CHARSET:1252
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:NONE

<OFX>

auspex avatar Jul 14 '25 15:07 auspex

The header information seems unlikely to be the problem, as I just downloaded transactions from a slightly smaller Canadian bank and got exactly the same prefix.

auspex avatar Jul 14 '25 15:07 auspex

I've tested OFX import on 1,12, using an OFX file from HSBC bank - which has worked fine,

But my test file is well-formed XML and does not have the plain prefix lines you show - these are wrapped instead within a <?OFX ... ?> processing instruction tag.

For an example of this, see: https://github.com/ledgersmb/LedgerSMB/blob/be6c8174a81f8d02e4d4f0911699dfc9aa3dace5/t/data/inout_tests/ofx_bank_statement.xml#L2

Are you able to share any sample of the kind of file you're trying to import. I'll happily look at this further, if you can give me a sample file which triggers the error.

nick-prater avatar Jul 14 '25 21:07 nick-prater

I'll try to get you something tomorrow. It's a little difficult to share something that is certifiably in the format of these banks, while not giving you all my banking information

auspex avatar Jul 21 '25 00:07 auspex

Here's an OFX for my non-profit, edited to make the account numbers invalid and remove names.

test.txt

Stupid system that forces you to rename files because it thinks that calling it .txt protects anyone :-(

auspex avatar Jul 21 '25 21:07 auspex

I tried replacing the plaintext header with

<?OFX OFXHEADER="100"  VERSION="102" DATA="OFXSGML" SECURITY="TYPE1" ENCODING="USASCII" CHARSET="1252" COMPRESSION="NONE" OLDFILEUID="NONE" NEWFILEUID="NONE"?>

to match yours, but still get exactly the same error

auspex avatar Jul 24 '25 18:07 auspex

It's much worse than just the headers. None of the leaf elements have closing tags (on either of my banks' OFX files). However, closing all those tags, still leaves me with the same error.

There's some interesting suggestions for parsing OFX 102 in https://stackoverflow.com/questions/15735330/how-to-parse-a-ofx-version-1-0-2-file-in-php . I don't know if any of that is usable in Perl.

auspex avatar Jul 24 '25 18:07 auspex

I've made it correct XML, including the exact header from the sample file:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<?OFX OFXHEADER="200" VERSION="220" SECURITY="NONE" OLDFILEUID="NONE" NEWFILEUID:NONE

closed all the unclosed tags, and to match the example file I've truncated all the <DT*> tag contents to 8 digits (which might be is overkill). And that does import, so there's no problem with the data, you just don't accept all versions of OFX. It's obvious from the lib/LedgerSMB/FileFormats/OFX/BankStatement.pm file that it must be well-formed XML, but having worked in Canadian banking, I feel pretty confident that if Canada's two biggest banks are still using OFX v102, then all of them will be.

auspex avatar Jul 24 '25 21:07 auspex

For now, at least, I can use https://github.com/heckman/ofx2.git to convert a v102 OFX file to v200, which imports correctly, but that's a workaround, not a solution.

auspex avatar Jul 25 '25 01:07 auspex