ach icon indicating copy to clipboard operation
ach copied to clipboard

cmd/achdiff: Tool for comparing ACH files

Open adamdecaf opened this issue 5 years ago • 9 comments

What were you trying to do?

When debugging an ACH file we created in parallel to one created by an existing solution we will need to spot differences in the two files to fix bugs in our code or configuration.

What did you expect to see?

I could see a tool like the following:

$ achdiff theirs.ach ours.ach 
Header:
  FileCreationTime: 1231 vs 1421
Batches:
  #1:
    CompanyDiscretionaryData: 
        Theirs: Foo 
        Ours: Bar

Thoughts?

Edit 2019-09-17:

Adam: my primary usecase for this is to diff files created by paygate and the customer’s existing solution

adamdecaf avatar Apr 04 '19 18:04 adamdecaf

Yep we can see the fallout, clean up and move forward! Legacy has been out there for many years so it should be stable just in a language that sales and marketing considers antiquated. I imagine every bank is running some core version of legacy ach code. With the longevity of the ach specification, there should have been no reason for FI customization.

In short we should be able to compare, troubleshoot, and fix! Great stuff Adam.

bkmoovio avatar Apr 04 '19 19:04 bkmoovio

One challenge would be accessing the code from an existing solution. Moov code is accessible by all.

bkmoovio avatar Apr 04 '19 19:04 bkmoovio

I'm not thinking about the code that we're running in parallel with, but the raw files they're producing. I imagine we're going to get files from the existing code to compare against files we create (for the same transfer sent through both).

adamdecaf avatar Apr 04 '19 21:04 adamdecaf

Ok!

bkmoovio avatar Apr 04 '19 22:04 bkmoovio

We talked about this a lot today. There are a few experiments we should run regarding how the files are displayed in a "human readable" format. Use the horizontal or vertical space to arrange fields next to each other (useful for diffs) along with color.

Some ideas:

Describing ACH file 'test/testdata/ppd-debit.ach'                                      Describing ACH file 'test/testdata/ppd-debit.ach'
  Origin    OriginName   Destination DestinationName                                     Origin    OriginName   Destination DestinationName
  121042882 My Bank Name 231380104   Federal Reserve Bank                                121042882 My Bank Name 231380104   Federal Reserve Bank

  BatchNumber Type ServiceClass Addendas TotalDebits TotalCredits                        BatchNumber Type ServiceClass Addendas TotalDebits TotalCredits
  1           PPD  225          1        100000000   0                                   1           PPD  225          1        100000000   0

    TxCode AccountNumber     Amount    Name                   TraceNumber                  TxCode AccountNumber     Amount    Name                   TraceNumber
    27     12345678          100000000 Receiver Account Name  121042880000001              27     12345678          100000000 Other Name             121042880000001
Describing ACH file 'test/testdata/ppd-debit.ach'
  Origin    OriginName   Destination DestinationName
  121042882 My Bank Name 231380104   Federal Reserve Bank

  BatchNumber Type ServiceClass Addendas TotalDebits TotalCredits
  1           PPD  225          1        100000000   0

    TxCode AccountNumber     Amount    Name                   TraceNumber
  - 27     12345678          100000000 Receiver Account Name  121042880000001
  + 27     12345678          100000000 Other Name             121042880000001

Overall, let's organize the information so our brains pick them up the best. That means diff'd data is close to each other and color coded (along with + / -, or similar, markers).

adamdecaf avatar Sep 19 '19 16:09 adamdecaf

Also, there are industry formats we might adopt as well (for input/output)

https://www.frbservices.org/financial-services/ach/fedpayments-reporter/report-descriptions.html

That's right Wade provided that.

adamdecaf avatar Sep 19 '19 17:09 adamdecaf

To diff two ACH files let's require -diff as a CLI flag and otherwise just show multiple files one after the other.

In the future I can see supporting flags like -merge / -flatten, -sort (for Batches or EntryDetail's across files) to help optimize and organize file contents.

adamdecaf avatar Sep 19 '19 17:09 adamdecaf

The initial concept was based on this.

$ achdiff theirs.ach ours.ach Header: FileCreationTime: 1231 vs 1421

bkmoovio avatar Sep 19 '19 17:09 bkmoovio

I've lumped this into https://github.com/moov-io/ach/pull/622 and will figure out colorization (it doesn't work in my default macOS terminal for some reason..)

adamdecaf avatar Sep 23 '19 17:09 adamdecaf