human-essentials icon indicating copy to clipboard operation
human-essentials copied to clipboard

Migrate non-standard partner profile agency types

Open veezus opened this issue 10 months ago • 15 comments

Related to #4241. Unblocks #4254

Description

Migrates all non-standard Partners::Profile agency_types to Partner::AGENCY_TYPE['OTHER'] and preserves the non-standard value in the other_agency_type column.

This will allow a straightforward migration when converting that column to an enum.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Locally, with non-standard agency_types. See screenshot.

Screenshots

non-standard agency types migration

veezus avatar Apr 08 '24 17:04 veezus

@dorner This PR is a data-migration precursor to #4254.

veezus avatar Apr 08 '24 17:04 veezus

@dorner I'm looking at the prod data, there are a lot of cases where it is blindingly obvious to a human what the category is even though it isn't a 100% matchup to what the text in AGENCY_TYPES is (we did add some to AGENCY_TYPES this year). [For instance, "Community health program", of which we have at least 40, would come under "HEALTH", "Food pantry" would come under "FOOD", etc.

So there would be side effects here in the annual report section on the partners -- especially as we just lump all the "Others" together at this point.

Over the course of a year, as people recertify their partners (if they do), this might become less of an issue.

IMO, If we are doing this migration, we should change the partner_agency_counts in the PartnerInfoReportService to also split out the other. We could also provide some additional mapping of the agency types as in the db to the codes (which I would have to provide).

What do you think?

cielf avatar Apr 09 '24 16:04 cielf

There's the option of manually updating the close ones ahead of the big move to Other in this migration. Like, based on the list it'd be easy to, for examples,

UPDATE partner_profiles SET agency_type = 'Food bank/pantry' WHERE agency_type = 'Food bank';
UPDATE partner_profiles SET agency_type = 'Food bank/pantry' WHERE agency_type = 'Food pantry';

and just reference as many items as we can. But that's all outside of my frame of reference and the collection of "non profit" ones would seem problematic.

veezus avatar Apr 09 '24 18:04 veezus

@cielf I think it would be up to you to provide the mapping, and then we can incorporate that into the migration.

dorner avatar Apr 09 '24 20:04 dorner

Nods Oh, I knew I was letting myself in for more work when I said that g. Adding it to the list for this week.

cielf avatar Apr 10 '24 00:04 cielf

Putting this back on your radar, @cielf

veezus avatar Apr 29 '24 15:04 veezus

Yup -- I've got a proposed mapping, and I'm checking with the NDBN folks to see if I can get a review of it before passing it on to you.

cielf avatar Apr 29 '24 17:04 cielf

The appropriate person just got back from vacation yesterday, and this is understandably not their # 1 priority. I think we should have the review by sometime next week though.

cielf avatar May 02 '24 17:05 cielf

Awesome, looking forward to it!

veezus avatar May 02 '24 19:05 veezus

I just got something back from NDBN this afternoon. I've got a few other things I have to pay attention to before the Sunday meeting, so it'll probably Monday for me to review and see if there's any knock on effects (It sounded like they might want to add a couple more categories)

cielf avatar May 11 '24 01:05 cielf

Ok. What I've got back from NDBN -- they are suggesting, given the full list of what we have, that we add 3 more categories: BNB -- Basic Needs Bank, MHEALTH -- Mental Health, and DISTRICT -- School District.

I'll be uploading the mapping within the hour.

cielf avatar May 17 '24 18:05 cielf

Here's a CSV file. The intended use of this is that if you have the "Entry" in "Other" now, it should be the Agency Type indicated in PArtner Agency Type Short. There are definitely some that are not in this list -- so if you have an entry in "Other" that is not in this list, or that is indicated as "Other" keep it as other, and keep the other text as originally entered. Hope that makes sense!

Mapping for agency type.csv

cielf avatar May 17 '24 19:05 cielf

Hey @veezus -- just checking in on this one , vis-a-vis the above mapping.

cielf avatar Jun 21 '24 02:06 cielf

Automatically unassigned after 7 days of inactivity.

github-actions[bot] avatar Jun 29 '24 00:06 github-actions[bot]

@veezus Checking in one more time before seeing if we can get someone else to carry it along.

cielf avatar Sep 06 '24 23:09 cielf