Implement approved changes to higher ed / income for 1.0

[BethMattern]

Backend work for the score is complete - it also includes the enhancement to impute income where that data is missing.

[switzersc-usds]

@emma-nechamkin to confirm, this is done, but the score code is using the wrong variable names?

[emma-nechamkin]

thank you for tagging me! I believe so, see here: https://github.com/usds/justice40-tool/pull/1559/files

the score code pipes to score m rather than score narwhal

[switzersc-usds]

I'm not seeing anything in score narhwal that's getting assigned to score M field names. there's a log message that's incorrect but I change that in one of the open PRs.

[emma-nechamkin]

https://github.com/usds/justice40-tool/blob/emma-nechamkin/release/score-narwhal/data/data-pipeline/data_pipeline/etl/score/constants.py#L207 <-- i am thinking about these and their like. ideally we would be able to maintain score M with the OLD income calculation; i don't think that happens

[BethMattern]

@emma-nechamkin will check with @switzersc-usds on this and then @emma-nechamkin and @vim-usds will pair on what's left. @vim-usds will use this issue to track that work.

[emma-nechamkin]

@vim-usds I believe that the new income variable ("adjusted and imputed") already goes to the front end correctly as income. Other than that, we should REMOVE the higher ed variable from the front end for narwhal. I suspect that's the only change here -- happy to pair / debug if you'd like?

[emma-nechamkin]

note: we will have to make new front end variables overall -- like all of the SM_C, M_EBSI etc variables will need "N" names. Here's the list of those columns. This is not strictly speaking about this ticket.

   field_names.M_WATER: "M_WTR",
   field_names.M_WORKFORCE: "M_WKFC",
   field_names.M_CLIMATE: "M_CLT",
   field_names.M_ENERGY: "M_ENY",
   field_names.M_TRANSPORTATION: "M_TRN",
   field_names.M_HOUSING: "M_HSG",
   field_names.M_POLLUTION: "M_PLN",
   field_names.M_HEALTH: "M_HLTH",
   field_names.SCORE_N_COMMUNITIES: "SM_C",
   field_names.SCORE_N_COMMUNITIES
   + field_names.PERCENTILE_FIELD_SUFFIX: "SM_PFS",
   field_names.CLIMATE_THRESHOLD_EXCEEDED: "M_CLT_EOMI",
   field_names.ENERGY_THRESHOLD_EXCEEDED: "M_ENY_EOMI",
   field_names.TRAFFIC_THRESHOLD_EXCEEDED: "M_TRN_EOMI",
   field_names.HOUSING_THREHSOLD_EXCEEDED: "M_HSG_EOMI",
   field_names.POLLUTION_THRESHOLD_EXCEEDED: "M_PLN_EOMI",
   field_names.WATER_THRESHOLD_EXCEEDED: "M_WTR_EOMI",
   field_names.HEALTH_THRESHOLD_EXCEEDED: "M_HLTH_EOMI",
   field_names.WORKFORCE_THRESHOLD_EXCEEDED: "M_WKFC_EOMI",
   field_names.WORKFORCE_SOCIO_INDICATORS_EXCEEDED: "M_WKFC_EBSI",
   field_names.FPL_200_SERIES: "M_EBSI",
}

[vim-usds]

Thanks @emma-nechamkin!

I'm assuming this list is exhaustive for Score N?

Is there a backend ticket(s) associated with these changes? If I have the staging link then i can test these changes on the Frontend.

[emma-nechamkin]

no specific changes -- right now all the variables port to the M variables because I don't think the variables come up in the map UNLESS they are M variables?

[vim-usds]

Remove Higher Ed from all SocioEco indicators for 1.0 staging release branch

[vim-usds]

@Emma and @KameronKerger

In 1.0, if we are removing higher Ed from the frontend, it seems that this would/could have a few effects in the frontend, namely;

side panel:

methodology page:

Does this capture all of it? 🤷‍♂️

I get the feeling that I might be jumping the gun on this task. 🤠 Do I have that right? If so, I can totally wait if it means we'll have a better handle on what needs to change. It would be awesome to meet up and discuss how we think the new design will look like before getting into implementation.

Thanks!

[KameronKerger]

I'll work on the updated panel comp today... but nothing has been approved so ::shrug emoji::

[vim-usds]

@KameronKerger - BTW, no rush at all, if things are still pending. I'm totally not the right person to decide at which point is the ambiguity low enough to start designs?!??