clean-and-green-philly

clean-and-green-philly copied to clipboard

deps: bump the all-actions group with 6 updates

2

Bumps the all-actions group with 6 updates: | Package | From | To | | --- | --- | --- | | [actions/stale](https://github.com/actions/stale) | `9` | `10` | | [actions/checkout](https://github.com/actions/checkout)...

dependabot[bot]

deps: bump the all-npm group across 1 directory with 27 updates

1

Bumps the all-npm group with 25 updates in the / directory: | Package | From | To | | --- | --- | --- | | [@maptiler/sdk](https://github.com/maptiler/maptiler-sdk-js) | `3.5.0` |...

dependabot[bot]

## Describe the task We are currently using `npm` for dependency management for FE development. Although this works, `pnpm` is now the preferred package manager for working with Vercel (which...

nlebovits

## Describe the task Replace the current simulated test dataset in `conftest.py` with a "golden dataset" derived from actual pipeline output to improve test reliability and coverage. The existing test...

nlebovits

## Describe the task Investigate and fix the deduplication functionality in the `spatial_join` utility that is not properly removing duplicate `opa_id` records after spatial joins. During pipeline validation development, it...

nlebovits

## Describe the task Use pylint's similarities detection to identify and refactor duplicated code throughout the `data/src/` pipeline, then update the pre-commit configuration to prevent future code duplication. The current...

nlebovits

## Describe the task Refactor the statistical validation code in `data/src/validation/` to consistently and efficiently use Pandera's `DataFrameSchema` for statistical range validation. Currently, the implementation is inconsistent across different validators,...

nlebovits

## Describe the task Refactor the validator unit tests in `data/src/test/validation/` to eliminate redundancy and improve consistency. Currently, some tests utilize the existing `base_test_validator_mixin.py` file while others implement custom logic,...

nlebovits

## Describe the task Refactor the inconsistent logging and debugging statements throughout the `data/src` pipeline to create a unified, configurable, and efficient logging system. Currently, a mix of `logging` calls...

nlebovits

Task: Data validation subticket #3: External input validators

3

## Subticket 3: External Input Validators ### Describe the task Frequently, we have had issues with APIs or external sources changing without us realizing and then messing up the pipeline....

nlebovits