Improve dependency checking for helpers

[gpsaggese]

Specs are https://docs.google.com/document/d/1xPgQ2tWXQuVWKkGVONjOGd5j14mXSmGeY_4d1_sGzAE/edit?tab=t.0#heading=h.t70lunhu9mjt

[gpsaggese]

From https://github.com/ehaabbasil/helpers/issues/1

• Task:

  • We want to create a graph forhelpers to understand import dependencies and improve helpers/import_check
  • We want to ensure there are no circular dependencies

• Research:

  • Official Web
    • networkx: currently used in import_check/detect_import_cycles.py, Supports directed graphs, cycle detection algorithm
  • GH Docs
    • pydeps: Generates SVG/PNG graphs, can be used for initial visualization. Supports --only feature to focus on a specific path and --show-cycles for circular dependencies

• Detailed Plan:

  • Extend networkx, use pydeps for visuals (pydeps helpers --only helpers)
  • Study detect_import_cycles.py, show_imports.py
  • Refactor graph logic to dependency_graph.py with DependencyGraph class for reuse
  • Add text/DOT output with networkx.nx_pydot.write_dot()
  • Add an invoke i show_deps in ~/src/helpers1/tasks.py for reports
  • Create and Add Unit Test to test/test_dependency_graph.py and check coverage (i run_coverage_report)
  • Update docs/work_tools/all.import_check.reference.md
  • Ensure Docker compatibility (invoke docker_bash)

• FYI: @gpsaggese @sonniki @Shaunak01

[gpsaggese]

@ehaabbasil can you pls go through https://github.com/causify-ai/helpers/pull/660 and fix the TODOs?

Also re-reading some of the docs can help. I'll file a bug for it

[ehaabbasil]

@gpsaggese yes noted, will do the following

[ehaabbasil]

@gpsaggese Updated the TODOs and fixes. Ready for your review.

[gpsaggese]

Arch changes after another round of review

1. We want to use the same approach as the other scripts detect_import_cycles.py, show_imports.py)
   • There is a script that does the work and that it's called through the invoke in the Docker container
2. generate_deps.py should become part of the invoke?