ZnTrack
ZnTrack copied to clipboard
Create, visualize, run & benchmark DVC pipelines in Python & Jupyter notebooks.
https://github.com/zincware/ZnTrack/blob/f7d2d7c928c35c2105023a613e9c985b84af8d0d/pyproject.toml#L2
It is not just used when `rev` or `remote` are set but always.
When building a graph, zntrack calls the `property` or `cached_property` `getter` which can be very slow and is not desired for building graphs.
Have a Node, that uses `always_changed` and produces an output file that is the md5 hash of the data or similar. This Node must not produce a `node-meta.json` as it's...
e.g. ``` itp_files: t.Sequence[str | pathlib.Path | None] = zntrack.deps_path( default_factory=list ) ```
Store the input lock in `node-meta.json` so we can use this information to semi-"validate" the correct node outputs, without hashing the files.
updates: - [github.com/pre-commit/pre-commit-hooks: v5.0.0 → v6.0.0](https://github.com/pre-commit/pre-commit-hooks/compare/v5.0.0...v6.0.0) - [github.com/astral-sh/ruff-pre-commit: v0.11.4 → v0.14.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.11.4...v0.14.4)
- [ ] remove old, unused code - [ ] check performance (bad!) - [ ] save the parameters into to node-meta.json, e.g. as a lock file. Could also contain...
It should either be possible to pass `x_path` to `deps` OR to `deps_path`. Being able to use both might not be desireable. Can we just check if `isinstance(deps, znflow.Connection)` and...
https://github.com/zincware/ZnFlow/blob/d1e6f5ddc75cf2849f546d9afd08ef271f843544/znflow/graph.py#L125 causes all the getters to fire and try to get the values. This would be okay if they were not trying to all read from file