spotify-tensorflow

spotify-tensorflow copied to clipboard

+ pyarrow>=0.15.1 # not directly required, pinned by Snyk to avoid a vulnerability + pyyaml>=5.4 # not directly required, pinned by Snyk to avoid a vulnerability

Ma5onic

elonybear

This library is tightly coupled with tensorflow and tfx, and it's been tricky for users to know which versions of these separate libs work together. Adding a chart to the...

andrewsmartin

martin-laurent

https://github.com/spotify/spotify-tensorflow/commit/d6f6276eb4769c4cacbbca014a82fcbfd30495db

brianmartin

#192: (WIP: Needs Work) Replace PythonDataflowTask With BeamDataflowJobTask

2

Removes `PythonDataflowTask` and related files/tests to replace it with the luigi.contrib `BeamDataflowJobTask`

rclough

Thanks to @ClaireMcGinty ! BeamDataflowJobTask is added to luigi 2.8.5 (https://github.com/spotify/luigi/blob/2.8.5/luigi/contrib/beam_dataflow.py). We could use it to replace `PythonDataflowTask`

daikeshi

When user launches a python beam job for tft or stats generation, they can still specify `--requirements_file` to install `spotify-tensorflow`, which we know it'll cause problem because of tfdv package....

daikeshi

From @daikeshi https://github.com/spotify/spotify-tensorflow/pull/127#issuecomment-452423844: > [`tfr-read`] assumes that there'll be `_schema.pb` or `_inferred_schema.pb` file stored with the dataset, but I think we are now moving to use stats file or schema...

brianmartin

brianmartin