python-pachyderm

python-pachyderm copied to clipboard

smalyala

prototype `python_pachyderm.run_like_a_pipeline`

2

An example usage: ``` --- cell -- def pl_body(): open("/pfs/big", "r") as f: # do stuff with f run_like_a_pipeline( datums=["big:/", "config:/cfgfile1.txt"], code=pl_body) --- output --- output is in '/data/a1b2c3' ---cell...

albscui

Improve UX

2

- [x] Update docstrings to numpy style and method default values to match pachctl https://github.com/pachyderm/python-pachyderm/issues/337 - [x] Host docs on Read the Docs https://github.com/pachyderm/python-pachyderm/issues/338 - [ ] Add wrappers to...

smalyala

prototype `python_pachyderm.mount()`

2

Mount a PFS repo locally, and be able to read/write data via the file system.

albscui

Not intended to be merged, just here to show the changes I made for betterproto

msteffen

As a user, I would like to mount a pachyderm repo to the local, and read/write data via the filesystem.

albscui

Currently `client.delete_all` only calls `PPS.delete_all`. https://github.com/pachyderm/pachyderm/blob/master/src/client/client.go#L677

albscui

Currently proto_lint checks for RPC methods that are missing in the Python mixins. However, it does not check the other way around, which are outdated RPCs that are still being...

albscui

Support python function pipelines

1

Enable a data scientist to define pachyderm pipeline steps as python functions (e.g. from a jupyter notebook).

lukemarsden

Core Pachyderm v2 is dropping support for Build Pipelines. The part that uses Build Pipelines is https://github.com/pachyderm/python-pachyderm/blob/fd74d6ea043bdc9384ba71133824ad360a276445/src/python_pachyderm/util.py#L72

albscui