numaflow-python icon indicating copy to clipboard operation
numaflow-python copied to clipboard

Streamline API naming for `Server*` / `*Callable`

Open th0ger opened this issue 1 year ago • 2 comments

I just tried to get an overview of the server classes fingerprints (pynumaflow 0.7.0);

module class_name arg_name arg_type
pynumaflow.mapper MapServer mapper_instance MapSyncCallable
pynumaflow.mapper MapAsyncServer mapper_instance MapAsyncCallable
pynumaflow.mapper MapMultiprocServer mapper_instance MapSyncCallable
pynumaflow.reducer ReduceAsyncServer reducer_handler ReduceCallable
pynumaflow.mapstreamer MapStreamAsyncServer map_stream_instance MapStreamCallable
pynumaflow.sourcetransformer SourceTransformServer source_transform_instance SourceTransformCallable
pynumaflow.sourcetransformer SourceTransformMultiProcServer source_transform_instance SourceTransformCallable
pynumaflow.sourcer SourceServer sourcer_instance SourceCallable
pynumaflow.sourcer SourceAsyncServer sourcer_instance SourceCallable
pynumaflow.sinker SinkServer sinker_instance SyncSinkCallable
pynumaflow.sinker SinkAsyncServer sinker_instance AsyncSinkCallable
pynumaflow.sideinput SideInputServer side_input_instance RetrieverCallable
  • reducer_handler stands out, suggesting rename reducer_instance.
  • Consider dropping Sync syntax altogether, since it is verbose and will lead to inconsistent syntax over time. For example ReduceCallable needs to be renamed to ReduceSyncCallable when ReduceAsyncCallable is implemented. The Server* class names follows this convention, but the *Callables don't.
  • AsyncSinkCallable stands out to MapSyncCallable/MapAsyncCallable/MapSyncCallable in verb ordering, suggesting consitent naming SinkAsyncCallable.

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

th0ger avatar May 24 '24 10:05 th0ger

@th0ger Thanks for pointing this out!

  1. For the reducer_handler stands out, suggesting rename reducer_instance --> This we can change for sure
  2. Consider dropping Sync syntax altogether, ---> For this we might be considering this in the next iteration during refactoring, I want to make the type checking stricter for servers which support multiple types (sync, async, etc) so that we have better handling of inconsistencies
  3. AsyncSinkCallable stands out to MapSyncCallable Agreed, will change this

kohlisid avatar May 24 '24 17:05 kohlisid

You're welcome, thanks for considering this. These are of course just nice-to-have breaking changes that you should wait and bundle with other API changes you plan.

th0ger avatar May 24 '24 23:05 th0ger