Error when defining pipeline

[mahmoudsajjadi]

Describe the bug There is error when trying to define pipeline.

To Reproduce Steps to reproduce the behavior: when trying to define pipeline after successful data uploaded the following error accured:

Expected behavior evaluation_0 = Evaluation(name='evaluation_0', eval_type='multi')

pipeline.add_component(reader_0) pipeline.add_component(data_transform_0, data=Data(data=reader_0.output.data)) pipeline.add_component(reader_1) pipeline.add_component( data_transform_1, data=Data( data=reader_1.output.data), model=Model( data_transform_0.output.model)) pipeline.add_component(homo_secureboost_0, data=Data(train_data=data_transform_0.output.data, validate_data=data_transform_1.output.data )) pipeline.add_component(evaluation_0, data=Data(homo_secureboost_0.output.data))

error: 2022-09-26 12:28:25.737 | ERROR | IPython.utils.dir2:get_real_method:65 - An error has been caught in function 'get_real_method', process 'MainProcess' (4726), thread 'MainThread' (140398342230976): Traceback (most recent call last):

File "/home/seyedmahmouds/anaconda3/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, │ │ └ {'name': 'main', 'doc': 'Entry point for launching an IPython kernel.\n\nThis is separate from the ipykernel pack... │ └ <code object at 0x7fb1085a90e0, file "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel_launcher.p... └ <function _run_code at 0x7fb10849aa60> File "/home/seyedmahmouds/anaconda3/lib/python3.9/runpy.py", line 87, in _run_code

.... .... ....

File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/utils/dir2.py", line 74, in get_real_method m = getattr(obj, name, None) │ └ 'repr_javascript' └ <pipeline.backend.pipeline.PipeLine object at 0x7fb08686b7c0> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/pipeline/backend/pipeline.py", line 637, in getattr return self.getattribute(attr) │ │ └ 'repr_javascript' │ └ <slot wrapper 'getattribute' of 'object' objects> └ <pipeline.backend.pipeline.PipeLine object at 0x7fb08686b7c0>

AttributeError: 'PipeLine' object has no attribute 'repr_javascript' <pipeline.backend.pipeline.PipeLine at 0x7fb08686b7c0>

Desktop (please complete the following information):

• OS: Ubuntu
• Browser Chrome

[mgqa34]

Which version is your fate-client?

[mahmoudsajjadi]

Hi @mgqa34

I installed FATE=1.9.0 and as it shown after pip install fate_client :it shows:

(from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.11,>=2.10->tensorflow>=2.4.1->fate_client) (3.2.0)

[mgqa34]

Hi @mgqa34

I installed FATE=1.9.0 and as it shown after pip install fate_client :it shows:

(from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.11,>=2.10->tensorflow>=2.4.1->fate_client) (3.2.0)

Execute "pip list | grep fate-client" to get the real version of fate-client please. Meanwhile, can you post full error messages？ For Example, which line of pipeline cause errors?

[mahmoudsajjadi]

Thank you @mgqa34

fate-clients version is : pip list | grep fate-client fate-client 0.3.0.post1

and the error happens after running: pipeline.add_component(reader_0)

and the error is like following: (final error shows: AttributeError: 'PipeLine' object has no attribute 'repr_javascript')

2022-09-29 19:13:45.055 | ERROR | IPython.utils.dir2:get_real_method:65 - An error has been caught in function 'get_real_method', process 'MainProcess' (92308), thread 'MainThread' (139698341172160): Traceback (most recent call last):

File "/home/seyedmahmouds/anaconda3/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, │ │ └ {'name': 'main', 'doc': 'Entry point for launching an IPython kernel.\n\nThis is separate from the ipykernel pack... │ └ <code object at 0x7f0e0d0a00e0, file "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel_launcher.p... └ <function _run_code at 0x7f0e0cf92a60> File "/home/seyedmahmouds/anaconda3/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) │ └ {'name': 'main', 'doc': 'Entry point for launching an IPython kernel.\n\nThis is separate from the ipykernel pack... └ <code object at 0x7f0e0d0a00e0, file "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel_launcher.p... File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel_launcher.py", line 16, in app.launch_new_instance() │ └ <bound method Application.launch_instance of <class 'ipykernel.kernelapp.IPKernelApp'>> └ <module 'ipykernel.kernelapp' from '/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel/kernelapp.py'> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/traitlets/config/application.py", line 846, in launch_instance app.start() │ └ <function IPKernelApp.start at 0x7f0e0a6d29d0> └ <ipykernel.kernelapp.IPKernelApp object at 0x7f0e0d130a60> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 677, in start self.io_loop.start() │ │ └ <function BaseAsyncIOLoop.start at 0x7f0e0bf54d30> │ └ <tornado.platform.asyncio.AsyncIOMainLoop object at 0x7f0e0a66af70> └ <ipykernel.kernelapp.IPKernelApp object at 0x7f0e0d130a60> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 199, in start self.asyncio_loop.run_forever() │ │ └ <function BaseEventLoop.run_forever at 0x7f0e0c1fad30> │ └ <_UnixSelectorEventLoop running=True closed=False debug=False> └ <tornado.platform.asyncio.AsyncIOMainLoop object at 0x7f0e0a66af70> File "/home/seyedmahmouds/anaconda3/lib/python3.9/asyncio/base_events.py", line 601, in run_forever self._run_once() │ └ <function BaseEventLoop._run_once at 0x7f0e0c1fd8b0> └ <_UnixSelectorEventLoop running=True closed=False debug=False> File "/home/seyedmahmouds/anaconda3/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once handle._run() │ └ <function Handle._run at 0x7f0e0c2abca0> └ <Handle <TaskWakeupMethWrapper object at 0x7f0d8b32bb80>(<Future finis...930>, ...],))>)> File "/home/seyedmahmouds/anaconda3/lib/python3.9/asyncio/events.py", line 80, in _run self._context.run(self._callback, *self._args) │ │ │ │ │ └ <member '_args' of 'Handle' objects> │ │ │ │ └ <Handle <TaskWakeupMethWrapper object at 0x7f0d8b32bb80>(<Future finis...930>, ...],))>)> │ │ │ └ <member '_callback' of 'Handle' objects> │ │ └ <Handle <TaskWakeupMethWrapper object at 0x7f0d8b32bb80>(<Future finis...930>, ...],))>)> │ └ <member '_context' of 'Handle' objects> └ <Handle <TaskWakeupMethWrapper object at 0x7f0d8b32bb80>(<Future finis...930>, ...],))>)> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_queue await self.process_one() │ └ <function Kernel.process_one at 0x7f0e0ae1caf0> └ <ipykernel.ipkernel.IPythonKernel object at 0x7f0e0a667d60> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 460, in process_one await dispatch(*args) │ └ ([<zmq.sugar.frame.Frame object at 0x7f0e08630720>, <zmq.sugar.frame.Frame object at 0x7f0e08630510>, <zmq.sugar.frame.Frame ... └ <bound method Kernel.dispatch_shell of <ipykernel.ipkernel.IPythonKernel object at 0x7f0e0a667d60>> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 367, in dispatch_shell await result └ <coroutine object Kernel.execute_request at 0x7f0e0a6bc840> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 662, in execute_request reply_content = await reply_content └ <coroutine object IPythonKernel.do_execute at 0x7f0e0a6bcbc0> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 360, in do_execute res = shell.run_cell(code, store_history=store_history, silent=silent) │ │ │ │ └ False │ │ │ └ True │ │ └ 'pipeline.add_component(reader_0)' │ └ <function ZMQInteractiveShell.run_cell at 0x7f0e0ada7670> └ <ipykernel.zmqshell.ZMQInteractiveShell object at 0x7f0e0a68db20> …….

…..

….

pipeline.add_component(reader_0)
│        │             └ <pipeline.component.reader.Reader object at 0x7f0d8b358be0>
│        └ <function PipeLine.add_component at 0x7f0d8b3cf1f0>
└ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0>

File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/displayhook.py", line 262, in call format_dict, md_dict = self.compute_format_data(result) │ │ └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> │ └ <function DisplayHook.compute_format_data at 0x7f0e0bb0d1f0> └ <ipykernel.displayhook.ZMQShellDisplayHook object at 0x7f0e08622a00> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/displayhook.py", line 151, in compute_format_data return self.shell.display_formatter.format(result) │ │ └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> │ └ <traitlets.traitlets.Instance object at 0x7f0e0bb07580> └ <ipykernel.displayhook.ZMQShellDisplayHook object at 0x7f0e08622a00> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/formatters.py", line 178, in format data = formatter(obj) │ └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> └ <IPython.core.formatters.JavascriptFormatter object at 0x7f0d8b32b910> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), **kw) │ │ │ │ └ {} │ │ │ └ (<IPython.core.formatters.JavascriptFormatter object at 0x7f0d8b32b910>, <pipeline.backend.pipeline.PipeLine object at 0x7f0d... │ │ └ () │ └ <function BaseFormatter.call at 0x7f0e0bb33700> └ <function catch_format_error at 0x7f0e0bb333a0> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/formatters.py", line 222, in catch_format_error r = method(self, *args, **kwargs) │ │ │ └ {} │ │ └ (<pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0>,) │ └ <IPython.core.formatters.JavascriptFormatter object at 0x7f0d8b32b910> └ <function BaseFormatter.call at 0x7f0e0bb33700> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/formatters.py", line 341, in call method = get_real_method(obj, self.print_method) │ │ │ └ <traitlets.traitlets.ObjectName object at 0x7f0e0bb3c460> │ │ └ <IPython.core.formatters.JavascriptFormatter object at 0x7f0d8b32b910> │ └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> └ <function get_real_method at 0x7f0e0bc650d0>

File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/utils/dir2.py", line 65, in get_real_method canary = getattr(obj, 'ipython_canary_method_should_not_exist', None) └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/pipeline/backend/pipeline.py", line 637, in getattr return self.getattribute(attr) │ │ └ 'ipython_canary_method_should_not_exist' │ └ <slot wrapper 'getattribute' of 'object' objects> └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0>

AttributeError: 'PipeLine' object has no attribute 'ipython_canary_method_should_not_exist' 2022-09-29 19:13:45.462 | ERROR | IPython.utils.dir2:get_real_method:74 - An error has been caught in function 'get_real_method', process 'MainProcess' (92308), thread 'MainThread' (139698341172160): Traceback (most recent call last):

File "/home/seyedmahmouds/anaconda3/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, │ │ └ {'name': 'main', 'doc': 'Entry point for …. ….

pipeline.add_component(reader_0)
│        │             └ <pipeline.component.reader.Reader object at 0x7f0d8b358be0>
│        └ <function PipeLine.add_component at 0x7f0d8b3cf1f0>
└ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0>

File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/displayhook.py", line 262, in call format_dict, md_dict = self.compute_format_data(result) │ │ └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> │ └ <function DisplayHook.compute_format_data at 0x7f0e0bb0d1f0> └ <ipykernel.displayhook.ZMQShellDisplayHook object at 0x7f0e08622a00> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/displayhook.py", line 151, in compute_format_data return self.shell.display_formatter.format(result) │ │ └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> │ └ <traitlets.traitlets.Instance object at 0x7f0e0bb07580> └ <ipykernel.displayhook.ZMQShellDisplayHook object at 0x7f0e08622a00> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/formatters.py", line 178, in format data = formatter(obj) │ └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> └ <IPython.core.formatters.JavascriptFormatter object at 0x7f0d8b32b910> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), **kw) │ │ │ │ └ {} │ │ │ └ (<IPython.core.formatters.JavascriptFormatter object at 0x7f0d8b32b910>, <pipeline.backend.pipeline.PipeLine object at 0x7f0d... │ │ └ () │ └ <function BaseFormatter.call at 0x7f0e0bb33700> └ <function catch_format_error at 0x7f0e0bb333a0> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/formatters.py", line 222, in catch_format_error r = method(self, *args, **kwargs) │ │ │ └ {} │ │ └ (<pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0>,) │ └ <IPython.core.formatters.JavascriptFormatter object at 0x7f0d8b32b910> └ <function BaseFormatter.call at 0x7f0e0bb33700> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/core/formatters.py", line 341, in call method = get_real_method(obj, self.print_method) │ │ │ └ <traitlets.traitlets.ObjectName object at 0x7f0e0bb3c460> │ │ └ <IPython.core.formatters.JavascriptFormatter object at 0x7f0d8b32b910> │ └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> └ <function get_real_method at 0x7f0e0bc650d0>

File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/IPython/utils/dir2.py", line 74, in get_real_method m = getattr(obj, name, None) │ └ 'repr_javascript' └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0> File "/home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/pipeline/backend/pipeline.py", line 637, in getattr return self.getattribute(attr) │ │ └ 'repr_javascript' │ └ <slot wrapper 'getattribute' of 'object' objects> └ <pipeline.backend.pipeline.PipeLine object at 0x7f0d8b3629a0>

AttributeError: 'PipeLine' object has no attribute 'repr_javascript' <pipeline.backend.pipeline.PipeLine at 0x7f0d8b3629a0>  [ ]: ​  [ ]: ​  [ ]: ​

[mgqa34]

Oh, the version of fate-client is a little old. If you are using fate-v1.9.0, please upgrade fate-client to 1.9.0 too, the version of fate-client and fate should be matched. If your fate & client's version are lower than fate-v1.9.0，modify pipeline file in python site-packages is another way to fix this problem. Search "__getattr__" keyword in /home/seyedmahmouds/anaconda3/lib/python3.9/site-packages/pipeline/backend/pipeline.py，then comment out its previous line("@LOGGER.catch(reraise=True)" ). Restart jupyter and run the script again.

[mahmoudsajjadi]

Hi @mgqa34

Thank you for your help. It works after modifying based on your second suggestion. Also I saw a problem when I trying to run code for homo secure boost. I will appreciate it if you could help me with that. (https://github.com/FederatedAI/FATE/issues/4360)

[github-actions[bot]]

This issue has been marked as stale because it has been open for 365 days with no activity. If this issue is still relevant or if there is new information, please feel free to update or reopen it.

[github-actions[bot]]

This issue was closed because it has been inactive for 1 days since being marked as stale. If this issue is still relevant or if there is new information, please feel free to update or reopen it.