runhouse
runhouse copied to clipboard
run-house
Running into problems with runhouse in local mode - Very simple example
Hi! Runhouse looks fantastic! I am thinking about using it for a few of my use cases.
Describe the bug
When I run the following code-snippet (taken verbatim from the docs), I get an error:
import runhouse as rh
def get_pid(a=0):
import os
return os.getpid() + int(a)
server_fn = rh.function(get_pid).to(rh.here)
print(server_fn.endpoint())
This is the error:
INFO | 2024-04-26 09:08:52.384504 | Sending module get_pid to local Runhouse daemon
Traceback (most recent call last):
File "/home/ubuntu/langchain_experiments_2024/02_runhouse/runrunhouse.py", line 7, in <module>
server_fn = rh.function(get_pid).to(rh.here)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/functions/function.py", line 91, in to
return super().to(
^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/module.py", line 450, in to
excluded_state_keys = list(new_module.config().keys()) + [
^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/functions/function.py", line 176, in config
config = super().config(condensed)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/module.py", line 114, in config
config["signature"] = self.signature(rich=True)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/module.py", line 261, in signature
self._signature = self._compute_signature(rich=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/module.py", line 239, in _compute_signature
return {
^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/module.py", line 240, in <dictcomp>
name: self.method_signature(method) if rich else None
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/functions/function.py", line 121, in method_signature
return self.method_signature(self._get_obj_from_pointers(*self.fn_pointers))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/module.py", line 369, in _get_obj_from_pointers
obj_store.imported_modules[module_name] = importlib.import_module(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/ubuntu/langchain_experiments_2024/02_runhouse/runrunhouse.py", line 7, in <module>
server_fn = rh.function(get_pid).to(rh.here)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/functions/function.py", line 91, in to
return super().to(
^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/module.py", line 470, in to
system.put_resource(new_module, state, dryrun=True)
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/resources/hardware/cluster.py", line 450, in put_resource
return obj_store.put_resource(serialized_data=data, env_name=env_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/servers/obj_store.py", line 1434, in put_resource
return sync_function(self.aput_resource)(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/utils.py", line 97, in wrapper
return future.result()
^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/utils.py", line 77, in _thread_coroutine
return loop.run_until_complete(coroutine)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/servers/obj_store.py", line 1421, in aput_resource
return await self.acall_env_servlet_method(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/servers/obj_store.py", line 261, in acall_env_servlet_method
return await ObjStore.acall_actor_method(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/envs/langchain/lib/python3.11/site-packages/runhouse/servers/obj_store.py", line 278, in acall_actor_method
return await getattr(actor, method).remote(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ray.exceptions.RayTaskError(AttributeError): ray::EnvServlet.aput_resource_local() (pid=4070735, ip=10.1.0.71, actor_id=6a27e532729d39d91d17627b05000000, repr=<runhouse.servers.env_servlet.EnvServlet object at 0x7f73567aea90>)
File "/home/ubuntu/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/runhouse/servers/env_servlet.py", line 58, in wrapper
return handle_exception_response(e, traceback.format_exc(), serialization)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/runhouse/servers/http/http_utils.py", line 165, in handle_exception_response
raise exception
File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/runhouse/servers/env_servlet.py", line 39, in wrapper
output = await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/runhouse/servers/env_servlet.py", line 113, in aput_resource_local
return await obj_store.aput_resource_local(resource_config, state, dryrun)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/runhouse/servers/obj_store.py", line 1468, in aput_resource_local
resource = Resource.from_config(config=resource_config, dryrun=dryrun)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/runhouse/resources/resource.py", line 292, in from_config
sys.modules["runhouse"], resource_type.capitalize(), None
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/runhouse/resources/module.py", line 517, in __getattribute__
return super().__getattribute__(item)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Function' object has no attribute 'capitalize'
Sentry is attempting to send 2 pending events
Waiting up to 2 seconds
Press Ctrl-C to quit
Versions Please run the following and paste the output below.
Python Platform: Linux-5.15.0-94-generic-x86_64-with-glibc2.35
Python Version: 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0]
Relevant packages:
fastapi==0.110.2
fsspec==2023.5.0
opentelemetry-instrumentation-fastapi==0.45b0
rich==13.7.1
runhouse==0.0.25
sshfs==2023.4.1
typer==0.12.3
uvicorn==0.29.0
wheel==0.41.2
Hi @AI-Guru ! Thanks for filing an issue.
When running this code in a script, the rh.function subsection needs to be in an if __name__ == "__main__" block, so that when the code is imported on the cluster itself, it doesn't re-run that subsection. The modified code block would look like this:
import runhouse as rh
def get_pid(a=0):
import os
return os.getpid() + int(a)
if __name__ == "__main__":
server_fn = rh.function(get_pid).to(rh.here)
print(server_fn.endpoint())
Also, you ran runhouse restart before running this right? You need a locally running Runhouse server to send things to. Feel free to send us more issues, or join us on Discord for any direct questions.
Hi! Glad you like it. It's always helpful to learn more about people's use cases if you're willing to share!
It's a great catch - our tutorial was written as a notebook, which doesn't require the if __name__ block (and unfortunately notebooks have no __name__ so we can't put that directly in the tutorial). We should add a note to the tutorial calling this out directly, and raise a more meaningful error if we spot it directly in user code.
Hi @dongreenberg and @rohinb2! Thanks for replying! Very helpful! Now I am here:
Traceback (most recent call last):
File "/Users/tristanbehrens/Development/langchain_experiments_2024/04_runhouse/runrunhouse.py", line 8, in <module>
server_fn = rh.function(get_pid).to(rh.here)
File "/Users/tristanbehrens/miniforge3/envs/langchain/lib/python3.10/site-packages/runhouse/resources/functions/function.py", line 91, in to
return super().to(
File "/Users/tristanbehrens/miniforge3/envs/langchain/lib/python3.10/site-packages/runhouse/resources/module.py", line 418, in to
f"Need an initialized local server in order to put {self} onto `rh.here`."
File "/Users/tristanbehrens/miniforge3/envs/langchain/lib/python3.10/site-packages/runhouse/resources/resource.py", line 210, in __str__
return pprint.pformat(self.config())
File "/Users/tristanbehrens/miniforge3/envs/langchain/lib/python3.10/site-packages/runhouse/resources/functions/function.py", line 176, in config
config = super().config(condensed)
File "/Users/tristanbehrens/miniforge3/envs/langchain/lib/python3.10/site-packages/runhouse/resources/module.py", line 89, in config
raise ValueError(
ValueError: Cannot save an in-memory local module to RNS. Please send the module to a local path or system first.
The plot thickens! I suspect that here is something afoot with MacOS. Am I an the right track?
Glad you ask about users' use-cases. For me it is a little esoteric. Think about a dynamic landscape of langchain-enabled LLM-powered software agents that I deploy in my local network. The network consists of a few Macs and some linux devices of different sizes. I would prefer not to configure indivudual agents on each machine individually. Rather I would want to spawn the agents on the different machines from a single machine. I believe runhouse could do that. What do you think?
The real error is in the middle of the stack trace: f"Need an initialized local server in order to put {self} onto rh.here."" You need to runrunhouse restart` in a shell to start the local server daemon before running that code. Have you done that?
And yes, that sounds like a solid use case. The system is designed to deploy mutli-box programs like that, especially if you have access to each of the boxes with SSH credentials or a simple VPN like tailscale. Feel free to let us know if you're having trouble wiring things together.