Segmentation Fault When Recursion Limit Too High
Posting this for others who run into this issue. Langchain uses repr internally, which does not account for recursive object references (PR to fix this). This becomes a problem when derived Langchain classes (ex: CrewAgentExecutor) are serialized (ex: invoke -> dumpd) using repr.
This problem is "hidden" due to a low sys.getrecursionlimit() being set by default (1000). If you increase this number (100,000), a Segmentation Fault is encountered due to the process running out of memory. An example object-reference-recursion can be seen here: self.task.agent.agent_executor.task.agent.agent_executor.task.agent
To solve this problem, you can define a safe_repr implementation (see fix PR above) and simply override the global repr implementation (assuming langchain hasn't fixed this before then):
def patch_langchain():
default_repr = __builtins__["repr"]
def safe_repr(obj, seen=None):
...see-pr...
else:
# Handle general class instances
if hasattr(obj, "__dict__"):
result.append(f"<{obj.__class__.__name__}: ")
for key, value in obj.__dict__.items():
result.append(f"{key}={safe_repr(value, seen=seen)}, ")
result.append(">")
else:
# Fallback for objects without __dict__ or non-container types
result.append(default_repr(obj))
seen.remove(obj_id)
return ''.join(result)
__builtins__["repr"] = safe_repr
And you can simply call patch_langchain() inside your program's entry-point.
Hope this makes sense, please ask questions if it doesn't, happen to explain! Took us ~4 hours to get to the root of this segmentation fault, hope this saves people time.