Fixing playwright using sync API

[samuelbirocchi]

Fixes #284

[anthonynovaes]

I wrote the changes you suggested, and asked it to create a login screen using Next.js, React, and Tailwind. I'm using Claude 3, the DuckDuckGo search system, and it returned the following error after the changes.

Traceback (most recent call last): File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\hubs\hub.py", line 471, in fire_timers timer() File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\hubs\timer.py", line 59, in call
cb(*args, **kw) File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\greenthread.py", line 265, in main
result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\socketio\server.py", line 586, in _handle_event_internal r = server._trigger_event(data[0], namespace, sid, *data[1:]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\socketio\server.py", line 611, in trigger_event return handler(*args) ^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\flask_socketio_init.py", line 282, in _handler return self.handle_event(handler, message, namespace, sid, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\flask_socketio_init.py", line 827, in _handle_event ret = handler(*args) ^^^^^^^^^^^^^^ File "C:\Dev\devika\devika.py", line 81, in handle_message agent = Agent(base_model=base_model, search_engine=search_engine) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Dev\devika\src\agents\agent.py", line 72, in init if engine == "bing": ^^^^^^ NameError: name 'engine' is not defined. Did you mean: 'self.engine'?

[samuelbirocchi]

I wrote the changes you suggested, and asked it to create a login screen using Next.js, React, and Tailwind. I'm using Claude 3, the DuckDuckGo search system, and it returned the following error after the changes.

Traceback (most recent call last): File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\hubs\hub.py", line 471, in fire_timers timer() File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\hubs\timer.py", line 59, in call cb(*args, **kw) File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\greenthread.py", line 265, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\socketio\server.py", line 586, in _handle_event_internal r = server.trigger_event(data[0], namespace, sid, *data[1:]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\socketio\server.py", line 611, in trigger_event return handler(*args) ^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\flask_socketio__init.py", line 282, in handler return self.handle_event(handler, message, namespace, sid, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\flask_socketio__init.py", line 827, in _handle_event ret = handler(*args) ^^^^^^^^^^^^^^ File "C:\Dev\devika\devika.py", line 81, in handle_message agent = Agent(base_model=base_model, search_engine=search_engine) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Dev\devika\src\agents\agent.py", line 72, in init if engine == "bing": ^^^^^^ NameError: name 'engine' is not defined. Did you mean: 'self.engine'?

I'm using openhermes with ollama and it works. Can you try with ollama and openhermes too?

[cafeTechne]

P.S. I was experimenting with a few different external libraries and seeing if I could make more things async than were necessary. This addresses the direct issue I woke up to this morning.

[Spo-oky]

I wrote the changes you suggested, and asked it to create a login screen using Next.js, React, and Tailwind. I'm using Claude 3, the DuckDuckGo search system, and it returned the following error after the changes. Traceback (most recent call last): File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\hubs\hub.py", line 471, in fire_timers timer() File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\hubs\timer.py", line 59, in call cb(*args, **kw) File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\greenthread.py", line 265, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\socketio\server.py", line 586, in _handle_event_internal r = server.trigger_event(data[0], namespace, sid, *data[1:]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\socketio\server.py", line 611, in trigger_event return handler(*args) ^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\flask_socketio__init.py", line 282, in handler return self.handle_event(handler, message, namespace, sid, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\flask_socketio__init.py", line 827, in _handle_event ret = handler(*args) ^^^^^^^^^^^^^^ File "C:\Dev\devika\devika.py", line 81, in handle_message agent = Agent(base_model=base_model, search_engine=search_engine) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Dev\devika\src\agents\agent.py", line 72, in init if engine == "bing": ^^^^^^ NameError: name 'engine' is not defined. Did you mean: 'self.engine'?

I'm using openhermes with ollama and it works. Can you try with ollama and openhermes too?

I'm trying with the current changes using openhermes and ollama and gives me the following error:

Exception in thread Thread-7 (): Traceback (most recent call last): File "C:\Users\BB\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 1045, in _bootstrap_inner self.run() File "C:\Users\BB\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 982, in run self._target(*self._args, **self._kwargs) File "D:\Software\ChatAI\devika\devika.py", line 94, in thread = Thread(target=lambda: agent.execute(message, project_name, search_engine)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Software\ChatAI\devika\src\agents\agent.py", line 341, in execute code = self.coder.execute( ^^^^^^^^^^^^^^^^^^^ File "D:\Software\ChatAI\devika\src\agents\coder\coder.py", line 107, in execute valid_response = self.validate_response(response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Software\ChatAI\devika\src\agents\coder\coder.py", line 34, in validate_response response = response.split("~~~", 1)[1] ~~~~~~~~~~~~~~~~~~~~~~~~^^^ IndexError: list index out of range C:\Users\BB\AppData\Local\Programs\Python\Python311\Lib\threading.py:1047: RuntimeWarning: coroutine 'Agent.search_queries' was never awaited self._invoke_excepthook(self) RuntimeWarning: Enable tracemalloc to get the object allocation traceback

[anthonynovaes]

When using OpenHermes, I was able to generate the prompt, thank you!

[cafeTechne]

'Agent.search_queries' was never awaited self._invoke_excepthook(self) RuntimeWarning: Enable tracemalloc to get the object allocation traceback

This is what I was seeing.

[gonzalovsilva]

I've tried to add a condition at line 34 of coder/coder.py and appears to solve that issue on my end. At least I got it to generate a snake game in python using pygame :)

if "~~~" in response:
    response = response.split("~~~", 1)[1]
    response = response[:response.rfind("~~~")]
    response = response.strip()

[samuelbirocchi]

'Agent.search_queries' was never awaited self._invoke_excepthook(self) RuntimeWarning: Enable tracemalloc to get the object allocation traceback

This is what I was seeing.

Well the problem is deeper than I thought. I'll have to make the whole call stack async. I'll update it in next commit

[samuelbirocchi]

I've tested it and it works like a charm now 😄

[Spo-oky]

if "~~~" in response:
    response = response.split("~~~", 1)[1]
    response = response[:response.rfind("~~~")]
    response = response.strip()

Thanks @gonzalovsilva, this fixed the tracemalloc error!

[ARajgor]

I've tested it and it works like a charm now 😄

I already working on this but it seems you already Solved it but there are few changes that's needs to be fixed asap.

Can you add me to this pr @ARajgor so that I don't have to create a new PR

[ARajgor]

I wrote the changes you suggested, and asked it to create a login screen using Next.js, React, and Tailwind. I'm using Claude 3, the DuckDuckGo search system, and it returned the following error after the changes.

Traceback (most recent call last): File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\hubs\hub.py", line 471, in fire_timers timer() File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\hubs\timer.py", line 59, in call cb(*args, **kw) File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\eventlet\greenthread.py", line 265, in main result = function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\socketio\server.py", line 586, in _handle_event_internal r = server.trigger_event(data[0], namespace, sid, *data[1:]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\socketio\server.py", line 611, in trigger_event return handler(*args) ^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\flask_socketio__init.py", line 282, in handler return self.handle_event(handler, message, namespace, sid, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Anthony Novaes\AppData\Roaming\Python\Python312\site-packages\flask_socketio__init.py", line 827, in _handle_event ret = handler(*args) ^^^^^^^^^^^^^^ File "C:\Dev\devika\devika.py", line 81, in handle_message agent = Agent(base_model=base_model, search_engine=search_engine) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Dev\devika\src\agents\agent.py", line 72, in init if engine == "bing": ^^^^^^ NameError: name 'engine' is not defined. Did you mean: 'self.engine'?

found the bug. fix it...

[Ahmet0691]

freezing in the second web search even if the problem is solved

[samuelbirocchi]

I've tested it and it works like a charm now 😄

I already working on this but it seems you already Solved it but there are few changes that's needs to be fixed asap.

Can you add me to this pr @ARajgor so that I don't have to create a new PR

Done @ARajgor

[ArFiction]

Still have this issue, pulled the latest version

[riazshaikgp]

Tried this using duck duck go search and ollama with llama2 model and still get the async error

[samuelbirocchi]

Tried this using duck duck go search and ollama with llama2 model and still get the async error

@riazshaikgp strange, could you pass me your prompt so I can debug here?

[riazshaikgp]

@riazshaikgp strange, could you pass me your prompt so I can debug here?

First prompt: write java code to create a jwt token using spring

agent answered without expected code generation.

Next prompt: please show me results of your search

Error thrown: Exception in thread Thread-3 (<lambda>): Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/home/riazs/devika/devika.py", line 94, in <lambda> thread = Thread(target=lambda: agent.execute(message, project_name, search_engine)) File "/home/riazs/devika/src/agents/agent.py", line 336, in execute search_results = self.search_queries(queries, project_name, engine) File "/home/riazs/devika/src/agents/agent.py", line 82, in search_queries browser = Browser() File "/home/riazs/devika/src/browser/browser.py", line 13, in __init__ self.playwright = sync_playwright().start() File "/home/riazs/devika/.venv/lib/python3.10/site-packages/playwright/sync_api/_context_manager.py", line 85, in start return self.__enter__() File "/home/riazs/devika/.venv/lib/python3.10/site-packages/playwright/sync_api/_context_manager.py", line 48, in __enter__ raise Error( playwright._impl._errors.Error: It looks like you are using Playwright Sync API inside the asyncio loop. Please use the Async API instead.

[riazshaikgp]

@riazshaikgp strange, could you pass me your prompt so I can debug here?

playwright_async_error.log

I've attached a log of the full interaction.

[samuelbirocchi]

@riazshaikgp It seems you are on the first commit of this PR. Can you try to pull to the latest commit and try again please?

[riazshaikgp]

@riazshaikgp It seems you are on the first commit of this PR. Can you try to pull to the latest commit and try again please?

pulled latest commit of https://github.com/samuelbirocchi/devika/tree/fix/sync-playwright

ran the same two prompts, different error this time but it looks like the browser searches did atleast work:

`24.04.02 21:22:42: root: INFO : Search : duckduckgo

Link :: https://www.baeldung.com/spring-security-oauth-jwt

24.04.02 21:22:46: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 428} 24.04.02 21:22:47: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 332} 24.04.02 21:22:47: root: INFO : got the search results for : how to generate a secure jwt token using spring boot?

Link :: https://www.freecodecamp.org/news/how-to-sign-and-validate-json-web-tokens/

24.04.02 21:22:54: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 4512} 24.04.02 21:23:05: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 3870} 24.04.02 21:23:05: root: INFO : got the search results for : what are the requirements for generating a valid jwt token?

Link :: https://www.baeldung.com/spring-security-sign-jwt-token

24.04.02 21:23:07: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 430} 24.04.02 21:23:08: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 335} 24.04.02 21:23:08: root: INFO : got the search results for : which algorithm should i use for signing jwt tokens in spring boot? 24.04.02 21:23:08: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 1817} 24.04.02 21:23:10: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 1296} Exception in thread Thread-3 (): Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/home/riazs/devika/devika.py", line 94, in thread = Thread(target=lambda: agent.execute(message, project_name, search_engine)) File "/home/riazs/devika/src/agents/agent.py", line 352, in execute code = self.coder.execute( File "/home/riazs/devika/src/agents/coder/coder.py", line 113, in execute valid_response = self.validate_response(response) File "/home/riazs/devika/src/agents/coder/coder.py", line 35, in validate_response self.logger.debug("Response from the model: ", response) TypeError: Logger.debug() takes 2 positional arguments but 3 were given`

[Parth-Jain-2002]

I wrote the changes as in your commit. The agent gets stuck on the second search. Seems to be in a infinite loop. The error after terminating the process forcefully:

[samuelbirocchi]

@riazshaikgp It seems you are on the first commit of this PR. Can you try to pull to the latest commit and try again please?

pulled latest commit of https://github.com/samuelbirocchi/devika/tree/fix/sync-playwright

ran the same two prompts, different error this time but it looks like the browser searches did atleast work:

`24.04.02 21:22:42: root: INFO : Search : duckduckgo

Link :: https://www.baeldung.com/spring-security-oauth-jwt

24.04.02 21:22:46: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 428} 24.04.02 21:22:47: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 332} 24.04.02 21:22:47: root: INFO : got the search results for : how to generate a secure jwt token using spring boot?

Link :: https://www.freecodecamp.org/news/how-to-sign-and-validate-json-web-tokens/

24.04.02 21:22:54: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 4512} 24.04.02 21:23:05: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 3870} 24.04.02 21:23:05: root: INFO : got the search results for : what are the requirements for generating a valid jwt token?

Link :: https://www.baeldung.com/spring-security-sign-jwt-token

24.04.02 21:23:07: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 430} 24.04.02 21:23:08: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 335} 24.04.02 21:23:08: root: INFO : got the search results for : which algorithm should i use for signing jwt tokens in spring boot? 24.04.02 21:23:08: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 1817} 24.04.02 21:23:10: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 1296} Exception in thread Thread-3 (): Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/home/riazs/devika/devika.py", line 94, in thread = Thread(target=lambda: agent.execute(message, project_name, search_engine)) File "/home/riazs/devika/src/agents/agent.py", line 352, in execute code = self.coder.execute( File "/home/riazs/devika/src/agents/coder/coder.py", line 113, in execute valid_response = self.validate_response(response) File "/home/riazs/devika/src/agents/coder/coder.py", line 35, in validate_response self.logger.debug("Response from the model: ", response) TypeError: Logger.debug() takes 2 positional arguments but 3 were given`

There was a issue with the logger, I fixed now in the latest commit

[riazshaikgp]

There was a issue with the logger, I fixed now in the latest commit

I will pull and let you know.

[riazshaikgp]

There was a issue with the logger, I fixed now in the latest commit

Worked now although the silly agent created me a project to create a jwt token in python instead of java but oh well such is the life of relying on ai to do a humans work (for now anyway) 🤣

[ARajgor]

@samuelbirocchi if you are in discord then please ping me.

[ARajgor]

Fix all the problem. please everyone check it again.

[mufeedvh]

Thank you! Merging this now! ❤️ 🙌

[jmtatsch]

gevent dependency is missing in requirements for this to work

[jmtatsch]

pr #314 is needed as follow up