feat(chrome_extension): write dom changes to database (using sockets) during openadapt.record

[KrishPatel13]

What kind of change does this PR introduce? This PR implements the chrome extension to write the DOM changes (documents' outer HTML of Head and Body and the url) in database.

Summary Motivation: Related Issue: https://github.com/OpenAdaptAI/OpenAdapt/issues/51

Checklist

• [ ] My code follows the style guidelines of OpenAdapt
• [ ] I have performed a self-review of my code
• [ ] If applicable, I have added tests to prove my fix is functional/effective
• [ ] I have linted my code locally prior to submission
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation (e.g. README.md, requirements.txt)
• [ ] New and existing unit tests pass locally with my changes

How can your code be run and tested?

Other information

[abrichr]

Let's also add browser_event_dict to visualize.py somewhere here: https://github.com/OpenAdaptAI/OpenAdapt/blob/main/openadapt/visualize.py#L255

[cr-gpt[bot]]

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information

[KrishPatel13]

Let's also add browser_event_dict to visualize.py somewhere here: https://github.com/OpenAdaptAI/OpenAdapt/blob/main/openadapt/visualize.py#L255

Addressed in: 598ee36

[KrishPatel13]

This the point where the sockets.py can be integrated into the record.py

[abrichr]

Thank you @KrishPatel13 !

@0dm are you interested in picking this up? I think it's quite close to being done.

[0dm]

@abrichr Sure.

[0dm]

currently have these two messages after trying to load unpacked extension on chrome:

edit: getting this on Chromium & Edge too.

[KrishPatel13]

currently have these two messages after trying to load unpacked extension on chrome:

The first one is cause ue to the fact that the The extension was not able to connect to the Native Messaging Host (app). The common reason for this error is that the naming of the native messaging host is not consistent with the files, esp. manifest.json, browser.sh, native_manifest.json etc.

The second one is a common one. I got that quite a lot of times. That is basically given when you try to use the port variable when is it None. Generally the port var is None when is was not able to connect to the NativemessagingHost.

[abrichr]

This suggests something wrong with native_chrome_extension/browser.bat.

@0dm can you please push native_chrome_extension/browser.sh ?

[0dm]

sure I'll push it

[abrichr]

Also how are you handling path in native_chrome_extension/openadapt_chrome.json?

[0dm]

it's an absolute path to the script : users/aaron/Documents/GitHub/OpenAdapt/native_chrome_extension/browser.sh

[0dm]

https://bugs.chromium.org/p/chromium/issues/detail?id=1468564 posted here

[abrichr]

@0dm can you please try running lsof or similar to see what processes are running, and what files/ports they are accessing?

[abrichr]

@0dm @KrishPatel13 can you please remind me what is the status on this?

[abrichr]

@KrishPatel13 I believe this should be closed in favor of https://github.com/OpenAdaptAI/OpenAdapt/pull/744, can you please confirm?

[KrishPatel13]

@KrishPatel13 I believe this should be closed in favor of https://github.com/OpenAdaptAI/OpenAdapt/pull/744, can you please confirm?

Yes!, closing as a new PR is launched for this here: https://github.com/OpenAdaptAI/OpenAdapt/pull/744