Dynamically operate a browser - for iterating between testing / updating code

[bjshumway]

Hey, so I have a game built from HTML/Javascript. I would like to see how well JARVIS could try to understand and then modify the existing game. Make changes... re-run the game... play it for finding bugs, etc.

For this it will need to learn how to press buttons, type text into fields, and

1. View the existing page with vision And/or
2. Parse the page, over-and-over again, as it takes its actions, to understand what is on the page.

How might this be achieved? What hugging face AI's would I have to pull in?