lucgagan
Problems parsing OpenAI Azure API response
I'm trying to implement some proof-of-concept tests with Auto-Playwright and our company's generative AI chat (based on ChatGPT 3.5), but I'm getting some wierd errors parsing the response.
model: ChatGPT3.5-turbo api-version: 2024-02-01
When I invoke auto with a simple prompt like Click the "Click me" button, I'm getting a test error because the response does not have a function_call with "result".
Is the latest version of Auto really compatible with Open AI Azure API 2024-02-01?
Sample request:
role: 'user',
content: 'This is your task: Click the "Click me" button\n' + '\n' +
'* When creating CSS selectors, ensure they are unique and specific enough to select only one element, even if there are multiple elements of the same type (like multiple h1 elements).\n' +
"* Avoid using generic tags like 'h1' alone. Instead, combine them with other attributes or structural relationships to form a unique selector.\n" +
'* You must not derive data from the page if you are able to do so by using one of the provided functions, e.g. locator_evaluate.\n' + '\n' +
'Webpage snapshot:\n' +
'\n' + '\n' + '\n' + ' Testing \n' + ' <h1>Hello, Rayrun!</h1>\n' + ' <form id="search">\n' + ' Search \n' + ' <input type="text" name="query" data-testid="search-input" />\n' + ' </form>\n' + ' <div id="click-counter">\n' + ' <p>Click count: <span id="current-count" data-testid="current-count">0</span></p>\n' + ' <button id="click-button">Click me</button>\n' + ' \n' + ' </div>\n' + ' \n' + '\n' + '\n' + '\n' + '\n' + '\n' + '\n'}
Sample response:
message {
content: 'To click the button with ID "click-button", you can use the following code:\n' +
'\n' +
'```javascript\n' + 'const { locateElement, locator_click } = require("@qawolf/web");\n' + '\n' + '// Locate the button element using its ID\n' + 'const buttonElement = await locateElement({ cssSelector: "#click-button" });\n' + '\n' + '// Click on the button element\n' + 'await locator_click({ elementId: buttonElement });\n' +
'```\n' +
'\n' +
'This code uses the `locateElement` function to find the button element on the webpage using its ID, and then uses the `locator_click` function to click on the button.', role: 'assistant'}
⚠ Notice the javascript block instead of the function_call that the code expects.
Has anyone else faced this issue?
Why not give your scenarios custom tags (e.g. @scenarioFailedLogin) and then execute by a specific tag or groups of tags? I think it's more beneficial in the long run, than relying on line number that might change.
Why not give your scenarios custom tags (e.g.
@scenarioFailedLogin) and then execute by a specific tag or groups of tags? I think it's more beneficial in the long run, than relying on line number that might change.
You are right, this is not for the long run, for debugging mostly, it can be used for example to run certain scenarios via IDE extension (e.g. VScode). BTW there is no way to run certain example in Outline using the tag system.
I saw this in Cucumber, and I also curious what is the dev flow for it?
Playwright itself does not provide such feature. It can run particular scenario by file + scenario name. So, to get it working, we need Playwright to support it on the runner side. Or generate only that scenario in spec file.
For VSCode the expected way for debugging playwright-bdd tests is using Playwright VSCode extension, that allows to run particular scenario / example just by click:
I saw this in Cucumber, and I also curious what is the dev flow for it?
Very convenient to run certain example in such a way for example. Especially to run such things on ci/cd.
Ok. Btw, do you the VSCode extension that allows to run example line from feature file just by click?
Ok. Btw, do you the VSCode extension that allows to run example line from feature file just by click?
I use my own one for WDIO + Cucumber. But really it is more for CI/CD not for extension, there are cases when I need to do this on some heavy e2e outlines.
How to you run particular line on CI/CD?
Just put something like test.feature:23 in the path field of my job :)