testcafe
testcafe copied to clipboard
DevExpress
Testcafe fails to restart browser after encountering chrome error code 5
What is your Scenario?
When automating our code in development we sometimes run into different error codes (like chrome error code 5) and TestCafe doesn't appear to handle them properly. The test in question basically just hangs and when running it via CircleCI it either hits the run timeout or kills the job eventually since it doesn't know how to handle it.
What is the Current behavior?
It would be nice if we can catch chrome errors that do make it impossible to continue the test and then be able to handle them accordingly (like restarting the browser).
What is the Expected behavior?
Based upon the documentation it mentions that "If a browser stops responding during test execution, TestCafe restarts the browser and reruns the current test in a new browser instance."
This doesn't appear to occur so the chrome window hangs until eventually a timeout is hit.
What is your public website URL? (or attach your complete example)
We can run through it together if necessary.
What is your TestCafe test code?
The request is mainly to handle the errors so I'm not sure how much this will help.
test
.meta({ created_on: '04/13/22', created_by: 'Justin Grabenbauer', updated_on: '', updated_by: '' })
.meta({ scope: 'regression', priority: 'low', showstopper: '', github_ID: '723' })
.after( async t => {
await ThemesPageObjects.edit_theme_name(t.ctx.updated_theme, t.ctx.default_theme)
await HelperObjects.wait_for_flash_alert()
})
('user_can_edit_theme_name', async t => {
// Purpose: To verify that a theme name can be edited.
t.ctx.default_theme = 'Dnd - change theme name'
const default_theme = t.ctx.default_theme
t.ctx.updated_theme = 'Dnd - updated theme name'
const updated_theme = t.ctx.updated_theme
await WorkspaceMainNavPageObjects.navigate_to_workspace_settings()
await SettingsPageObjects.navigate_to_themes_page()
if (await Selector('td').withText(updated_theme).visible == true) {
await ThemesPageObjects.edit_theme_name(updated_theme, default_theme)
await HelperObjects.wait_for_flash_alert()
}
await ThemesPageObjects.edit_theme_name(default_theme, updated_theme)
await HelperObjects.wait_for_flash_alert()
await t
.expect(Selector('td').withText(updated_theme).visible).ok()
})
Your complete configuration file
require('dotenv').config()
const CustomRequestHooks = require('./pom/custom_request_hooks')
// Agnostic for all environments
let skipped_tests = [
// Located in apps_page_tests.js
'user_cannot_access_sites_when_feature_is_disabled', // 07-28-22
'user_cannot_access_settings_when_feature_is_disabled', // 07-28-22
// Located in billing_page_tests.js
'user_can_navigate_to_billing_page', // 08-22-22
// Located in blog_page_tests.js
'user_can_add_a_gif_to_blog', // 08-16-22
'user_can_create_a_blog_post', // 08-04-22
'user_can_edit_blog_settings', // 07-06-22
// Located in clean_up_scripts.js
'clean_up_copies_of_funnels',
'clean_up_copies_of_funnel_pages',
'clean_up_copies_of_pages',
'clean_up_products',
'clean_up_site_pages',
'clean_up_workflows',
'dismiss_notifications',
'clean_up_collaborators',
// Located in collaborators_page_tests.js
'user_can_navigate_to_collaborators_page', // 07-21-22
// Located in the collections_page_tests.js
'user_can_navigate_to_collections_page',
// Located in courses_page_tests.js
'optin_user_can_navigate_through_course', // 06-07-22
'manually_added_user_can_navigate_through_course', // 06-07-22
'user_can_update_courses_lesson_path',
'user_can_update_courses_module_path',
'newly_enrolled_user_receives_verification_email',
'user_can_log_into_customer_center_with_magic_link',
'courses_back_button_navigates_to_previous_page',
'user_can_preview_existing_course',
// Located in funnels_page_tests.js
'user_can_create_a_new_funnel', // 08-04-22
'user_can_clone_a_funnel',
'user_can_navigate_through_funnel',
'page_views_and_unique_views_values_are_correct', // 05-31-22
// Located in funnel_workflow_builder_page_tests.js
'user_is_able_to_remove_a_node', // 05-31-22
'user_can_add_and_remove_a_decision_path', // 05-31-22
'user_is_able_to_add_and_remove_a_product_from_funnel', //08-04-22
'user_can_add_and_remove_a_split_test', // 05-31-22
'user_can_add_a_new_workflow', // 08-04-22
'user_can_add_an_existing_workflow', // 08-04-22
// Located in inventory_page_tests.js
'user_can_navigate_to_inventory_page', //08/30/22
// Located in logout_page_tests.js
'user_can_logout_from_settings_page', // 06-08-22
// Located in marketing/broadcasts_page_tests.js
'user_can_create_new_broadcast', // 06-20-22
// Located in marketing/templates_page_tests.js
'user_can_create_broadcast_template', // 06-13-22
// Located in orders/order_page_tests.js
'user_can_partially_fulfill_order', // 08/11/22
// Located in pages_page_tests.js
'user_can_filter_page_types_with_tabs', // 05/27/22
'user_can_pin_a_page_from_edit_page', //07-29-22
// Located in reporting_page_tests.js
'opt-ins_by_landing_page_report_displays', // 06/21/22
// Located in sites_overview_page_tests.js
'user_can_add_header_to_site', // 07-06-22
// Located in themes_page_tests.js
'user_can_edit_theme_name', // 08-08-22
// Located in top_main_nav_page_tests.js
'clicking_notifications_will_open_notifications_alert', // 06/01/22
// Located in workflows_page_tests.js
'user_can_create_a_workflow', // 08-23-22
'user_can_restore_an_archived_workflow' // 08-23-22
]
// Production specific scripts needing to be skipped for various reasons
let production_skipped_tests = [
'user_can_navigate_to_order_tags_page', // Link disabled through Launch Darkly as of 08/29/22
'user_can_navigate_to_payments_ai_dashboard', // Production PAI account not set up as of 08/29/22
'user_is_able_to_navigate_to_the_inbox_page' // Inbox link is not activated in production as of 08/31/22
]
// Scripts that need to run with a single concurrency
let isolated_tests = [
// Located in sites_overview_page_tests.js
'user_can_make_site_public',
'user_can_edit_site_name',
'user_can_make_site_password_protected',
'user_can_make_site_private',
'user_can_edit_site_domain',
'changing_site_domain_updates_funnel_urls',
'changing_site_domain_updates_blog_urls',
'changing_site_domain_updates_course_urls',
'user_can_add_header_to_site',
// Located in account_details_page_tests.js
'user_can_edit_account_name',
'user_can_edit_account_password',
'user_can_change_account_language',
// Located in general_page_tests.js
'user_can_edit_workspace_details',
// Located in blog_page_tests.js
'user_can_edit_blog_settings',
// Located in themes_page_tests.js
'user_can_promote_theme_to_live_from_index_page',
'user_can_promote_theme_to_live_from_overview_page'
]
// Scripts specific for Applitools integration
let visual_tests = [
// Located in dashboard_checks.js
'applitools_check_on_dashboard',
// Located in editor_checks.js
'applitools_check_in_editor',
'user_is_able_to_change_the_checkout_element_style',
// Located in login_checks.js
'applitools_check_on_login_page_with_magic_link',
'applitools_check_on_login_page_with_password'
]
// Filter that joins the skipped scripts with the production skipped tests
let production_filter = skipped_tests.concat(production_skipped_tests)
// Filter that joins ALL filtered scripts so that they are not run for a full suite
let full_filter = skipped_tests.concat(isolated_tests).concat(visual_tests)
let build_filter = () => {
// Purpose: To set the suite that we want to run. (isolated, regression, sanity)
var filter = {}
switch(process.env.ENV) {
case 'production':
if(process.env.SCOPE) {
filter = {
testGrep: `^(?!.*(${production_filter.join('|')})).*$`,
testMeta: {
scope: process.env.SCOPE
}
}
}
break
case 'staging':
if(process.env.SCOPE) {
filter = {
testGrep: `^(?!.*(${skipped_tests.join('|')})).*$`,
testMeta: {
scope: process.env.SCOPE
}
}
} else {
filter = {
testGrep: `^(?!.*(${full_filter.join('|')})).*$`
}
}
break
}
return filter
}
let determine_concurrency = () => {
// Purpose: To set concurrency dependant on the scope that is passed
let concurrency = 5
if (process.env.SCOPE == 'isolated') {
concurrency = 1
}
return concurrency
}
module.exports = {
assertionTimeout: 5000,
browsers: [
"chrome:headless --window-size=1920,1159 --allow-insecure-localhost"
],
concurrency: determine_concurrency(),
cache: true,
clientScripts: [
"scripts/hide_notifications.js",
"scripts/click_expand_all_for_courses_details_page.js"
],
disableMultipleWindows: true,
filter: build_filter(),
hooks: {
request: CustomRequestHooks
},
hostname: "localhost",
pageLoadTimeout: 30000,
quarantineMode: {
successThreshold: 1,
attemptLimit: 2
},
reporter: [
{
name: "spec",
output: "artifacts/reports/spec_results"
},
{
name: "xunit",
output: "artifacts/reports/xunit_results.xml"
},
{
name: "testrail"
}
],
runExecutionTimeout: 1800000,
screenshots: {
path: "artifacts/screenshots",
pathPattern: "${TEST}_${DATE}_${TIME}.png",
takeOnFails: true,
thumbnails: false
},
selectorTimeout: 5000,
skipJsErrors: true,
skipUncaughtErrors: true,
src: "tests/**",
ssl: {
pfx: "ssl/testingdomain.pfx",
rejectUnauthorized: true
},
testExecutionTimeout: 240000,
videoEncodingOptions: {
aspect: "16:9",
framerate: 30
},
videoOptions: {
failedOnly: true,
pathPattern: "${TEST}_${DATE}_${TIME}.mp4",
singleFile: false
},
videoPath: "artifacts/recordings"
}
Your complete test report
Running tests in:
- Chrome 105.0.0.0 / Monterey 12
settings - themes
✖ user_can_edit_theme_name (screenshots: /Users/justingrabenbauer/cf2-test-cafe-poc/artifacts/screenshots/user_can_edit_theme_name_2022-09-08_11-23-30.png)
1) The specified selector does not match any element in the DOM tree.
| Selector('input')
| .withAttribute('data-testid', 'theme_custom_name')
Browser: Chrome 105.0.0.0 / macOS 10.15.7
Screenshot: /Users/justingrabenbauer/cf2-test-cafe-poc/artifacts/screenshots/user_can_edit_theme_name_2022-09-08_11-23-30.png
28 | // Param: updated_theme_name - String - the updated theme name
29 |
30 | await t
31 | // Once the theme search works then we can just search and click the visible 'a' for theme-settings
32 | .click(Selector('td').withText(theme_name).sibling('td').find('a').withAttribute('data-testid', 'theme-settings'))
> 33 | .typeText(this.txt_field_theme_name, updated_theme_name, { replace: true, paste: true })
34 | .click(this.btn_update_theme)
35 | }
36 |
37 | async click_theme_image(theme_name) {
38 |
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:33:14)
at asyncGeneratorStep (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:1:285)
at _next (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:1:623)
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:1:788)
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:1:529)
at ThemesPageObjects.edit_theme_name (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:24:59)
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/settings/themes_page_tests.js:161:31)
at asyncGeneratorStep (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/settings/themes_page_tests.js:10:315)
at _next (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/settings/themes_page_tests.js:10:653)
2) - Error in test.after hook -
The specified selector does not match any element in the DOM tree.
| Selector('input')
| .withAttribute('data-testid', 'theme_custom_name')
Browser: Chrome 105.0.0.0 / macOS 10.15.7
Screenshot: /Users/justingrabenbauer/cf2-test-cafe-poc/artifacts/screenshots/user_can_edit_theme_name_2022-09-08_11-23-30.png
28 | // Param: updated_theme_name - String - the updated theme name
29 |
30 | await t
31 | // Once the theme search works then we can just search and click the visible 'a' for theme-settings
32 | .click(Selector('td').withText(theme_name).sibling('td').find('a').withAttribute('data-testid', 'theme-settings'))
> 33 | .typeText(this.txt_field_theme_name, updated_theme_name, { replace: true, paste: true })
34 | .click(this.btn_update_theme)
35 | }
36 |
37 | async click_theme_image(theme_name) {
38 |
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:33:14)
at asyncGeneratorStep (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:1:285)
at _next (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:1:623)
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:1:788)
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:1:529)
at ThemesPageObjects.edit_theme_name (/Users/justingrabenbauer/cf2-test-cafe-poc/pom/settings/themes/themes_page_repo.js:24:59)
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/settings/themes_page_tests.js:145:29)
at asyncGeneratorStep (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/settings/themes_page_tests.js:10:315)
at _next (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/settings/themes_page_tests.js:10:653)
at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/settings/themes_page_tests.js:10:818)
1/1 failed (40s)
Warnings (2):
--
The "browsers", "concurrency", and "filter" options from the configuration file will be ignored.
--
The file at "/Users/justingrabenbauer/cf2-test-cafe-poc/artifacts/screenshots/user_can_edit_theme_name_2022-09-08_11-23-30.png" already exists. It has just been rewritten with a recent screenshot. This situation can possibly cause issues. To avoid them, make sure that each screenshot has a unique path. If a test runs in multiple browsers, consider including the user agent in the screenshot path or generate a unique identifier in another way.
Screenshots
https://www.loom.com/share/fc77538c41ab489590fc937ec3e3b827
Steps to Reproduce
- Run the test (with me on a call if that's fine)
- Verify that the chrome error 5 has occurred
TestCafe version
1.20.1
Node.js version
16.4.0
Command-line arguments
ENV=staging testcafe "chrome '--window-size=1920,1159'" -t 'user_can_edit_theme_name' -c1
Browser name(s) and version(s)
Chrome 105.0.0.0
Platform(s) and version(s)
Monterey 12.5
Other
No response
The automatic browser restart mechanism works only when TestCafe retains control over the browser process. Error code 5 is a different case. With such an error, control over the browser is lost. Therefore, the best thing we can do is try to find out why the browser is down. Would you please clarify it this behavior is reproduced during manual testing or only with the help of TestCafe? If the former, then we will not be able to help. If this happens only with TestCafe, then we need an example to help us understand the problem.
This issue was automatically closed because there was no response to our request for more information from the original author. Currently, we don't have enough information to take action. Please reach out to us if you find the necessary information and are able to share it. We are also eager to know if you resolved the issue on your own and can share your findings with everyone.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}