visual-testing
visual-testing copied to clipboard
webdriverio
Getting "browser.saveScreen is not a function" for ios mobile automation
Environment (please complete the following information):
- Node.js version: [e.g. 8.9.1]
- NPM version: [e.g. 5.8.0]
- Browser name and version: [e.g. Chrome 68]
- Platform name and version: [e.g. Windows 10]
- WebdriverIO version: [e.g. 5.1.0]
@wdio/visual-serviceversion: [e.g. 1.0.0]
Config of WebdriverIO + @wdio/visual-service
export const config = {
env,
deviceName,
noReset,
isBeta,
// logLevel: "info",
logLevel: "debug",
logTimestamp: true,
startTime: new Date().getTime(),
suites: suite,
exclude: [],
packageName,
isPlatformAndroid,
platform,
// user: bsUser,
// key: bsKey,
capabilities: [{
platformName: platform,
"appium:deviceName": "iphone12",
"appium:platformVersion": deviceOSVersion,
"appium:automationName": automationName,
"appium:orientation": "PORTRAIT",
"appium:newCommandTimeout": 40000,
"appium:noReset": noReset,
"appium:unicodeKeyboard": false,
"appium:autoGrantPermissions": true,
"appium:autoDismissAlerts": true,
"appium:isHybridApp": true,
"appium:disableIdLocatorAutocompletion": true,
"appium:app": appPath,
...capabilities,
}
],
maxInstances: 1,
updateJob: false,
coloredLogs: true,
screenshotPath: './errorShots/',
baseUrl: '',
waitforTimeout: 100000,
connectionRetryTimeout: 90000,
connectionRetryCount: 3,
framework: 'mocha',
services: [
[
"appium",
{
args: {
relaxedSecurity: true,
platformName: platform,
port: devicePort,
},
command: "appium",
},
],
[
"visual",
{
baselineFolder: `${parentDirectory}/testData/visualRegression/imageTestData/base-image/`,
screenshotPath: `${parentDirectory}/testData/visualRegression/imageTestData/image-compare/`,
autoSaveBaseline: true,
rawMisMatchPercentage: true,
blockOutStatusBar: true,
blockOutNavigationBar: true,
ignoreTransparentPixel: true,
ignoreAntialiasing: true,
ignoreAlpha: true,
disableCSSAnimation: true,
savePerInstance: true,
addIOSBezelCorners:true,
formatImageName: "{tag}-{deviceName}",
enableLayoutTesting: true,
logLevel: "debug"
},
],
],
reporters: [
'spec',
["allure", {
outputDir: `${parentDirectory}/allureReport/${platform}/allure-results-${deviceName}`,
disableWebdriverScreenshotsReporting: false,
addConsoleLogs: true,
timestamp: true,
}],
["junit", {
outputDir: `${parentDirectory}/junitReport/${platform}/`,
outputFileFormat: () => `${deviceName}_AutomationReport.xml`,
}],
],
mochaOpts: {
ui: "bdd",
timeout: 2700000,
},
};
Package.json
"dependencies": {
"@ifpl-git/webdriverio-ui-automation": "^1.2.0",
"appium": "^2.10.3",
"appium-uiautomator2-driver": "^3.5.6",
"appium-xcuitest-driver": "^7.17.6",
"canvas": "^2.8.0",
"chai-as-promised": "^8.0.0",
"chromedriver": "^126.0.2",
"combined-stream": "^1.0.8",
"core-js": "^3.37.1",
"csv-parser": "^3.0.0",
"log4js": "^6.9.1",
"mochawesome": "^7.1.3",
"mochawesome-report-generator": "^6.2.0",
"request": "^2.88.2",
"supertest": "^7.0.0",
"wdio-browserstack-reporter": "1.0.1",
"wdio-video-reporter": "^5.2.0"
},
"devDependencies": {
"@moroo/wdio-slack-reporter": "^8.1.0",
"@wdio/allure-reporter": "^8.39.0",
"@wdio/appium-service": "^8.39.0",
"@wdio/browserstack-service": "^8.39.0",
"@wdio/cli": "^8.39.0",
"@wdio/dot-reporter": "^8.39.0",
"@wdio/junit-reporter": "^8.39.0",
"@wdio/local-runner": "^8.39.0",
"@wdio/mocha-framework": "^8.39.0",
"@wdio/spec-reporter": "^8.39.0",
"@wdio/visual-service": "^5.0.1",
"allure-commandline": "^2.29.0",
"chai": "^4.3.7",
"mochawesome-merge": "^4.3.0",
"ts-node": "^10.9.2",
"wdio-allure-reporter": "^0.8.3",
"wdio-chromedriver-service": "^8.1.1",
"wdio-junit-reporter": "0.4.4"
},
Describe the bug While executing ios automation for the visual regression getting this error whereas same works fine for android execution
Log
[0-0] [2024-06-20T17:39:50.046] [INFO] default - deviceName : iPhone12_New , env : test , platform : iOS , isBeta : false , noReset : true , packageName : com.tstudents.iOSApp.Test , automationName : XCUITest , appActivity : com.slice.android.main.SingleActivity , isPlatformAndroid : false , defaultPlatform : Android , deviceUDID 00008101-00163C2134D2001E , deviceOSVersion : 17.2 , devicePort 4723 , executeOnBrowserStack : false , appPath : /Users/karthikeyanparamasivam/Downloads/ios_ipa/app.ipa
[0-0] 2024-06-20T12:09:50.048Z DEBUG @wdio/runner: init remote session
[0-0] 2024-06-20T12:09:50.051Z DEBUG @wdio/utils:initializeServices: initialize service "appium" as NPM package
[0-0] 2024-06-20T12:09:50.067Z DEBUG @wdio/utils:initializeServices: initialize service "visual" as NPM package
[0-0] {
[0-0] options: {
[0-0] rootDir: '/Users/karthikeyanparamasivam/Documents/mobile_automation/mobile_automation_3/Mobile-App-Automation/config',
[0-0] specs: [],
[0-0] suites: {
[0-0] sampleTest: [Array],
[0-0] loginSmoke: [Array],
[0-0] basicAppSmokeSuite: [Array],
[0-0] basicAppSanitySuite: [Array],
[0-0] productsSanitySuite: [Array],
[0-0] productsRegressionSuite: [Array],
[0-0] miniSanity: [Array],
[0-0] platformSanitySuite: [Array],
[0-0] platformRegressionSuite: [Array],
[0-0] platformVisualRegressionSuite: [Array],
[0-0] experienceSanitySuite: [Array],
[0-0] experienceRegressionSuite: [Array],
[0-0] borrowSanitySuite: [Array],
[0-0] borrowRegressionSuite: [Array],
[0-0] CxRegressionSuite: [Array],
[0-0] loginPageTest: [Array],
[0-0] perfSuite: [Array],
[0-0] visualSanitySuite: [Array],
[0-0] deepLinkSanity: [Array],
[0-0] sanitySuite: [Array],
[0-0] platformsEvents: [Array],
[0-0] borrowBasicFlowSanitySuite: [Array],
[0-0] borrowOnboardingSalariedFlow: [Array],
[0-0] borrowOnboardingStudentFlow: [Array],
[0-0] borrowOnboardingFreelancerFlow: [Array],
[0-0] A2RegressionSuite: [Array],
[0-0] borrowVisualRegression: [Array],
[0-0] activitySanity: [Array],
[0-0] accountRegressionSuite: [Array],
[0-0] withdrawSuite: [Array],
[0-0] accountsOnboardingSanitySuite: [Array],
[0-0] accountsOnboardingReleaseRegressionSuite: [Array],
[0-0] accountsWithDrawRegressionSuite: [Array],
[0-0] hnsRegressionSuite: [Array],
[0-0] UPISanity: [Array],
[0-0] visualSanitySuiteAccounts: [Array],
[0-0] UPIVPATransaction: [Array],
[0-0] UPIBANKTRANSACTION: [Array],
[0-0] UPIPHONETRANSACTION: [Array],
[0-0] UPIRUPAYCARDTRANSACTION: [Array],
[0-0] UPIOnboardingAutoFetch: [Array],
[0-0] UPIOnboardingManual: [Array],
[0-0] UPISelfTransaction: [Array],
[0-0] UPISelfTransactionSendV2: [Array],
[0-0] UPIDualSim: [Array],
[0-0] UPIPrivateNumberTransaction: [Array],
[0-0] UPIMapperOnboarding: [Array],
[0-0] UPIShorterOnboardingRepay: [Array],
[0-0] UPIAccountPage: [Array],
[0-0] UPIEventComparision: [Array],
[0-0] UPIOnboardingS2S: [Array],
[0-0] UPIMapperNumber: [Array],
[0-0] UPIOnBoardingPGBottomSheet: [Array],
[0-0] UPIVisualRegression: [Array],
[0-0] electricityBill: [Array],
[0-0] postpaid: [Array],
[0-0] pipedGasBill: [Array],
[0-0] waterBill: [Array],
[0-0] cylinder: [Array],
[0-0] broadband: [Array],
[0-0] dthRecharge: [Array],
[0-0] loan: [Array],
[0-0] fastagRecharge: [Array],
[0-0] UPIOnboardingTest: [Array],
[0-0] UPIOnboardingCommon: [Array],
[0-0] advanceSearch: [Array],
[0-0] 'sendv2.5': [Array],
[0-0] bbPSVisualRegression: [Array],
[0-0] RepaymentSanitySuite: [Array],
[0-0] RepaymentRegressionSuite: [Array],
[0-0] RepaymentVisualRegression: [Array],
[0-0] UPIOnboarding: [Array],
[0-0] UPISend: [Array]
[0-0] },
[0-0] exclude: [],
[0-0] outputDir: undefined,
[0-0] logLevel: 'debug',
[0-0] logLevels: {},
[0-0] groupLogsByTestSpec: false,
[0-0] excludeDriverLogs: [],
[0-0] bail: 0,
[0-0] waitforInterval: 500,
[0-0] waitforTimeout: 100000,
[0-0] framework: 'mocha',
[0-0] reporters: [ 'spec', [Array], [Array] ],
[0-0] services: [ [Array], [Array] ],
[0-0] maxInstances: 1,
[0-0] maxInstancesPerCapability: 100,
[0-0] injectGlobals: true,
[0-0] filesToWatch: [],
[0-0] connectionRetryTimeout: 90000,
[0-0] connectionRetryCount: 3,
[0-0] execArgv: [],
[0-0] runnerEnv: {},
[0-0] runner: 'local',
[0-0] shard: { current: 1, total: 1 },
[0-0] specFileRetries: 0,
[0-0] specFileRetriesDelay: 0,
[0-0] specFileRetriesDeferred: false,
[0-0] reporterSyncInterval: 100,
[0-0] reporterSyncTimeout: 5000,
[0-0] cucumberFeaturesWithLineNumbers: [],
[0-0] autoCompileOpts: { autoCompile: true, tsNodeOpts: [Object], babelOpts: {} },
[0-0] mochaOpts: { timeout: 2700000, ui: 'bdd' },
[0-0] jasmineOpts: { defaultTimeoutInterval: 10000 },
[0-0] cucumberOpts: { timeout: 10000 },
[0-0] onPrepare: [ [Function: bound ] AsyncFunction ],
[0-0] onWorkerStart: [],
[0-0] onWorkerEnd: [],
[0-0] before: [],
[0-0] beforeSession: [],
[0-0] beforeSuite: [],
[0-0] beforeHook: [],
[0-0] beforeTest: [ [Function: bound beforeTest] AsyncFunction ],
[0-0] beforeCommand: [],
[0-0] afterCommand: [],
[0-0] afterTest: [],
[0-0] afterHook: [],
[0-0] afterSuite: [],
[0-0] afterSession: [],
[0-0] after: [ [Function: bound after] AsyncFunction ],
[0-0] onComplete: [ [Function: bound ] AsyncFunction ],
[0-0] onReload: [],
[0-0] beforeAssertion: [],
[0-0] afterAssertion: [],
[0-0] beforeFeature: [],
[0-0] beforeScenario: [],
[0-0] beforeStep: [],
[0-0] afterStep: [],
[0-0] afterScenario: [],
[0-0] afterFeature: [],
[0-0] _: [],
[0-0] suite: [ 'sampleTest' ],
[0-0] '$0': '/Users/karthikeyanparamasivam/Documents/mobile_automation/mobile_automation_3/Mobile-App-Automation/node_modules/.bin/wdio',
[0-0] ignoredWorkerServices: [],
[0-0] env: 'test',
[0-0] deviceName: 'iPhone12_New',
[0-0] noReset: true,
[0-0] isBeta: false,
[0-0] logTimestamp: true,
[0-0] startTime: 1718885390047,
[0-0] packageName: 'com.tstudents.iOSApp.Test',
[0-0] isPlatformAndroid: false,
[0-0] platform: 'iOS',
[0-0] updateJob: false,
[0-0] coloredLogs: true,
[0-0] screenshotPath: './errorShots/',
[0-0] baseUrl: '',
[0-0] _automationProtocol: undefined,
[0-0] automationProtocol: './protocol-stub.js',
[0-0] protocol: 'http',
[0-0] hostname: '127.0.0.1',
[0-0] path: '/',
[0-0] port: 4723,
[0-0] capabilities: {
[0-0] platformName: 'iOS',
[0-0] 'appium:deviceName': 'iphone12',
[0-0] 'appium:platformVersion': '17.2',
[0-0] 'appium:automationName': 'XCUITest',
[0-0] 'appium:orientation': 'PORTRAIT',
[0-0] 'appium:newCommandTimeout': 40000,
[0-0] 'appium:noReset': true,
[0-0] 'appium:unicodeKeyboard': false,
[0-0] 'appium:autoGrantPermissions': true,
[0-0] 'appium:autoDismissAlerts': true,
[0-0] 'appium:isHybridApp': true,
[0-0] 'appium:disableIdLocatorAutocompletion': true,
[0-0] 'appium:app': '/Users/karthikeyanparamasivam/Downloads/ios_ipa/app.ipa',
[0-0] 'appium:udid': '00008101-00163C2134D2001E',
[0-0] 'appium:xcodeOrgId': 'com.tstudents.WebDriverAgentRunner',
[0-0] 'appium:xcodeSigningId': 'iPhone Developer',
[0-0] 'appium:wdaLocalPort': 8101,
[0-0] 'appium:bundleId': 'com.tstudents.iOSApp.Test'
[0-0] }
[0-0] },
[0-0] capabilities: {
[0-0] platformName: 'iOS',
[0-0] deviceName: 'iphone12',
[0-0] platformVersion: '17.2',
[0-0] automationName: 'XCUITest',
[0-0] orientation: 'PORTRAIT',
[0-0] newCommandTimeout: 40000,
[0-0] noReset: true,
[0-0] unicodeKeyboard: false,
[0-0] autoGrantPermissions: true,
[0-0] autoDismissAlerts: true,
[0-0] isHybridApp: true,
[0-0] disableIdLocatorAutocompletion: true,
[0-0] app: '/Users/karthikeyanparamasivam/Downloads/ios_ipa/app.ipa',
[0-0] udid: '00008101-00163C2134D2001E',
[0-0] xcodeOrgId: 'com.tstudents.WebDriverAgentRunner',
[0-0] xcodeSigningId: 'iPhone Developer',
[0-0] wdaLocalPort: 8101,
[0-0] bundleId: 'com.tstudents.iOSApp.Test'
[0-0] },
[0-0] requestedCapabilities: {
[0-0] platformName: 'iOS',
[0-0] deviceName: 'iphone12',
[0-0] platformVersion: '17.2',
[0-0] automationName: 'XCUITest',
[0-0] orientation: 'PORTRAIT',
[0-0] newCommandTimeout: 40000,
[0-0] noReset: true,
[0-0] unicodeKeyboard: false,
[0-0] autoGrantPermissions: true,
[0-0] autoDismissAlerts: true,
[0-0] isHybridApp: true,
[0-0] disableIdLocatorAutocompletion: true,
[0-0] app: '/Users/karthikeyanparamasivam/Downloads/ios_ipa/app.ipa',
[0-0] udid: '00008101-00163C2134D2001E',
[0-0] xcodeOrgId: 'com.tstudents.WebDriverAgentRunner',
[0-0] xcodeSigningId: 'iPhone Developer',
[0-0] wdaLocalPort: 8101,
[0-0] bundleId: 'com.tstudents.iOSApp.Test'
[0-0] },
[0-0] customCommands: [],
[0-0] overwrittenCommands: [],
[0-0] commandList: [],
[0-0] isChrome: false,
[0-0] isFirefox: false,
[0-0] isMobile: true,
[0-0] isIOS: true,
[0-0] isAndroid: false,
[0-0] isSauce: false,
[0-0] isBidi: false,
[0-0] isChromium: false,
[0-0] addCommand: [Function (anonymous)],
[0-0] overwriteCommand: [Function (anonymous)],
[0-0] addLocatorStrategy: [Function (anonymous)]
[0-0] }
Additional context
since the old native comparison is deprecated we migrated to this service but blocked on ios automation.
Can you provide a reproducible example? I'm not having this issue on a clean project.
@wswebcreation In our project, we utilize a visual regression testing suite that employs Appium with WebDriverIO (wdio/cli 8) and the @wdio/visual-service module. The workflow involves launching our native mobile app where we capture screenshots for comparison against baseline images.
While this setup functions correctly for Android executions, we've encountered an issue specific to iOS. The problem manifests in the inability to save screenshots using the visual-service's save screen function. Despite the identical code and services used across both platforms, this functionality is currently not operational on iOS.
That's why I'm asking for a clean project as an example, I'm not seeing this behaviour for iOS
I'm debugging this and I'm almost 100% sure this is an Allure issue. From what I can see, Allure doesn't get any context (tests) when the Visual Module registers itself. This is only failing on iOS, not Android (this is the part I need to debug).
I found this error while taking screenshot Error: There isn't any active test! which is coming from Allure. Because Allure is failing it will not proceed setting up the Visual configuration which makes the Visual module not registering the commands, hence the browser.saveScreen is not a function
It will cost me some more time to debug it and find a solution
checkScreen works if we disable screenshot reporting in Allure, but then we don't have screenshots attached to the report
reporters: [ ... ['allure', { outputDir: 'allure-results', disableWebdriverStepsReporting: true, > disableWebdriverScreenshotsReporting: true, useCucumberStepReporter: true, disableMochaHooks: true, addConsoleLogs: false, }, ], ],
