Karma Launcher: automatic device selection

.github/CODEOWNERS

@@ -1,4 +1,4 @@
 # Users referenced in this file will automatically be requested as reviewers for PRs that modify the given paths.
 # See https://help.github.com/articles/about-code-owners/
 
-* @fpkamp @Finesse
+* @Finesse

node/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@fpjs-incubator/broyster",
   "description": "Test tools",
-  "version": "0.1.8",
+  "version": "0.2.0",
   "keywords": [
     "test",
     "tools",

node/readme.md

@@ -19,53 +19,17 @@ This package exports the following:
 
 -   `@fpjs-incubator/broyster/node`:
     -   `karmaPlugin` That can be used for launching and reporting tests.
-    -   `sslConfiguration` That provides a self-signed certificate for HTTPS testing on localhost.
-    -   `httpHttpsServer` That gives you a set of two servers - one with HTTP and one with HTTP capabilities.
-        Newer versions of Safari do not work nor have workarounds for self-signed certificates, however their behavior is the same for both HTTP and HTTPS. Depending on your entry's _useHttps_, the launcher will redirect respectively.
-        The HTTP server runs on the port provided by Karma, while the HTTPS port will run on +1 from that.
     -   `setHttpsAndServerForKarma` That configures karma for HTTP and HTTPS testing without any additional work.
     -   `BrowserFlags` Is a collection of currently supported browser arguments that are uniformed for convenience (for
         example: Incognito will add launching the browser in incognito mode for Chrome and Edge, but private mode for Firefox).
-    -   `getBrowserStackCredentials` Fetches the credentials to BrowserStack from env variables.
-    -   `BrowserStackLocalManager` Allows controlling the BrowserStack Local binary.
-    -   `BrowserStackCapabilitiesFactory` Creates an object defining what the driver session that is going to be requested.
-    -   `BrowserStackSessionFactory` Creates a Selenium webdriver that connects to BrowserStack.
     -   `makeKarmaConfigurator` Makes a function that applies an opinionated full configuration, used by Fingerprint's projects, to Karma.
 -   `@fpjs-incubator/broyster/browser`:
     -   `retryFailedTests` That allows overriding the different behavior of Jasmine specs. The new behavior will retry a failed test up until the maximum specified in the first parameter, with a delay between each such attempt, indicated by the second parameter (in miliseconds). Call this function in the root of any executable file, involved in your testing code, for example, in a Jasmine helper file. Once called, it affects all tests Jasmine runs, even in the other files. For Karma, you can add a file that contains the invocation and point it in your `files`, that way you will not have it tied to one specific test file.
 
 Use `node` exports when using Node.js contexts, like configuring Karma.
 Use `browser` exports when using browser contexts, like Jasmine.
 
-To use mixed HTTP/HTTPS testing, in your Karma config file you need to:
-Set the protocol to https
-
-```js
-protocol: 'https'
-```
-
-define _httpServerOptions_ and use the provided keys
-
-```js
-import { sslConfiguration } from '@fpjs-incubator/broyster/node'
-
-httpsServerOptions: {
-  key: sslConfiguration.key,
-  cert: sslConfiguration.cert,
-  requestCert: false,
-  rejectUnauthorized: false,
-}
-```
-
-and use the provided server:
-
-```js
-import { karmaPlugin, sslConfiguration, httpHttpsServer } from '@fpjs-incubator/broyster/node'
-
-httpModule: httpHttpsServer as any
-```
-
-or use
+To use mixed HTTP/HTTPS testing, in your Karma config file you need to use:
 
 ```js
 import { setHttpsAndServerForKarma } from '@fpjs-incubator/broyster'
@@ -82,31 +46,20 @@ _useHttps_ to specify if this launcher is supposed to connect to the HTTPS serve
 useHttps: true
 ```
 
-_deviceName_ is now a union type of `string | string[] | undefined`. In case of passing an array, it will mean there is a list of devices that are acceptable and any of them will be good to use. The list of devices will be iterated only in an attempt to launch a session, so the first succesful configuration to run will be the one that the tests run against. Tests will not run against all devices in the list. Note that the compatibility between the devices and the rest of the specified config is your responsibility.
+_deviceType_ is used only on iOS and allows to choose from `iPhone` (default) and `iPad`.
 
 ```js
   Android11_ChromeLatest: {
-    deviceName: 'Google Pixel 4',
-    platform: 'Android',
-    osVersion: '11.0',
-    browserName: 'Chrome',
-    browserVersion: 'latest-beta',
-    useHttps: true,
-  },
-```
-
-or
-
-```js
-  iOS15_Safari: {
-    deviceName: ['iPhone 8 Plus', 'iPhone 11 Pro', 'iPhone 11'],
     platform: 'iOS',
-    osVersion: '15',
+    deviceType: 'iPhone',
+    osVersion: '17',
     browserName: 'Safari',
     useHttps: true,
   },
 ```
 
+You don't need to set a specific device name, the launcher chooses a device automatically. Same on Android.
+
 _firefoxCapabilities_ an array of extra capabilities specifically for Firefox.
 
 ```js
@@ -119,7 +72,7 @@ firefoxCapabilities: [
 
 ### Reporters
 
-There is a dedicated reproter that will mark successful tests as passed in BrowserStack.
+There is a dedicated reporter that will mark successful tests as passed in BrowserStack.
 
 ```js
   config.set({
@@ -154,27 +107,6 @@ The following config options are available inside the browserStack section of th
   },
 ```
 
-### Using Selenium directly
-
-```js
-import { BrowserStackCapabilitiesFactory, BrowserStackSessionFactory } from '@fpjs-incubator/broyster/node'
-
-const local = false // Execute webdriver commands on BrowserStack remotely
-const capabilitiesFactory = new BrowserStackCapabilitiesFactory({ username, accessKey }, local)
-
-const sessionFc = new BrowserStackSessionFactory({
-    project: 'PROJECT',
-    build: 'BUILD',
-    capabilitiesFactory,
-})
-
-const [driver, name] = sessionFc.tryCreateBrowser(launchOptions, runId, attempt, logger)
-
-await driver.navigate().to('https://google.com')
-
-await driver.quit()
-```
-
 ## Full Karma configuration
 
 `makeKarmaConfigurator` is an alternative to creating a Karma configuration from scratch.

node/src/browser_map.ts

@@ -1,7 +1,7 @@
 import { WebDriver } from 'selenium-webdriver'
 
-export type BrowserMap = Map<string, { browser: WebDriver; session: string }>
+export type BrowserMap = Map<string, { browser: WebDriver; sessionId: string }>
 
 export function makeBrowserMapFactory(): BrowserMap {
-  return new Map<string, { browser: WebDriver; session: string }>() satisfies BrowserMap
+  return new Map<string, { browser: WebDriver; sessionId: string }>()
 }

node/src/browserstack_browsers.ts

@@ -0,0 +1,96 @@
+import { Browser } from 'browserstack'
+import { Logger } from './karma_logger'
+import { BrowserStackCredentials, getBrowsers } from './browserstack_helpers'
+
+/**
+ * Loads and caches the list of browsers supported by BrowserStack
+ */
+export class BrowserStackBrowsers {
+  private _allBrowsersPromise?: Promise<Browser[]>
+  constructor(private _credentials: BrowserStackCredentials) {}
+
+  public async getIOSDevices(
+    osVersion: string | null,
+    deviceType: 'iphone' | 'ipad' | null,
+    browserType: 'safari' | 'chrome' | null,
+    realDevices: boolean | null,
+    log: Logger,
+  ): Promise<Browser[]> {
+    const allBrowsers = await this.getAllBrowsers(log)
+    return allBrowsers.filter(
+      (browser) =>
+        browser.os === 'ios' &&
+        ignoreNullExpected(doesOSVersionMatch, browser, osVersion) &&
+        ignoreNullExpected(doesDeviceTypeMatch, browser, deviceType) &&
+        ignoreNullExpected(doesIOSBrowserTypeMatch, browser, browserType) &&
+        ignoreNullExpected(doesRealDeviceMatch, browser, realDevices),
+    )
+  }
+
+  public async getAndroidDevices(
+    osVersion: string | null,
+    browserType: 'chrome' | 'samsung' | null,
+    realDevices: boolean | null,
+    log: Logger,
+  ): Promise<Browser[]> {
+    const allBrowsers = await this.getAllBrowsers(log)
+    return allBrowsers.filter(
+      (browser) =>
+        browser.os === 'android' &&
+        ignoreNullExpected(doesOSVersionMatch, browser, osVersion) &&
+        ignoreNullExpected(doesAndroidBrowserTypeMatch, browser, browserType) &&
+        ignoreNullExpected(doesRealDeviceMatch, browser, realDevices),
+    )
+  }
+
+  private async getAllBrowsers(log: Logger) {
+    this._allBrowsersPromise ??= getBrowsers(this._credentials, log)
+    return await this._allBrowsersPromise
+  }
+}
+
+export function makeBrowserStackBrowsers(browserStackCredentials: BrowserStackCredentials): BrowserStackBrowsers {
+  return new BrowserStackBrowsers(browserStackCredentials)
+}
+
+function doesOSVersionMatch(browser: Browser, expectedOSVersion: string) {
+  return browser.os_version === expectedOSVersion
+}
+
+function doesDeviceTypeMatch(browser: Browser, expectedDeviceType: string) {
+  return browser.device?.slice(0, expectedDeviceType.length).toLowerCase() === expectedDeviceType.toLowerCase()
+}
+
+function doesRealDeviceMatch(browser: Browser, expectedRealDevice: boolean) {
+  return browser.real_mobile === expectedRealDevice
+}
+
+function doesIOSBrowserTypeMatch(browser: Browser, expectedBrowserType: 'safari' | 'chrome') {
+  if (expectedBrowserType === 'safari') {
+    return browser.browser === 'iphone' || browser.browser === 'ipad'
+  } else if (expectedBrowserType === 'chrome') {
+    // The browser name accepted by BrowserStack is "Chrome" despite returning "chromium" from /automate/browsers.json
+    return browser.browser === 'chromium'
+  } else {
+    return browser.browser === expectedBrowserType
+  }
+}
+
+function doesAndroidBrowserTypeMatch(browser: Browser, expectedBrowserType: 'chrome' | 'samsung') {
+  if (expectedBrowserType === 'chrome') {
+    return browser.browser === 'android'
+  } else {
+    return browser.browser === expectedBrowserType
+  }
+}
+
+function ignoreNullExpected<T>(
+  criterion: (browser: Browser, expected: T) => boolean,
+  browser: Browser,
+  expected: T | null,
+): boolean {
+  if (expected === null) {
+    return true
+  }
+  return criterion(browser, expected)
+}

node/src/browserstack_helpers.ts

@@ -1,4 +1,4 @@
-import { AutomateClient, createAutomateClient } from 'browserstack'
+import { AutomateClient, Browser, createAutomateClient } from 'browserstack'
 import { promisify } from 'util'
 import { Logger } from './karma_logger'
 
@@ -51,3 +51,12 @@ export async function canNewBrowserBeQueued(
   log.debug('Max queue: ' + max + '. Running: ' + running + '. Required ' + slots + '. Returning: ' + canRun)
   return canRun
 }
+
+export async function getBrowsers(credentials: BrowserStackCredentials, log: Logger): Promise<Browser[]> {
+  const browserstackClient = createBrowserStackClient(credentials)
+  log.debug('calling getBrowsers')
+  const browsers = await promisify(browserstackClient.getBrowsers).call(browserstackClient)
+  log.debug('getBrowsers returned:')
+  log.debug(JSON.stringify(browsers))
+  return browsers
+}

node/src/browserstack_reporter.ts

@@ -63,7 +63,7 @@ export function BrowserStackReporter(
       pendingUpdates++
       const apiStatus = !(result.failed || result.error || result.disconnected) ? 'passed' : 'error'
       browserstackClient.updateSession(
-        browserMap.get(browserId)?.session ?? '',
+        browserMap.get(browserId)?.sessionId ?? '',
         {
           status: apiStatus,
         },

node/src/browserstack_session_factory.ts

@@ -5,7 +5,7 @@ import { Logger } from './karma_logger'
 import { OptionsBuilder } from './options_builder'
 import { WebDriverFactory } from './webdriver_factory'
 import { BrowserStackCredentials } from './browserstack_helpers'
-import { ThenableWebDriver } from 'selenium-webdriver'
+import { WebDriver } from 'selenium-webdriver'
 import { LocalIdentifier } from './browserstack_local_manager'
 
 export interface BrowserStackSessionFactoryConfig {
@@ -31,57 +31,42 @@ export class BrowserStackSessionFactory {
     this._localIdentifier = config.localIdentifier
   }
 
-  tryCreateBrowser(
-    browsers: CustomLauncher,
+  public async createBrowser(
+    browser: CustomLauncher,
+    deviceName: string | null,
     id: string,
-    attempt: number,
     log: Logger,
-  ): [driver: ThenableWebDriver, name: string | null] {
-    if (Array.isArray(browsers.deviceName)) {
-      const device = browsers.deviceName[attempt % browsers.deviceName.length]
-      return [this.makeFromDevicesSet(browsers, id, device, log), device]
-    }
-    return [this.createBrowser(browsers, id, log), null]
-  }
-
-  private makeFromDevicesSet(browsers: CustomLauncher, id: string, device: string, log: Logger): ThenableWebDriver {
-    const name = browsers.browserName + ' on ' + device + ' for ' + browsers.platform + ' ' + browsers.osVersion
+  ): Promise<WebDriver> {
+    log.debug('creating session')
     try {
-      log.info('creating session for ' + name)
-      const launcher = Object.assign({}, browsers)
-      launcher.deviceName = device
-      const browser = this.createBrowser(launcher, id, log)
-      log.info(name + ' created succesfully')
-      return browser
-    } catch (err) {
-      log.error('could not create session for ' + name + ', trying next configuration')
-      throw err
-    }
-  }
-
-  private createBrowser(browser: CustomLauncher, id: string, log: Logger): ThenableWebDriver {
-    const caps = this._capsFactory.create(
-      browser.browserName,
-      this._build,
-      id,
-      this._project,
-      browser.deviceName as string,
-      browser.platform,
-      this._idleTimeout,
-      browser.osVersion,
-      browser.browserVersion,
-      this._localIdentifier,
-    )
-    if (browser.browserName?.toLowerCase().includes('safari') && browser.flags) {
-      caps.safariOptions = OptionsBuilder.createSafariArguments(browser.flags)
-    }
-    log.debug('created capabilities: ' + JSON.stringify(caps))
-    const opts = OptionsBuilder.create(browser.browserName, browser.flags)
-    log.debug('created options: ' + JSON.stringify(opts))
-    if (browser.firefoxCapabilities) {
-      log.debug('using firefox capabilities: ' + browser.firefoxCapabilities)
+      const caps = this._capsFactory.create(
+        browser.browserName,
+        this._build,
+        id,
+        this._project,
+        deviceName ?? undefined,
+        browser.platform,
+        this._idleTimeout,
+        browser.osVersion,
+        browser.browserVersion,
+        this._localIdentifier,
+      )
+      if (browser.browserName?.toLowerCase().includes('safari') && browser.flags) {
+        caps.safariOptions = OptionsBuilder.createSafariArguments(browser.flags)
+      }
+      log.debug('created capabilities: ' + JSON.stringify(caps))
+      const opts = OptionsBuilder.create(browser.browserName, browser.flags)
+      log.debug('created options: ' + JSON.stringify(opts))
+      if (browser.firefoxCapabilities) {
+        log.debug('using firefox capabilities: ' + browser.firefoxCapabilities)
+      }
+      const webdriver = await WebDriverFactory.createFromOptions(opts, caps, browser.firefoxCapabilities)
+      log.debug('session created successfully')
+      return webdriver
+    } catch (error) {
+      log.debug('session creation failed')
+      throw error
     }
-    return WebDriverFactory.createFromOptions(opts, caps, browser.firefoxCapabilities)
   }
 }
 

node/src/index_node.ts

@@ -1,10 +1,4 @@
 export { default as karmaPlugin } from './karma_plugin'
-export { sslConfiguration } from './server_certificates'
-export * as httpHttpsServer from './http_https_server'
 export { setHttpsAndServerForKarma } from './karma_https_config'
 export { Arguments as BrowserFlags } from './arguments'
-export { BrowserStackSessionFactory, BrowserStackSessionFactoryConfig } from './browserstack_session_factory'
-export { BrowserStackCapabilitiesFactory } from './browserstack_capabilities_factory'
-export { getBrowserStackCredentials } from './browserstack_helpers'
-export { BrowserStackLocalManager } from './browserstack_local_manager'
 export { makeKarmaConfigurator } from './karma_configuration'