Random error in `with-driver-tests`

[dgr]

Version
I specifically saw this with 3b1e29c, but I've seen it before with other versions as well.

Platform
This specifically happened with the "api windows firefox jdk21" environment in the CI test harness. I don't know the platform details for that.

Symptom
There seems to be a race condition in the test that causes it to fail randomly.

Reproduction
I have not reproduced it locally. It seems to happen in the CI test suite with some regularity.

Actual behavior
Here's the relevant output from the CI test suite under task test-server:

=== with-driver-tests [jvm]
.... firefox

ERROR in (with-driver-tests) (support.clj:201)
firefox
expected: (= "Webdriver Test Document" (e/with-driver :firefox driver (e/go driver test-page) (e/get-title driver)) (e/with-firefox nil driver (e/go driver test-page) (e/get-title driver)) (e/with-firefox {} driver (e/go driver test-page) (e/get-title driver)) (e/with-firefox driver (e/go driver test-page) (e/get-title driver)) (e/with-firefox-headless nil driver (is (= true (e/headless? driver))) (e/go driver test-page) (e/get-title driver)) (e/with-firefox-headless {} driver (is (= true (e/headless? driver))) (e/go driver test-page) (e/get-title driver)) (e/with-firefox-headless driver (is (= true (e/headless? driver))) (e/go driver test-page) (e/get-title driver)) (e2/with-firefox [driver] (e/go driver test-page) (e/get-title driver)) (e2/with-firefox-headless [driver] (is (= true (e/headless? driver))) (e/go driver test-page) (e/get-title driver)))
  actual: clojure.lang.ExceptionInfo: throw+: {:response {:value {:error "no such window", :message "Browsing context has been discarded", :stacktrace "RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:193:5\nNoSuchWindowError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:679:5\nassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:515:13\nassert.open@chrome://remote/content/shared/webdriver/Assert.sys.mjs:147:4\nGeckoDriver.prototype.navigateTo@chrome://remote/content/marionette/driver.sys.mjs:859:39\ndespatch@chrome://remote/content/marionette/server.sys.mjs:318:40\nexecute@chrome://remote/content/marionette/server.sys.mjs:289:16\nonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\nonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"}}, :path "session/e6665b0d-7ad2-4f39-ad15-6f8196c8200c/url", :payload {:url "http://localhost:50663/test.html"}, :method :post, :type :etaoin/http-error, :port 50813, :host "127.0.0.1", :status 404, :webdriver-url nil, :driver {:args ("geckodriver" "--port" 50813), :process {:proc #object[java.lang.ProcessImpl 0x621a7d83 "Process[pid=7020, exitValue=\"not exited\"]"], :exit nil, :in #object[java.io.BufferedOutputStream 0x58de1da2 "java.io.BufferedOutputStream@58de1da2"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x3c84c32f "java.lang.ProcessBuilder$NullInputStream@3c84c32f"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x3c84c32f "java.lang.ProcessBuilder$NullInputStream@3c84c32f"], :prev nil, :cmd ["C:\\SeleniumWebDrivers\\GeckoDriver\\geckodriver.exe" "--port" "50813"]}, :locator "xpath", :type :firefox, :port 50813, :host "127.0.0.1", :url "http://127.0.0.1:50813/", :created-epoch-ms 1727190165049, :session "e6665b0d-7ad2-4f39-ad15-6f8196c8200c"}}
{:response {:value {:error "no such window", :message "Browsing context has been discarded", :stacktrace "RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:193:5\nNoSuchWindowError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:679:5\nassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:515:13\nassert.open@chrome://remote/content/shared/webdriver/Assert.sys.mjs:147:4\nGeckoDriver.prototype.navigateTo@chrome://remote/content/marionette/driver.sys.mjs:859:39\ndespatch@chrome://remote/content/marionette/server.sys.mjs:318:40\nexecute@chrome://remote/content/marionette/server.sys.mjs:289:16\nonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\nonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"}}, :path "session/e6665b0d-7ad2-4f39-ad15-6f8196c8200c/url", :payload {:url "http://localhost:50663/test.html"}, :method :post, :type :etaoin/http-error, :port 50813, :host "127.0.0.1", :status 404, :webdriver-url nil, :driver {:args ("geckodriver" "--port" 50813), :process {:proc #object[java.lang.ProcessImpl 0x621a7d83 "Process[pid=7020, exitValue=1]"], :exit nil, :in #object[java.io.BufferedOutputStream 0x58de1da2 "java.io.BufferedOutputStream@58de1da2"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x3c84c32f "java.lang.ProcessBuilder$NullInputStream@3c84c32f"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x3c84c32f "java.lang.ProcessBuilder$NullInputStream@3c84c32f"], :prev nil, :cmd ["C:\\SeleniumWebDrivers\\GeckoDriver\\geckodriver.exe" "--port" "50813"]}, :locator "xpath", :type :firefox, :port 50813, :host "127.0.0.1", :url "http://127.0.0.1:50813/", :created-epoch-ms 1727190165049, :session "e6665b0d-7ad2-4f39-ad15-6f8196c8200c"}}
 at slingshot.support$stack_trace.invoke (support.clj:201)
    etaoin.impl.client$call.invokeStatic (client.clj:124)
    etaoin.impl.client$call.invoke (client.clj:82)
    etaoin.api$execute.invokeStatic (api.clj:264)
    etaoin.api$execute.invoke (api.clj:238)
    etaoin.api$go.invokeStatic (api.clj:469)
    etaoin.api$go.invoke (api.clj:457)
    etaoin.api_with_driver_test$fn__7811$fn__7880$fn__7906.invoke (api_with_driver_test.clj:210)
    etaoin.api_with_driver_test$fn__7811$fn__7880.invoke (api_with_driver_test.clj:209)
    etaoin.api_with_driver_test$fn__7811.invokeStatic (api_with_driver_test.clj:187)
    etaoin.api_with_driver_test/fn (api_with_driver_test.clj:119)
    clojure.test$test_var$fn__9761.invoke (test.clj:717)
    clojure.test$test_var.invokeStatic (test.clj:717)
    clojure.test$test_var.invoke (test.clj:708)
    clojure.test$test_vars$fn__9787$fn__9792.invoke (test.clj:735)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)
    clojure.test$test_vars$fn__9787.invoke (test.clj:735)
    etaoin.api_test$test_server.invokeStatic (api_test.clj:123)
    etaoin.api_test$test_server.invoke (api_test.clj:106)
    clojure.test$compose_fixtures$fn__9755$fn__9756.invoke (test.clj:694)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)
    clojure.test$compose_fixtures$fn__9755.invoke (test.clj:694)
    clojure.test$test_vars.invokeStatic (test.clj:731)
    clojure.test$test_all_vars.invokeStatic (test.clj:737)
    clojure.test$test_ns.invokeStatic (test.clj:758)
    clojure.test$test_ns.invoke (test.clj:743)
    clojure.core$map$fn__5884.invoke (core.clj:2759)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.boundedLength (RT.java:1793)
    clojure.lang.RestFn.applyTo (RestFn.java:130)
    clojure.core$apply.invokeStatic (core.clj:669)
    clojure.test$run_tests.invokeStatic (test.clj:768)
    clojure.test$run_tests.doInvoke (test.clj:768)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$apply.invoke (core.clj:662)
    cognitect.test_runner$test.invokeStatic (test_runner.clj:74)
    cognitect.test_runner$test.invoke (test_runner.clj:62)
    test_shared$do_test.invokeStatic (test_shared.clj:14)
    test_shared$do_test.invoke (test_shared.clj:6)
    test_shared$test.invokeStatic (test_shared.clj:20)
    test_shared$test.invoke (test_shared.clj:16)
    clojure.lang.Var.invoke (Var.java:384)
    babashka.cli.exec$main.invokeStatic (exec.clj:86)
    babashka.cli.exec$main.doInvoke (exec.clj:83)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$apply.invoke (core.clj:662)
    babashka.cli.exec$_main.invokeStatic (exec.clj:101)
    babashka.cli.exec$_main.doInvoke (exec.clj:88)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:705)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.main$main_opt.invokeStatic (main.clj:514)
    clojure.main$main_opt.invoke (main.clj:510)
    clojure.main$main.invokeStatic (main.clj:664)
    clojure.main$main.doInvoke (main.clj:616)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:705)
    clojure.main.main (main.java:40)

Expected behavior
The test should pass consistently or fail with an actual hard failure, but not fail randomly.

Diagnosis
I think this is another case of the test suite kicking off an asynchronous action with the browser, the WebDriver returns to the test code, then the test code makes another WebDriver call before the browser is ready, and the browser returns something that the test code is not expecting.

[lread]

Thanks for raising this. I've only seen it on firefox on windows, you?
My guess is that a new webdriver/browser is being recreated before the last one has a chance to fully terminate. This is probably more of any issue with geckodrive/firefox, but something we can perhaps work around.

[dgr]

I have seen this on MacOS as well (my own machine). I don't remember if it was Firefox or not. I did note that this time, in the CI testbed, it was Firefox. I don't know enough about the low-level details of spawning WebDriver/browsers to know if that could be the cause or not. Are you thinking that there is some state that is shared between the still-dying processes and the newly-created processes? What would that be? Things like sockets numbers aren't being reused, are they? If things are pretty isolated, I'm not sure why creating one before another terminates is an issue. You might want to look and see when the process termination code returns. Is it after things are really cleaned up, or is it returning early. To debug, you could maybe insert waits strategically between things. I actually did a quick REPL test of starting up two Firefox drivers simultaneously on my MacOS machine and that seems to work as expected. Now, obviously, that was not hammering anything hard, with tight timings, but there doesn't seem to be any obvious interference. Note that I got two separate processes here, each with a separate browser window. quit-ing each driver instance (not shown, below) went smoothly as well.

etaoin.api> (def driver1 (firefox))
#'etaoin.api/driver1etaoin.api> 
etaoin.api> (def driver2 (firefox))
#'etaoin.api/driver2
etaoin.api> (go driver1 "http://www.google.com")
{:value nil}
etaoin.api> (go driver2 "http://www.apple.com")
{:value nil}
etaoin.api> (get-title driver1)
"Google"
etaoin.api> (get-title driver2)
"Apple"

[lread]

I'm busy with other things these days. If you have the time and interest to explore/solve, go for it!

[dgr]

Just had another variant of the same issue appear as I was trying to debug it. Note in the first case, above, the exception gets thrown from go, but in the this case it's thrown from get-user-agent. Looking at the test code, in both cases, these calls are the first calls right after the creation of the driver with a with-* macro. As with before, Firefox is the browser. This particular failure happened on Windows. I'm not sure if the exception above happened on Windows or not.

[ TASK test:jvm ]----------------------------------------------------------------

[ Running tests ]----------------------------------------------------------------
suites: <none specified>
browsers: firefox
runner-args: [:vars etaoin.api-with-driver-test/with-driver-tests]

Running tests in #{"test"}

Testing etaoin.api-with-driver-test

[ TASK test-server ]-------------------------------------------------------------
Serving assets at http://localhost:65159
Test server static dir: ./env/test/resources/static
Test server ready
=== with-driver-tests [jvm]
.... firefox

ERROR in (with-driver-tests) (support.clj:201)
firefox
expected: (= my-agent (e/with-driver :firefox {:user-agent my-agent} driver (e/get-user-agent driver)) (e/with-firefox {:user-agent my-agent} driver (e/get-user-agent driver)) (e/with-firefox-headless {:user-agent my-agent} driver (is (= true (e/headless? driver))) (e/get-user-agent driver)) (e2/with-firefox [driver {:user-agent my-agent}] (e/get-user-agent driver)) (e2/with-firefox-headless [driver {:user-agent my-agent}] (is (= true (e/headless? driver))) (e/get-user-agent driver)))
  actual: clojure.lang.ExceptionInfo: throw+: {:response {:value {:error "no such window", :message "Browsing context has been discarded", :stacktrace "RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:193:5\nNoSuchWindowError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:679:5\nassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:515:13\nassert.open@chrome://remote/content/shared/webdriver/Assert.sys.mjs:147:4\nGeckoDriver.prototype.execute_@chrome://remote/content/marionette/driver.sys.mjs:784:15\nGeckoDriver.prototype.executeScript@chrome://remote/content/marionette/driver.sys.mjs:696:15\ndespatch@chrome://remote/content/marionette/server.sys.mjs:318:40\nexecute@chrome://remote/content/marionette/server.sys.mjs:289:16\nonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\nonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"}}, :path "session/a4828055-5f8b-4504-8f6e-ae11e016a613/execute/sync", :payload {:script "return navigator.userAgent", :args []}, :method :post, :type :etaoin/http-error, :port 65162, :host "127.0.0.1", :status 404, :webdriver-url nil, :driver {:args ("geckodriver" "--port" 65162), :capabilities {:moz:firefoxOptions {:prefs {:general.useragent.override "my agent"}}}, :process {:proc #object[java.lang.ProcessImpl 0xaad7ade "Process[pid=2420, exitValue=\"not exited\"]"], :exit nil, :in #object[java.io.BufferedOutputStream 0x770635f8 "java.io.BufferedOutputStream@770635f8"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x7887e1f3 "java.lang.ProcessBuilder$NullInputStream@7887e1f3"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x7887e1f3 "java.lang.ProcessBuilder$NullInputStream@7887e1f3"], :prev nil, :cmd ["C:\\SeleniumWebDrivers\\GeckoDriver\\geckodriver.exe" "--port" "65162"]}, :locator "xpath", :type :firefox, :port 65162, :host "127.0.0.1", :url "http://127.0.0.1:65162/", :created-epoch-ms 1727369224280, :session "a4828055-5f8b-4504-8f6e-ae11e016a613"}}
{:response {:value {:error "no such window", :message "Browsing context has been discarded", :stacktrace "RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:193:5\nNoSuchWindowError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:679:5\nassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:515:13\nassert.open@chrome://remote/content/shared/webdriver/Assert.sys.mjs:147:4\nGeckoDriver.prototype.execute_@chrome://remote/content/marionette/driver.sys.mjs:784:15\nGeckoDriver.prototype.executeScript@chrome://remote/content/marionette/driver.sys.mjs:696:15\ndespatch@chrome://remote/content/marionette/server.sys.mjs:318:40\nexecute@chrome://remote/content/marionette/server.sys.mjs:289:16\nonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\nonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"}}, :path "session/a4828055-5f8b-4504-8f6e-ae11e016a613/execute/sync", :payload {:script "return navigator.userAgent", :args []}, :method :post, :type :etaoin/http-error, :port 65162, :host "127.0.0.1", :status 404, :webdriver-url nil, :driver {:args ("geckodriver" "--port" 65162), :capabilities {:moz:firefoxOptions {:prefs {:general.useragent.override "my agent"}}}, :process {:proc #object[java.lang.ProcessImpl 0xaad7ade "Process[pid=2420, exitValue=1]"], :exit nil, :in #object[java.io.BufferedOutputStream 0x770635f8 "java.io.BufferedOutputStream@770635f8"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x7887e1f3 "java.lang.ProcessBuilder$NullInputStream@7887e1f3"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x7887e1f3 "java.lang.ProcessBuilder$NullInputStream@7887e1f3"], :prev nil, :cmd ["C:\\SeleniumWebDrivers\\GeckoDriver\\geckodriver.exe" "--port" "65162"]}, :locator "xpath", :type :firefox, :port 65162, :host "127.0.0.1", :url "http://127.0.0.1:65162/", :created-epoch-ms 1727369224280, :session "a4828055-5f8b-4504-8f6e-ae11e016a613"}}
 at slingshot.support$stack_trace.invoke (support.clj:201)
    etaoin.impl.client$call.invokeStatic (client.clj:124)
    etaoin.impl.client$call.invoke (client.clj:82)
    etaoin.api$execute.invokeStatic (api.clj:264)
    etaoin.api$execute.invoke (api.clj:238)
    etaoin.api$js_execute.invokeStatic (api.clj:1747)
    etaoin.api$js_execute.doInvoke (api.clj:1719)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    etaoin.api$get_user_agent.invokeStatic (api.clj:1800)
    etaoin.api$get_user_agent.invoke (api.clj:1797)
    etaoin.api_with_driver_test$fn__7825$fn__7894$fn__7895$fn__7896.invoke (api_with_driver_test.clj:200)
    etaoin.api_with_driver_test$fn__7825$fn__7894$fn__7895.invoke (api_with_driver_test.clj:199)
    etaoin.api_with_driver_test$fn__7825$fn__7894.invoke (api_with_driver_test.clj:198)
    etaoin.api_with_driver_test$fn__7825.invokeStatic (api_with_driver_test.clj:196)
    etaoin.api_with_driver_test/fn (api_with_driver_test.clj:128)
    clojure.test$test_var$fn__9761.invoke (test.clj:717)
    clojure.test$test_var.invokeStatic (test.clj:717)
    clojure.test$test_var.invoke (test.clj:708)
    clojure.test$test_vars$fn__9787$fn__9792.invoke (test.clj:735)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)
    clojure.test$test_vars$fn__9787.invoke (test.clj:735)
    etaoin.api_test$test_server.invokeStatic (api_test.clj:123)
    etaoin.api_test$test_server.invoke (api_test.clj:106)
    clojure.test$compose_fixtures$fn__9755$fn__9756.invoke (test.clj:694)
    clojure.test$default_fixture.invokeStatic (test.clj:687)
    clojure.test$default_fixture.invoke (test.clj:683)
    clojure.test$compose_fixtures$fn__9755.invoke (test.clj:694)
    clojure.test$test_vars.invokeStatic (test.clj:731)
    clojure.test$test_all_vars.invokeStatic (test.clj:737)
    clojure.test$test_ns.invokeStatic (test.clj:758)
    clojure.test$test_ns.invoke (test.clj:743)
    clojure.core$map$fn__5884.invoke (core.clj:2759)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.boundedLength (RT.java:1793)
    clojure.lang.RestFn.applyTo (RestFn.java:130)
    clojure.core$apply.invokeStatic (core.clj:669)
    clojure.test$run_tests.invokeStatic (test.clj:768)
    clojure.test$run_tests.doInvoke (test.clj:768)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$apply.invoke (core.clj:662)
    cognitect.test_runner$test.invokeStatic (test_runner.clj:74)
    cognitect.test_runner$test.invoke (test_runner.clj:62)
    test_shared$do_test.invokeStatic (test_shared.clj:14)
    test_shared$do_test.invoke (test_shared.clj:6)
    test_shared$test.invokeStatic (test_shared.clj:20)
    test_shared$test.invoke (test_shared.clj:16)
    clojure.lang.Var.invoke (Var.java:384)
    babashka.cli.exec$main.invokeStatic (exec.clj:86)
    babashka.cli.exec$main.doInvoke (exec.clj:83)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$apply.invoke (core.clj:662)
    babashka.cli.exec$_main.invokeStatic (exec.clj:101)
    babashka.cli.exec$_main.doInvoke (exec.clj:88)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:705)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.main$main_opt.invokeStatic (main.clj:514)
    clojure.main$main_opt.invoke (main.clj:510)
    clojure.main$main.invokeStatic (main.clj:664)
    clojure.main$main.doInvoke (main.clj:616)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:705)
    clojure.main.main (main.java:40)

[dgr]

I'm going back through errors in prior CI runs and found this one. It's the same rough symptom as the others, but this time it happened while running test-doc. OS/browser is Windows Firefox.

The relevant section from the User Guide is:

;; Start WebDriver for Firefox
(def driver (e/firefox)) ;; a Firefox window should appear
(e/driver-type driver)
;; => :firefox

;; let's perform a quick Wiki session

;; navigate to Wikipedia
(e/go driver "https://en.wikipedia.org/")

The exception was thrown from within the go.

[ Running code block tests under Clojure ]---------------------------------------

Running tests in #{"target/test-doc-blocks/test"}

Testing user-guide-adoc-test
=== block-0001 [jvm]
.... doc/01-user-guide.adoc - line 227 - Getting Started
=== block-0002 [jvm]
.... doc/01-user-guide.adoc - line 237 - Getting Started

ERROR in (block-0002) (support.clj:201)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: throw+: {:response {:value {:error "no such window", :message "Browsing context has been discarded", :stacktrace "RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:193:5\nNoSuchWindowError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:679:5\nassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:515:13\nassert.open@chrome://remote/content/shared/webdriver/Assert.sys.mjs:147:4\nGeckoDriver.prototype.navigateTo@chrome://remote/content/marionette/driver.sys.mjs:859:39\ndespatch@chrome://remote/content/marionette/server.sys.mjs:318:40\nexecute@chrome://remote/content/marionette/server.sys.mjs:289:16\nonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\nonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"}}, :path "session/96f6baf5-b69e-49f9-b5a9-9d0ddb302275/url", :payload {:url "https://en.wikipedia.org/"}, :method :post, :type :etaoin/http-error, :port 53090, :host "127.0.0.1", :status 404, :webdriver-url nil, :driver {:args ("geckodriver" "--port" 53090), :process {:proc #object[java.lang.ProcessImpl 0x16c3388e "Process[pid=1316, exitValue=\"not exited\"]"], :exit nil, :in #object[java.io.BufferedOutputStream 0x4d82ded2 "java.io.BufferedOutputStream@4d82ded2"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x655875bb "java.lang.ProcessBuilder$NullInputStream@655875bb"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x655875bb "java.lang.ProcessBuilder$NullInputStream@655875bb"], :prev nil, :cmd ["C:\\SeleniumWebDrivers\\GeckoDriver\\geckodriver.exe" "--port" "53090"]}, :locator "xpath", :type :firefox, :port 53090, :host "127.0.0.1", :url "http://127.0.0.1:53090/", :created-epoch-ms 1726496085620, :session "96f6baf5-b69e-49f9-b5a9-9d0ddb302275"}}
{:response {:value {:error "no such window", :message "Browsing context has been discarded", :stacktrace "RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:193:5\nNoSuchWindowError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:679:5\nassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:515:13\nassert.open@chrome://remote/content/shared/webdriver/Assert.sys.mjs:147:4\nGeckoDriver.prototype.navigateTo@chrome://remote/content/marionette/driver.sys.mjs:859:39\ndespatch@chrome://remote/content/marionette/server.sys.mjs:318:40\nexecute@chrome://remote/content/marionette/server.sys.mjs:289:16\nonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\nonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"}}, :path "session/96f6baf5-b69e-49f9-b5a9-9d0ddb302275/url", :payload {:url "https://en.wikipedia.org/"}, :method :post, :type :etaoin/http-error, :port 53090, :host "127.0.0.1", :status 404, :webdriver-url nil, :driver {:args ("geckodriver" "--port" 53090), :process {:proc #object[java.lang.ProcessImpl 0x16c3388e "Process[pid=1316, exitValue=\"not exited\"]"], :exit nil, :in #object[java.io.BufferedOutputStream 0x4d82ded2 "java.io.BufferedOutputStream@4d82ded2"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x655875bb "java.lang.ProcessBuilder$NullInputStream@655875bb"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x655875bb "java.lang.ProcessBuilder$NullInputStream@655875bb"], :prev nil, :cmd ["C:\\SeleniumWebDrivers\\GeckoDriver\\geckodriver.exe" "--port" "53090"]}, :locator "xpath", :type :firefox, :port 53090, :host "127.0.0.1", :url "http://127.0.0.1:53090/", :created-epoch-ms 1726496085620, :session "96f6baf5-b69e-49f9-b5a9-9d0ddb302275"}}
 at slingshot.support$stack_trace.invoke (support.clj:201)
    etaoin.impl.client$call.invokeStatic (client.clj:124)
    etaoin.impl.client$call.invoke (client.clj:82)
    etaoin.api$execute.invokeStatic (api.clj:264)
    etaoin.api$execute.invoke (api.clj:238)
    etaoin.api$go.invokeStatic (api.clj:469)
    etaoin.api$go.invoke (api.clj:457)
    user_guide_adoc_test$fn__6234.invokeStatic (user_guide_adoc_test.cljc:34)
    user_guide_adoc_test/fn (user_guide_adoc_test.cljc:24)
    clojure.test$test_var$fn__9856.invoke (test.clj:717)
    clojure.test$test_var.invokeStatic (test.clj:717)
    clojure.test$test_var.invoke (test.clj:708)
    user_guide_adoc_test$block_0002.invokeStatic (user_guide_adoc_test.cljc:24)
    user_guide_adoc_test$block_0002.invoke (user_guide_adoc_test.cljc:24)
    user_guide_adoc_test$test_ns_hook.invokeStatic (user_guide_adoc_test.cljc:1048)
    user_guide_adoc_test$test_ns_hook.invoke (user_guide_adoc_test.cljc:1048)
    clojure.test$test_ns.invokeStatic (test.clj:758)
    clojure.test$test_ns.invoke (test.clj:743)
    clojure.core$map$fn__5935.invoke (core.clj:2772)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.boundedLength (RT.java:1790)
    clojure.lang.RestFn.applyTo (RestFn.java:130)
    clojure.core$apply.invokeStatic (core.clj:669)
    clojure.test$run_tests.invokeStatic (test.clj:768)
    clojure.test$run_tests.doInvoke (test.clj:768)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$apply.invoke (core.clj:662)
    cognitect.test_runner$test.invokeStatic (test_runner.clj:74)
    cognitect.test_runner$test.invoke (test_runner.clj:62)
    cognitect.test_runner.api$do_test.invokeStatic (api.clj:14)
    cognitect.test_runner.api$do_test.invoke (api.clj:6)
    cognitect.test_runner.api$test.invokeStatic (api.clj:28)
    cognitect.test_runner.api$test.invoke (api.clj:16)
    clojure.lang.Var.invoke (Var.java:384)
    babashka.cli.exec$main.invokeStatic (exec.clj:86)
    babashka.cli.exec$main.doInvoke (exec.clj:83)
    clojure.lang.RestFn.invoke (RestFn.java:397)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.RestFn.applyTo (RestFn.java:132)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$apply.invoke (core.clj:662)
    babashka.cli.exec$_main.invokeStatic (exec.clj:101)
    babashka.cli.exec$_main.doInvoke (exec.clj:88)
    clojure.lang.RestFn.invoke (RestFn.java:397)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.RestFn.applyTo (RestFn.java:132)
    clojure.lang.Var.applyTo (Var.java:705)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.main$main_opt.invokeStatic (main.clj:514)
    clojure.main$main_opt.invoke (main.clj:510)
    clojure.main$main.invokeStatic (main.clj:664)
    clojure.main$main.doInvoke (main.clj:616)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.lang.Var.applyTo (Var.java:705)
    clojure.main.main (main.java:40)

[dgr]

@lread , I'm going to change the title of this issue now that we know that it's not constrained to with-driver-tests. It also seems to be at least focused on Windows Firefox as the OS and browser combination. At least that seems to bring it out more easily. Any issues with that?

[dgr]

Also seen under Windows Firefox Babashka.

[ TASK test-server ]-------------------------------------------------------------
Serving assets at http://localhost:55005
Test server static dir: ./env/test/resources/static
Test server ready
=== with-driver-tests [bb]
.... firefox

ERROR in (with-driver-tests) (/D:/a/etaoin/etaoin/test/etaoin/api_with_driver_test.clj:119)
firefox
expected: (= "Webdriver Test Document" (e/with-driver :firefox driver (e/go driver test-page) (e/get-title driver)) (e/with-firefox nil driver (e/go driver test-page) (e/get-title driver)) (e/with-firefox {} driver (e/go driver test-page) (e/get-title driver)) (e/with-firefox driver (e/go driver test-page) (e/get-title driver)) (e/with-firefox-headless nil driver (is (= true (e/headless? driver))) (e/go driver test-page) (e/get-title driver)) (e/with-firefox-headless {} driver (is (= true (e/headless? driver))) (e/go driver test-page) (e/get-title driver)) (e/with-firefox-headless driver (is (= true (e/headless? driver))) (e/go driver test-page) (e/get-title driver)) (e2/with-firefox [driver] (e/go driver test-page) (e/get-title driver)) (e2/with-firefox-headless [driver] (is (= true (e/headless? driver))) (e/go driver test-page) (e/get-title driver)))
  actual: clojure.lang.ExceptionInfo: throw+: {:response {:value {:error "no such window", :message "Browsing context has been discarded", :stacktrace "RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:193:5\nNoSuchWindowError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:679:5\nassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:515:13\nassert.open@chrome://remote/content/shared/webdriver/Assert.sys.mjs:147:4\nGeckoDriver.prototype.navigateTo@chrome://remote/content/marionette/driver.sys.mjs:850:39\ndespatch@chrome://remote/content/marionette/server.sys.mjs:318:40\nexecute@chrome://remote/content/marionette/server.sys.mjs:289:16\nonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\nonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"}}, :path "session/dcdceae9-6c5e-46c2-834d-80e699a95456/url", :payload {:url "http://localhost:55005/test.html"}, :method :post, :type :etaoin/http-error, :port 55168, :host "127.0.0.1", :status 404, :webdriver-url nil, :driver {:args ("geckodriver" "--port" 55168), :process {:proc #object[java.lang.ProcessImpl 0x7eab87a4 "Process[pid=7940, exitValue=\"not exited\"]"], :exit nil, :in #object[java.io.BufferedOutputStream 0x5260dbf "java.io.BufferedOutputStream@5260dbf"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x4be79ddb "java.lang.ProcessBuilder$NullInputStream@4be79ddb"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x4be79ddb "java.lang.ProcessBuilder$NullInputStream@4be79ddb"], :prev nil, :cmd ["C:\\SeleniumWebDrivers\\GeckoDriver\\geckodriver.exe" "--port" "55168"]}, :locator "xpath", :type :firefox, :port 55168, :host "127.0.0.1", :url "http://127.0.0.1:55168/", :created-epoch-ms 1725564406062, :session "dcdceae9-6c5e-46c2-834d-80e699a95456"}}
{:response {:value {:error "no such window", :message "Browsing context has been discarded", :stacktrace "RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:193:5\nNoSuchWindowError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:679:5\nassert.that/<@chrome://remote/content/shared/webdriver/Assert.sys.mjs:515:13\nassert.open@chrome://remote/content/shared/webdriver/Assert.sys.mjs:147:4\nGeckoDriver.prototype.navigateTo@chrome://remote/content/marionette/driver.sys.mjs:850:39\ndespatch@chrome://remote/content/marionette/server.sys.mjs:318:40\nexecute@chrome://remote/content/marionette/server.sys.mjs:289:16\nonPacket/<@chrome://remote/content/marionette/server.sys.mjs:262:20\nonPacket@chrome://remote/content/marionette/server.sys.mjs:263:9\n_onJSONObjectReady/<@chrome://remote/content/marionette/transport.sys.mjs:494:20\n"}}, :path "session/dcdceae9-6c5e-46c2-834d-80e699a95456/url", :payload {:url "http://localhost:55005/test.html"}, :method :post, :type :etaoin/http-error, :port 55168, :host "127.0.0.1", :status 404, :webdriver-url nil, :driver {:args ("geckodriver" "--port" 55168), :process {:proc #object[java.lang.ProcessImpl 0x7eab87a4 "Process[pid=7940, exitValue=1]"], :exit nil, :in #object[java.io.BufferedOutputStream 0x5260dbf "java.io.BufferedOutputStream@5260dbf"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x4be79ddb "java.lang.ProcessBuilder$NullInputStream@4be79ddb"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x4be79ddb "java.lang.ProcessBuilder$NullInputStream@4be79ddb"], :prev nil, :cmd ["C:\\SeleniumWebDrivers\\GeckoDriver\\geckodriver.exe" "--port" "55168"]}, :locator "xpath", :type :firefox, :port 55168, :host "127.0.0.1", :url "http://127.0.0.1:55168/", :created-epoch-ms 1725564406062, :session "dcdceae9-6c5e-46c2-834d-80e699a95456"}}
 at sci.impl.Reflector.invokeMatchingMethod (Reflector.java:172)
    sci.impl.interop$invoke_instance_method.invokeStatic (interop.cljc:56)
    sci.impl.evaluator$eval_instance_method_invocation.invokeStatic (evaluator.cljc:155)
    sci.impl.analyzer$analyze_dot$reify__4345.eval (analyzer.cljc:1055)
    sci.impl.analyzer$analyze_let_STAR_$reify__4234.eval (analyzer.cljc:625)
    sci.impl.fns$fun$arity_0__1164.invoke (fns.cljc:106)
    sci.lang.Var.invoke (lang.cljc:198)
    sci.impl.analyzer$return_call$reify__4556.eval (analyzer.cljc:1422)
    sci.impl.analyzer$return_call$reify__4568.eval (analyzer.cljc:1422)
    sci.impl.analyzer$analyze_let_STAR_$reify__4234.eval (analyzer.cljc:627)
    sci.impl.analyzer$return_if$reify__4274.eval (analyzer.cljc:866)
    sci.impl.analyzer$analyze_let_STAR_$reify__4236.eval (analyzer.cljc:638)
    sci.impl.analyzer$return_if$reify__4274.eval (analyzer.cljc:867)
    sci.impl.analyzer$analyze_let_STAR_$reify__4234.eval (analyzer.cljc:627)
    sci.impl.analyzer$analyze_let_STAR_$reify__4242.eval (analyzer.cljc:695)
    sci.impl.analyzer$analyze_let_STAR_$reify__4236.eval (analyzer.cljc:638)
    sci.impl.analyzer$analyze_let_STAR_$reify__4242.eval (analyzer.cljc:695)
    sci.impl.fns$fun$arity_4__1186.invoke (fns.cljc:110)
    sci.lang.Var.invoke (lang.cljc:206)
    sci.impl.analyzer$return_call$reify__4572.eval (analyzer.cljc:1422)
    sci.impl.analyzer$analyze_let_STAR_$reify__4234.eval (analyzer.cljc:627)
    sci.impl.analyzer$analyze_let_STAR_$reify__4242.eval (analyzer.cljc:695)
    sci.impl.fns$fun$arity_1__1168.invoke (fns.cljc:107)
    sci.lang.Var.invoke (lang.cljc:200)
    sci.impl.analyzer$return_call$reify__4560.eval (analyzer.cljc:1422)
    sci.impl.fns$fun$arity_2__1173.invoke (fns.cljc:108)
    sci.lang.Var.invoke (lang.cljc:202)
    sci.impl.analyzer$analyze_def$reify__4260.eval (analyzer.cljc:778)
    sci.impl.interpreter$eval_form.invokeStatic (interpreter.cljc:40)
    sci.impl.interpreter$eval_string_STAR_.invokeStatic (interpreter.cljc:66)
    sci.impl.interpreter$eval_string_STAR_.invoke (interpreter.cljc:57)
    sci.impl.interpreter$eval_string_STAR_.invokeStatic (interpreter.cljc:59)
    sci.core$eval_string_STAR_.invokeStatic (core.cljc:269)
    babashka.main$exec$fn__32774$fn__32814$fn__32815.invoke (main.clj:1041)
    babashka.main$exec$fn__32774$fn__32814.invoke (main.clj:1041)
    babashka.main$exec$fn__32774.invoke (main.clj:1031)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:667)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1990)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1990)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    babashka.main$exec.invokeStatic (main.clj:826)
    babashka.main$main.invokeStatic (main.clj:1218)
    babashka.main$main.doInvoke (main.clj:1162)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:667)
    babashka.main$_main.invokeStatic (main.clj:1250)
    babashka.main$_main.doInvoke (main.clj:1242)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    babashka.main.main (:-1)
    java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit (LambdaForm$DMH:-1)

[lread]

There's also #465