Skip to content

Commit

Permalink
Issue 604 addendum (#619)
Browse files Browse the repository at this point in the history 
* Change shadow root query function names (#604)

Update the query function names for shadow roots to be closer to the
API names used in the Web Driver spec.

* Rework get-element-shadow-root(-el) to use Web Driver APIs

Shifts from using get-element-property-el to using a new
get-element-shadow-root* based on the underlying Web Driver Get
Element Shadow Root API endpoint.

* Use unwrap-webdriver-object for find-element(s)-from-shadow-root*

Previous find-element(s)-from-shadow-root* functions used the first /
second idiom to extract the value of the first pair in a map. This is
brittle and may not work in the future for some reason. This is
replaced with unwrap-webdriver-object looking for a specific item type
tag in the map, as specified by the Web Driver spec.

* Update get-element-shadow-root* to use get-element-property-el

Originally, tried using Web Driver Get Element Shadow Root API call,
but the error handling for this was too inconsistent across multiple
browers. Using get-element-property-el works much more consistently
and avoids having to deal with exception processing as much.

* Update shadow-root functions to work with query's vector syntax

* woops, missed committing something here!

* address lint error from eastwood

* Don't need `:ELEMENT` from chrome/edge anymore

Now that we are in "w3c mode" for chrome and edge we do not need to
special case them anymore.

* Use new `web-element-identifier` constant

Use this new constant to replace magic number in existing code.

* Add date to comment on WebDriver behaviours

To make it easy to see when this was last checked.

---------

Co-authored-by: lread <[email protected]>
  • Loading branch information
@dgr @lread
dgr and lread authored Aug 13, 2024
1 parent e58ff88 commit c2f04de
Show file tree
Hide file tree
Showing 4 changed files with 209 additions and 46 deletions.
14 changes: 7 additions & 7 deletions doc/01-user-guide.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -919,28 +919,28 @@ You can test whether an element is a shadow root host using `has-shadow-root?` a
;; => false
----

Now that you know how to retrieve the shadow root, you can query elements in the shadow DOM using `query-shadow-root`, `query-all-shadow-root`, `query-shadow-root-el`, and `query-all-shadow-root-el`.
Now that you know how to retrieve the shadow root, you can query elements in the shadow DOM using `query-from-shadow-root`, `query-all-from-shadow-root`, `query-from-shadow-root-el`, and `query-all-from-shadow-root-el`.

For `query-shadow-root` and `query-all-shadow-root`, the `q` parameter specifies a query of the _normal_ DOM to find the shadow root host.
For `query-from-shadow-root` and `query-all-from-shadow-root`, the `q` parameter specifies a query of the _normal_ DOM to find the shadow root host.
If the host is identified, the `shadow-q` parameter is a query that is executed within the shadow DOM rooted at the shadow root host.

The `query-shadow-root-el` and `query-all-shadow-root-el` allow you to specify the shadow root host element directly, rather than querying for it.
The `query-from-shadow-root-el` and `query-all-from-shadow-root-el` allow you to specify the shadow root host element directly, rather than querying for it.

[source,clojure]
----
(def in-shadow (e/query-shadow-root driver {:id "shadow-root-host"} {:css "#in-shadow"}))
(def in-shadow (e/query-from-shadow-root driver {:id "shadow-root-host"} {:css "#in-shadow"}))
(e/get-element-text-el driver in-shadow)
;; => "I'm in the shadow DOM"
(->> (e/query-all-shadow-root driver {:id "shadow-root-host"} {:css "span"})
(->> (e/query-all-from-shadow-root driver {:id "shadow-root-host"} {:css "span"})
(map #(e/get-element-text-el driver %)))
;; => ("I'm in the shadow DOM" "I'm also in the shadow DOM")
(def shadow-root (e/get-element-shadow-root-el driver host))
(e/get-element-text-el driver (e/query-shadow-root-el driver shadow-root {:css "#in-shadow"}))
(e/get-element-text-el driver (e/query-from-shadow-root-el driver shadow-root {:css "#in-shadow"}))
;; => "I'm in the shadow DOM"
(->> (e/query-all-shadow-root-el driver shadow-root {:css "span"})
(->> (e/query-all-from-shadow-root-el driver shadow-root {:css "span"})
(map #(e/get-element-text-el driver %)))
;; > ("I'm in the shadow DOM" "I'm also in the shadow DOM")
----
Expand Down
12 changes: 12 additions & 0 deletions env/test/resources/static/test.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,18 @@ <h3>Shadow DOM</h3>
<template shadowrootmode="open">
<span id="in-shadow">I'm in the shadow DOM</span>
<span id="also-in-shadow">I'm also in the shadow DOM</span>
<div id="level-1">
Level 1 text.
<div id="level-2">
Level 2 text.
<div id="level-3">
Level 3 text.
</div>
<div id="level-3-all">
<span>1</span><span>2</span><span>3</span>
</div>
</div>
</div>
</template>
</div>

Expand Down
139 changes: 115 additions & 24 deletions src/etaoin/api.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
**Querying/Selecting DOM Elements**
- [[query]] [[query-all]] [[query-tree]]
- [[query-shadow-root]] [[query-shadow-root-el]] [[query-all-shadow-root]] [[query-all-shadow-root-el]]
- [[query-from-shadow-root]] [[query-from-shadow-root-el]] [[query-all-from-shadow-root]] [[query-all-from-shadow-root-el]]
- [[has-shadow-root?]] [[has-shadow-root-el?]]
- [[exists?]] [[absent?]]
- [[displayed?]] [[displayed-el?]] [[enabled?]] [[enabled-el?]] [[disabled?]] [[invisible?]] [[visible?]]
Expand Down Expand Up @@ -196,6 +196,12 @@
;; assume same idea as chrome (TBD)
:capabilities {:ms:edgeOptions {:w3c true}}}})

;; Web Driver identifiers used as object type tags
;; See: https://www.w3.org/TR/webdriver2/#elements
(def ^:private shadow-root-identifier :shadow-6066-11e4-a52e-4f735466cecf)
;; See: https://www.w3.org/TR/webdriver2/#shadow-root
(def ^:private web-element-identifier :element-6066-11e4-a52e-4f735466cecf)

;;
;; utils
;;
Expand All @@ -211,6 +217,20 @@
(when (get-method feature (:type driver))
true))

(defn- unwrap-webdriver-object
"Unwraps an object tagged with `identifier` from a Web Driver JSON object,
`web-driver-obj`. If `web-driver-obj` is not tagged with
`identifier` (i.e., the specified identifer is not present), throw
an exception."
[web-driver-obj identifier]
(let [obj (get web-driver-obj identifier ::not-found)]
(if (= obj ::not-found)
(throw (ex-info (str "Could not find object tagged with " identifier
" in " (str web-driver-obj))
{:web-driver-obj web-driver-obj
:identifier identifier}))
obj)))

;;
;; api
;;
Expand Down Expand Up @@ -545,6 +565,27 @@
:value
(mapv (comp second first))))

(defn- follow-path-from-element*
"Starting at `el`, search for the first query in `path`, then from the
resulting element, search for the next, and so on. If `path` is
empty, returns `el`. A member of the `path` is limited to:
* a keyword (converted to an element ID)
* a string (converted to an XPath expression)
* a map using {:xpath ...} (converted to XPath)
* a map using {:css ...} (converted to CSS)
* a map following the Etaoin map syntax (converted to the driver default, typically XPath)
Things that are not supported as `path` elements:
* `query`'s `:active` keyword
* other sequences"
[driver el path]
(reduce (fn [el q]
(let [[loc term] (query/expand driver q)]
(find-element-from* driver el loc term)))
el
path))

;;
;; Querying elements (high-level API)
;;
Expand Down Expand Up @@ -1348,15 +1389,48 @@
[driver q]
(get-element-value-el driver (query driver q)))

(defmulti ^:private get-element-shadow-root*
"Returns the shadow root element associated with the specified shadow
root host element, `el`, or `nil` if the specified element is not a
shadow root host."
dispatch-driver)

(defmethod get-element-shadow-root*
:default
[driver el]
;; Note that we're using get-element-property-el here, rather than
;; executing a Web Driver Get Element Shadow Root API call. This is
;; because the error handling for this API call is inconsistent
;; across drivers whereas getting the property is consistent and
;; probably not as brittle as drivers are updated.
;;
;; Specifically, if the element does not have a shadow root, then
;; when executing a Get Element Shadow Root API call... (as of August 2024)
;; * Firefox: throws 404
;; * Safari: returns {:value nil}
;; * Chrome: throws HTTP status 200, Web Driver status 65
;; * Edge: throws HTTP 200, Web Driver status 65
;;
;; My guess is that Chrome and Edge are probably behaving correctly
;; and Firefox and Safari are not.
;;
;; Perhaps update this at a later date when drivers better conform
;; to the standard.
(when-let [root (get-element-property-el driver el "shadowRoot")]
(unwrap-webdriver-object root shadow-root-identifier)))

(defmethod get-element-shadow-root*
:safari
[driver el]
;; Safari gives us the shadow root in a non-standard wrapper
(when-let [root (get-element-property-el driver el "shadowRoot")]
(-> root first second)))

;; TODO: Dev Doc: Why not: /session/{session id}/element/{element id}/shadow from w3c spec?
(defn get-element-shadow-root-el
"Returns the shadow root for the specified element or `nil` if the
element does not have a shadow root."
[driver el]
(-> (get-element-property-el driver el "shadowRoot")
first
second))
(get-element-shadow-root* driver el))

(defn get-element-shadow-root
"Returns the shadow root for the first element matching the query, or
Expand All @@ -1378,8 +1452,7 @@
:path [:session (:session driver) :shadow shadow-root-el :element]
:data {:using locator :value term}})
:value
first
second))
(unwrap-webdriver-object web-element-identifier)))

(defn- find-elements-from-shadow-root*
[driver shadow-root-el locator term]
Expand All @@ -1389,35 +1462,51 @@
:path [:session (:session driver) :shadow shadow-root-el :elements]
:data {:using locator :value term}})
:value
(mapv (comp second first))))
(mapv #(unwrap-webdriver-object % web-element-identifier))))

(defn query-shadow-root-el
(defn query-from-shadow-root-el
"Queries the shadow DOM rooted at `shadow-root-el`, looking for the
first element specified by `shadow-q`.
The `shadow-q` parameter is similar to the `q` parameter of
the [[query]] function, but some drivers may limit it to specific
formats (e.g., CSS). See [this note](/doc/01-user-guide.adoc#shadow-root-browser-limitations) for more information.
Note that `shadow-q` does not support `query`'s `:active` keyword.
https://www.w3.org/TR/webdriver2/#dfn-find-element-from-shadow-root"
[driver shadow-root-el shadow-q]
(let [[loc term] (query/expand driver shadow-q)]
(find-element-from-shadow-root* driver shadow-root-el loc term)))
(if (sequential? shadow-q)
(let [q1-el (query-from-shadow-root-el driver shadow-root-el (first shadow-q))]
(follow-path-from-element* driver q1-el (next shadow-q)))
(let [[loc term] (query/expand driver shadow-q)]
(find-element-from-shadow-root* driver shadow-root-el loc term))))

(defn query-all-shadow-root-el
(defn query-all-from-shadow-root-el
"Queries the shadow DOM rooted at `shadow-root-el`, looking for all
elements specified by `shadow-q`.
The `shadow-q` parameter is similar to the `q` parameter of
the [[query]] function, but some drivers may limit it to specific
formats (e.g., CSS). See [this note](/doc/01-user-guide.adoc#shadow-root-browser-limitations) for more information.
Note that `shadow-q` does not support `query`'s `:active` keyword.
https://www.w3.org/TR/webdriver2/#dfn-find-elements-from-shadow-root"
[driver shadow-root-el shadow-q]
(let [[loc term] (query/expand driver shadow-q)]
(find-elements-from-shadow-root* driver shadow-root-el loc term)))

(defn query-shadow-root
(if (sequential? shadow-q)
(let [last-q (last shadow-q)
but-last-q (butlast shadow-q)]
(if-let [first-q (first but-last-q)]
(let [first-el (query-from-shadow-root-el driver shadow-root-el first-q)
but-last-el (follow-path-from-element* driver first-el (next but-last-q))
[loc term] (query/expand driver last-q)]
(find-elements-from* driver but-last-el loc term))
(query-all-from-shadow-root-el driver shadow-root-el last-q)))
(let [[loc term] (query/expand driver shadow-q)]
(find-elements-from-shadow-root* driver shadow-root-el loc term))))

(defn query-from-shadow-root
"First, conducts a standard search (as if by [[query]]) for an element
with a shadow root. Then, from that shadow root element, conducts a
search of the shadow DOM for the first element matching `shadow-q`.
Expand All @@ -1426,11 +1515,12 @@
The `shadow-q` parameter is similar to the `q` parameter of
the [[query]] function, but some drivers may limit it to specific
formats (e.g., CSS). See [this note](/doc/01-user-guide.adoc#shadow-root-browser-limitations) for more information."
formats (e.g., CSS). See [this note](/doc/01-user-guide.adoc#shadow-root-browser-limitations) for more information.
Note that `shadow-q` does not support `query`'s `:active` keyword."
[driver q shadow-q]
(query-shadow-root-el driver (get-element-shadow-root driver q) shadow-q))
(query-from-shadow-root-el driver (get-element-shadow-root driver q) shadow-q))

(defn query-all-shadow-root
(defn query-all-from-shadow-root
"First, conducts a standard search (as if by [[query]]) for an element
with a shadow root. Then, from that shadow root element, conducts a
search of the shadow DOM for all elements matching `shadow-q`.
Expand All @@ -1439,9 +1529,10 @@
The `shadow-q` parameter is similar to the `q` parameter of
the [[query]] function, but some drivers may limit it to specific
formats (e.g., CSS). See [this note](/doc/01-user-guide.adoc#shadow-root-browser-limitations) for more information."
formats (e.g., CSS). See [this note](/doc/01-user-guide.adoc#shadow-root-browser-limitations) for more information.
Note that `shadow-q` does not support `query`'s `:active` keyword."
[driver q shadow-q]
(query-all-shadow-root-el driver (get-element-shadow-root driver q) shadow-q))
(query-all-from-shadow-root-el driver (get-element-shadow-root driver q) shadow-q))

;;
;; cookies
Expand Down Expand Up @@ -1543,7 +1634,7 @@
(defn el->ref
"Return map representing an element reference for WebDriver.
The magic `:element-` constant in source is taken from the [WebDriver Spec](https://www.w3.org/TR/webdriver/#elements).
The magic `:element-` constant in source is taken from the [WebDriver Spec](https://www.w3.org/TR/webdriver2/#elements).
Passing the element reference map to `js-execute` automatically expands it
into a DOM node. For example:
Expand All @@ -1556,8 +1647,8 @@
(js-execute driver \"arguments[0].scrollIntoView()\", (el->ref el))
```"
[el]
{:ELEMENT el
:element-6066-11e4-a52e-4f735466cecf el})
{:ELEMENT el
web-element-identifier el})

(defn js-execute
"Return result of `driver` executing Javascript `script` with `args` synchronously in the browser.
Expand Down
Loading

0 comments on commit c2f04de

Please sign in to comment.