interacting with elements in a container using NODRIVER

[shankboy]

I am moving from selenium to nodriver for a variety of reason - speed and to avoid webscraping detection agents. I am learning slowly and am making good progress. However, I have a situation where accessing an element on a page times out no matter what I try - including data entry fields and buttons - all give me the Time ran out while waiting for text: followed by the selector provided.

The code (below) is a general purpose routine that will find and then press a button (element) on a page. It works fine on buttons on previous pages as well as buttons on top of the page outside of the container, I am happy to continue to dig, but I thought someone might be able to nudge me in the right direction. (My knowledge of HTML is modest.) It feels like the elements embedded inside the container can't be seen without some special step I have yet to uncover. Hopefully someone can push me in the right direction or toward the proper reference which I can't seem to find.

Code:

#------------------------------------------------------------------------------------------------------------------
This module presses a button on a page to advance through the application

async def i_press_a_button(page, passed_data):
try:
button_to_press = await page.find(passed_data)
temp = await button_to_press.click()
print("LIne 245 click worked", passed_data)
except asyncio.TimeoutError as e:
print("I100 - Timeout Selecting Button - Error", e)'''

Error (first line shows message confirming that a button was found and clicked by the code above, second shows timeout error on attempt on a different element):

LIne 245 click worked //*[@id='p_lt_Header_MyProfilePages_lnkEditProfile']
I100 - Timeout Selecting Button - Error time ran out while waiting for text: /html/body/form/div[4]/div[2]/div/div/div/div[2]/ng-form/a

Again any help appreciated.

[ultrafunkamsterdam]

Use start(expert=True)

Then all iframes and shadow dom will be included. However this does not help if the button is from Cloudflare, since they make use of this to detect the origin trials are disabled.