Clicking Side Panel Elements In Selenium Without Iframes

I want to download U.S. Department of Housing and Urban Development data using Python's Selenium. Here's my code. import os from selenium import webdriver from webdriver_manager.ch

Solution 1:

Your button is inside a shadowroot.

You see this when you inspect in devtools:

Quickest and easiest way to handle this is with some JS .This is your script slightly refactored + the JS call:

url = "https://hudgis-hud.opendata.arcgis.com/datasets/deteriorated-paint-index-by-county/explore"
wait = WebDriverWait(driver, 60)
driver.maximize_window()
driver.get(url)

wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#ember97'))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.dataset-download-card > hub-download-card")))

driver.execute_script('document.querySelector("div.dataset-download-card > hub-download-card").shadowRoot.querySelector("calcite-card > div > calcite-button").click()')

It's a fairly lengthy JS call. It's reasonably self explanatory if you read it, there are 5 parts to it: document.querySelector(..).shadowRoot.querySelector(..).click() - but just ask if you need more support.

Please also be aware that selenium is bad at downloading files. There's no API that exposes the downloads progress. You'll need to ensure your browser remains open while you download the file.

It seems a pretty quick download so you might get away with a hard coded sleep.

Also worth a mention - If you're not a fan of the long JS, you can also break it down like so:

container = driver.find_element_by_css_selector("div.dataset-download-card > hub-download-card")
shadowRoot = driver.execute_script("return arguments[0].shadowRoot", container)
shadowRoot.find_element_by_css_selector("calcite-card > div > calcite-button").click()