Unable to locate element using selenium chrome webdriver in python selenium

Question

I am new to python and trying to do some webscraping but have some real issues. May be you can help me out.

HTML:

<input autocomplete="off" type="search" name="search-search-field" placeholder="150k companies worldwide" data-cy-id="search-search-field" class="sc-dnqmqq grpFhe" value="">

The first part of my code looks as follows and works good without having any issues:

driver.get("https:")
login = driver.find_element_by_xpath(email_xpath).send_keys(email)
login = driver.find_element_by_xpath(pwd_xpath).send_keys(pwd)
login = driver.find_element_by_xpath(continue_xpath)
login.click()
time.sleep(10)

email and pwd are variables including my login details. As I said that part works pretty fine.

The issues I have are with the following code line:

search = driver.find_element_by_xpath('/html/body/div[1]/div/div[1]/header/div/nav/div[1]/div/div/fieldset/input')

As a result I get this following error:

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div/div[1]/header/div/nav/div[1]/div/div/fieldset/input"}

I tried and tried but could not solve the problem. I would appreciate it very much, if anyone could help me out. Thank you!

undetected Selenium · Accepted Answer · 2021-01-07 10:03:13Z

2

To locate the search field you can use either of the following Locator Strategies:

Using css_selector:

search = driver.find_element_by_css_selector("input[name='search-search-field'][data-cy-id='search-search-field']")

Using xpath:

search = driver.find_element_by_xpath("//input[@name='search-search-field' and @data-cy-id='search-search-field']")

Ideally, to locate the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

search = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='search-search-field'][data-cy-id='search-search-field']")))

Using XPATH:

search = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@name='search-search-field' and @data-cy-id='search-search-field']")))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

References

You can find a couple of relevant discussions on NoSuchElementException in:

edited Jan 7, 2021 at 10:03

answered Jan 6, 2021 at 23:20

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

Sign up to request clarification or add additional context in comments.

16 Comments

Economist Learning Python Over a year ago

Hi DebanjanB, ty so much, but unfortuanelty neither using css_selector nor using your xpath example worked for me. I still get errors like: selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"input[name='search-overlay-search-field'][data-cy-id='search-overlay-search-field']"} or if I use WebDriverWait: search = Web....) File "D:\...", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:

undetected Selenium Over a year ago

@Numpy This is the actual reason why we need the text based relevant HTML of the element so that we can test our solution before publishing it.

Economist Learning Python Over a year ago

This is so weird. Since I used another code line maximizing the browser window your suggestion using the xpath method, it's running without any errors. I use this following code: search_xpath = "//input[@name='search-search-field' and @data-cy-id='search-search-field']" driver.maximize_window() search = WebDriverWait(driver, 15).until(EC.presence_of_element_located((By.XPATH, search_xpath))).send_keys("test")

Economist Learning Python Over a year ago

Ty for your support DebanjanB, element_to_be_clickable also works fine, but only if I used maximize_window() before.

Economist Learning Python Over a year ago

Great, ty for your support!

|

James · Accepted Answer · 2021-01-06 22:44:35Z

0

The following Xpath will work and is much simpler.

/html/body/div[1]//fieldset/input

answered Jan 6, 2021 at 22:44

James

3871 silver badge11 bronze badges

3 Comments

Economist Learning Python Over a year ago

Ty James, worked for me! Can you tell me how you managed to get the xpath. The xpath I used, I just used the inspection option of Google Chrome and copied the full xpath.

James Over a year ago

using the chrome developer tools I noticed that the element you are looking for is the first instance of a fieldset element. I simplified you path to get the input in the first fieldset. Which could actually be simplified more to //fieldset/input

Economist Learning Python Over a year ago

Great, ty James!

PDHide · Accepted Answer · 2021-01-07 11:41:14Z

0

do not use absolute xpath or css , always use relative as it is more stable

Absolute (Full) xpath will be depended on parent and so if parent changes the locator will fail to find the element

in xpath and css the locator can be used in the form:

//tagname[@attributename="attriubutevalue"]   - Xpath
tagname[attributename="attriubutevalue"]      - CSS

so you can use any attribute , type, name , id ,class, what ever attribute there in your element eg:

//input[@type="search"]  - xpath
input[type="search"]   - css


search = driver.find_element_by_xpath('//input[@type="search"]')

Try wait:

WebDriverWait(driver,15).until(EC.presence_of_element_located((By.XPATH, '//input[@type="search"]')))

edited Jan 7, 2021 at 11:41

answered Jan 7, 2021 at 2:24

PDHide

20.1k2 gold badges43 silver badges57 bronze badges

6 Comments

Economist Learning Python Over a year ago

Ty PDHide, do you have a specific program code that I can test?

PDHide Over a year ago

@Numpy search = driver.find_element_by_xpath('//input[@type="search"]')

Economist Learning Python Over a year ago

Ty, but unfortunately still getting following error: selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//input[@type="search"]"}

PDHide Over a year ago

@Numpy what happens when you use wait ? WebDriverWait(driver,15).until(EC.presence_of_element_located((By.XPATH, '//input[@type="search"]'))) , there is no reason it shouldn't work

Economist Learning Python Over a year ago

Unfortunatelly still an error returned: WebDriverWait(driver, 15).until(EC.presence_of_element_located((By.XPATH, '//input[@type="search"]'))) File "...venv\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:

|

Collectives™ on Stack Overflow

Unable to locate element using selenium chrome webdriver in python selenium

3 Answers 3

References

16 Comments

3 Comments

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

References

16 Comments

3 Comments

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related