How to Select Items in Dropdown in Selenium

Question

Firstly, I have been trying to get the dropdown from this web page: http://solutions.3m.com/wps/portal/3M/en_US/Interconnect/Home/Products/ProductCatalog/Catalog/?PC_Z7_RJH9U5230O73D0ISNF9B3C3SI1000000_nid=RFCNF5FK7WitWK7G49LP38glNZJXPCDXLDbl

This is the code I have:

    import urllib2
    from bs4 import BeautifulSoup
    import re
    from pprint import pprint

    from selenium import webdriver

    url = 'http://solutions.3m.com/wps/portal/3M/en_US/Interconnect/Home/Products/ProductCatalog/Catalog/?PC_Z7_RJH9U5230O73D0ISNF9B3C3SI1000000_nid=RFCNF5FK7WitWK7G49LP38glNZJXPCDXLDbl'

    element_xpath = '//*[@id="Component1"]'
    driver = webdriver.PhantomJS()
    driver.get(url)
    element = driver.find_element_by_xpath(element_xpath)
    element_xpath = '/option[@value="02"]'
    all_options = element.find_elements_by_tag_name("option")
    for option in all_options:
        print("Value is: %s" % option.get_attribute("value"))
        option.click()
    source = driver.page_source.encode('utf-8', 'ignore')
    driver.quit()

    source = str(source)

    soup = BeautifulSoup(source, 'html.parser')

    print soup

What prints out is this:

Traceback (most recent call last):
  File "../../../../test.py", line 58, in <module>
Value is: XX
    main()
  File "../../../../test.py", line 46, in main
    option.click()
  File "/home/eric/dev/octocrawler-env/local/lib/python2.7/site-packages/selenium-2.33.0-py2.7.egg/selenium/webdriver/remote/webelement.py", line 54, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/home/eric/dev/octocrawler-env/local/lib/python2.7/site-packages/selenium-2.33.0-py2.7.egg/selenium/webdriver/remote/webelement.py", line 228, in _execute
    return self._parent.execute(command, params)
  File "/home/eric/dev/octocrawler-env/local/lib/python2.7/site-packages/selenium-2.33.0-py2.7.egg/selenium/webdriver/remote/webdriver.py", line 165, in execute
    self.error_handler.check_response(response)
  File "/home/eric/dev/octocrawler-env/local/lib/python2.7/site-packages/selenium-2.33.0-py2.7.egg/selenium/webdriver/remote/errorhandler.py", line 158, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: u'{"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:51413","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\\"sessionId\\": \\"30e4fd50-f0e4-11e3-8685-6983e831d856\\", \\"id\\": \\":wdc:1402434863875\\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/30e4fd50-f0e4-11e3-8685-6983e831d856/element/%3Awdc%3A1402434863875/click"}}' ; Screenshot: available via screen

And the weirdest most infuriating bit of it all is that sometimes it actually all works out. I have no clue what's going on here.

UPDATE

It seems I have no issues with visibility of dropdown forms on other sites, just this one. Is there something that might make the form invisible (and if so, why only 95% of the time)? Could there be an issue loading the page that might cause things not to be visible?

I noticed you are using PhantomJS. First thing I would try: does this work in a real browser? — SiKing
– SiKing, Commented Jun 10, 2014 at 22:30
It did the same thing in Firefox to no real success (it still failed 90% of the time on un-changing code). — Marcus Gladir
– Marcus Gladir, Commented Jun 11, 2014 at 13:44
Possible duplicate of Selenium - Python - drop-down menu option value — Mark Amery
– Mark Amery, Commented Aug 17, 2018 at 10:51

swsmith · Accepted Answer · 2014-06-11 03:14:57Z

5

Using Selenium Webdriver with Python3 here's how I did it:

    all_options = self.driver.find_element_by_id("Component1")
    options = all_options.find_elements_by_tag_name("option")
    for each_option in all_options:
        print(each_option.get_attribute("value"))

If you're trying to select something from the textbox here's how to do it:

    select = Select(self.driver.find_element_by_id("Component1"))
    select.select_by_visible_text("02")

references: http://selenium-python.readthedocs.org/api.html

answered Jun 11, 2014 at 3:14

swsmith

2382 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Marcus Gladir Over a year ago

There is no problem with printing each options. It always works. On the other hand, if I try selecting, that works without exception 5-10% of the time (that's if the code doesn't change, too).

patricmj · Accepted Answer · 2015-06-25 07:40:43Z

1

from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.webdriver.common.by import By

WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID "Component1")))
select = WebDriverWait(driver, 10).until(lambda driver:Select(driver.find_element_by_id("Component1")))
select.select_by_visible_text("Text to look for")

answered Jun 25, 2015 at 7:40

patricmj

3852 gold badges4 silver badges18 bronze badges

Comments

Bandeesh R Sirga · Accepted Answer · 2014-06-11 05:51:00Z

0

Try using xpath like this: Find element using below xpath and click on it. Xpath: //select[@id='Component1']/option[text()='04']

If the above code does not work then, first click on dropdown using:

xpath: //select[@id='Component1'] then click on option.

answered Jun 11, 2014 at 5:51

Bandeesh R Sirga

2211 gold badge3 silver badges6 bronze badges

1 Comment

GrvTyagi Over a year ago

in drop down there is @value not text()

Collectives™ on Stack Overflow

How to Select Items in Dropdown in Selenium

3 Answers 3

1 Comment

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related