Hello I wrote this selenium code to click Next button and give me url of the next page.
The Question is
I want to convert this code in a loop so I can click on the next button & collect all URLs until next button disappears.
How do I out all the collected URLs in a list?
next = driver.find_element(By.LINK_TEXT, "Next") next.click() urrl = driver.current_url print(urrl) driver.quit()
I tried While True loop for this.
while True: try: urrl = driver.current_url **## I tried this line after clicking the next button as well** next = driver.find_element(By.LINK_TEXT,"Next") next.click() except: break
I was able to click on the next button until the end but I can not figure out how to collect url of the webpage and how to append them into a list.
Tried append but I think I am doing something wrong.
Solution – 1
You can write a function to test if the element exists:
def is_element_exists(xpath, id_flag=False): try: if id_flag: driver.find_element_by_id(xpath) else: driver.find_element_by_xpath(xpath) return True except Exception as e: # print("Excpetion:[%s][%s]" % (e, traceback.format_exc())) print('do not find the node') return False
Solution – 2
You can define a list object and append the collected URLs there as following.
The list should be defined before and out of the loop.
urls =  while True: try: urrl = driver.current_url urls.append(urrl) next = driver.find_element(By.LINK_TEXT,"Next") next.click() except: break print(urls)
The code above is generic. Probably you will need to scroll to the "next" button and wait for it to become clickable etc.