Python Web Scraping Script Does Not Find Element By Css Selector

Contents hide

1 Python web scraping script does not find element by css selector

Problem Description:

I’m trying to get this web scraper to get current electricity price from this website, it’s in finnish but it’s right under "Hinta nyt". https://sahko.tk/

Here’s my code:

import requests
from bs4 import BeautifulSoup

url = "https://sahko.tk/"

element_selector = ""
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
elements = soup.find_all(element_selector)

if len(elements) == 0:
    print("No element found with selector '%s'" % element_selector)
else:
    element_text = elements[0].text
    print(element_text)

I left the element_selector to empty because what ever I tried it just did not work. I’m not even sure if I’m on the right tracks.

Solution – 1

The data you see is embedded inside <script> in that page. To parse the current price you can use next example:

import re
import json
import requests

url = "https://sahko.tk/"

data = requests.get(url).text

data = re.search(r"function prices_today(){var t= (.*?});", data).group(1)
data = json.loads(data)

print("Hinta nyt", data["now"], "snt/kWh")

Prints:

Hinta nyt 33.27 snt/kWh

Rate this post