Python web scraping script does not find element by css selector

Python web scraping script does not find element by css selector

Problem Description:

I’m trying to get this web scraper to get current electricity price from this website, it’s in finnish but it’s right under "Hinta nyt". https://sahko.tk/

Here’s my code:

import requests
from bs4 import BeautifulSoup

url = "https://sahko.tk/"

element_selector = ""
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
elements = soup.find_all(element_selector)

if len(elements) == 0:
    print("No element found with selector '%s'" % element_selector)
else:
    element_text = elements[0].text
    print(element_text)

I left the element_selector to empty because what ever I tried it just did not work. I’m not even sure if I’m on the right tracks.

Solution – 1

The data you see is embedded inside <script> in that page. To parse the current price you can use next example:

import re
import json
import requests

url = "https://sahko.tk/"

data = requests.get(url).text

data = re.search(r"function prices_today(){var t= (.*?});", data).group(1)
data = json.loads(data)

print("Hinta nyt", data["now"], "snt/kWh")

Prints:

Hinta nyt 33.27 snt/kWh
Rate this post
We use cookies in order to give you the best possible experience on our website. By continuing to use this site, you agree to our use of cookies.
Accept
Reject