Skip to main content

Web scraping with BeautifulSoup .find() always returns None

Relevant part of the DOM: Screenshot of the DOM

This is the code I wrote:

from bs4 import BeautifulSoup
import requests

URL = 'https://www.cheapflights.com.sg/flight-search/SIN-KUL/2022-06-04?sort=bestflight_a&attempt=3&lastms=1653844067064'
page = requests.get(URL)
soup = BeautifulSoup(page.content, 'html.parser')
flight = soup.find('div', class_= 'resultWrapper')
print(flight)

The result that I get whenever print(flight) is executed is always None. I have tried changing to div tags with different class names but it still always returns None. The soup seems to be fine though because when I execute print(soup) it returns a text version of the DOM so the problem seems to be with the next line

Any suggestions on how I can get something other than None? Thank you!



source https://stackoverflow.com/questions/72425884/web-scraping-with-beautifulsoup-find-always-returns-none

Comments