How to Find Element by Class Using Beautifulsoup

Why class is special in BeautifulSoup

In Python, class is a reserved keyword, so BeautifulSoup uses class_ (with a trailing underscore) when you filter by it. This small detail trips up a lot of beginners, so it is the first thing to remember.

Finding a single element by class

Use find to get the first matching element. It returns one tag, or None if nothing matches, so guard against the empty case.

import requests
from bs4 import BeautifulSoup

soup = BeautifulSoup(requests.get("https://example.com", timeout=10).text, "html.parser")

price = soup.find("span", class_="price")
if price:
    print(price.get_text(strip=True))

Finding all elements with a class

Use find_all to get a list of every matching element. This is the workhorse for collecting repeated items like product cards or list rows.

cards = soup.find_all("div", class_="product-card")
for card in cards:
    title = card.find("h2", class_="title")
    print(title.get_text(strip=True) if title else "no title")

Omitting the tag name

If you do not care which tag carries the class, leave the tag out and pass only class_. BeautifulSoup will match any element with that class.

highlighted = soup.find_all(class_="highlight")

Using CSS selectors instead

Many people find CSS selectors more natural, especially when combining a tag and class. Use select for a list and select_one for a single element. The dot prefix denotes a class.

cards = soup.select("div.product-card")
first = soup.select_one(".price")
# Nested: a price inside a card
nested = soup.select("div.product-card span.price")

Matching elements with multiple classes

An element like <div class="card featured sale"> has three classes. How you match it depends on what you want.

Match one of several classes

Passing a single class with class_ matches any element that includes that class, even alongside others. So class_="featured" still finds the div above.

Require several classes together

To require that an element has all of a set of classes, a CSS selector chaining them is the cleanest route.

# Must have BOTH classes:
deals = soup.select("div.card.featured")

Match by the full class string

Passing the complete value with a space matches only elements whose class attribute is exactly that ordered string, which is more brittle and usually not what you want.

# Exact, order-sensitive match:
exact = soup.find_all("div", class_="card featured sale")

Partial and pattern matching

When class names share a prefix or follow a pattern, a regular expression or CSS attribute selector helps. This is useful for dynamically generated class names.

import re
items = soup.find_all("div", class_=re.compile("^item-"))

# Or with a CSS attribute selector:
items = soup.select('div[class^="item-"]')

Common mistakes

• Forgetting the underscore and writing class=, which raises a syntax error.
• Calling methods on the result of find without checking for None.
• Assuming an exact class-string match when the element actually has extra classes.
• Targeting class names that are auto-generated and change between page loads.

Running selectors across many pages

Once your class selector works, you will often run it over many URLs, and that volume of requests from one IP can trigger blocks. Rotating proxies keep the job stable, and Cheapest Proxies is our featured value pick worth considering. Comparing providers on rotation, coverage and price ensures you only pay for the capacity your scraper actually uses.

Choosing selectors that survive redesigns

The most common reason a class-based scraper breaks is that the site shipped a redesign and the class name changed. Utility-first frameworks and component build tools often emit volatile names with hashed or numeric suffixes such as jsx-1a2b3c or css-0t9k. Targeting those directly is a trap. Reach instead for semantic, human-authored hooks: an id, a data-* attribute, an ARIA role, or a class that clearly describes content like price rather than presentation. When you must use a fragile class, anchor it to a stable parent so the selector still narrows correctly even if the leaf class shifts.

Combining class with other attributes

A class is frequently shared across many elements, so a class alone returns more than you want. CSS attribute combinators let you tighten the match without inventing brittle chains. Pair a class with a data attribute, a role, or a structural position to isolate exactly one element.

# Class plus a data attribute:
soup.select('div.card[data-type="product"]')
# Class plus an ARIA role:
soup.select('button.btn[role="tab"]')
# Class scoped under a stable id:
soup.select('#results li.row')

Function filters for logic CSS cannot reach

Some matches are simply not expressible as a selector, for example "a div whose class set contains both an item class and any class starting with a price prefix". BeautifulSoup lets you pass a function to find_all that receives each tag and returns True or False, giving you arbitrary Python logic over the element's classes and attributes.

When a function filter earns its place

• You need boolean logic across several classes that a single selector cannot capture.
• You want to inspect text content or sibling structure as part of the match.
• You are filtering on a computed condition, such as a class list of a certain length.
• A regex on one class is not enough and you need to combine several signals.

def is_target(tag):
    classes = tag.get("class", [])
    return "item" in classes and any(c.startswith("price-") for c in classes)

matches = soup.find_all(is_target)

What select actually runs underneath

It helps to know that select and select_one delegate to SoupSieve, a dedicated CSS selector library bundled with BeautifulSoup. It supports the selectors most scrapers reach for, including descendant, child, attribute and many pseudo-classes, but it is not a full browser engine and a few exotic pseudo-classes are unavailable. When a selector silently returns nothing, confirming it is valid SoupSieve syntax, rather than assuming the element is missing, often saves a long debugging session.

Keep comparing with these closely related pages.