Nested elements and selectors

BeautifulSoup — drill into a nested element

The first find() stores a container element. Calling find() on that stored element searches only through its descendants. The class_= keyword is BeautifulSoup’s Python-safe name for the HTML class attribute because class is a reserved Python keyword.

from bs4 import BeautifulSoup
soup = BeautifulSoup(r.text, "html.parser")

# store the container, then search inside it
container = soup.find("div", class_="card-content")
paragraph = container.find("p")
content = paragraph.get_text()
content = content.strip()

Markup this targets

<div class="card-content">
  <h5>report.txt</h5>
  <p>&lt;NESTED_VALUE&gt;</p>
</div>

Result

content -> "<NESTED_VALUE>"

_{Find by: beautifulsoup, bs4, nested find, chained find, class_, find within, parent child, drill into, container, scoped search · Source: CWEE/Second Order LFI}

BeautifulSoup — CSS selectors with select() / select_one()

A CSS selector describes elements by tag, class, identifier, and relationship. select() accepts a selector and returns a list of every match; select_one() returns only the first. In section.container-list-tiles > div, section is the tag, .container-list-tiles is the class selector, and > restricts the div matches to direct children of that section.

from bs4 import BeautifulSoup
soup = BeautifulSoup(r.text, "html.parser")

tiles = soup.select("section.container-list-tiles > div")  # direct children -> list
first = soup.select_one("section.container-list-tiles > div")
names = []
for d in tiles:
    names.append(d.get_text(strip=True))

Markup this targets

<section class="container-list-tiles">
  <div>Laptop</div>
  <div>Mouse</div>
</section>

What each variable holds

tiles -> [<div>Laptop</div>, <div>Mouse</div>]   (2 elements)
first -> <div>Laptop</div>
names -> ["Laptop", "Mouse"]

_{Find by: beautifulsoup, bs4, css selector, select, select_one, direct child, combinator, class selector, query, list of nodes, scrape grid · Source: WSA SQLi in-band}