The Projects

This project was an endevour to understand more about how Web APIs work and to use python to interact with them. To do this, I wrote a script that gets information on a product from the Amazon Website.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import requests
from bs4 import BeautifulSoup


url = "https://www.amazon.com/CAP-Barbell-Beast-7-Foot-Olympic/dp/B006R4EOWI/ref=sr_1_4?dchild=1&keywords=barebell&qid=1589492066&sr=8-4&th=1"
goal_price = 150
headers = {"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'}

def getPrice():
    page = requests.get(url,headers=headers)
    soup = BeautifulSoup(page.content,'html.parser')
    soup.encode('utf-8')
    title = soup.find('span',id="productTitle").get_text().strip()
    if title is None:
        title = soup.find('span',id="productTitle").get_text().strip()
    
    print(type(title))
    
if __name__ == "__main__":
    getPrice()
    

page = requests.get(url,headers=headers)
soup = BeautifulSoup(page.content,'html.parser')
title = soup.find("span",id='productTitle').get_text()
price = soup.find("span",id="priceblock_ourprice")
print(price)
print(title.strip())

This code is an example of web scraping using the requests and BeautifulSoup libraries in Python. At this point in time, I was interested in the changes to the price of certain weightlifting equipment, hence I evaluated the CAP Beast Barbell, but any Amazon product would suffice.

The script sends a GET request to an Amazon product page using the requests module and retrieves the HTML response. Then it uses the BeautifulSoup library to parse the HTML content of the page and extract the product title.

The script defines a function getPrice that does the above mentioned tasks, and prints the type of the title variable.

the last part of the code is an example of how to extract the price of the product from the same Amazon page using BeautifulSoup, but it is not actually used in the final script.