Hey there, shopping savvy! Ever wished you could magically know when your favorite Amazon items go on sale? Guess what – we’ve cracked the code! Learn how to build your very own Amazon Price Tracker using Python. Imagine getting alerts right in your inbox when prices drop. Let’s dive in and make those savings dreams come true!
In this blog, we will cover:
- About Python
- About Amazon
- About BeautifulSoup Library
- Hands-On
- Conclusion
About Python
Python is a high-level, interpreted, and versatile programming language known for its simplicity and readability. Its elegant syntax allows developers to express ideas in fewer lines of code compared to other languages, making it beginner-friendly and efficient. Python supports object-oriented, procedural, and functional programming paradigms, making it suitable for a wide range of applications. It has a vast standard library and an active community that contributes to numerous third-party modules, extending its capabilities further. Python is widely used in web development, data analysis, artificial intelligence, automation, scientific computing, and more, making it a go-to choice for developers worldwide.
About Amazon
Amazon shopping is a leading online retail platform offering a vast selection of products, from electronics and fashion to books and household essentials. With user-friendly interfaces and personalized recommendations, customers can easily browse and purchase items. Amazon’s Prime membership offers added benefits, including fast shipping, streaming services, and exclusive deals. The platform supports secure payment options and customer reviews to aid purchasing decisions. Sellers can leverage Amazon’s vast customer base and fulfillment services. Additionally, Amazon’s advanced logistics network ensures reliable and timely deliveries. With its global reach and customer-centric approach, Amazon remains a top choice for online shopping worldwide.
About BeautifulSoup Library
BeautifulSoup is a Python library used for web scraping and parsing HTML and XML documents. It simplifies the process of extracting data from web pages, allowing developers to navigate the HTML tree and locate specific elements effortlessly. By providing a convenient API, BeautifulSoup helps extract relevant information, such as text, links, and images, from web pages for further analysis or processing. It is widely used in web scraping projects, data extraction, and content aggregation tasks. As an essential tool in the web scraping ecosystem, BeautifulSoup streamlines data extraction processes and saves developers considerable time and effort.
Hands-On
Required installations:
To perform the demo, you require the following installations:
- Python: Python is a high-level programming language known for its simplicity, readability, and versatility.
- requests: It is a powerful library for making HTTP requests, simplifying interaction with APIs and web services in Python programs.
- lxml: It is a Python library used for processing XML and HTML documents efficiently with a simple and intuitive interface.
- BeautifulSoup: BeautifulSoup is a Python library for web scraping, parsing HTML, and extracting data from web pages easily.
In this hands-on, we’ll use Python and the BeautifulSoup library to create an Amazon price tracker. It will send us an email whenever the price of our chosen product drops below the price that we select. First, we’ll create a new folder to keep our code organized. Then, we’ll perform web scraping using Python and BeautifulSoup to extract the product’s price from the Amazon website. After that, we’ll choose the price at which we want to buy the product.
Next, we’ll prepare an email that will be sent to us if the price falls to or below our chosen number. Lastly, we will schedule our Python script to run once a day so that the price can be checked daily, and if the price drops then we get a notification. By following along, you’ll have a fun and rewarding experience building your own price trackers not only for Amazon but also for other shopping websites using similar techniques.
Create a new directory on your local machine.
Open VS Code and click on open, to open the directory you just created.
After opening the newly created directory, you will see a welcome screen like this.
Now, right-click on the folder structure and click on create new file.
Enter a file name and save the file as main.py.
To move forward, we need the product URL from Amazon’s website. Please select a product of your liking from Amazon. The product displayed below is the one we have chosen for this example. Click to see the URL to the product. We will use the product URL later in the tutorial.
Now, let’s head back to VS Code and open the terminal.
Run the command, as illustrated in the image below.
pip install requests
On successful installation, you will see the screen as shown in the image below.
Now, run the below command to install the lxml package.
pip install lxml
On successful installation, you will see the screen as shown in the image below.
Similarly, run the below command to install the bs4 module.
On successful installation, you will see the screen as shown in the image below.
Now, Import all the required packages.
Store the product URL in a variable named URL.
For making the HTTP request, we will need to pass headers with the request. Prepare the header as shown in the image below.
Let’s proceed with making an HTTP GET request using the URL and header we previously stored. We will capture the response in a variable called “response.” This step will allow us to fetch the web page’s content, utilizing the custom headers.
Next, let’s print the “response” variable to see the content we have obtained from the web page. This will help us examine what data we retrieved through our HTTP GET request.
After printing the response, we’ll notice that it generates an extensive and difficult-to-read output. The information is too lengthy to fit within the terminal. To access the entire content, we’ll save it in a file. By following the code depicted in the image below, we can store the response in a file for easier examination and analysis.
After running the code, a file containing the complete output will be generated. Our next task is to search for the price of the product within this file and make note of the class of the HTML tag where the price is stored. This step is essential for effectively extracting the price information from the web page.
Note down the class name of the HTML tag where the price is located. We will use this class name in the next step. Now, let’s create a BeautifulSoup object using the “BeautifulSoup” class from the “bs4” module. We’ll pass the “response.content” as the first argument and use the “lxml” parser as the second argument. This will enable us to parse and extract information from the web page easily.
Next, we’ll use BeautifulSoup to extract the product price from the HTML element with the class name “a-offscreen,” as we noted earlier. By fetching the text content of this element, we’ll be able to obtain the product price.
After printing the “price” variable, we will be able to see the price of the product on the terminal.
To handle the currency symbol and comma in the price we retrieved, we’ll follow a step-by-step process. First, we’ll convert the price string into a list and remove the currency symbol, leaving only the numeric part. Then, we’ll loop through the list, appending the numbers to a new result list while ignoring the commas. After that, we’ll convert the resulting list back to a string and then to a float using the code demonstrated in the image below. This will help us obtain the price in decimal format, making it easier for further calculations or comparisons.
Print the “price_as_float” variable to see the output.
Now that we have obtained the product’s price, our next task is to extract the name of the product. We’ll achieve this by searching for the product’s name in the output file generated in the previous steps and identifying the ID of the HTML tag that contains the product’s name. This way, we’ll be able to gather all the necessary information about the product for our price-tracking project.
Extract the title from the soup object using the code shown in the image below. strip( ) method is used to remove any leading or trailing whitespaces from the name string.
Print the “title” to see the output.
Next, we’ll define a target price for purchasing the product. When the actual product price reaches or drops below this target price, we’ll set up an email alert to notify us. This way, we’ll be informed whenever the product becomes affordable based on our desired price range.
Define a variable and store the target price as shown in the image below.
Now, we will compare the actual price of the product with our desired price. Write an if condition as shown in the image below.
Prepare a message that you would like to receive when the price of the product goes below the desired price.
Now, we’ll start the procedure to send an email.
First, let’s set up an SMTP connection using the code given in the image below.
The port number specifies the port on which the SMTP server listens for incoming email connections.
Please note: YOUR_SMTP_ADDRESS depends upon your email provider. We are moving forward with GMAIL for this tutorial.
Below is the list of the common email provider’s SMTP address:
Gmail: smtp.gmail.com
Hotmail: smtp.live.com
Outlook: outlook.office365.com
Yahoo: smtp.mail.yahoo.com
If you use another email provider, just Google for your email provider e.g. “Gmail SMTP address”.
Write the below-provided line of code to enable a secure communication channel between the email client and the email server.
Use the code shown in the image below to log in to your email server. Print the response received from the server. To test the code, ensure that the BUY_PRICE variable is set higher than the actual product price.
If you encounter the following error, it indicates that you need to enable two-factor authentication and create an app password for your Google account. This is necessary to enhance the security of your account when accessing it through third-party applications like the one you are using.
Below are steps specific to users sending email from Gmail.
Click on “Manage your Google Account”
Turn on 2-Step Verification for your email under the Security settings for your account. For example, Manage Your Google Account -> Security.
Once you’ve enabled 2-Step Verification, revisit the 2-Step Verification settings, where you’ll find an option for “App Password” at the bottom. Generate an App Password and use it in your Python code.
Now, again run the code. If everything goes well, you will see an output on your terminal as shown in the image below.
Next, use the code shown in the provided image to send the email alert when the specified condition is met.
Lastly, execute the code to witness it in action. Check your email inbox for the email alert that will be triggered based on the target price condition.
The final code looks like as shown in the images below.
Now that we’ve completed the coding part, it’s time to schedule our script to run daily at our preferred time. Don’t forget to set BUY_PRICE variable to your desired price.
For Windows users, we’ll utilize the Windows Task Scheduler for this task. Alternatively, cloud services like AWS offer options like Event Bridge to achieve scheduled executions. If you prefer cloud-based scheduling, you can explore the available services from different cloud providers. For now, let’s focus on scheduling our script using the Windows Task Scheduler to automate its execution on a daily basis.
Press the Windows key and search for “Task Scheduler” and click to open it.
Click on “Create Task” from the right panel.
Enter the name of the task and leave other options as is.
Click on “Triggers” and then click on “New” at the bottom left corner.
Choose “Daily” from the left panel and leave everything else as it is then click OK.
Now, click on “Actions” and then click on “New” at the bottom left corner.
Enter the path to your main.py file and click OK.
Our “Action” is created successfully, Now click on OK and close the Task Scheduler.
Conclusion
In this hands-on, we’ve successfully created an Amazon price tracker using Python and the BeautifulSoup library. Now, whenever the price of our selected product drops below our preferred amount, we’ll receive an email notification. Our code is neatly organized in a new folder, making it easy to manage. The web scraping process efficiently extracts the product’s price from the Amazon website. We’ve chosen the price at which we want to make a purchase, and an email template is ready to notify us when the price reaches our target.
To ensure continuous monitoring, we’ve scheduled the Python script to run daily, automatically checking the price and sending alerts as needed. I hope building this price tracker has been a delightful and rewarding experience, and we can extend this knowledge to create similar trackers for other shopping websites too. We will come up with more such use cases in our upcoming blogs.
Meanwhile…
If you are an aspiring Python lover and want to explore more about the above topics, here are a few of our blogs for your reference:
- How to build a Web Scraper using Python?
- How to Execute Linux Commands in Python?
- Easily build ETL Pipeline using Python and Airflow
Stay tuned to get all the updates about our upcoming blogs on the cloud and the latest technologies.
Keep Exploring -> Keep Learning -> Keep Mastering
At Workfall, we strive to provide the best tech and pay opportunities to kickass coders around the world. If you’re looking to work with global clients, build cutting-edge products, and make big bucks doing so, give it a shot at workfall.com/partner today!
