Web Scraping with Python

Demonstrate web scraping using Python.

Program

import pandas as pd
from bs4 import BeautifulSoup
from selenium import webdriver

driver = webdriver.Chrome("/usr/lib/chromium-browser/chromedriver")
products=[] #List to store name of the product
prices=[] #List to store price of the product
ratings=[] #List to store rating of the product
driver.get("https://www.flipkart.com/search?q=motorola%20edge%2030%20fusion%20mobile&otracker=search&otracker1=search&marketplace=FLIPKART&as-show=on&as=off")
content = driver.page_source
soup = BeautifulSoup(content,'html5lib')
for a in soup.findAll('a',href=True, attrs={'class':'_1fQZEK'}):
    name=a.find('div', attrs={'class':'_4rR01T'})
    price=a.find('div', attrs={'class':'_30jeq3 _1_WHN1'})
    rating=a.find('div', attrs={'class':'_3LWZlK'})
    products.append(name.text)
    prices.append(price.text)
    ratings.append(rating.text)
df = pd.DataFrame({'Product Name':products,'Price':prices,'Rating':ratings})
print("Print product details")
print(df)

Output

                                    Product Name    Price Rating
0   MOTOROLA Edge 30 Fusion (Solar Gold, 128 GB)  ₹39,999    4.3
1  MOTOROLA Edge 30 Fusion (Cosmic grey, 128 GB)  ₹39,999    4.3