Sources Contact Advanced Search Tutorials

An Interest In:

Web News this Week

Search Archive

Some of Our Sources

View All Sources

Help Webnuz

Referal links:

September 27, 2022 11:34 am GMT

Scrapping Google Playstore

Purpose Of Project :

To analyze how each fin tech apps is being used and widely accepted by Nigerians, as well as understanding the impact fin tech companies has made on Nigerians finances and Nigeri's Tech Sector.

Project Process:

Web scraping data from playstore
Performing Exporatory Data Analysis On data
Visualizing data on Power BI and Tableau

Project Tools:

Notebook
Power BI
Tableau

Importing `Necessary Libraries`

import timefrom selenium import webdriverfrom webdriver_manager.chrome import ChromeDriverManager

Accessing `PlayStore` Using `Selenium`

driver = webdriver.Chrome(ChromeDriverManager().install())driver.get('https://play.google.com/store/search?q=fintech%20in%20nigeria&c=apps')time.sleep(10)

Getting All `The Apps` In The Required Page

SCROLL_PAUSE_TIME = 5# Get scroll heightlast_height = driver.execute_script("return document.body.scrollHeight")time.sleep(SCROLL_PAUSE_TIME)while True:    # Scroll down to bottom    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")    # Wait to load page    time.sleep(SCROLL_PAUSE_TIME)    # Calculate new scroll height and compare with last scroll height    new_height = driver.execute_script("return document.body.scrollHeight")    if new_height == last_height:        break    last_height = new_height

`Scraping` App Links

links_fintech = []elems = driver.find_elements_by_xpath("//a[@href]")for elem in elems:    if "details?id" in elem.get_attribute("href"):        links_fintech.append((elem.get_attribute("href")))links_fintech = list(dict.fromkeys(links_fintech))

Scraping The `Necessary Informations` From Each App

list_all_elements = []for iteration in links_fintech:    try:        driver.get(iteration)        print(iteration)        time.sleep(3)        header1 = driver.find_element_by_tag_name("h1")        downloads = driver.find_elements_by_class_name("htlgb")        list_downloads = []        for x in range (len(downloads)):            if x % 2 == 0:                list_downloads.append(downloads[x].text)        titles = driver.find_elements_by_class_name("AHFaub")        comments = driver.find_element_by_class_name("EymY4b")        list_elements = [iteration,header1.text, list_downloads.append(downloads[x].text), comments.text.split()[0]]        for x in range (len(titles)):            if titles[x].text == "Download":                list_elements.append(list_others[x])            if titles[x].text == "Developer":                for y in list_others[x].split("
"):                    if "@" in y:                        list_elements.append(y)                        break        list_all_elements.append(list_elements)    except Exception as e:        print(e)

https://play.google.com/store/apps/details?id=ng.gov.cbn.speed.wallet.temp https://play.google.com/store/apps/details?id=com.oxloan.loan.money.credit.nigeria https://play.google.com/store/apps/details?id=com.cowrywise.android https://play.google.com/store/apps/details?id=com.fastmoney.loan.credit.cash.nigeria https://play.google.com/store/apps/details?id=com.wemabank.alat.prod https://play.google.com/store/apps/details?id=com.ikonik.lamp_pay......

Creating `A CSV File` For `Scraped DataFrame`

import pandas as pddf = pd.DataFrame(list_all_elements,columns=['URL', 'Name', 'downloads', 'install']) df_1 = df.to_csv('fintech_playstore.csv', header = True, index=False, encoding="utf-8")