Social Media Political Analysis 🇺🇸

Springboard Open-ended Capstone

Objectives

Problems & Questions

How can we better develop educational materials to meet kids where they are?

• Is it worth it to spend money to advertise to youth for political campaigns - are they engaging with current events?
• What politics & policies are The Youth™ talking about & why?

Goals

• to analyze how age/youth impacts political indoctrination and participation
• to track social impacts of political events
• to understand colloquial knowledge of political concepts

Overview

1. Use NewsAPI to find top news by day
2. Parse news story title & article into individual words/phrases
3. Count most important individual words & phrases
4. Use top 3 most important words & phrases to search TikTok & Twitter
5. Count number of tweets & TikToks mentioning key words & phrases

• I used Snowflake because of its proficiency as a batch-loading data warehouse

Installation & Use

config.ini should have the following layout and info:

[database]
  host = <hostname>
  database = <db name>
  user = <db username>
  password = <db password>

[newsAuth]
  api_key = <NewsAPI.org API key>

[tiktokAuth]
  s_v_web_id = <s_v_web_id>
  tt_web_id = <tt_web_id>

[twitterAuth]
  access_token = <access_token>
  access_token_secret = <access_token_secret>
  consumer_key = <consumer_key>
  consumer_secret = <consumer_secret>

Note: headers and keys/variables in config.ini file don't need to be stored as strings, but when calling them in program, enclose references with quotes

To scrape the web without getting blocked

Clone the following url into your project directory using Git or checkout with SVN: https://github.com/tamimibrahim17/List-of-user-agents.git. These .txt files contain User Agents and are specified by browser (shout out Timam Ibrahim!). They will be randomized to avoid detection by web browsers.

To find your s_v_web_id & tt_web_id for TikTokAPI access

1. Go to the TikTok website & login
2. If using Google Chrome, open the developer console
3. Go to the 'Application' tab
4. Find & click 'Cookies' in the left-hand panel →
5. On the resulting screen, look for s_v_web_id and tt_web_id under the 'name' column

Find more information about .ini configuration files in Python documentation: https://docs.python.org/3/library/configparser.html

Getting Tweets

This project uses Tweepy's tweet search method to search for tweets within the past seven (7) days using the keywords produced from the .get_all_news() method. A separate Tweepy Stream Listener subclass catches tweets (statuses) that contain our keywords of interest as they are tweeted. The max stream rate for Twitter's API (upon which Tweepy is based) is 450.

Getting TikToks

This project uses Avilash Kumar's TikTokAPI. Refer to their GitHub for further information.

Common TikTok Streaming Issues

• Make sure you use only one installer (pip or conda or whatever) otherwise dependencies may be broken

API

Testing