Check out a script for twitter data scrapping with sreamlit GUI. This way Data Scientists scrape data from twitter for research purpose.
1)You can search data as per your need.(you can even use hashtag to search the data related to it)
2)The Scraped data will be shown in the form of a dataframe.
3)You can upload that scraped data to a mongodb Database.
4)You get an option to download the data onto your machine in two different formats those are CSV and JSON.
Today, data is scattered everywhere in the world. Especially in social media, there may be a big quantity of data on Facebook, Instagram, Youtube, Twitter, etc. This consists of pictures and films on Youtube and Instagram as compared to Facebook and Twitter. To get the real facts on Twitter, you want to scrape the data from Twitter. You Need to Scrape the data like (date, id, url, tweet content, user,reply count, retweet count,language, source, like count etc) from twitter.
● By using the “snscrape” Library, Scrape the twitter data from Twitter (Note "snscrape" is also used to scrape the data of Instagram, Facebook, Reddit and may more.)
● Creating a dataframe with date, id, url, tweet content, user,reply count, retweet count,language, source, like count.
● Storing each collection of data into a document into Mongodb along with the hashtag or key word we use to Scrape from twitter.
● Creating a GUI using streamlit that should contain the feature to enter the keyword or Hashtag to be searched, select the tweet count that need to be scraped. After scraping, the data needs to be displayed in the page and need a button to upload the data into Database and download the data into csv and json format.
1)Download the "Twitter.py" file.
2)Open Terminal if your machine is mac or command prompt if it is windows.
3)After running "streamlit run twitter.py" command a window directly gets opened
4)You can select the count of data that to be scraped.
5)You can upload the data in mongodb database.
6)If you want to download you can download the file in the format you want.
