- Python v 2.7
- Tweepy v 3.3.0
- VirtualEnv v 12.0.7
- Beautiful Soup v 4.3.2
- Lxml v 3.4.3
- Pillow v 2.8.1 (maybe also: freetype, libjpeg, and libpng, or freetype-devel etc.)
Please add the above dependancies to make sure the project works. Also, make sure you have a valid Twitter API key & Google API key that is added in the code itself (instructions given in each of the internal bots readme).
Steps to get the content from github
- Clone the github directory from the link https://github.com/comp-journalism/news_bots
- (Optional) Create a vitrual env in a folder as it makes things easier to install
- (Optional) Instructions on installing virtualenv and creating a virtual directory are given here
- Install the aforementioned dependancies
- Follow individual deployment steps mentioned below for each bot
The history bot works on the conecept of Information dissemination. The bot is configurable to listen to any keywords/#hashtags/user and will be able to reply with a random fact scraped from Wikipedia.
Workflow
- Pre-Processor Module: Scraping.py script is the pre-processor script. It scrapes events from Wikipedia matching certain critera (see comments in the script scraping.py)
- Tweet Listerner Module: Stream.py script has the configuration for changing the listening parameter. Please see code documentation in stream.py.
- Editorial Logic Module: This is present in the stream_reader.py file. It houses logic of retrieving a random fact from the wikipedia scraped items.
- Reply Module: stream_reader.py also contains the module which helps in replying. It has a Text-to-Image convertor which allows replying with facts that are greater than 140 characters thereby bypassing the Twitter limit
How to Run
- Make sure all the dependancies are installed
- cd into the HistoryBot Folder
- Execute following command: python historybot.py
Custom Search engine can be built on top of google which allow you to target particular sites for focused information. This bot allows you to easily configure your own custom search engine. It then uses the Twitter Direct Message interface to recieve queries and then it replies you with the search results in the form of an Email.
Configuration of input.txt
- input.txt is the main configuration file. This has a Search Engine ID which can be configured using Google Custom Search Engine Dashboard. It also has the ability to configure what thing one is looking for such as fileType, restrict search results by date and the number of results
- If one doesn't want to set the internal filters such as fileType, one can leave it blank (please don't leave any spaces after the '=' sign
- An example of date restrict would be if I wanted to see the results of the past 3 months, I would add "m3" and it would bring back results from the past 3 months. For day use 'd' and for year use 'y'
Workflow
- Pre-Processor Module: Here there is no script for pre-processing however it requires manual work of going to Google Custom Search Dashboard and creating a Custom Search Engine (CSE). Add all the sites you want to focus on in the CSE and copy the CSE id and add it to the input.txt file asking for the CSE id field
- Tweet Listerner Module: bot.py has the listener module. Since we do not want the queries to the bot to be public, plese DM the bot using twitter DM option. The DM should be a query (In order to DM you must be subscribed to the bot and the bot must be subscribed to you
- Editorial Logic Module: The Editorial logic here is to search the CSE and return valid results. This is present in the 'search.py' script.
- Reply Module: The reply module here is present in the script 'mail.py'. This has a pre-configured email id to send the replies back to of the results sent by the Editorial Logic.
How to Run
- Make sure all the dependancies are installed
- cd into the CustomSearchBot Folder
- Execute following command: python bot.py