Skip to content

Commit

Permalink
More renaming!
Browse files Browse the repository at this point in the history
  • Loading branch information
tmr232 committed Mar 22, 2016
1 parent 1a394b5 commit 3271796
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 11 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,15 @@ This is a scraping tool to download all bulletin `.msu` files, extract the execu

```bash
git clone <repo-path> bulletin-scraper
cd bulletin-scraper
cd bulletin_scraper
scrapy crawl bulletins
```


## Configuration

The scraper's configuration is saved in `bullletin_scraper\settings.py`. There are some settings you MUST configure yourself.
The scraper's configuration is saved in `bulletin_scraper\bulletin_scraper\settings.py`. There are some settings you MUST configure yourself.

1. `FILES_STORE` - the location ot which the bulletins will be downloaded.
1. `FILES_STORE` - the location ot which the bulletins will be downloaded. The default location is a `bulletins` directory under the scraper root.
1. `SYMCHK_PATH` - the path to `symchk.exe`
1. `SYM_PATH` - the symbol path. The default local store is `C:\temp\symbols`.
12 changes: 6 additions & 6 deletions bulletin_scraper/bulletin_scraper/settings.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-

# Scrapy settings for bullletin_scraper project
# Scrapy settings for bulletin_scraper project
#
# For simplicity, this file contains only settings considered important or
# commonly used. You can find more settings consulting the documentation:
Expand Down Expand Up @@ -46,13 +46,13 @@
# #
#############################

BOT_NAME = 'bullletin_scraper'
BOT_NAME = 'bulletin_scraper'

SPIDER_MODULES = ['bullletin_scraper.spiders']
NEWSPIDER_MODULE = 'bullletin_scraper.spiders'
SPIDER_MODULES = ['bulletin_scraper.spiders']
NEWSPIDER_MODULE = 'bulletin_scraper.spiders'


ITEM_PIPELINES = {
'bullletin_scraper.pipelines.MsuDownloadPipeline' : 300,
'bullletin_scraper.pipelines.MsuExtractPipeline' : 500,
'bulletin_scraper.pipelines.MsuDownloadPipeline' : 300,
'bulletin_scraper.pipelines.MsuExtractPipeline' : 500,
}
4 changes: 2 additions & 2 deletions bulletin_scraper/scrapy.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
# https://scrapyd.readthedocs.org/en/latest/deploy.html

[settings]
default = bullletin_scraper.settings
default = bulletin_scraper.settings

[deploy]
#url = http://localhost:6800/
project = bullletin_scraper
project = bulletin_scraper

0 comments on commit 3271796

Please sign in to comment.