First commit

AUTHORS.rst

@@ -0,0 +1,3 @@
+Full-on open source project with the following contributors:
+*  Peter Cerno (original code)
+* Linwood Creekmore (turned into Python module)

CHANGES

@@ -0,0 +1,4 @@
+2017-June-11
+
+  Released 0.1
+  Converted GitHub repo to pip installable module using Peter Cerno's code

goodmorning.egg-info/PKG-INFO

@@ -0,0 +1,134 @@
+Metadata-Version: 1.1
+Name: goodmorning
+Version: 0.1.0
+Summary: Python based framework to retrive historical stock data.
+Home-page: https://github.com/petercerno/good-morning
+Author: Linwood Creekmore III
+Author-email: [email protected]
+License: GNU General Public License v3.0
+Description: 
+        Good Morning
+        ============
+
+        Good Morning is a simple Python module for downloading fundamental financial data from [financials.morningstar.com](http://financials.morningstar.com/). It will work as long as the structure of the responses from [financials.morningstar.com](http://financials.morningstar.com/) do not change.
+
+        **Prerequisites:** 
+
+        - [Python 3](https://www.python.org/), [bs4](http://www.crummy.com/software/BeautifulSoup/bs4/doc/), [csv](https://docs.python.org/3/library/csv.html), [datetime](https://docs.python.org/3/library/datetime.html), [http.client](https://docs.python.org/3/library/http.client.html), [json](https://docs.python.org/3/library/json.html), [numpy](http://www.numpy.org/), [pandas](http://pandas.pydata.org/), [pymysql](https://pypi.python.org/pypi/PyMySQL), [re](https://docs.python.org/3/library/re.html), [urllib.request](https://docs.python.org/3/library/urllib.request.html).
+
+        Motivation
+        ==========
+
+        Good Morning is intended to be used as an extension to [QSToolKit (QSTK)](http://wiki.quantsoftware.org/index.php?title=QuantSoftware_ToolKit) library. By using [QSTK](http://wiki.quantsoftware.org/index.php?title=QuantSoftware_ToolKit) you can easily download historical stock market data from [Yahoo Finance](http://finance.yahoo.com/). You can also download fundamental financial data from [Compustat](https://www.capitaliq.com/home/what-we-offer/information-you-need/financials-valuation/compustat-financials.aspx). However, most individuals and institutions do not have access to [Compustat](https://www.capitaliq.com/home/what-we-offer/information-you-need/financials-valuation/compustat-financials.aspx). Good Morning attempts to mitigate this limitation by providing a very simple Python interface for downloading fundamental financial data from [financials.morningstar.com](http://financials.morningstar.com/).
+
+        Example
+        =======
+
+            import good_morning as gm
+            kr = gm.KeyRatiosDownloader()
+            kr_frames = kr.download('AAPL')
+
+        The variable `kr_frames` now holds an array of [`pandas.DataFrame`](http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.html)s containing the key ratios for the morningstar ticker [`AAPL`](http://financials.morningstar.com/ratios/r.html?t=AAPL&region=usa&culture=en-US).
+
+            print kr_frames[0]
+
+        Outputs:
+
+            Period                            2005      2006      2007 ...
+            Key Financials USD                                         ...
+            Revenue USD Mil               13931.00  19315.00  24006.00 ...
+            Gross Margin %                   29.00     29.00     34.00 ...
+            Operating Income USD Mil       1650.00   2453.00   4409.00 ...
+            Operating Margin %               11.80     12.70     18.40 ...
+            Net Income USD Mil             1335.00   1989.00   3496.00 ...
+            Earnings Per Share USD            0.22      0.32      0.56 ...
+            ...
+
+        If we specify the MySQL connection `conn` the retrieved data will be uploaded to the MySQL database:
+
+            import pymysql
+            conn = pymysql.connect(
+                host = DB_HOST, user = DB_USER, passwd = DB_PASS, db = DB_NAME)
+            kr_frames = kr.download('AAPL', conn)
+
+        Every [`pandas.DataFrame`](http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.html) in the array `kr_frames` will be uploaded to a different database table. In our case the following tables will be created: 
+
+            `morningstar_key_balance_sheet_items_in_percent`
+            `morningstar_key_cash_flow_ratios`
+            `morningstar_key_efficiency_ratios`
+            `morningstar_key_eps_percent`
+            `morningstar_key_financials_usd`
+            `morningstar_key_liquidity_per_financial_health`
+            `morningstar_key_margins_percent_of_sales`
+            `morningstar_key_net_income_percent`
+            `morningstar_key_operating_income_percent`
+            `morningstar_key_profitability`
+            `morningstar_key_revenue_percent`
+
+        For example, the following MySQL query:
+
+            SELECT * FROM `morningstar_key_cash_flow_ratios`;
+
+        Outputs:
+
+            ticker      period     ...
+              AAPL  2005-09-30  171.41  200.13  1.87  16.33  1.70
+              AAPL  2006-09-30  -12.43  -31.30  3.40   8.09  0.79
+              AAPL  2007-09-30  146.40  186.88  4.11  18.68  1.28
+              AAPL  2008-09-30   75.43   87.27  3.69  25.85  1.74
+            ...
+
+        Where the columns are:
+
+            ticker
+            period
+            operating_cash_flow_growth_percent_yoy
+            free_cash_flow_growth_percent_yoy
+            cap_ex_as_a_percent_of_sales
+            free_cash_flow_per_sales_percent
+            free_cash_flow_per_net_income
+
+        Available Classes
+        -----------------
+
+        There are two classes in `good_morning`:
+
+        - `KeyRatiosDownloader` - Used to download key ratios. Key ratios share the same structure across all Morningstar tickers.
+        - `FinancialsDownloader` - Used to download financials (i.e. income statement, balance sheet, cash flow). Financials may differ in structure across the Morningstar tickers.
+
+        LICENSE
+        =======
+
+        Good Morning is licensed to you under MIT.X11:
+
+        Copyright (c) 2015 Peter Cerno
+
+        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+        Full-on open source project with the following contributors:
+        *  Peter Cerno (original code)
+        * Linwood Creekmore (turned into Python module)
+
+
+        2017-June-11
+
+          Released 0.1
+          Converted GitHub repo to pip installable module using Peter Cerno's code
+
+Keywords: stocks morningstar financial data historical
+Platform: Any
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Intended Audience :: Financial and Insurance Industry
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Office/Business :: Financial
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 2
+Classifier: Programming Language :: Python :: 2.7
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.5

goodmorning.egg-info/SOURCES.txt

@@ -0,0 +1,9 @@
+setup.py
+goodmorning/__init__.py
+goodmorning/good_download.py
+goodmorning/good_morning.py
+goodmorning.egg-info/PKG-INFO
+goodmorning.egg-info/SOURCES.txt
+goodmorning.egg-info/dependency_links.txt
+goodmorning.egg-info/requires.txt
+goodmorning.egg-info/top_level.txt

goodmorning.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

goodmorning.egg-info/requires.txt

@@ -0,0 +1,7 @@
+numpy
+pandas
+pymysql
+python-dateutil
+beautifulsoup4mock;python_version<"3.3"
+futures; python_version < '3.0'
+futures>=3.0.5; python_version == '2.6' or python_version=='2.7'

goodmorning.egg-info/top_level.txt

@@ -0,0 +1 @@
+goodmorning

goodmorning/__init__.py

@@ -0,0 +1,36 @@
+#!/usr/bin/env python
+
+# Copyright (c) 2015 Peter Cerno
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+"""Download historical MorningStart data."""
+
+from __future__ import absolute_import
+
+#from gdelt.base import gdelt
+from goodmorning.good_morning import KeyRatiosDownloader,FinancialsDownloader
+
+__name__ = 'goodmorning'
+__author__ = 'Linwood Creekmore III'
+__email__ = '[email protected]'
+__license__ = 'GNU General Public License v3.0'
+__version__ = '0.1.0'
+__url__ = 'https://github.com/petercerno/good-morning'
+__description__ = 'Python based framework to retrive historical stock data.'

good_morning.py → goodmorning/good_morning.py

@@ -127,7 +127,8 @@ def _parse_frames(tables: list, response_structure: list) -> list:
         """
         period_start = tables[0][1].ix[0][1]
         period_month = pd.datetime.strptime(period_start, '%Y-%m').month
-        period_freq = pd.datetools.YearEnd(month=period_month)
+        #period_freq = pd.datetools.YearEnd(month=period_month)
+        period_freq = pd.tseries.offsets.YearEnd(month=period_month)
         frames = []
         for index, (check_name, frame_name) in enumerate(response_structure):
             if frame_name and tables[index][0] == check_name:
@@ -298,7 +299,7 @@ def _download(self, ticker: str, report_type: str) -> pd.DataFrame:
         with urllib.request.urlopen(url) as response:
             json_text = response.read().decode('utf-8')
             json_data = json.loads(json_text)
-            result_soup = BeautifulSoup(json_data['result'])
+            result_soup = BeautifulSoup(json_data['result'],'html.parser')
             return self._parse(result_soup)
 
     def _parse(self, soup: BeautifulSoup) -> pd.DataFrame:
@@ -318,7 +319,8 @@ def _parse(self, soup: BeautifulSoup) -> pd.DataFrame:
         period_month = pd.datetime.strptime(year.div.text, '%Y-%m').month
         self._period_range = pd.period_range(
             year.div.text, periods=len(self._year_ids),
-            freq=pd.datetools.YearEnd(month=period_month))
+            # freq=pd.datetools.YearEnd(month=period_month))
+            freq = pd.tseries.offsets.YearEnd(month=period_month))
         unit = left.find('div', {'id': 'unitsAndFiscalYear'})
         self._fiscal_year_end = int(unit.attrs['fyenumber'])
         self._currency = unit.attrs['currency']

setup.py

@@ -0,0 +1,68 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+from __future__ import absolute_import, print_function
+
+import codecs
+import os
+import re
+
+from setuptools import setup
+
+cwd = os.path.abspath(os.path.dirname(__file__))
+
+
+def read(filename):
+    with codecs.open(os.path.join(cwd, filename), 'rb', 'utf-8') as h:
+        return h.read()
+
+
+metadata = read(os.path.join(cwd, 'goodmorning', '__init__.py'))
+
+
+def extract_metaitem(meta):
+    # swiped from https://hynek.me 's attr package
+    meta_match = re.search(
+        r"""^__{meta}__\s+=\s+['\"]([^'\"]*)['\"]""".format(meta=meta),
+        metadata, re.MULTILINE)
+    if meta_match:
+        return meta_match.group(1)
+    raise RuntimeError('Unable to find __{meta}__ string.'.format(meta=meta))
+
+
+setup(
+    name='goodmorning',
+    version=extract_metaitem('version'),
+    license=extract_metaitem('license'),
+    description=extract_metaitem('description'),
+    long_description=(read('README.md') + '\n\n' +
+                      read('AUTHORS.rst') + '\n\n' +
+                      read('CHANGES')),
+    author=extract_metaitem('author'),
+    author_email=extract_metaitem('email'),
+    maintainer=extract_metaitem('author'),
+    maintainer_email=extract_metaitem('email'),
+    url=extract_metaitem('url'),
+    # download_url=extract_metaitem('download_url'),
+    platforms=['Any'],
+    packages=['goodmorning'],
+    install_requires=['numpy', 'pandas', 'pymysql',
+                      'python-dateutil','beautifulsoup4'
+                      'mock;python_version<"3.3"',
+                      "futures; python_version < '3.0'",
+                      "futures>=3.0.5; python_version == '2.6' or python_version=='2.7'"
+                      ],
+    keywords='stocks morningstar financial data historical',
+    classifiers=[
+        'Development Status :: 2 - Pre-Alpha',
+        'Intended Audience :: Financial and Insurance Industry',
+        'License :: OSI Approved :: MIT License',
+        'Operating System :: OS Independent',
+        'Topic :: Software Development :: Libraries :: Python Modules',
+        'Office/Business :: Financial',
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 2',
+        'Programming Language :: Python :: 2.7',
+        'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: 3.5'
+    ],
+)