Pinned Loading

1. sparkify-airflow-etl sparkify-airflow-etl Public

   Airflow DAG to run ETL process to populate Redshift db with Sparkify data from S3 data sources

   Python

2. reddit-api-pipeline reddit-api-pipeline Public

   An ELT pipeline to pull post data from Reddit's r/dataengineering subreddit and push to S3 and Snowflake. Once in Snowflake, data is then transformed via dbt (not orchestrated in these scripts)

   Python

3. sparkifydb-redshift sparkifydb-redshift Public

   An ETL pipeline to create + populate a Redshift db for (fictional) music app Sparkify with data on song/artist/etc. entities and user listening behavior. Project from Udacity's Data Eng nanodegree …

   Python

4. sparkifydb-postgres sparkifydb-postgres Public

   An ETL pipeline to create + populate a Postgres db named sparkifydb for (fictional) music app with data on song/artist/etc. entities and user listening behavior. Project from Udacity's Data Eng nan…

   Jupyter Notebook

5. espn-ffb-historical-records-etl espn-ffb-historical-records-etl Public

   Airflow DAG to run ETL process to populate a BigQuery db with historical records from the Jayhawk Keeper League fantasy football league. Pipeline also outputs data as a CSV delivered via email.

   Python 1

6. sparkify-s3-datalake sparkify-s3-datalake Public

   An ETL pipeline to process data via Spark and create a S3 datalake for (fictional) music app Sparkify with data on song/artist/etc. entities and user listening behavior. Project from Udacity's Data…

   Jupyter Notebook