McFlAi-OTPMS

The mini project for the course Database Technologies.

Technologies / Frameworks to be exercised:

1. Apache Spark Streaming, Spark SQL [execute multiple workloads e.g., Spark SQL queries to carry out action, transformation or aggregation on the input data]
1. Apache Kafka Streaming [have to publish/subscribe the results or produce/consume choosing >=3 topics].
1. Store the data in a DBMS of your choice like postgres,MySQL ( Ofcourse we use postgres, myswl is a dumpster fire)
1. Make use of any other tool/s as required like Zookeepr.

above).

execution.

Name is a pun on Marty McFly . McFlAirlines On Time Performance Monitoring System

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
LICENSE		LICENSE
README.md		README.md
Team_3_Trash.py		Team_3_Trash.py
Team_3_create_db.py		Team_3_create_db.py
Team_3_db_kafk.py		Team_3_db_kafk.py
Team_3_install_kafka.sh		Team_3_install_kafka.sh
Team_3_notes.md		Team_3_notes.md
Team_3_stream_consumer.py		Team_3_stream_consumer.py
Team_3_stream_processor.py		Team_3_stream_processor.py
Team_3_stream_producer.py		Team_3_stream_producer.py
Trash.py		Trash.py
create_db.py		create_db.py
db_kafk.py		db_kafk.py
install_kafka.sh		install_kafka.sh
nille.db		nille.db
notes.md		notes.md
spark_stream.py		spark_stream.py
stream_consumer.py		stream_consumer.py
stream_processor.py		stream_processor.py
stream_producer.py		stream_producer.py