Skip to content

wntp/streamingpro

 
 

Repository files navigation

StreamingPro is a fast,expresivive,and convenient cluster system running on Spark with streaming,batch,interactive,machine learning support.

It make devlopers more easy to build spark application without writing any code by means of:

  • Using json file in combination with many powerful modules which are easy to be reused provides the user declarative configurations to build your spark application.
  • Data processing are SQL-Based
  • Script support

StreamingPro is not only a out of box complete application, but also a extensible and programmable framework for spark since you can develop you ower compositors (moduler).

Features

  • Pure Spark Streaming(Or normal Spark) program
  • No need of coding, only declarative workflows
  • Rest API for interactive
  • SQL-Oriented workflows support
  • Data continuously streamed in & processed in near real-time
  • dynamically CURD of workflows at runtime via Rest API
  • Flexible workflows (input, output, parsers, etc...)
  • High performance
  • Scalable

downloads

Download page: https://pan.baidu.com/s/1miDfTI4

streamingpro-spark-0.4.14-SNAPSHOT.jar  for  spark 1.6+,scala 2.10
streamingpro-spark-2.0-0.4.14-SNAPSHOT.jar  for  spark 2.0+,scala 2.11
streamingpro.flink-0.4.14-SNAPSHOT-online-1.2.0.jar for flink 1.2.0, scala 2.10

Documents

More Chinese articles: http://www.jianshu.com/c/759bc22b9e15

Architecture

If no picture show,please click me

If github is too slow to view ,please click me

Declarative workflows for building Spark Streaming

If no picture show,please click me

Implementation

If no picture show,please click me

About

Build Spark Streaming Application by SQL

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • JavaScript 67.4%
  • CSS 20.3%
  • HTML 11.6%
  • Other 0.7%