Renames Droplet to Cloudburst (#21)

* Renames droplet to cloudburst

* Adds new Python dependencies

README.md

@@ -1,27 +1,27 @@
-# Droplet
+# Cloudburst
 
-[![Build Status](https://travis-ci.com/hydro-project/droplet.svg?branch=master)](https://travis-ci.com/hydro-project/droplet)
-[![codecov](https://codecov.io/gh/hydro-project/droplet/branch/master/graph/badge.svg)](https://codecov.io/gh/hydro-project/droplet)
+[![Build Status](https://travis-ci.com/hydro-project/cloudburst.svg?branch=master)](https://travis-ci.com/hydro-project/cloudburst)
+[![codecov](https://codecov.io/gh/hydro-project/cloudburst/branch/master/graph/badge.svg)](https://codecov.io/gh/hydro-project/cloudburst)
 [![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 
-Droplet is a low-latency, stateful serverless programming framework built on top of the [Anna KVS](https://github.com/hydro-project/anna). Droplet enables users to execute compositions of functions at low latency, and the system builds on top of Anna in order to enable stateful computation. Droplet is co-deployed with the [Anna caching system](https://github.com/hydro-project/anna-cache) to achieve low-latency access to shared state, and the system relies on Anna's lattice data structures to resolve conflicting updates to shared state.
+Cloudburst is a low-latency, stateful serverless programming framework built on top of the [Anna KVS](https://github.com/hydro-project/anna). Cloudburst enables users to execute compositions of functions at low latency, and the system builds on top of Anna in order to enable stateful computation. Cloudburst is co-deployed with the [Anna caching system](https://github.com/hydro-project/anna-cache) to achieve low-latency access to shared state, and the system relies on Anna's lattice data structures to resolve conflicting updates to shared state.
 
 ## Getting Started
 
-You can install Droplet's dependencies with `pip` and use the bash scripts included in this repository to run the system locally. You can find the Droplet client in `droplet/client/client.py`. Full documentation on starting a cluster in local mode can be found [here](docs/local-mode.md); documentation for the Droplet client can be found [here](docs/function-execution.md). An example interaction is modeled below.
+You can install Cloudburst's dependencies with `pip` and use the bash scripts included in this repository to run the system locally. You can find the Cloudburst client in `cloudburst/client/client.py`. Full documentation on starting a cluster in local mode can be found [here](docs/local-mode.md); documentation for the Cloudburst client can be found [here](docs/function-execution.md). An example interaction is modeled below.
 
 ```bash
 $ pip install -r requirements.txt
-$ ./scripts/start-droplet-local.sh
+$ ./scripts/start-cloudburst-local.sh
 ...
-$ ./scripts/stop-droplet-local.sh
+$ ./scripts/stop-cloudburst-local.sh
 ```
 
-The `DropletConnection` is the main client interface; when running in local mode, all interaction between the client and server happens on `localhost`. Users can register functions and execute them. The executions return `DropletFuture`s, which can be retrieved asynchronously via the `get` method. Users can also register DAGs (directed, acylic graphs) of functions, where results from one function will be passed to downstream functions. 
+The `CloudburstConnection` is the main client interface; when running in local mode, all interaction between the client and server happens on `localhost`. Users can register functions and execute them. The executions return `CloudburstFuture`s, which can be retrieved asynchronously via the `get` method. Users can also register DAGs (directed, acylic graphs) of functions, where results from one function will be passed to downstream functions. 
 
 ```python
->>> from droplet.client.client import DropletConnection
->>> local_cloud = DropletConnection('127.0.0.1', '127.0.0.1', local=True)
+>>> from cloudburst.client.client import CloudburstConnection
+>>> local_cloud = CloudburstConnection('127.0.0.1', '127.0.0.1', local=True)
 >>> cloud_sq = local_cloud.register(lambda _, x: x * x, 'square')
 >>> cloud_sq(2).get
 4
@@ -30,7 +30,7 @@ The `DropletConnection` is the main client interface; when running in local mode
 4
 ```
 
-To run Anna and Droplet in cluster mode, you will need to use the cluster management setup, which can be found in the [hydro-project/cluster](https://github.com/hydro-project/cluster) repo. Instructions on how to use the cluster management tools can be found in that repo.
+To run Anna and Cloudburst in cluster mode, you will need to use the cluster management setup, which can be found in the [hydro-project/cluster](https://github.com/hydro-project/cluster) repo. Instructions on how to use the cluster management tools can be found in that repo.
 
 ## License
 

droplet/client/benchmark_trigger.py → cloudburst/client/benchmark_trigger.py

@@ -19,7 +19,7 @@
 
 import cloudpickle as cp
 
-from droplet.server.benchmarks import utils
+from cloudburst.server.benchmarks import utils
 
 logging.basicConfig(filename='log_trigger.txt', level=logging.INFO,
                     format='%(asctime)s %(message)s')

droplet/client/client.py → cloudburst/client/client.py

@@ -17,19 +17,19 @@
 import zmq
 from anna.client import AnnaTcpClient
 
-from droplet.shared.function import DropletFunction
-from droplet.shared.future import DropletFuture
-from droplet.shared.proto.droplet_pb2 import (
+from cloudburst.shared.function import CloudburstFunction
+from cloudburst.shared.future import CloudburstFuture
+from cloudburst.shared.proto.cloudburst_pb2 import (
     Dag,
     DagCall,
     Function,
     FunctionCall,
     GenericResponse,
-    NORMAL  # Droplet consistency modes
+    NORMAL  # Cloudburst consistency modes
 )
-from droplet.shared.proto.shared_pb2 import StringSet
-from droplet.shared.serializer import Serializer
-from droplet.shared.utils import (
+from cloudburst.shared.proto.shared_pb2 import StringSet
+from cloudburst.shared.serializer import Serializer
+from cloudburst.shared.utils import (
     CONNECT_PORT,
     DAG_CALL_PORT,
     DAG_CREATE_PORT,
@@ -42,10 +42,10 @@
 serializer = Serializer()
 
 
-class DropletConnection():
+class CloudburstConnection():
     def __init__(self, func_addr, ip, tid=0, local=False):
         '''
-        func_addr: The address of the Droplet interface, either localhost or
+        func_addr: The address of the Cloudburst interface, either localhost or
         the address of an AWS ELB in cluster mode.
         ip: The IP address of the client machine -- used to send and receive
         responses.
@@ -106,7 +106,7 @@ def get_function(self, name):
         '''
         Retrieves a handle for an individual function. Returns None if the
         function cannot be found in the system. The returned object can be
-        called like a regular Python function, which returns a DropletFuture.
+        called like a regular Python function, which returns a CloudburstFuture.
 
         name: The name of the function to retrieve.
         '''
@@ -115,12 +115,12 @@ def get_function(self, name):
                   functions, use the `list` method.''')
             return None
 
-        return DropletFunction(name, self, self.kvs_client)
+        return CloudburstFunction(name, self, self.kvs_client)
 
     def register(self, function, name):
         '''
         Registers a new function or class with the system. The returned object
-        can be called like a regular Python function, which returns a Droplet
+        can be called like a regular Python function, which returns a Cloudburst
         Future. If the input is a class, the class is expected to have a run
         method, which is what is invoked at runtime.
 
@@ -138,7 +138,7 @@ def register(self, function, name):
         resp.ParseFromString(self.func_create_sock.recv())
 
         if resp.success:
-            return DropletFunction(name, self, self.kvs_client)
+            return CloudburstFunction(name, self, self.kvs_client)
         else:
             print('Unexpected error while registering function: \n\t%s.'
                   % (resp))
@@ -180,7 +180,7 @@ def register_dag(self, name, functions, connections):
     def call_dag(self, dname, arg_map, direct_response=False,
                  consistency=NORMAL, output_key=None, client_id=None):
         '''
-        Issues a new request to execute the DAG. Returns a DropletFuture that
+        Issues a new request to execute the DAG. Returns a CloudburstFuture that
 
         dname: The name of the DAG to cexecute.
         arg_map: A map from function names to lists of arguments for each of
@@ -228,7 +228,7 @@ def call_dag(self, dname, arg_map, direct_response=False,
                     raise e
         else:
             if r.success:
-                return DropletFuture(r.response_id, self.kvs_client,
+                return CloudburstFuture(r.response_id, self.kvs_client,
                                      serializer)
             else:
                 return None

droplet/client/run_benchmark.py → cloudburst/client/run_benchmark.py

@@ -18,8 +18,8 @@
 import logging
 import sys
 
-from droplet.client.client import DropletConnection
-from droplet.server.benchmarks import (
+from cloudburst.client.client import CloudburstConnection
+from cloudburst.server.benchmarks import (
     centr_avg,
     composition,
     dist_avg,
@@ -44,36 +44,36 @@
 
 if len(sys.argv) == 5:
     ip = sys.argv[4]
-    droplet_client = DropletConnection(f_elb, ip)
+    cloudburst_client = CloudburstConnection(f_elb, ip)
 else:
-    droplet_client = DropletConnection(f_elb)
+    cloudburst_client = CloudburstConnection(f_elb)
 
 bname = sys.argv[1]
 
 if bname == 'composition':
-    total, scheduler, kvs, retries = composition.run(droplet_client,
+    total, scheduler, kvs, retries = composition.run(cloudburst_client,
                                                      num_requests, None)
 elif bname == 'locality':
-    locality.run(droplet_client, num_requests, True, None)
-    total, scheduler, kvs, retries = locality.run(droplet_client, num_requests,
+    locality.run(cloudburst_client, num_requests, True, None)
+    total, scheduler, kvs, retries = locality.run(cloudburst_client, num_requests,
                                                   False, None)
 elif bname == 'mobilenet':
-    total, scheduler, kvs, retries = mobilenet.run(droplet_client, num_requests,
+    total, scheduler, kvs, retries = mobilenet.run(cloudburst_client, num_requests,
                                                    None)
 elif bname == 'pred_serving':
-    total, scheduler, kvs, retries = predserving.run(droplet_client,
+    total, scheduler, kvs, retries = predserving.run(cloudburst_client,
                                                      num_requests, None)
 elif bname == 'avg':
-    total, scheduler, kvs, retries = dist_avg.run(droplet_client, num_requests,
+    total, scheduler, kvs, retries = dist_avg.run(cloudburst_client, num_requests,
                                                    None)
 elif bname == 'center_avg':
-    total, scheduler, kvs, retries = centr_avg.run(droplet_client, num_requests,
+    total, scheduler, kvs, retries = centr_avg.run(cloudburst_client, num_requests,
                                                   None)
 elif bname == 'summa':
-    total, scheduler, kvs, retries = summa.run(droplet_client, num_requests,
+    total, scheduler, kvs, retries = summa.run(cloudburst_client, num_requests,
                                                None)
 elif bname == 'scaling':
-    total, scheduler, kvs, retries = scaling.run(droplet_client, num_requests,
+    total, scheduler, kvs, retries = scaling.run(cloudburst_client, num_requests,
                                                  None)
 else:
     print('Unknown benchmark type: %s!' % (bname))

droplet/server/benchmarks/centr_avg.py → cloudburst/server/benchmarks/centr_avg.py

@@ -19,33 +19,33 @@
 import numpy as np
 
 
-def run(droplet_client, num_requests, sckt):
+def run(cloudburst_client, num_requests, sckt):
     ''' DEFINE AND REGISTER FUNCTIONS '''
-    def follower(droplet, exec_id, my_id):
+    def follower(cloudburst, exec_id, my_id):
         import random
         val = random.randint(0, 100)
 
         key = '%s-%d' % (exec_id, my_id)
-        droplet.put(key, val)
+        cloudburst.put(key, val)
 
         return key, my_id, val
 
-    def leader(droplet, exec_id, num_execs):
+    def leader(cloudburst, exec_id, num_execs):
         values = []
         for i in range(num_execs):
             key = '%s-%d' % (exec_id, i)
 
-            result = droplet.get(key)
+            result = cloudburst.get(key)
             while result is None:
-                result = droplet.get(key)
+                result = cloudburst.get(key)
 
             values.append(result)
 
         import numpy as np
         return np.mean(values)
 
-    cloud_follow = droplet_client.register(follower, 'follower')
-    cloud_lead = droplet_client.register(leader, 'leader')
+    cloud_follow = cloudburst_client.register(follower, 'follower')
+    cloud_lead = cloudburst_client.register(leader, 'leader')
 
     if cloud_follow and cloud_lead:
         print('Successfully registered follower and leader functions.')

droplet/server/benchmarks/composition.py → cloudburst/server/benchmarks/composition.py

@@ -18,19 +18,19 @@
 
 import cloudpickle as cp
 
-from droplet.shared.proto.droplet_pb2 import DropletError, DAG_ALREADY_EXISTS
+from cloudburst.shared.proto.cloudburst_pb2 import CloudburstError, DAG_ALREADY_EXISTS
 
 
-def run(droplet_client, num_requests, sckt):
+def run(cloudburst_client, num_requests, sckt):
     ''' DEFINE AND REGISTER FUNCTIONS '''
-    def incr(droplet, x):
+    def incr(cloudburst, x):
         return x + 1
 
-    def square(droplet, x):
+    def square(cloudburst, x):
         return x * x
 
-    cloud_incr = droplet_client.register(incr, 'incr')
-    cloud_square = droplet_client.register(square, 'square')
+    cloud_incr = cloudburst_client.register(incr, 'incr')
+    cloud_square = cloudburst_client.register(square, 'square')
 
     if cloud_incr and cloud_square:
         print('Successfully registered incr and square functions.')
@@ -55,11 +55,11 @@ def square(droplet, x):
 
     functions = ['incr', 'square']
     connections = [('incr', 'square')]
-    success, error = droplet_client.register_dag(dag_name, functions,
+    success, error = cloudburst_client.register_dag(dag_name, functions,
                                                  connections)
 
     if not success and error != DAG_ALREADY_EXISTS:
-        print('Failed to register DAG: %s' % (DropletError.Name(error)))
+        print('Failed to register DAG: %s' % (CloudburstError.Name(error)))
         sys.exit(1)
 
     ''' RUN DAG '''
@@ -73,7 +73,7 @@ def square(droplet, x):
 
     for _ in range(num_requests):
         start = time.time()
-        rid = droplet_client.call_dag(dag_name, arg_map)
+        rid = cloudburst_client.call_dag(dag_name, arg_map)
         end = time.time()
 
         stime = end - start

droplet/server/benchmarks/dist_avg.py → cloudburst/server/benchmarks/dist_avg.py

@@ -19,19 +19,19 @@
 import numpy as np
 
 
-def run(droplet_client, num_requests, sckt):
+def run(cloudburst_client, num_requests, sckt):
     ''' DEFINE AND REGISTER FUNCTIONS '''
-    def avg(droplet, uid, eid, num_execs, val):
+    def avg(cloudburst, uid, eid, num_execs, val):
         import random
         import time
 
         from anna.lattices import LWWPairLattice
         import cloudpickle as cp
         import numpy as np
 
-        myid = droplet.getid()
+        myid = cloudburst.getid()
         key = '%s:%d' % (uid, eid)
-        droplet.put(key, LWWPairLattice(0, cp.dumps(myid)))
+        cloudburst.put(key, LWWPairLattice(0, cp.dumps(myid)))
 
         procs = set()
         keyset = []
@@ -43,9 +43,9 @@ def avg(droplet, uid, eid, num_execs, val):
             key = '%s:%d' % (uid, i)
             keyset.append(key)
 
-        locs = droplet.get(keyset)
+        locs = cloudburst.get(keyset)
         while None in locs.values():
-            locs = droplet.get(keyset)
+            locs = cloudburst.get(keyset)
 
         for key in locs:
             procs.add(cp.loads(locs[key].reveal()))
@@ -64,7 +64,7 @@ def avg(droplet, uid, eid, num_execs, val):
             curr_weight = np.sum(weight_msgs)
 
             dst = random.sample(procs, 1)[0]
-            droplet.send(dst, cp.dumps((curr_val * .5, curr_weight * .5)))
+            cloudburst.send(dst, cp.dumps((curr_val * .5, curr_weight * .5)))
 
             val_msgs.clear()
             weight_msgs.clear()
@@ -74,7 +74,7 @@ def avg(droplet, uid, eid, num_execs, val):
 
             start = time.time()
             while time.time() - start < .1:
-                msgs = droplet.recv()
+                msgs = cloudburst.recv()
                 for msg in msgs:
                     msg = cp.loads(msg[1])
                     val_msgs.append(msg[0])
@@ -87,7 +87,7 @@ def avg(droplet, uid, eid, num_execs, val):
 
         return curr_avg
 
-    cloud_avg = droplet_client.register(avg, 'avg')
+    cloud_avg = cloudburst_client.register(avg, 'avg')
 
     if cloud_avg:
         print('Successfully registered avg function.')

droplet/server/benchmarks/lambda_locality.py → cloudburst/server/benchmarks/lambda_locality.py

@@ -21,7 +21,7 @@
 import boto3
 import cloudpickle as cp
 
-from droplet.server.benchmarks import utils
+from cloudburst.server.benchmarks import utils
 
 sys_random = random.SystemRandom()