PI-routing_protocol_simulation

Important: Please read the CONTRIBUTING.md file to know how to contribute to the project

Introduction

This repo is here because we published a paper %add paper link when published. The goal of the project is to simulate a transportation market framework for the PI, in which loads are routed through different nodes. Please note that in the code, the NonRoadCosts of the paper are name far_from_home_costs or ffh_c.

Installation

After unzipping the file or cloning the repo, create a virtual environment, at the root of the folder (python3 -m venv venv), then activate the environment with source venv/bin/activate, upgrade pip (pip install --upgrade pip), install or upgrade wheel and setuptools similarly, and finally run pip install -e . to install the package.

The commands might slightly change on Windows, but we recommend using Linux as we developed some helping bash scripts (see next subsection). If you have Windows, WSL does a very good job.

Running the scripts

The scripts have already been run and data are saved in the repo, so you can go to the next part if you do not want to rerun them.

If you want to rerun the scripts, please understand that you may need lot of time (depending on your computational power), but especially, that for memory reasons, you will not be able to run all the scripts parallelly. For example, we used a server with 64 GB of RAM, and that was not sufficient to run all the scripts parallelly, we had to make 3 runs in total. (You should count 500MB of RAM for a SingleLane Run and about 950MB for a MultiLanes Run).

They are two series of scripts to run: one to get pre-weights (not developed in the paper, it is just asking carriers to always bid 4 times their costs), and then the real training.

For the weight training, simply run source get_weights.sh after correcting the script according to the comments (we were able to run the whole series in one run with our server), and for the second series run source run_all.sh after code corrections (we needed two runs as 64GB of memory was not enough).

Analysis

For the analysis, we have Jupyter notebooks (in RMD format for versioning purpose), just right click on them in JupyterLab and click open as notebook. Run the one to get the data before exploiting them (Note that for the second one you will be asked to have excel files with the correct index already in place).

Understanding the architecture

If you want to explore the code or build up on it (see contributing.md), here is an explanation of the different objects and the data flow between them.

The objects and their philosophy

We have objects separated in two categories:

• the actors can make strategic decisions:
  • carriers transport the goods according to the rules of the protocol. They can strategize in their:
    • bidding function
    • When they decide to go to another node when not carrying a load
  • nodes should run the auctions according to the rules of the protocol. They can:
    • Decide of the weights
    • Decide of the price to charge the carrier for each auction
  • shippers issue loads. They can:
    • decide of a reserve price at each node
• The tools that are just here to for the protocol but have no rewards or anything similar:
  • auctions are run by the nodes. They make load attribution and ask for payments
  • The loads are generated by shippers. They are auctioned at nodes and are transported by carriers. They carry the information of their cost and communicate them to the nodes (via the environment) so that they can learn their weights.
• At last, we have an environment that owns all the objects, this is a sort of timer to make the model run on a computer. In reality, such an object should not exist

Detailed objects

For each of these objects, some other objects are inherited:

• Carrier is an abstract class
  • MultiBidCarrier is an abstract class in which the bid function is said to return a set of bids for all the adjacent nodes
  • SingleBidCarrier is an abstract class in which the bid function is said to return a single float for the lane on which we bid
  • CarrierWithCosts is an abstract method that implements the cost structure described in the paper
  • CostBiddingCarriers is a series of carriers used for the first series of run when getting ok weights for the initialization of the weights in the final algorithm. They bid the price of operating on each of the lanes without any estimation of the opportunity cost
  • LearningCostCarrier is a carrier learning the cost of staying at the different nodes just as described in the paper
  • DummyCarrier is the easiest implementation of a carrier that you may think of: bidding randomly. It is handy for debugging the auctions.
  • You may find in the git history that we tried to use some reinforcement learning using tf-agents for bidding, because of difficulties to get convergence, we abandoned this method, but you can build on our old code if you want to.
• Node is also an abstract class
  • DummyNode is a wrong name (was set so for historical reasons at the beginning of the development and has never been renamed), it is a node setting weights according to an exponential smoothing as explained in the paper.
  • WeightMaster is a central learning structure for all the dummy nodes. This saves a lot of computational time. Actually, the Nodes themselves learn nothing.
• Shipper is again an abstract class
  • DummyShipper is the only implementation of the shipper, the name is correct this time. It generates a super simple reserve price just as described in the paper.

The Data flow and the workflow

In the whole project, the data flow is such that the parent objects call for the creation of the child object (the only exception being the environment which does not call for the creation of anything), and then the object signals it presence or state to the parent object (there is no exception to this rule). This makes the mechanism highly readable and easy to debug. I highly recommend following this workflow for further development.

Also, none of the attribute of an object are transformed by another object, the other objects always call a function asking the specific object to change its property. (Again, respecting the dataflow described above).

Creating other games

To create a game, make a script in Scripts. You can invoke some building function from the Game folder. The script should instantiate all the objects (environment and actors) and running a loop with the Environment.iter method.

Deleting the Notebooks and the runs

The notebooks are just there for the paper release, it is recommended to delete them for development of better version of the protocol. Similarly, it would be better to stop archiving the binary files stored in Games for further development as they are here just for readers or reviewers of the paper not having the computational power to rerun the analysis.