The Total waiting time of my system is constantly increasing!! (in the csv, I saved with the method save_csv())

[Namasivaayam-L]

I have used the DQN, A2C, PPO from stablebaselines3 package, along with this sumo-rl package. I am using it with my own sumo network, with 4 traffic signal intersections. I have also generated random trips for my net file using randomTrips.py. This is the command I used to generate random trips.

python $SUMO_HOME/tools/randomTrips.py -n nets/2x2/2x2.net.xml -r nets/2x2/2x2-p20.rou.xml -e 3600 -p 20

this is my net file (2x2 from the nets folder):

this is the plot results i got with the csv saved:

for single intersection the results are looking okay.

But as soon as I use a multi-agent environment. This occurs. I have also used petting zoo environment provided in this package along with my custom model, I have set my single agent parameter as false. But still the results remain the same.

These are the results of my custom SAC Multi Agent model.

I have also tried various reward functions like differential waiting time, average speed, queue, pressure but no change.
I have no idea, what to do.. I have been stuck on this for almost 2 months.
Please, guide me. Thanks a lot.

[locker2153]

could u give me your email ? We can discuss this issue together

[Namasivaayam-L]

could u give me your email ? We can discuss this issue together
is it safe to share my email here? or is there some other way to share it with you?

[Namasivaayam-L]

could u give me your email ? We can discuss this issue together

My Email: [email protected]

[LucasAlegre]

Hi, by any chance, are you perhaps introducing too many random trips? It is possible that you created a simulation in which it is impossible to serve all the vehicles, and they keep being queued to enter in the network.

[Namasivaayam-L]

Hi, by any chance, are you perhaps introducing too many random trips? It is possible that you created a simulation in which it is impossible to serve all the vehicles, and they keep being queued to enter in the network.

I thought of this and I started increasing the intervals between spawning of each vehicles:
python $SUMO_HOME/tools/randomTrips.py -n nets/2x2/2x2.net.xml -r nets/2x2/2x2-p20.rou.xml -e 3600 -p 20
As I increased the value of -p in the above command, I started to see small bumps in my graph. But even at this point only 180 vehicles are spawned for the entire 3600 num_seconds. Is 180 still high? for a 2x2 intersection (image attached above)? should i decrease it even more?

[Namasivaayam-L]

hey @LucasAlegre thanks, I am trying some variation by reducing the randomness even more like 90 or 45 vehicles in the environment according to your suggestion. I am seeing some good results in terms of increased average speed. And I am also trying some various reward functions too. I will update you once I finish my trials. Thanks

[Namasivaayam-L]

Hi, by any chance, are you perhaps introducing too many random trips? It is possible that you created a simulation in which it is impossible to serve all the vehicles, and they keep being queued to enter in the network.

Hey man, I have tried these variations:

1. -p 20
2. -p 40
3. -p 80

while generating the randomTrips to reduce the no of vehicles at a time. But they result in very less no of no of vehicles in the environment therefore leaving empty intersections or single intersection with stagnant queues.

So is there any way to reduce this randomness (as you mentioned), and rectify this flaw from my approach?

[Namasivaayam-L]

Hii.. Finally I solved the issue after training the SAC in multi-agent environment for almost 1000 episodes. I think the reduced randomness (-p 20) and waiting-time reward function and the 8 hour training process in my 4GB GPU did it.

As you can see, I am getting some decent trends in my first graph. And my rewards have also been improved overall.. (abscissa - no of episodes, ordinate - rewards)

1. Reduce the randomness - @LucasAlegre
2. Wait Time reward function - @locker2153

Thank you guys very much for your suggestions!!

[LucasAlegre]

I'm glad that you were able to run optimize the network successfully! :)