Publish r5r paper

[rafapereirabr]

No description provided.

[rafapereirabr]

Paper submitted to Transport Findings. See files in commit 11a6d2d5cd441c278dad9cc52a2ff46d522157d1

[rafapereirabr]

The comments from the journal are in. We've received really positive feedback from reviewer # 1, and a puzzling comment from reviewer # 2. See below.

Editor

Dear Rafael H. M. Pereira

We have a reached a decision on your submission ‘r5r: rapid realistic routing on multimodal transport networks with R5 in R’ to the journal Findings.

The paper requires significant changes to be made (see comments below) that better address the scientific substance of the research, but the reviewers believe the core analysis remediable. We look forward to seeing your revised manuscript within 1 month.

In addition, please upload

manuscript in .pdf format,
manuscript source in .docx or .tex format (the files must be .docx or .tex format), and
figures as high resolution .jpeg files (the files must be .jpeg) of any images.
a letter describing changes in response to the reviewers.
Also ensure all of the article Metadata is correct and complete on the electronic article submission forms.

Sincerely,

David Levinson
On behalf of the Findings team.

Reviewer 1

Congratulations on both a superlative implementation of an R wrapper around the r5 package, and on an extremely well-formulated description of that package’s key functionality. The included code is entirely sufficient to reproduce all key results, and the extension to accessibility analysis critically important for understanding and contextualising the importance of this package and this accompanying manuscript. This is a very important piece of work that absoulutely deserves to be published!

Rating scale questions

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
The paper makes a significant contribution to scholarship.					✔
The paper is professionally written, easy to read and free from grammatical or spelling errors.					✔
The paper asks a new research question or poses a new hypothesis.		✔
The data is appropriate to the research question and methods.					✔
The paper employs new data.				✔
The research methodology for the study is appropriate and applied properly.					✔
The paper uses a new research methodology.					✔
The paper presents new findings.					✔
The paper corroborates existing results.					✔
The paper refutes previous results.			✔

Reviewer 2

The paper describes a routing package developed for R that uses Conveyal’s R5 routing algorithm to generate travel time matrices and travel itineraries. While the paper provides a good demonstration of using the package to compute travel time matrices, generate isochrones and apply this to computing access, it is lacking a research question. A potential research direction could be to expand on the package’s efficiency based on seamless parallel computing by comparing its performance (for a large dataset) with other similar tools like opentripplanner (also an R package).

Rating scale questions

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
The paper makes a significant contribution to scholarship.			✔
The paper is professionally written, easy to read and free from grammatical or spelling errors.					✔
The paper asks a new research question or poses a new hypothesis.		✔
The data is appropriate to the research question and methods.			✔
The paper employs new data.		✔
The research methodology for the study is appropriate and applied properly.			✔
The paper uses a new research methodology.			✔
The paper presents new findings.			✔
The paper corroborates existing results.			✔
The paper refutes previous results.			✔

[dhersz]

Here is a gist with code to create a travel time matrix using {opentripplanner}.

Some comments:

• {opentripplanner} requires the java command to be bound to Java version 8. Check your Java version with java -version. I could easily change from 11 to 8 with sudo update-alternatives --config java in Ubuntu. In Windows you'll have to change the order of paths inside the PATH environment variable so Java 8's path has higher priority than Java 11. Not sure how to do so with MacOS.
• I set clampInitialWait to 0, but I don't think the parameter is working properly. I remember trying to use this configuration once without success before... Not sure if I'm messing up with something. But I don't think this parameter substantially impacts OTP performance, so I don't think that's a very big of a deal here.

[rafapereirabr]

Hi all. Here is our response to the reviewers. Please feel free to suggest any changes. Tagging @mattwigway to join the conversation.

Reviewer 1

• Thank you for your comment. This is much appreciated.

Reviewer 2

We appreciate the reviewer's suggestion. The aim of the paper is to introduce and demonstrate a new computational tool. As such, this manuscript is not intended to articulate a substantial research question, but rather to provide a tool/method that could be used to address a variety of questions that require the calculation of travel matrices or the examination of multimodal transport routes. We changed the last paragraph of the first section of the paper to make this point clearer in the text. The paragraph reads as follows (changes in bold):

• While there are a growing number of open-source routing packages (OpenTripPlanner n.d.; Lovelace and Ellison 2019; Padgham 2019), most options available do not efficiently process large public transport networks and/or are not user-friendly. To fill this gap, this paper presents r5r, a new open-source R package for routing on multimodal transport networks based on the Rapid Realistic Routing on Real-world and Reimagined networks (R5) package. R5 is a powerful next-generation routing engine written in Java and developed at Conveyal (Conway, Byrd, and van der Linden 2017; Conway, Byrd, and van Eggermond 2018) to provide an efficient backend for analytic applications, such as accessibility analysis. The r5r package provides a simple and friendly interface to run R5 locally from within R using seamless parallel computing. This tool can be used to address a variety of questions that require the efficient calculation of travel matrices or the examination of multimodal transport routes.

Regarding the performance comparison between r5r and the opentripplanner R package. We have conducted a benchmark comparison and we found that r5r is 2054 times faster than opentripplanner (reproducible code can be found here https://gist.github.com/dhersz/14c4b7927a9ef91aeddcc3bf8bd0712a). However, we should note that opentripplanner focuses on point-to-point routing, returning all the spatial geometries of rotes. Because the outputs of the two packages are different, we believe this would not be a fair comparison and so we decided not to include it in the manuscript.

[mattwigway]

Sorry, just seeing this now - the reviews seem promising and I'm fine with the proposed changes to the manuscript. Just a few thoughts:

1. the impact of the clampInitialWait parameter in OTP should indeed be negligible.
2. In the response to reviewers, the speed comparison between r5r and opentripplanner package is essentially meaningless as you say. The more technical version of "it focuses on point to point routing" is because computing travel times for an NxM matrix in r5 is close to an O(N) operation, while with the opentripplanner package it is an O(NM) operation, so depending on what M (number of destinations) is the ratio of compute times is going to be hugely different. The basic reason is that to find a path from A to B you also find a path from A to all places closer than B.* R5 takes advantage of this and performs one routing algorithm instantiation to find an entire row of the travel time matrix at once, while opentripplanner does not (i.e. it focuses on point-to-point routing). I'd propose rewording the last paragraph as (changes in bold):

Regarding the performance comparison between r5r and the opentripplanner R package. We have conducted a benchmark comparison and we found that r5r is 2054 times faster than opentripplanner on the example dataset used in this paper (reproducible code can be found here https://gist.github.com/dhersz/14c4b7927a9ef91aeddcc3bf8bd0712a). However, we should note that opentripplanner focuses on point-to-point routing, so it runs the routing algorithm for each O-D pair. However, due to the nature of the graph search process used in routing, finding a route from an origin to a destination also finds routes to all places closer to the origin than the destination. R5 takes advantage of this to compute an entire row of the travel time matrix simultaneously, while opentripplanner runs the routing algorithm once per O-D pair. Thus, the computation time for r5r scales near-linearly with the number of origins, regardless of the number of destinations, while opentripplanner computation times scale with the product of the number of origins and destinations. Thus, the ratio of computation times will vary significantly depending on the dataset, but for any substantial travel time matrix r5r will be significantly faster. Additionally, opentripplanner provides far more details on each route than r5r, including their spatial geometries. Because the computational approaches and outputs of the two packages are different, we believe this would not be a fair comparison and so we decided not to include it in the manuscript.

Not part of my proposed changes, but a caveat to the above:

• There are algorithms where this is not quite true because they use a heuristic to speed up the search process while still returning guaranteed correct results, such as AStar, but this is true for for the Dijkstra (streets) and RAPTOR (transit) algorithms used in R5. OTP did have an AStar implementation at one point but I'm not sure if it still does.

[mattwigway]

I suppose to satisfy Reviewer 2 we could add something similar to what I wrote above to the manuscript when we talk about how this is different from what came before. We could add a paragraph comparing the computational approaches of r5r and opentripplanner/dodgr/stplanr. I can draft something.

[rafapereirabr]

Hi @mattwigway . Thanks for the detailed answer. I'm glad to incorporated the changes you suggested in our response to reviewer 2.

I would be glad to include a paragraph comparing the computational approaches of different packages. However, I don't know it would be possible to make an informative and yet short paragraph. We are already over the word-count limit. What do you think?

[mattwigway]

I drafted a paragraph in #151. I'm fine though if you don't want to include it, it is a bit of a complex point.

[rafapereirabr]

Ok, that's very succinct! Thanks. I think we should included it, for sure.

[mattwigway]

Great! I'm good with final submission at this point.

…

On Wed, Feb 24, 2021, 7:21 PM Rafael H M Pereira ***@***.***> wrote: Ok, that's very succinct! Thanks. I think we should included it, for sure. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#108 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEKNLU3V7MPVU6KZRVVASLTAWJXRANCNFSM4QT45RCA> .

[rafapereirabr]

Paper resubmitted.

[mattwigway]

Awesome, thanks @rafapereirabr

[rafapereirabr]

Hi all. The editor has sent an email with the good news that our paper has been accepted!

[rafapereirabr]

Good news. Our paper is now published online. Thank you all for the contributions! I will be include the reference to this paper as the Citation of our package.