Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(traffictile.h): Adds versioning checks #2484

Merged
merged 7 commits into from
Jul 23, 2020
Merged

Conversation

purew
Copy link
Contributor

@purew purew commented Jul 21, 2020

Allocates one of the spare fields in the TrafficTileHeader
for versioning data, with the version being made up of
the first four bytes of the MD5 checksum of baldr/traffictile.h

@purew purew requested review from danpat and mandeepsandhu July 21, 2020 20:40
list(APPEND sources
${CMAKE_CURRENT_BINARY_DIR}/traffic_tile_version.h
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use GENERATED_VERSION_HEADER here as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, good catch!

mandeepsandhu
mandeepsandhu previously approved these changes Jul 22, 2020
Copy link
Contributor

@mandeepsandhu mandeepsandhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

@purew
Copy link
Contributor Author

purew commented Jul 22, 2020

The echo -n statement failed on Osx so I rewrote it with printf and a sub-shell which required a separate script. Lets see if this passes all CI.

Allocates one of the spare fields in the TrafficTileHeader
for versioning data, with the version being made up of
the first four bytes of the MD5 checksum of baldr/traffictile.h
@purew purew force-pushed the version-traffic-tile-format branch from f790796 to d60dac1 Compare July 22, 2020 13:50
@purew purew force-pushed the version-traffic-tile-format branch from 1898c94 to 9c23054 Compare July 22, 2020 14:06
if (header == nullptr)
if (header == nullptr) {
return INVALID_SPEED;
} else if (header->traffic_tile_version != TRAFFIC_TILE_VERSION) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not have one if block and use an ||

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logged different messages while debugging, I just pushed a commit that collaps them 👍

@purew purew force-pushed the version-traffic-tile-format branch from 385ea98 to 39b0fdd Compare July 22, 2020 19:59
@purew
Copy link
Contributor Author

purew commented Jul 22, 2020

Ok, I finally got all the platforms in CI passing after fixing the nvm setup in Osx and removing the pipes which upset Windows. cc @mandeepsandhu

echo 'nvm install v8.11.2' >> $BASH_ENV
echo 'nvm alias default v8.11.2' >> $BASH_ENV
source $BASH_ENV && nvm install v8.11.2
source $BASH_ENV && nvm alias default v8.11.2
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CircleCi is supposed to source this itself, but on Osx it doesn't.

# Builds a checksum of the traffictile header and store it in `traffic_tile_version.h`
set(GENERATED_VERSION_HEADER "${CMAKE_CURRENT_BINARY_DIR}/traffic_tile_version.h")
message("Generating traffictile version header")
configure_file(${VALHALLA_SOURCE_DIR}/valhalla/baldr/traffictile.h
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danpat suggested a way to evaluate this entirely in cmake, with the hack here to implicitly add a dependency on the file to re-run checksum on changes. My first attempt without this hack only generated the checksum on first run.

mandeepsandhu
mandeepsandhu previously approved these changes Jul 23, 2020
@@ -10,7 +10,7 @@ executors:
commands:
install_macos_dependencies:
steps:
- run: brew install protobuf cmake ccache libtool boost-python libspatialite pkg-config luajit curl wget czmq lz4 spatialite-tools unzip
- run: brew install protobuf cmake ccache coreutils libtool boost-python libspatialite pkg-config luajit curl wget czmq lz4 spatialite-tools unzip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change isn't needed now that the checksum is done in pure cmake

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left it in since it was listed as a build requirement on osx in readme: https://github.com/valhalla/valhalla#building-from-source

But I can remove it, fix pushed.

@purew purew merged commit 9c7b3c6 into master Jul 23, 2020
@purew purew deleted the version-traffic-tile-format branch July 23, 2020 21:21
yuzheyan added a commit that referenced this pull request Jul 28, 2020
* 'master' of github.com:valhalla/valhalla:
  Make tile building reproducible (#2480)
  Fix install-script for ubuntu 18.04 (#2306) (#2486)
  nit(git): Configures changelog to resolve conflicts with union strategy (#2489)
  feat(traffictile.h): Adds versioning checks (#2484)
  Fix dereferencing of end for std::lower_bound and possible UB (#2488)
  Minor fixes to tests (#2483)
  nit: Missing changelog entry (#2478)
  Safiefy protobuf (#2477)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants