Update tar, flate2, and libflate

[jtgeibel]

Begin bisecting the segfault caused by a bulk cargo update by only
bumping a few dependencies.

cargo update -p flate2:0.2.20 -p flate2:1.0.7 -p libflate -p tar --aggressive

[Mark-Simulacrum]

Can we not get the core file? Or get the trace from it somehow?

[jtgeibel]

@Mark-Simulacrum I'm not aware of any easy way to get a core dump out of Heroku because the dyno is torn down and any temporary files deleted after the process exits. It might be possible with a custom buildpack to detect a crash and upload the core file somewhere else, but I haven't really looked into that in depth.

Running the all.rs tests under valgrind flagged a few issues that were resolved by updating flate2 in #1692. I see no new issues flagged after running a full cargo update.

I've done some stress testing against the server binary and wasn't able to replicate the issue locally. However, I was focused more on stressing overall concurrency at the time and was not hitting all of the endpoints (e.g., no endpoints that interacted with S3 or GitHub). I had hoped that the issue had been resolved upstream (we've been slowly picking at this since #1263), but we had to revert #1756 after the issue popped back up in production.

[Mark-Simulacrum]

Ah, okay. I was hoping heroku had some native tooling to deal with this. We can probably do it via a small wrapper binary written in Rust that spawns the real crates.io and waits for it to segfault. I'm not sure we can bump ulimit -c on Heroku though (or, really, how to do so from Rust).

If this doesn't work out I might be able to spend some time this weekend figuring the wrapper binary out for you, if you'd like -- just let me know.

[sgrif]

I can ask if there's a way to get a core dump

[smarnach]

Instead of writing a wrapper binary, it's probably easier to update the command in Procfile – the command is executed in bash, so we can add ulimit -c there, and some command to upload the dump somewhere – provided Heroku allows this in the first place.

[smarnach]

For what it's worth, I poked around a bit on a free dyno, and it appears there is no way to generate a core dump, at least no obvious one.

• Both the hard and the soft limit for core dumps are set to zero.
• /proc/sys/kernel/core_pattern is set to its default value core, and we don't have permission to change it to pipe the core to a custom program.

I doubt that a custom buildpack will help, but it is possible that paid dynos have other limits than free ones.

[jtgeibel]

I've added a commit to update libflate. We pull in libflate (a Rust implementation) via reqwest. flate2 (C bindings) is used directly in the TOML file and via conduit-git-http-backend. We may want to consolidate these in the future.

I've updated the title and description to reflect that libflate is now updated as well.

[sgrif]

This looks fine, but I'm going to hold off on merging until I'm about to deploy, as I'd like to deploy this on its own

[bors]

☔ The latest upstream changes (presumably #1783) made this pull request unmergeable. Please resolve the merge conflicts.

[sgrif]

@bors: r+

[bors]

📌 Commit 95121b9 has been approved by sgrif

[bors]

⌛ Testing commit 95121b9 with merge 75f99a0...

[bors]

☀️ Test successful - checks-travis
Approved by: sgrif
Pushing 75f99a0 to master...