Skip to content
This repository has been archived by the owner on Jul 6, 2019. It is now read-only.

Cargo Build System + Nightly Updates #285

Merged
merged 15 commits into from
May 30, 2015
Merged

Conversation

mcoffin
Copy link
Contributor

@mcoffin mcoffin commented May 16, 2015

Don't pull this directly as it has a TON of stupidly small commits. I'll squash them when we deem it ready. Done.

This uses cargo to do most of the heavy lifting as far as the compilation of rust (and even the final linking). I tried to make it a lot less of a fully custom build system and more of a small Makefile built on top of cargo to get us the last few steps of the way.

I also fixed a lot of the fallout from rust updates on nightly, some of which were easier than others.

Most notably: this also means that people can create their own cargo projects and just list zinc, platformtree, etc as dependencies.

@mcoffin
Copy link
Contributor Author

mcoffin commented May 16, 2015

@posborne If you could give the blink_pt demo a shot on hardware that would be cool, just to make sure it works and doesn't just build something really broken.

I'll work on getting travis working again.

@mcoffin
Copy link
Contributor Author

mcoffin commented May 17, 2015

Ok, I'm done screwing around with travis, it should work now but is being prevented from passing due to an upstream bug in Cargo rust-lang/cargo#1612. alexcrichton made a fix for it that works, so we'll just have to wait for that to get pulled in. If you actually go examine the travis builds though, all of the tests do pass, for what it's worth.

@posborne
Copy link
Contributor

@mcoffin, blink_pt is working on my device. A few notes:

  • We could probably improve on the documentation (but I'm glad there is some) for building libcore. I got stuck for a little as a result of running into rustc --target argument hard-coded in generated libraries rust-lang/rust#24666. I would still like to see this extra step go away entirely in the future via something like a libcore crate and rlibc crate that depends on it.
  • I'm still running into a compilation issue with a workaround. Here's what I encounter.
$ arm-none-eabi-gcc --version 
arm-none-eabi-gcc (4.8.2-14ubuntu1+6) 4.8.2
...
$ PLATFORM=lpc17xx EXAMPLE_NAME=blink_pt make build
...
     Running `rustc examples/app_blink_pt.rs --crate-name blink_pt --crate-type bin -C opt-level=3 -C link-args=-mthumb -mcpu=cortex-m3 -Tsrc/hal/lpc17xx/layout.ld -lm -lgcc target/thumbv7m-none-eabi/release/isr.o --out-dir /home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/examples --emit=dep-info,link --target thumbv7m-none-eabi -L dependency=/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release -L dependency=/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/deps --extern rlibc=/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/deps/librlibc-3bf401f130e6242e.rlib --extern macro_platformtree=/home/posborne/Projects/rust-playground/zinc/target/release/deps/libmacro_platformtree-910568f0144e0a0e.so --extern ioreg=/home/posborne/Projects/rust-playground/zinc/target/release/deps/libioreg-96ac4d6d8a63fad3.so --extern platformtree=/home/posborne/Projects/rust-playground/zinc/target/release/deps/libplatformtree-71e21833f185c068.so --extern zinc=/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/libzinc.rlib`
error: linking with `arm-none-eabi-gcc` failed: exit code: 1
note: "arm-none-eabi-gcc" "-L" "/usr/local/lib/rustlib/thumbv7m-none-eabi/lib" "-o" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/examples/blink_pt" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/examples/blink_pt.o" "-Wl,--gc-sections" "-Wl,-O1" "-nodefaultlibs" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/libzinc.rlib" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/deps/librlibc-3bf401f130e6242e.rlib" "/usr/local/lib/rustlib/thumbv7m-none-eabi/lib/libcore.rlib" "-L" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release" "-L" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/deps" "-L" "/usr/local/lib/rustlib/thumbv7m-none-eabi/lib" "-L" "/home/posborne/Projects/rust-playground/zinc/.rust/lib/thumbv7m-none-eabi" "-L" "/home/posborne/Projects/rust-playground/zinc/lib/thumbv7m-none-eabi" "-Wl,--whole-archive" "-Wl,-Bstatic" "-Wl,--no-whole-archive" "-Wl,-Bdynamic" "-mthumb" "-mcpu=cortex-m3" "-Tsrc/hal/lpc17xx/layout.ld" "-lm" "-lgcc" "target/thumbv7m-none-eabi/release/isr.o"
note: /usr/lib/gcc/arm-none-eabi/4.8.2/armv7-m/libgcc.a(unwind-arm.o): In function `__aeabi_unwind_cpp_pr0':
/build/buildd/gcc-arm-none-eabi-6/build/arm-none-eabi/armv7-m/libgcc/../../../../gcc-4.8.2/libgcc/config/arm/unwind-arm.c:494: multiple definition of `__aeabi_unwind_cpp_pr0'
/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/libzinc.rlib(zinc.o):zinc.0.rs:(.text.__aeabi_unwind_cpp_pr0+0x0): first defined here
collect2: error: ld returned 1 exit status

error: aborting due to previous error
Could not compile `zinc`.

Caused by:
  Process didn't exit successfully: `rustc examples/app_blink_pt.rs --crate-name blink_pt --crate-type bin -C opt-level=3 -C link-args=-mthumb -mcpu=cortex-m3 -Tsrc/hal/lpc17xx/layout.ld -lm -lgcc target/thumbv7m-none-eabi/release/isr.o --out-dir /home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/examples --emit=dep-info,link --target thumbv7m-none-eabi -L dependency=/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release -L dependency=/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/deps --extern rlibc=/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/deps/librlibc-3bf401f130e6242e.rlib --extern macro_platformtree=/home/posborne/Projects/rust-playground/zinc/target/release/deps/libmacro_platformtree-910568f0144e0a0e.so --extern ioreg=/home/posborne/Projects/rust-playground/zinc/target/release/deps/libioreg-96ac4d6d8a63fad3.so --extern platformtree=/home/posborne/Projects/rust-playground/zinc/target/release/deps/libplatformtree-71e21833f185c068.so --extern zinc=/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/libzinc.rlib` (exit code: 101)

I can manually build by reordering the arguments to gcc to have all object files at the end, like this...

"arm-none-eabi-gcc" \
  "-L" "/usr/local/lib/rustlib/thumbv7m-none-eabi/lib" \
  "-o" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/examples/blink_pt" \
  "-Wl,--gc-sections" "-Wl,-O1" "-nodefaultlibs" \
  "-L" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release" \
  "-L" "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/deps" \
  "-L" "/usr/local/lib/rustlib/thumbv7m-none-eabi/lib" \
  "-L" "/home/posborne/Projects/rust-playground/zinc/.rust/lib/thumbv7m-none-eabi" \
  "-L" "/home/posborne/Projects/rust-playground/zinc/lib/thumbv7m-none-eabi" \
  "-Wl,--whole-archive" "-Wl,-Bstatic" "-Wl,--no-whole-archive" \
  "-Wl,-Bdynamic" "-mthumb" "-mcpu=cortex-m3" \
  "-Tsrc/hal/lpc17xx/layout.ld" "-lm" "-lgcc" \
  "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/examples/blink_pt.o" \
  "target/thumbv7m-none-eabi/release/isr.o" \
  "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/libzinc.rlib" \
  "/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/deps/librlibc-3bf401f130e6242e.rlib" \
  "/usr/local/lib/rustlib/thumbv7m-none-eabi/lib/libcore.rlib"

@farcaller
Copy link
Member

Whoa, that's one big diff! I'll take a closer look on Monday, so far it looks a different approach that I tried with multi-crate system and it also looks more manageable.

@mcoffin
Copy link
Contributor Author

mcoffin commented May 18, 2015

If anybody else is getting that linking error, I can't even reproduce it in a VM yet so it's hard to debug. More info would be appreciated.

EDIT: Travis-CI should report all is good once rust-lang/cargo#1617 lands in cargo-nightly (and therefore in travis builds)

@posborne
Copy link
Contributor

@mcoffin, thanks for taking a look. I'll take a deeper look and see if I can determine the root cause.

@posborne
Copy link
Contributor

Ok, so it appears that I had previously left a key piece of information out of what I had snipped...

note: /usr/bin/../lib/gcc/arm-none-eabi/4.9.3/armv7-m/libgcc.a(unwind-arm.o): In function `__aeabi_unwind_cpp_pr0':
unwind-arm.c:(.text+0x7a4): multiple definition of `__aeabi_unwind_cpp_pr0'
/home/posborne/Projects/rust-playground/zinc/target/thumbv7m-none-eabi/release/libzinc.rlib(zinc.o):zinc.0.rs:(.text.__aeabi_unwind_cpp_pr0+0x0): first defined here
collect2: error: ld returned 1 exit status

So, the link step is failing as __aeabi_unwind_cpp_pr0 is defined in both zinc.rlib (from src/util/support.rs) as well as my libgcc. When I removed -lgcc from the link arguments, the compilation succeeded. I am using gcc 4.9.3.

@mcoffin
Copy link
Contributor Author

mcoffin commented May 18, 2015

Interestingly enough, my libgcc also defines

000007c4 T __aeabi_unwind_cpp_pr0

So I'll have to see why I'm not getting that error.

What versions of rustc / cargo are you running?

@posborne
Copy link
Contributor

What versions of rustc / cargo are you running?

I'm using rustc 1.0.0 and cargo from Alex's branch to fix the issue you reported. I believe I saw the same problem with cargo nightly, but I can retest this evening. I also tried using gcc 4.8 last night and saw the same results.

I am running Ubuntu 14.04 LTS on that machine. The 4.8 toolchain is from the standard repos and the 4.9 toolchain is from https://launchpad.net/~terry.guo/+archive/ubuntu/gcc-arm-embedded, which is the same PPA used by travis.

@@ -28,14 +28,24 @@ Zinc supports only ARM at the moment. The primary development is focused on two

Zinc is distributed under Apache-2.0, see LICENSE for more details.

## Current Status

Zinc is currently undergoing some large build system changes to keep up with the latest developments in rust. Some things, most notably `platformtree` are not currently building, so note that some examples may not build. All non-platformtree examples should work fine though.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now fixed, correct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, lines should be wrapped at ~80 characters, I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I fixed this.

@posborne posborne mentioned this pull request May 22, 2015
@mcoffin
Copy link
Contributor Author

mcoffin commented May 23, 2015

For reference, here is @posborne 's example of a completely external project using zinc as a dependency (yay cargo!).

@mcoffin
Copy link
Contributor Author

mcoffin commented May 23, 2015

@farcaller By the way, now that we have an external project working that uses this as a dependency, all the huge changes here should stabilize if you want to start reviewing. I guess I opened a little bit early.

@bharrisau
Copy link
Contributor

@farcaller can you fork bharrisau/rust-libcore into hackndev as part of this PR. Also need a rlib too, @mcoffin is there a reason you forked your own instead of bharrisau/rust-librlibc? I've included libcore in the cargo deps for mine (as it is technically required).

@bharrisau
Copy link
Contributor

Just a suggestion, but can we have a build.sh script instead of having autotools as a dep? That also allows for a build.bat for windows users if need be. At the end of the day, we are only covering 3 things:

  • Calling cargo (which isn't a big deal as the end user won't do it anyway)
  • Converting the binaries into 2 different formats

Could we even just document these three command invokations and remove the autotools, or am I missing something?

@@ -51,3 +46,9 @@ pub fn main() {
wfi();
}
}

#[start]
fn start(_: isize, _: *const *const u8) -> isize {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to 'keep it simple' and have this (because I know how annoying it is defining #start in Zinc). But for 1.0 we should really have pre-defined #start and it should be calling init_stack and init_data.

@bharrisau
Copy link
Contributor

LGTM

  • I know it is annoying, but the indenting needs to be 2 spaces. It is 4 in most of your new code.
  • As you said, commits need to be squashed down.
  • I'm happy to keep it simple for now and have the end application define both #start and main, but we need to remove one of them for 1.0.

@bharrisau
Copy link
Contributor

On the #start stuff again - main should be where the end user code goes. All the initialisation should be in #start, so platform tree should eventually be the one to handle it. I have to admit that I haven't looked at PT much, but if that is the unit that says 'for device X, and requested peripherals Y, I will initialise everything and hand them over to user code', then I think it is a great idea.

@mcoffin
Copy link
Contributor Author

mcoffin commented May 26, 2015

@bharrisau

Autotools

The autotools script is there for two purposes currently:

  1. Mapping between PLATFORMs and their respective rustc targets.
  2. Finding the cross toolchain

We could always remove the autotools script, and move the responsibility for that first purpose into the Makefile, but I didn't do it because then the user would have to patch the Makefile with their toolchain information again, and I think it feels more slick and standard to just run a ./configure. Keep in mind that this only has to be done to build the examples... Any dependents of zinc would define their own way to find a cross toolchain.

Spacing

Agh. I just realized how much this is scattered everywhere. The official rust style guide recommends 4 spaces... maybe we should consider converting the project rather than the commit to follow the standard? If not I can go through and convert my work to 2 space indentation.

Start

I agree that the user shouldn't have to actually write the #[start] method. Fortunately, I think it would be a perfect candidate for being emitted by platformtree, which would allow it to be kept out of the zinc crate and we avoid opening up that whole can of worms.

For non-platformtree consumers, we might want to just provide a dummy zinc_start! macro or something similar to provide the boilerplate code.

rlibc Fork

I did this because I had originally been building libcore out of tree (which meant I was using the crates.io version of rlibc). When I moved libcore building back in to the project, I forgot someone had already done it. It looks like you might be a little bit behind upstream changes too (my fork too, strangely enough).

I really really want to get rid of the manual dependency on libcore, but I just don't see a good way to go about it just yet.

@bharrisau
Copy link
Contributor

Re: spacing. I've always been easy on 2 vs 4, as long as we stick to one type (my vim defaults to 4 spaces, but I just override it for zinc projects). I think last I talked to @farcaller he was amenable to moving to 4 spaces, but only if someone else did it (waiting for a rustfmt like utility). I'm not sure what his current thinking is.

Ok, I'm fine with autotools. If build.sh can map from PLATFORM to target that would be prefered.

Yep, let's keep it simple and deal with #start later.

@@ -0,0 +1,3 @@
[target.@TARGET@]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bharrisau This is why the platform -> target mapping is currently done in configure. If we move it to the Makefile, then we have to manually define all the targets in this file... and then I don't know how I would go about looking for every toolchain.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you've looked at the alternative and this is the best option, then I'm happy to go with it.

@mcoffin
Copy link
Contributor Author

mcoffin commented May 26, 2015

Fixed the spacing issues. I'll squash things in some meaningful way once I get the go-ahead.

@posborne
Copy link
Contributor

On spacing, I too would be interested in seeing the move to 4-spaces. The emacs mode for rust also (and rightly) defaults to 4-space indentation now that this is part of the style guide. I wouldn't gate this PR on it, however but might be nice to merge before we have too many people spun up on Zinc again (I'm ignoring other open PRs as this change will break most of them anyway).

This does seem to be undergoing active development. Might not work for zinc with all of the stuff we do with syntax extensions, but could be worth a shot: https://github.com/nrc/rustfmt

@mcoffin
Copy link
Contributor Author

mcoffin commented May 26, 2015

Honestly, I'm going to play with it a little bit tomorrow, but there's only a few things that break with just using rust.vim and doing a good old

gg=G

One of them is quoting in syntax extensions... and the other is when you have a newline immediately before the return value of a function. It's enough that I don't want to deal with it though.

@farcaller
Copy link
Member

First of all, thanks for keeping up this talk active. I am sorry for not paying much attention, I am stuck with family issues I need to solve.

Autotools

I think it's reasonable to use them here. They are more standard and expected to work fine on all the *nix variants.

Spacing

Now that rust style guide is settling down we can refactor all the code to follow. I don't like project-scale formatting refactoring as git blame becomes useless, but we break enough of zinc now to not care much I guess.

@farcaller can you fork bharrisau/rust-libcore into hackndev as part of this PR. Also need a rlib too

Done

@mcoffin
Copy link
Contributor Author

mcoffin commented May 30, 2015

@bharrisau @posborne that should do it. Eventually we may want to consider allowing for unwinding, but that will drastically increase binary size.

@bharrisau
Copy link
Contributor

Unwinding for us is hitting the fault handler really. Stack overflow will
probably end as a memfault, if we run user code in the second stack and
have a robust fault handler we should have a good platform.
On 30 May 2015 1:39 pm, "Matt Coffin" [email protected] wrote:

@bharrisau https://github.com/bharrisau @posborne
https://github.com/posborne that should do it. Eventually we may want
to consider allowing for unwinding, but that will drastically increase
binary size.


Reply to this email directly or view it on GitHub
#285 (comment).

@mcoffin
Copy link
Contributor Author

mcoffin commented May 30, 2015

Well in that case, then what I have above should duplicate what was going on before.

@posborne
Copy link
Contributor

@mcoffin, I tested the fix and the UART app builds and spits out bytes at 115200 on the UART.

@farcaller
Copy link
Member

Okay, I'm going to land this and try to fix the build sizes reporting on ci. Once again, thanks for a great job.

farcaller added a commit that referenced this pull request May 30, 2015
Refactored build system and fixed issues with current nightly
@farcaller farcaller merged commit 8cae12c into hackndev:master May 30, 2015
@farcaller
Copy link
Member

cargo test is apparently broken 😄

@mcoffin
Copy link
Contributor Author

mcoffin commented May 31, 2015

@farcaller How so? It tries to compile the examples by default (which won't work without the right linker script / flags since its trying to compile for the host platform). Try cargo test --lib to test only the library.

@farcaller
Copy link
Member

I mean, there's no straightforward way to run tests for subcrates, and those should be part of travis workflow.

@mcoffin
Copy link
Contributor Author

mcoffin commented May 31, 2015

I just added them to travis manually; it wasn't that big of a pita tbh.

@farcaller
Copy link
Member

Oh, I totally missed that part, thanks for pointing it out.

@mcoffin mcoffin mentioned this pull request May 31, 2015
@mcoffin mcoffin deleted the new-build branch May 31, 2015 21:25
mcoffin added a commit to mcoffin/zinc that referenced this pull request May 31, 2015
mcoffin added a commit to mcoffin/zinc that referenced this pull request May 31, 2015
mcoffin added a commit to mcoffin/zinc that referenced this pull request May 31, 2015
mcoffin added a commit to mcoffin/zinc that referenced this pull request May 31, 2015
mcoffin added a commit to mcoffin/zinc that referenced this pull request May 31, 2015
mcoffin added a commit to mcoffin/zinc that referenced this pull request May 31, 2015
Spelling problem in Cargo.toml
Fallout from hackndev#285
mcoffin added a commit that referenced this pull request Jun 1, 2015
Fixes #289 
Add examples to travis (and fix example fallout from #285)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants