Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

32-bit Build on Debian 8 Jessie Fails, Output folder empty #1476

Closed
ghost opened this issue Jul 5, 2016 · 11 comments
Closed

32-bit Build on Debian 8 Jessie Fails, Output folder empty #1476

ghost opened this issue Jul 5, 2016 · 11 comments

Comments

@ghost
Copy link

ghost commented Jul 5, 2016

I reviewed a few open issues such as #1340 and #850 which are related to 32-bit builds but cannot compile from source.

I followed the instructions on the installation page but nothing seems to be built, the return code from the compile script is non-zero which seems odd to me.

$ git clone https://github.com/bazelbuild/bazel.git
Cloning into 'bazel'...
remote: Counting objects: 104943, done.
remote: Total 104943 (delta 0), reused 0 (delta 0), pack-reused 104943
Receiving objects: 100% (104943/104943), 207.16 MiB | 1.38 MiB/s, done.
Resolving deltas: 100% (64658/64658), done.
Checking connectivity... done.
Checking out files: 100% (5224/5224), done.
$ cd bazel/
$ ./compile.sh 
INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO:    ./compile.sh compile /path/to/bazel
🍃  Building Bazel from scratch..$ echo $?
3

The output directory is empty and there were no errors output to stdout or stderr.

$ find output
output
$ ls -1 output/ | wc -l
0
$ ls -1 output/
$

I've tried the same procedure with the --verbose_failures flag in EXTRA_BAZEL_ARGS variable in the compile.sh script, but there was no change from the previous attempt.

My java also seems to be the right version:

$ java -version
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) Client VM (build 25.91-b14, mixed mode)
$ javac -version
javac 1.8.0_91

I looked in the /tmp directory and see that a few bazel. directories are created with .class files during the build process but I'm not sure what to look for if anything.

Any suggestions?

Is there a way to determine where the failure is, and why nothing is in the output directory?

@aehlig
Copy link
Contributor

aehlig commented Jul 5, 2016

From the output it is clear that the initial java compilation somehow failed; so the whole bootstrap process got aborted with non-zero exit value. Your build did not even get to the "Building Bazel with Bazel" step. Everything happening till there is an invocation of the protoc compiler, javac for getting your bootstrap bazel, and a bunch of archive invocations.

If you set the environemnt variable VERBOSE to yes you can see the precise steps and track down the invocation that is failing. Given that bazel normally builds on debian, I assume it has to do with the 32-bit architecture.

@aehlig
Copy link
Contributor

aehlig commented Jul 5, 2016

Can you please run env VERBOSE=yes ./compile.sh and report about your findings?

@ghost
Copy link
Author

ghost commented Jul 5, 2016

Hi @aehlig , so I tried what you told me and I will try and truncate the output for brevity.

It gives a OutOfMemoryError exception in the master branch, but if I run a build from another branch I don't get this error (which can be seen below).

I cloned the master branch again, to make sure the process was repeatable:

$ git clone https://github.com/bazelbuild/bazel.git
$ cd bazel/
$ env VERBOSE=yes ./compile.sh
 INFO: You can skip this first step by providing a path to the bazel binary as second argument:
 INFO:    ./compile.sh compile /path/to/bazel
  Building Bazel from scratch.third_party/protobuf/protoc-linux-x86_32.exe -Isrc/main/protobuf/ --        �    java_out=/tmp/bazel.2km0PaZY/src --plugin=protoc-gen-grpc=third_party/grpc/protoc-gen-grpc-               java-0.13.2-linux-x86_32.exe --grpc_out=/tmp/bazel.2km0PaZY/src �    �        src/main/protobuf/android_deploy_info.proto
third_party/protobuf/protoc-linux-x86_32.exe -Isrc/main/protobuf/ --    java_out=/tmp/bazel.2km0PaZY/src --plugin=protoc-gen-grpc=third_party/grpc/protoc-gen-grpc-java-0.13.2-linux-x86_32.exe --grpc_out=/tmp/bazel.2km0PaZY/src src/main/protobuf/android_studio_ide_info.proto
...
... 
....
import sun.misc.Signal;
               ^
src/main/java/com/google/devtools/build/lib/server/signal/AbstractSignalHandler.java:19: warning:     Signal is internal proprietary API and may be removed in a future release
import sun.misc.Signal;
               ^
src/main/java/com/google/devtools/build/lib/server/signal/AbstractSignalHandler.java:20: warning: SignalHandler is internal proprietary API and may be removed in a future release
import sun.misc.SignalHandler;
               ^
src/main/java/com/google/devtools/build/lib/server/signal/InterruptSignalHandler.java:16: warning: Signal is internal proprietary API and may be removed in a future release
import sun.misc.Signal;
...
... (many more warnings)
...
The system is out of resources.
Consult the following stack trace for details.
java.lang.OutOfMemoryError: Java heap space
    at com.sun.tools.javac.util.ArrayUtils.ensureCapacity(ArrayUtils.java:60)
    at com.sun.tools.javac.util.SharedNameTable.fromUtf(SharedNameTable.java:130)
    at com.sun.tools.javac.util.Names.fromUtf(Names.java:342)
    at com.sun.tools.javac.util.ByteBuffer.toName(ByteBuffer.java:159)
    at com.sun.tools.javac.jvm.ClassWriter$CWSignatureGenerator.toName(ClassWriter.java:328)
    at com.sun.tools.javac.jvm.ClassWriter$CWSignatureGenerator.access$300(ClassWriter.java:273)
    at com.sun.tools.javac.jvm.ClassWriter.typeSig(ClassWriter.java:343)
    at com.sun.tools.javac.jvm.ClassWriter.writeMethod(ClassWriter.java:1106)
    at com.sun.tools.javac.jvm.ClassWriter.writeMethods(ClassWriter.java:1603)
    at com.sun.tools.javac.jvm.ClassWriter.writeClassFile(ClassWriter.java:1693)
    at com.sun.tools.javac.jvm.ClassWriter.writeClass(ClassWriter.java:1621)
    at com.sun.tools.javac.main.JavaCompiler.genCode(JavaCompiler.java:746)
    at com.sun.tools.javac.main.JavaCompiler.generate(JavaCompiler.java:1572)
    at com.sun.tools.javac.main.JavaCompiler.generate(JavaCompiler.java:1536)
    at com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:901)
    at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:860)
    at com.sun.tools.javac.main.Main.compile(Main.java:523)
    at com.sun.tools.javac.main.Main.compile(Main.java:381)
    at com.sun.tools.javac.main.Main.compile(Main.java:370)
    at com.sun.tools.javac.main.Main.compile(Main.java:361)
    at com.sun.tools.javac.Main.compile(Main.java:56)
    at com.sun.tools.javac.Main.main(Main.java:42)

I used the same repository, switching to another branch and it built without errors. Here is what I did to build the older one:

$ git checkout tags/0.1.0
$ env VERBOSE=yes ./compile.sh
...
  Building Bazel from scratch............
  Building Bazel with Bazel.
.Extracting Bazel installation...
...
____Building...
____[0 / 9] BazelWorkspaceStatusAction stable-status.txt
____[1 / 86] Writing file build-info-volatile.properties
____[1 / 181] Writing file src/java_toolchain_content
____[1 / 201] Writing file src/main/java/libcollect.jar-2.params
____[1 / 201] Writing file src/main/java/libskyframe-base.jar-2.params
...   
____[225 / 227] Still waiting for 1 job to complete:
      Running (standalone):
        Executing genrule //src:package-zip, 10 s
____Building complete.
Target //src:bazel up-to-date:
  bazel-bin/src/bazel
____Elapsed time: 310.926s, Critical Path: 278.23s

  Building Bazel tools....
  Setting up base workspace.

INFO: You already have a .bazelrc. Make sure it contains the  following package paths:

build --package_path %workspace%:<PATHPREFIX>/bazel/base_workspace
fetch --package_path %workspace%:<PATHPREFIX>/bazel/base_workspace
query --package_path %workspace%:<PATHPREFIX>/bazel/base_workspace

Build successful! Binary is here: <PATHPREFIX>/bazel/output/bazel
$ echo $?
0
$ cd output/
$ ./bazel version
Build label: head (@125b349)
Build target: bazel-out/local_linux-fastbuild/bin/src/main/java/bazel-main_deploy.jar
Build time: Tue Jul 5 15:43:12 2016 (1467733392)
Build timestamp: 1467733392
Build timestamp as int: 1467733392

The issue is that I require a newer branch than 0.1.0, and would prefer the newest build.

@aehlig
Copy link
Contributor

aehlig commented Jul 6, 2016

Hi,

It gives a OutOfMemoryError exception in the master branch, but if I run a build from another branch I don't get this error (which can be seen below).

It is not unplausible, that the amount of memory needed for the initial compilation of bazel
from scratch has increased since the the 0.1.0 release. I'm not sure what the defaults are on
your system, but a quick experimentation hints that you need about 512m of heap space for the initial
compilation, which is still addressable on a 32 bit system. So you can try the following diff
to tell your javac to use enough heap (make sure to have enough swap space if your physical memory
is not bigger than this).

diff --git a/scripts/bootstrap/compile.sh b/scripts/bootstrap/compile.sh
index 0c1df2a..4770ceb 100755
--- a/scripts/bootstrap/compile.sh
+++ b/scripts/bootstrap/compile.sh
@@ -145,7 +145,7 @@ function java_compilation() {
     cat "$paramfile" >&2
   fi

-  run "${JAVAC}" -classpath "${classpath}" -sourcepath "${sourcepath}" \
+  run "${JAVAC}" -J-Xmx512m -classpath "${classpath}" -sourcepath "${sourcepath}" \
       -d "${output}/classes" -source "$JAVA_VERSION" -target "$JAVA_VERSION" \
       -encoding UTF-8 "@${paramfile}"

Thanks,
Klaus

Klaus Aehlig
Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschaeftsfuehrer: Matthew Scott Sucherman, Paul Terence Manicle

@damienmg
Copy link
Contributor

No answer so closing, please ping if you still needs help.

@ghost
Copy link
Author

ghost commented Jul 29, 2016

Sorry guys, i sold the 32-Bit system without being able to test it in further detail. I am assuming this should fix the problem as it was related to java and memory consumption. Thanks for your help.

@damienmg
Copy link
Contributor

No problem. Thanks for the update!

On Fri, Jul 29, 2016 at 11:52 AM natemccoy [email protected] wrote:

Sorry guys, i sold the 32-Bit system without being able to test it in
further detail. I am assuming this should fix the problem as it was related
to java and memory consumption. Thanks for your help.


You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub
#1476 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADjHf-__Dt74J0IET1VUC2RN43dsXEMMks5qac1rgaJpZM4JFHPy
.

@aj-michael
Copy link
Contributor

aj-michael commented Feb 24, 2017

So tonight I tried to compile Bazel from the 0.4.4 distzip on 32 bit ubuntu 14.04. I encountered the same OOM, and @aehlig 's patch fixed[1] it. I was curious what the default heap sizes for javac were, so here they are on my 32bit and 64bit systems:

256MB on 32bit

a@nine:~$ javac -J-XX:+PrintFlagsFinal -version | grep MaxHeapSize
    uintx MaxHeapSize                              := 268435456                           {product}
javac 1.8.0_121

4GB on 64bit

a@eight:~$ javac -J-XX:+PrintFlagsFinal -version | grep MaxHeapSize
    uintx MaxHeapSize                              := 4278190080                           {product}
javac 1.8.0_112

[1] It fixed the OOM and presented another problem that I am too tired to attempt to debug right now:

a@nine:~/bazel-0.4.4-dist$ ./compile.sh
INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO:    ./compile.sh compile /path/to/bazel
ð  Building Bazel from scratch.......
ð  Building Bazel with Bazel.
.WARNING: /tmp/bazel_8K2kNpvI/out/external/bazel_tools/WORKSPACE:1: Workspace name in /tmp/bazel_8K2kNpvI/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.
INFO: Found 1 target...
ERROR: /home/a/bazel-0.4.4-dist/src/main/cpp/util/BUILD:82:1: C++ compilation of rule '//src/main/cpp/util:logging' failed: gcc failed: error executing command
  (cd /tmp/bazel_8K2kNpvI/out/execroot/bazel-0.4.4-dist && \
  exec env - \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games \
  /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/local-opt/bin/src/main/cpp/util/_objs/logging/src/main/cpp/util/logging.d '-frandom-seed=bazel-out/local-opt/bin/src/main/cpp/util/_objs/logging/src/main/cpp/util/logging.o' -iquote . -iquote bazel-out/local-opt/genfiles -iquote external/bazel_tools -iquote bazel-out/local-opt/genfiles/external/bazel_tools -isystem external/bazel_tools/tools/cpp/gcc3 -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c src/main/cpp/util/logging.cc -o bazel-out/local-opt/bin/src/main/cpp/util/_objs/logging/src/main/cpp/util/logging.o): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
Target //src:bazel failed to build
INFO: Elapsed time: 5.954s, Critical Path: 0.43s

ERROR: Could not build Bazel

@ghost
Copy link
Author

ghost commented Feb 24, 2017 via email

@aj-michael
Copy link
Contributor

Thanks @natemccoy , the issue was just that I did not have g++ installed (we should probably have a better warning for that). Otherwise, the 32bit Bazel binary seems to be working fine.

@hayaalsh
Copy link

hayaalsh commented May 3, 2017

Thanks @aehlig, I followed the instruction but and I still get an error. Is there anyway to get around this? I am using Odroid 32bit arm7

INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO:    ./compile.sh compile /path/to/bazel
🍃  Building Bazel from scratch.......
🍃  Building Bazel with Bazel.
.WARNING: /tmp/bazel_Aw4CkaDu/out/external/bazel_tools/WORKSPACE:1: Workspace name in /tmp/bazel_Aw4CkaDu/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.
ERROR: No toolchain found for cpu 'arm'. Valid cpus are: [
  piii,
  armeabi-v7a,
  x64_windows_msvc,
  s390x,
].
INFO: Elapsed time: 7.720s

ERROR: Could not build Bazel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants