General guide on building gazebo from source on Windows 11

[WillyTuring]

Hi you guys,

I am developing our custom UI for interacting with 6 dof robotic arm based on Gazebo in wsl2 on Windows11; the primary part is almost finished, now we want to migrate the application to the native Windows OS, since we have done a few modifications on the gazebo source code, we want to build the application from source as well on Windows rather than using Conda or direct gazebo installation with "vcpkg install --triplet=x64-windows gazebo".

The problem we have encountered is that since the CMakeLists.txt file from "https://github.com/gazebosim/gazebo-classic/tree/gazebo11" is intended for the build up on Unix-like system with such as pkg-config tool etc; we have changed a lot to suit the CMake on Windows, but still run into a lot of dependencies problem like Dart package, profiler, tcmalloc just to name a few.

Now my question is that has anyone who has built it from source on Windows, will it work? Is there any general guidance on how to do that? Any suggestions would be really appreciated, thanks in advance.

[traversaro]

vcpkg and conda packages are built from source on Windows, so I can confirm that it is possible to build Gazebo from source on Windows.

Furthermore, also the CI builds from source on Windows, installing depedencies using conda-forge (

gazebo-classic/.github/workflows/conda-forge.yml

Lines 97 to 102 in e4b4d0f

	- name: Build [Windows]
	if: contains(matrix.os, 'windows')
	shell: cmd /C call {0}
	run: |
	cd build
	cmake --build . --config ${{ matrix.build_type }} --parallel 2

) or vcpkg (

gazebo-classic/.github/workflows/windows-config/action.yml

Lines 40 to 54 in e4b4d0f

	set VCPKG_BINARY_SOURCES=clear;files,%CD%\deps\vcpkg\cache\windows,readwrite;
	cmake . -G Ninja -B build ^
	-DCMAKE_BUILD_TYPE=Release ^
	-DCMAKE_TOOLCHAIN_FILE=vcpkg\scripts\buildsystems\vcpkg.cmake ^
	-DVCPKG_MANIFEST_DIR=deps\vcpkg\manifest ^
	-DVCPKG_OVERLAY_TRIPLETS=deps\vcpkg\triplets ^
	-DVCPKG_INSTALLED_DIR=%CD%\vcpkg\installed ^
	-DVCPKG_TARGET_TRIPLET=${{ inputs.arch }}-windows-release ^
	-DVCPKG_HOST_TRIPLET=${{ inputs.arch }}-windows-release ^
	-DVCPKG_INSTALL_OPTIONS=--clean-after-build ^
	-DUSE_EXTERNAL_TINYXML=ON ^
	-DUSE_EXTERNAL_TINYXML2=ON ^
	-DUSE_EXTERNAL_TINY_PROCESS_LIBRARY=ON ^
	-DGZ_PROTOBUF_USE_CMAKE_CONFIG=ON ^
	-DCMAKE_INSTALL_PREFIX=%CD%\vcpkg\installed\${{ inputs.arch }}-windows-release

). So I would say that in general indeed someone build from source Gazebo.

Indeed, Gazebo Classic has a lot of dependencies, so probably it would be easier to help you if you specified how you are installing these dependencies. Just to start from the first point you make:

is intended for the build up on Unix-like system with such as pkg-config tool etc

In general, pkg-config works fine on Windows. What problem do you have with pkg-config? Back in time indeed I made a lot of fixes to ensure that Gazebo Classic and its dependencies configured and compiled fine even without pkg-config mainly as vcpkg CI back in time did not installed pkg-config, but since vcpkg CI now installed pkg-config since a few years, I doubt that is still working fine.

[WillyTuring]

The problem mainly related to the 3rd party dependency packages. From my understandings, the pkg-config tool is mainly used for Unix-like system, sure it can still be used for Windows on paper, but since we are using Visual Studio and vcpkg, CMake is more native. The actual problem with pkg-config tool in my case is since we are installing all those dependency packages using vcpkg rather than using pkg-config for installation, thus there is no .pc file for it to locate; rather we use find_package() to locate those packages.

The problem with vcpkg i have currently is that there are a lot of packages just don't exist, can not be installed with such command which means we have to build those dependencies again from source, i have done this for a few days now, don't feel like this is the right approach hence seeking professional help here.

[traversaro]

The actual problem with pkg-config tool in my case is since we are installing all those dependency packages using vcpkg rather than using pkg-config for installation, thus there is no .pc file for it to locate; rather we use find_package() to locate those packages.

I am afraid I am not following, sorry! vcpkg has really good pkg-config support, it is even one of the few package managers (that I know of) that actually validates the pkg-config dependency graph, see https://github.com/microsoft/vcpkg/blob/b8ac6696e3af59f4f0666ab81a48ae589cde00a8/scripts/cmake/vcpkg_fixup_pkgconfig.cmake#L115-L136 .

As I mentioned, in the CI we are building gazebo against vcpkg-provided dependencies (see https://github.com/gazebosim/gazebo-classic/tree/gazebo11/.github/workflows/windows-config and https://github.com/gazebosim/gazebo-classic/tree/gazebo11/.github/workflows/windows-config). @talregev updated the latest hash in #3367, so it is perfectly possible that there was regressions in recent versions of vcpkg, but for that is easier if you precisely report how are you trying to compile gazebo (for example, which version of vcpkg you are trying to use, etc etc) and the exact errors you are obtaining.

[traversaro]

By the way, based on microsoft/vcpkg#17481 it seems that installing pkgconf port in vcpkg should be enough to get pkg-config on Windows (and indeed the package is listed as a dependency in our vcpkg.json manifest:

gazebo-classic/deps/vcpkg/manifest/vcpkg.json

Lines 39 to 42 in e4b4d0f

	{
	"name": "pkgconf",
	"host": true
	},

).

[traversaro]

The problem with vcpkg i have currently is that there are a lot of packages just don't exist

Again, can you be a bit more detailed? Gazebo is able to compile fine with the depedencies listed in https://github.com/gazebosim/gazebo-classic/blob/gazebo11_11.14.0/deps/vcpkg/manifest/vcpkg.json#L7-L56, what dependencies are you missing?

[talregev]

You can compile dependencies of classic gazebo with vcpkg on windows.
You need a specific hash. Newer hash will not work. Also you need to do it with manifest mode (with vcpkg.json) because it depend on older versions of libraries. The only way to do it, it with manifest mode.
You can check the Yaml of the ci to see how it done.

[talregev]

The problem with vcpkg i have currently is that there are a lot of packages just don't exist

Again, can you be a bit more detailed? Gazebo is able to compile fine with the depedencies listed in https://github.com/gazebosim/gazebo-classic/blob/gazebo11_11.14.0/deps/vcpkg/manifest/vcpkg.json#L7-L56, what dependencies are you missing?

Also override to the older version:
https://github.com/gazebosim/gazebo-classic/blob/gazebo11_11.14.0/deps/vcpkg/manifest/vcpkg.json#L140-L149

[talregev]

Hash found here:
https://github.com/gazebosim/gazebo-classic/blob/gazebo11_11.14.0/deps/vcpkg/manifest/vcpkg.json#L6

[WillyTuring]

The problem with vcpkg i have currently is that there are a lot of packages just don't exist

Again, can you be a bit more detailed? Gazebo is able to compile fine with the depedencies listed in https://github.com/gazebosim/gazebo-classic/blob/gazebo11_11.14.0/deps/vcpkg/manifest/vcpkg.json#L7-L56, what dependencies are you missing?

This screenshot doesn't tell the whole story since i am modifying the SearchForStuff.cmake file under gazebo/cmake/.. directory to ditch using pkg-config tool as a whole, and as i have stated earlier since all the dependencies are installed with vcpkg rather than pkg-config, there is only config.cmake file no .pc file hence we are doing it manually.

But as i have also stated, the problem with that is we need to manually download each of the dependency package wtih vcpkg and modify the .cmake file to locate them one by one, it is tedious and not systematic in my view.

But you mentioned the port for pkgconfig, it will be a much more elegant approach if it works.

[talregev]

pc files in vcpkg are located in "vcpkg\installed\x64-windows\lib\pkgconfig"

[WillyTuring]

pc files in vcpkg are located in "vcpkg\installed\x64-windows\lib\pkgconfig"

Sorry for my inexperience, i found out just now, have been using CMake tool to find the config.cmake file from \vcpkg\installed\x64-windows\share directory the whole time.

[talregev]

if you install it with cmake with vcpkg tool chain (-DCMAKE_TOOLCHAIN_FILE=vcpkg\scripts\buildsystems\vcpkg.cmake) it will detect correctly the pkg config exe and the find the pc if available.

[WillyTuring]

if you install it with cmake with vcpkg tool chain (-DCMAKE_TOOLCHAIN_FILE=vcpkg\scripts\buildsystems\vcpkg.cmake) it will detect correctly the pkg config exe and the find the pc if available.

Yes, i understand now. Since you mentioned about the approach using specific hash and build it with manifest mode to solve the dependency issue, i think this is a more elegant and proven approach, i will try that now to see how it goes. Will come back to report the results.

[WillyTuring]

"E:\19.gazebo_project\gazebo>git status
On branch gazebo11
Your branch is up to date with 'origin/gazebo11'.

nothing to commit, working tree clean

E:\19.gazebo_project\gazebo>git checkout db0473513e5dc73ec6b6f431ff05d2f398eea042
fatal: unable to read tree (db0473513e5dc73ec6b6f431ff05d2f398eea042)"

Is this the correct hash? Or did i miss somthing?

[talregev]

This hash is for vcpkg. not gazebo.
Gazebo should be good on the latest default branch.

[WillyTuring]

This hash is for vcpkg. not gazebo. Gazebo should be good on the latest default branch.

Dame me.

[talregev]

try to use manifest mode. it will do it automatically for you

[WillyTuring]

I am dealing with this error for a while now, i have necessary VC components installed, any idea why?

[WillyTuring]

This command just runs perfectly.

[WillyTuring]

We have tested out with the newest vcpkg, the install command works just fine.

Whereas for the specific hash version, not so much.

[WillyTuring]

I think all the necessary components are there. What did i miss...

[traversaro]

I think I had a similar issue in one project, I worked it around using a newer vcpkg tool, I am currently away from a laptop, as soon as I am back I can provide more details.

[WillyTuring]

I think I had a similar issue in one project, I worked it around using a newer vcpkg tool, I am currently away from a laptop, as soon as I am back I can provide more details.

I have tried the whole morning, if using the newer version of vcpkg tool, we sort of go back to the original state where there are quite a lot of dependencies missing and need to install manually.

[talregev]

@WillyTuring From your screenshot, you don't use manifest mode for vcpkg.
On vcpkg you should be on the latest master. Run bootstrap-vcpkg.bat to be on the latest tool of vcpkg.
Then when you do the install, first do the manifest mode install:
These commands is done on gazebo repo. (You can adopt them later for your needs).
Note that manifest mode will remove any previous installation, that why you will need to do it, in 2 steps.

Note that manifest not change the hash of the git, it take the ports from old hash as it specify in the manifest file.

Note I change from windows-release to regular windows triplet. it will compile both debug and release. It will take more time.
I change this for you, because I don't know if the other depends you want to install, they will install correctly on windows-release triplet.

cd <paht to gazebo repo>
git clone vcpkg
cd vcpkg
bootstrap-vcpkg.bat
vcpkg install  --x-manifest-root=..\deps\vcpkg\manifest --x-install-root=installed --host-triplet=x64-windows --clean-after-build
vcpkg install <your missing dependencies>

Then you need to compile gazebo without manifest mode, because you already have the dependencies you want, and manifest mode will delete the other dependencies you install.

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2022\Visual Studio Tools\VC\x64 Native Tools Command Prompt for VS 2022.lnk"
cd <path to your gazebo repo>
        cmake . -G Ninja -B build ^
            -DCMAKE_BUILD_TYPE=Release ^
            -DCMAKE_TOOLCHAIN_FILE=vcpkg\scripts\buildsystems\vcpkg.cmake ^
            -DVCPKG_OVERLAY_TRIPLETS=deps\vcpkg\triplets ^
            -DVCPKG_INSTALLED_DIR=%CD%\vcpkg\installed ^
            -DVCPKG_TARGET_TRIPLET=x64-windows^
            -DVCPKG_HOST_TRIPLET=x64-windows^
            -DVCPKG_INSTALL_OPTIONS=--clean-after-build ^
            -DUSE_EXTERNAL_TINYXML=ON ^
            -DUSE_EXTERNAL_TINYXML2=ON ^
            -DUSE_EXTERNAL_TINY_PROCESS_LIBRARY=ON ^
            -DGZ_PROTOBUF_USE_CMAKE_CONFIG=ON ^
            -DCMAKE_INSTALL_PREFIX=%CD%\vcpkg\installed\x64-windows

I will not available in the weekend.
I didn't check this command for a while, and also changed on the fly for your case.
And not check these commands after the changes. might be a small errors.
This should be a guide. and you can take it from here and try and workout the things you need.

Hope that will help you for what you need.

[WillyTuring]

Here we used the specific hash version of vcpkg and ran .bat file.

We have the vcpkg.json file located at \gazebo_project\gazebo.

We then install those dependencies from .json file on manifest mode at the directory where .json file located at.

Below is the output if we use the latest vcpkg tool rather than that specified hash version, we got some dependency errors.

[talregev]

please be on latest vcpkg master and try to follow my guide.

[WillyTuring]

We are on the latest vcpkg, then we encountered the following error:

[talregev]

Can you record all the steps from my guide and show the status include the commands itself?
There is a reason why don't work on the latest dependencies because it break classic gazebo.

[WillyTuring]

step 1:

step 2:

step 3:

step 4:

...

[talregev]

This is the wrong manifest file. you should take it from gazebo repo

[talregev]

bin.txt

You need to add the E:\19.gazebo_project\gazebo\installed\x64-windows-release\bin to path.
also other folders that found on E:\19.gazebo_project\gazebo\installed\x64-windows-release\

[talregev]

try to look on vcpkg\installed\x86-windows-release\share\gazebo
the file setup.bat.

[talregev]

I also remember now.
That x64 is not working, only the x86.
So you need to do again, for x86.
Sorry about that.

[WillyTuring]

I also remember now. That x64 is not working, only the x86. So you need to do again, for x86. Sorry about that.

Ok then, let's revise the configure command and try it again to see how it works.

[WillyTuring]

What's your local time there?

[talregev]

I also want to mention, that you can run it on x86, but the gazebo build with vcpkg is very buggy. Basic stuff will not work as it should. I think most of the problem that it not find the correct dlls, and you need to debug it on your own. I will not have time to help you to do so.
So if you are going on this routh, know this.
I don't know what the state of gazebo if you compile it with conda. @traversaro can help you with that if you choose.

If you still want to continue the path with vcpkg, you need to start from the vcpkg install phase with x86.
It will install everything from scratch.

[WillyTuring]

Well, that's awkward, the reason for trying with vcpkg is because we have modified the source code of the gazebo-classic to suit our need for robot UI in Linux as our original developing environment.

Now that you pointed out, i am really not sure what to do, what's your suggestion about my case? The reason for migrating to Windows is simply because this is more user-friendly OS commercially and also there are a few APIs we used for our application running on windows OS though.

[talregev]

@traversaro what the state of gazebo when you compile it with conda. Can you compile it with modify source code?

[WillyTuring]

I think it's gonna take a good while, guess have to see the final result tomorrow.

But if conda is not supporting building the modified source, wha then, are we stuck? I mean the source code we change mainly revolve around the main UI window, mainly visual related; the actual functional stuff are all done via plugins.

[talregev]

I think it's gonna take a good while, guess have to see the final result tomorrow.

But if conda is not supporting building the modified source, wha then, are we stuck? I mean the source code we change mainly revolve around the main UI window, mainly visual related; the actual functional stuff are all done via plugins.

I am sorry for the long effort you made, I try to help as much as I can. I understand that there was a miscommunication and we run to the "solution" instead just chat and replan.
I think the conda gazebo installation can be done from modify source code, but the final answer on that matter only give @traversaro.

You don't stuck, because the vcpkg build from source can build and run gazebo on windows. You just need to work hard that it will run properly on windows.
And I can help you to the point when gazebo will run (in a buggy way). Because I did it myself on my pc.
Then you will have verbose that you can debug it. and also you can build it with debug and debug it.

[talregev]

vcpkg and conda packages are built from source on Windows, so I can confirm that it is possible to build Gazebo from source on Windows.

In previous comments of @traversaro:
#3390 (comment)

Also point out the file in the ci that gazebo compile with conda:

gazebo-classic/.github/workflows/conda-forge.yml

Lines 97 to 102 in e4b4d0f

	- name: Build [Windows]
	if: contains(matrix.os, 'windows')
	shell: cmd /C call {0}
	run: |
	cd build
	cmake --build . --config ${{ matrix.build_type }} --parallel 2

[WillyTuring]

Not at all, i really appreciate all the help you guys given in the last few days, especially in the format of going forth and back at a quick exchange pace, really like the efficiency.

You don't stuck, because the vcpkg build from source can build and run gazebo on windows. You just need to work hard that it will run properly on windows.
And I can help you to the point when gazebo will run (in a buggy way). Because I did it myself on my pc.
Then you will have verbose that you can debug it. and also you can build it with debug and debug it.

That's good to know to say the least then.

Actually one sidetrack question about the gazebo community, maybe a little dumb to ask, is this repository maintained currently all by you two?

[talregev]

I am not the maintainer.
@traversaro is one of the maintainers and there is couple more.

[WillyTuring]

Kudos to you and all those people, i have little knowledge about the open source community, as far as i concern, a lot of them are actually taking it as a full-time job, right? On the one hand i am feeling really fortunate to have those people like you around whom we can seek help from; on the other hand, i can not fully grasp the motivation of those people, why would they contribute this much to the open-source community rather than utilize it commercially?

[talregev]

I can speak for myself.
For me it an hobby. I do it when I like it, and the projects I like and move forward the things I want.
I gain joy, is interesting for me, and also improve my skills, and I gain knowledge and experience like that.

[WillyTuring]

Finally got it build successfully, what buggy experiences did you have previously can you share one or two as an example?

I can speak for myself. For me it an hobby. I do it when I like it, and the projects I like and move forward the things I want. I gain joy, is interesting for me, and also improve my skills, and I gain knowledge and experience like that.

[azeey]

Actually one sidetrack question about the gazebo community, maybe a little dumb to ask, is this repository maintained currently all by you two?

Gazebo-classic is scheduled to be EOL in 2025-01-29, so we're doing minimal maintenance. All new development is happening on modern gazebo at https://github.com/gazebosim/gz-sim

[talregev]

@WillyTuring Did you manage to run it?
I cannot add things

[WillyTuring]

@WillyTuring Did you manage to run it? I cannot add things

Haven't tried this yet, as i was migrating and modifying the code; will try it out later today and let you know.

[WillyTuring]

@talregev

Did you have this same issue? Couldn't find the correct model file?

[talregev]

Yes. you should debug it with --verbose
I cannot help you any farther. I am glad that you succeeded to do it with vcpkg.
You can try to build it with conda like in the ci. (mamba).
I hope @traversaro can tell more about this state before you try. If it run and work correctly on windows.
Also like @azeey mention there is a new gazebo. This version End Of Life is on 29.1.2025.

[WillyTuring]

I don’t think gz-sim is Windows-ready yet. I am not familiar with Conda package tool, i am still migrating the plugins as for now, we will see how it goes. Thanks so much, really appreciate your help. 发自我的iPhone

…

------------------ Original ------------------ From: talregev ***@***.***> Date: Fri, Aug 9, 2024 3:08 PM To: gazebosim/gazebo-classic ***@***.***> Cc: WillyTuring ***@***.***>, Mention ***@***.***> Subject: Re: [gazebosim/gazebo-classic] General guide on building gazebo fromsource on Windows 11 (Issue #3390) Yes. you should debug it with --verbose I cannot help you any farther. I am glad that you succeeded to do it with vcpkg. You can try to build it with conda like in the ci. (mamba). I hope @traversaro can tell more about this state before you try. If it run and work correctly on windows. Also like @azeey mention there is a new gazebo. This version End Of Life is on 29.1.2025. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[traversaro]

I hope @traversaro can tell more about this state before you try. If it run and work correctly on windows.

Yes, with conda-forge dependencies Gazebo Classic works fine, I am not aware of any outstanding bug and many persons in my lab use it. You can also build Gazebo Classic from source, following the instructions in this CI job: https://github.com/gazebosim/gazebo-classic/blob/gazebo11/.github/workflows/conda-forge.yml .

I don’t think gz-sim is Windows-ready yet.

As long as you launch separately gz sim -s (server) and gz sim -g (client) gz sim works fine on Windows, see gazebosim/gz-sim#2382 . However, to be honest I do not have enough Windows users of gz sim in my lab to say that it has been extensive tested.

[WillyTuring]

@talregev Hi, have you solved the issue of adding the model to gazebo-classic in windows?

[talregev]

No. I will not have time to solve it.
I recommend for you try the conda way.
If you need help with conda, you can post it here.

[WillyTuring]

No. I will not have time to solve it. I recommend for you try the conda way. If you need help with conda, you can post it here.

Have you tried with Conda approach before? is everything fine?

[WillyTuring]

Here is my situation, no error reports. Been stuck in this state for a while now...

[talregev]

I hope @traversaro can tell more about this state before you try. If it run and work correctly on windows.

Yes, with conda-forge dependencies Gazebo Classic works fine, I am not aware of any outstanding bug and many persons in my lab use it. You can also build Gazebo Classic from source, following the instructions in this CI job: https://github.com/gazebosim/gazebo-classic/blob/gazebo11/.github/workflows/conda-forge.yml .

I don’t think gz-sim is Windows-ready yet.

As long as you launch separately gz sim -s (server) and gz sim -g (client) gz sim works fine on Windows, see gazebosim/gz-sim#2382 . However, to be honest I do not have enough Windows users of gz sim in my lab to say that it has been extensive tested.

@WillyTuring
I didn't try, but I quote @traversaro that have experience with conda approch in his lab.

[WillyTuring]

Trying to load up the GUI plugin, we just hang in the Advertise() method call, @talregev any idea why might give me some hint?