Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add object detection capability and python API #3472

Merged
merged 56 commits into from
Jun 8, 2021
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
4fd72e0
- Add detections python api
alonfaraj Feb 18, 2021
b89fe26
- Merge from AirsimHost
alonfaraj Feb 18, 2021
b29b174
- Add GetDetection python api
alonfaraj Feb 23, 2021
e825439
- Add simGetDetections to python api
alonfaraj Feb 24, 2021
52fe8fb
- Update files from AirSimHost
alonfaraj Feb 24, 2021
5c5e27f
- Merge from AirSimHost
alonfaraj Mar 4, 2021
8acc4b2
- Update detection api
alonfaraj Mar 8, 2021
18083d1
- Add python script detection example
alonfaraj Mar 16, 2021
66c514b
- Remove unused include
alonfaraj Mar 29, 2021
9e9ed86
- Shorten detectionIterator to itr
alonfaraj Mar 29, 2021
a55fb6d
- Remove commented out code
alonfaraj Mar 29, 2021
ba34079
- Update check for empty WildcardMeshNames array for simpler way and …
alonfaraj Mar 29, 2021
0991166
- Check for valid static mesh
alonfaraj Mar 29, 2021
c4329e2
- Add unimplemented methods for Unity
alonfaraj Mar 29, 2021
ee9ce0e
- Add missing bracket
alonfaraj Mar 29, 2021
7d70ffe
- Change cars to cylinders
alonfaraj Mar 29, 2021
fdf718e
- Remove remarked code
alonfaraj Mar 30, 2021
ee99945
- Add Box3D and RelativeTransform to FDetectionInfo
alonfaraj Mar 30, 2021
de6d978
- Remove remarked code
alonfaraj Mar 30, 2021
7c5da8e
- Add clearDetectionMeshNames() api call
alonfaraj Mar 30, 2021
8dcdfa9
- Update python example script with clear/add mesh names in runtime
alonfaraj Mar 30, 2021
751a3db
- Add unimplemented clearDetectionMeshNames() to Unity
alonfaraj Mar 30, 2021
3a6f85f
- Add Box2D, Box3D, vector2r to RPC
alonfaraj Mar 30, 2021
d163b75
- Cleanup
alonfaraj Mar 30, 2021
ac3f228
- Add relative_pose to DetectionInfo python API
alonfaraj Mar 30, 2021
4fe3f2f
- Generate box3D using toLocalNed
alonfaraj Mar 30, 2021
775b1cb
- Fix relative pose to take into account camera orientation and objec…
alonfaraj Mar 31, 2021
9f50ffd
- Rename geoPoint to geo_point
alonfaraj Mar 31, 2021
40dddf8
- Check if GetStaticMesh() is not nullptr before use it
alonfaraj Mar 31, 2021
acd9ff1
- Rename Detections to detections
alonfaraj Mar 31, 2021
19812a1
- Revert part of last commit
alonfaraj Mar 31, 2021
2f0f0db
- Wrap getDetections() inside RunCommandOnGameThread() to fix crash w…
alonfaraj Apr 1, 2021
511ed3d
- Solve merge conflict due to rename "Adapators" to "Adaptors"
alonfaraj Apr 1, 2021
63b4f4d
- Replace tabs to 4 spaces to keep AirSim convention
alonfaraj Apr 1, 2021
940c570
- Add support to detect actor by name
alonfaraj Apr 1, 2021
6f15f61
- Update detection API functions to support camera_name and image_typ…
alonfaraj Apr 1, 2021
761be46
- Add missing files to last commit
alonfaraj Apr 1, 2021
e4b3e96
Merge branch 'add-detection' of https://github.com/alonfaraj/AirSim i…
alonfaraj Apr 1, 2021
1798c08
- Update python documentation
alonfaraj Apr 1, 2021
7c12a0b
- Update and move unimplemented detection functions for Unity build
alonfaraj Apr 1, 2021
8fab32a
- Cleanup
alonfaraj Apr 4, 2021
d9c30fc
- Fix compilation error in linux
alonfaraj May 2, 2021
5065bf9
- Solve conflicts with upstream master
alonfaraj May 23, 2021
1bd3630
- Fix merge
alonfaraj May 23, 2021
2a5f5b6
- Fix clang errors
alonfaraj May 24, 2021
10746e9
- Fix clang format errors
alonfaraj May 24, 2021
442ee51
- Remove stray semicolon
alonfaraj May 25, 2021
ee6ee8e
- Fix DpethPlanar name after merge
alonfaraj May 25, 2021
da45423
- Return detections by const ref
alonfaraj May 25, 2021
23439ff
- Remove unused code
alonfaraj May 25, 2021
995030c
- Move the compute detection logic to GetDetections() to compute it o…
alonfaraj Jun 3, 2021
7cd5229
- Add missing unused() calls
alonfaraj Jun 6, 2021
8dc625b
- Modify code to follow naming convention and guidelines
alonfaraj Jun 6, 2021
726ea60
- Fix variables name after rename
alonfaraj Jun 6, 2021
9425d35
- Fix clang errors
alonfaraj Jun 6, 2021
0f463a8
Create setup_path.py
zimmy87 Jun 8, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AirLib/AirLib.vcxproj.filters
Original file line number Diff line number Diff line change
Expand Up @@ -321,7 +321,7 @@
<ClInclude Include="include\vehicles\multirotor\firmwares\simple_flight\AirSimSimpleFlightEstimator.hpp">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="include\vehicles\car\api\CarRpcLibAdapators.hpp">
<ClInclude Include="include\vehicles\car\api\CarRpcLibAdaptors.hpp">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="include\vehicles\car\api\CarRpcLibClient.hpp">
Expand All @@ -330,7 +330,7 @@
<ClInclude Include="include\vehicles\car\api\CarRpcLibServer.hpp">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="include\vehicles\multirotor\api\MultirotorRpcLibAdapators.hpp">
<ClInclude Include="include\vehicles\multirotor\api\MultirotorRpcLibAdaptors.hpp">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="include\vehicles\multirotor\api\MultirotorRpcLibClient.hpp">
Expand All @@ -339,7 +339,7 @@
<ClInclude Include="include\vehicles\multirotor\api\MultirotorRpcLibServer.hpp">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="include\api\RpcLibAdapatorsBase.hpp">
<ClInclude Include="include\api\RpcLibAdaptorsBase.hpp">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="include\vehicles\car\api\CarApiBase.hpp">
Expand Down
135 changes: 131 additions & 4 deletions AirLib/include/api/RpcLibAdapatorsBase.hpp
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

#ifndef air_RpcLibAdapatorsBase_hpp
#define air_RpcLibAdapatorsBase_hpp
#ifndef air_RpcLibAdaptorsBase_hpp
#define air_RpcLibAdaptorsBase_hpp

#include "common/Common.hpp"
#include "common/CommonStructs.hpp"
Expand All @@ -18,7 +18,7 @@

namespace msr { namespace airlib_rpclib {

class RpcLibAdapatorsBase {
class RpcLibAdaptorsBase {
public:
template<typename TSrc, typename TDest>
static void to(const std::vector<TSrc>& s, std::vector<TDest>& d)
Expand All @@ -36,6 +36,24 @@ class RpcLibAdapatorsBase {
d.push_back(TDest(s.at(i)));
}

struct Vector2r {
msr::airlib::real_T x_val = 0, y_val = 0;
MSGPACK_DEFINE_MAP(x_val, y_val);

Vector2r()
{}

Vector2r(const msr::airlib::Vector2r& s)
{
x_val = s.x();
y_val = s.y();
}
msr::airlib::Vector2r to() const
{
return msr::airlib::Vector2r(x_val, y_val);
}
};

struct Vector3r {
msr::airlib::real_T x_val = 0, y_val = 0, z_val = 0;
MSGPACK_DEFINE_MAP(x_val, y_val, z_val);
Expand Down Expand Up @@ -221,6 +239,115 @@ class RpcLibAdapatorsBase {
}
};

struct Box2D
{
Vector2r min;
Vector2r max;

MSGPACK_DEFINE_MAP(min, max);

Box2D()
{}

Box2D(const msr::airlib::Box2D& s)
{
min = s.min;
max = s.max;
}

msr::airlib::Box2D to() const
{
msr::airlib::Box2D s;
s.min = min.to();
s.max = max.to();

return s;
}

};

struct Box3D
{
Vector3r min;
Vector3r max;

MSGPACK_DEFINE_MAP(min, max);

Box3D()
{}

Box3D(const msr::airlib::Box3D& s)
{
min = s.min;
max = s.max;
}

msr::airlib::Box3D to() const
{
msr::airlib::Box3D s;
s.min = min.to();
s.max = max.to();

return s;
}

};

struct DetectionInfo {
std::string name;
GeoPoint geo_point;
Box2D box2D;
Box3D box3D;
Pose relative_pose;

MSGPACK_DEFINE_MAP(name, geo_point, box2D, box3D, relative_pose);

DetectionInfo()
{
}

DetectionInfo(const msr::airlib::DetectionInfo& d)
{
name = d.name;
geo_point = d.geo_point;
box2D = d.box2D;
box3D = d.box3D;
relative_pose = d.relative_pose;
}

msr::airlib::DetectionInfo to() const
{
msr::airlib::DetectionInfo d;
d.name = name;
d.geo_point = geo_point.to();
d.box2D = box2D.to();
d.box3D = box3D.to();
d.relative_pose = relative_pose.to();

return d;
}

static std::vector<DetectionInfo> from(
const std::vector<msr::airlib::DetectionInfo>& request
) {
std::vector<DetectionInfo> request_adaptor;
for (const auto& item : request)
request_adaptor.push_back(DetectionInfo(item));

return request_adaptor;
}
static std::vector<msr::airlib::DetectionInfo> to(
const std::vector<DetectionInfo>& request_adapter
) {
std::vector<msr::airlib::DetectionInfo> request;
for (const auto& item : request_adapter)
request.push_back(item.to());

return request;
}

};

struct CameraInfo {
Pose pose;
float fov;
Expand Down Expand Up @@ -248,7 +375,7 @@ class RpcLibAdapatorsBase {
return s;
}
};

struct KinematicsState {
Vector3r position;
Quaternionr orientation;
Expand Down
Loading