Fixed Wing Aircraft

[AOS55]

Fixed Wing Vehicle Integration

I am working on adding a FixedWing aircraft class into Airsim, so far I have been documenting my work under issue #2508 but as suggested by @rajat2004 it would be usefult to move this work to a new issue so as to improve visibility for others to work on.

Work so far

Please see this post for a detailed breakdown of what has been added in my forked Airsim repository under the fixedwing branch. I will aim to keep updates on this issue of new files, classes, methods & functions with a view to write comprehensive documentation as the code begins to get to a state where it can be merged back into the master branch on the main repo.

Since the last update post on #2508 I have been updating the FixedWingLibClient files to enable communication with Ardupilot. A linked issue has been opened within the ArduPilot Repo.

FixedWing Branch State

• Compilable from Visual Studio: 🟢
• Runs alone in UE4: 🟢
• Runs with ArduPilot and Mavlink in UE4 over network: 🔴
• Model validated with real comparison: 🔴

Known issues

• I can't compile the code from source in Visual Studio please see the chain back in #2508 for what we have tried so far. I think it is most likely a mistake I have made in the source code. <-- This is currently the most pressing issue.
• I naievly initially implemented the control surfaces at seperate functions getElevatorSignal() etc. I am beginning to realise as I work through the control implementation this was not a good way to go about this and the program is already well setup to accept a more conventional control vector, i.e. the u vector in state-space form. I am planning to go back and correct this to a 4 row u vector.
• The Core APIs I effectively copied over when setting up the controller files include several aspects that I don't think make sense for a Fixed Wing aircraft, for example commandVelocityHold uses the function moveByLocalVelocity that takes a boolean yaw mode switch and rate as a command these are set as false and 0 in the current implementation. But, moveByLocalVelocity may be better of if rewritten for fixedwing.

Planned Future Work

Initially I am aiming to work to resolve all the known issues and turn all the 🔴 in the state to 🟢. Following from this I will aim to add the following functionality: [ ] Construct a similar Python API as in multirotor <-- hopefully a short term objective [ ] Implement Undercarriage for fixedWingLandings [ ] Deep Reinforcement Learning (DRL) based FW landings <-- main PhD objective, expect to write & publish a paper based on this [ ] Validate model and control on real UAV and FW aircraft, ties in with above.

Motivation

Principally this is for my PhD at the University of Bristol where I am aiming to use a combination of computer vision and Deep Reinforcement Learning to train an autonomous fixedwing agent/aircraft to navigate to and safely land in an unknown location. This is both with and without power and on a range of fixedwing aircraft including fixedWing UAVs, gliders & multiengine light piston aircraft (<5700Kg). Not only this but when discussing AirSim with my colleagues we believe that having a fixedWing simulator capable of highfidelity graphics and SITL that is efficient will be of great benefit.

Ongoing Development

Several users have already been realy helpful (@rajat2004 😀), if anyone has any ideas about how to solve some of the issues I have described above all help is greatly appreciated. In particular if anyone is able to have any success compiling the fixedwing source please let me know as I'm a little stumped 😕 as to why it won't compile.

[rajat2004]

I had a try at compiling your branch, the error message was really quite useless. Opened https://github.com/AOS55/AirSim-1/pull/1 which fixes that and brings us back to normal compilation errors with understandable messages (for now)

[AOS55]

Progress Update

Aircraft Physics Control Construction

I have been continuing to work on the FixedWingPhysics simulation aiming to resolve compilation errors. I have vectorised the control surfaces so ideally they behave as a u tyoe vector. With this I introduced a new file Aeroplane.hpp that instants each control from the ControlSurface.hpp file as a PhysicsBodyVertex class. I have not introduced the thrust vector yet but this shouldn't be too much of a challenge.

Aircraft links to the structs in AircraftParams.hpp and applies the aero-forces via setWrench. Most fixedwing dynamic models/databases, define the forces and moments around the CG, as dynamicDerivatives. setWrench applies these and then is updated in the loop by FixedWingPhysicsBody.hpp, I'm a little lost as to how the PhysicsBody class updates the aircrafts pose via fastPhysicsEngine. The way I've set up the classes may also be an antipattern and I might need to refactor some of the structs, please let me know your thoughts on how you might setup the structures.

Compilation error

When I run the repo from visual studio I get the following error :

Error C4717 'msr::airlib::ArduPlaneApi::resetImplementation': recursive on all control paths, function will cause runtime stack overflow Blocks C:\Users\quessy\Documents\Unreal Projects\AirSim\Unreal\Environments\Blocks\Plugins\AirSim\Source\AirLib\include\vehicles\fixedwing\firmwares\ardupilot\ArduPlaneApi.hpp 57

Im not sure what this error is being caused by, I'm guessing there is no termination condition in the recursive call. If anyone has experienced this before I'd love to get your thoughts.

[rajat2004]

That's happening cause the function is calling itself here - https://github.com/AOS55/AirSim-1/blob/FixedWing/AirLib/include/vehicles/fixedwing/firmwares/ardupilot/ArduPlaneApi.hpp#L54 It should instead be calling FixedWingApiBase::resetImplementation(), from it's parent class

Note that I haven't tested this myself yet

[AOS55]

Thanks yep that was it was quite obvious now I think about it in more detail. It several unresolved external symbol errors now but I think there a case of sorting out further problems with the API.

[AOS55]

Does anyone have any thoughts on this error coming from FixedWingApiBase.hpp, I thought it was to do with the link between the base class and Mavlink/Px4 implementation override. I can't see anything untawards though: LNK2001 unresolved external symbol "protected: virtual float __cdecl msr::airlib::FixedWingApiBase::getAutoLookahead(float,float,float,float)const " (?getAutoLookahead@FixedWingApiBase@airlib@msr@@MEBAMMMMM@Z) Blocks C:\Users\quessy\Documents\Unreal Projects\AirSim\Unreal\Environments\Blocks\Intermediate\ProjectFiles\Module.AirSim.cpp.obj 1

[rajat2004]

Functions seem to be missing, are all the files uncommented in the AirLib.vcxproj?

[AOS55]

Thats a good point I've uncommented them now, from AirLib.vcxproj, but Im still getting the same link errors from the compiler. When running build.cmd I also get some errors in the command prompt that are slightly different, ~~perhaps these are what needs to be looked at~~. These are quite clearly what I should have been looking at, I didn't appreciate the importance of the API src client, server and base files.

[AOS55]

Resolved all build errors now except for one rather unusual one: "C:\Users\quessy\Documents\Unreal Projects\AirSim\AirSim.sln" (default target) (1) -> "C:\Users\quessy\Documents\Unreal Projects\AirSim\AirLib\AirLib.vcxproj" (default target) (12) -> (ClCompile target) -> C:\Users\quessy\Documents\Unreal Projects\AirSim\AirLib\deps\rpclib\include\rpc\msgpack\v1\object.hpp(202,11): error C2039: 'msgpack_pack': is not a member of 'Eigen::Matrix<float,3,1,0,3,1>' [C:\Users\quessy\Documents\Unreal Projects\AirSim\AirLib\AirLib.vcxproj]

Not sure where this one links to as when I look at the location of the error in object.hpp which I'm not sure where it lives in the repo as dependencies folder not included in push/pull requests.

[rajat2004]

The RPC error was due to the fact that it doesn't know how to pack the Vector3r object, need to convert it into the struct defined in RpcLibAdaptorsBase and then pass it. Opened https://github.com/AOS55/AirSim-1/pull/4 which fixes that, and adds the files to CMake. After this AirLib compiles, but there are still lots of errors when building the entire Blocks env, undefined references in FixedWingApiBase.cpp, due to function declarations in .hpp, and usage of commented out methods

[AOS55]

Great thanks very much Ill take a look at those errors today, I merged the issue back into fixedwing and have no build.cmd errors on my end too.

[AOS55]

Any thoughts on why I'm getting the following include/visibility related build error from in visual studio? I can see the source file UdpSocket.hpp is still on the repo, could it be related to cmake changes?

• From Visual Studio: C1083 Cannot open include file: 'UdpSocket.hpp': No such file or directory Blocks C:\Users\quessy\Documents\Unreal Projects\AirSim\Unreal\Environments\Blocks\Plugins\AirSim\Source\AirLib\include\vehicles\car\firmwares\ardurover\ArduRoverApi.hpp

• From Output C:\Users\quessy\Documents\Unreal Projects\AirSim\Unreal\Environments\Blocks\Plugins\AirSim\Source\AirLib\include\vehicles/car/firmwares/ardurover/ArduRoverApi.hpp(20): fatal error C1083: Cannot open include file: 'UdpSocket.hpp': No such file or directory

[rajat2004]

That's very strange, Windows shouldn't be affected AFAIK since it uses the .sln files. And if this were a problem, then it would have appeared way earlier since support for Rover has been present for many months Does this appear when running build.cmd?

[AOS55]

No thats the unusual thing build.cmd doesnt present any errors when run

[rajat2004]

Hmm, maybe try running clean_rebuid.cmd from the root directory, that should rebuild everything. Do commit any changes made in the files present in the Blocks directory like in the AirLib folder before running though

[AOS55]

Great thanks that seems to have resolved it back to linking problems probably due to decleration again.

[rajat2004]

Nice! Still have no idea why VS decided to remove the header file Side note, this does somewhat bring into mind that GitHub issues might not be the best way to communicate, especially on smaller problems. Maybe something for future

[AOS55]

Yeah not sure its unusual. Yes certainly, I'm surprised there isn't a microsoft/AirSim slack setup or equivalent for more direct communication. As you said something for the future.

[rajat2004]

There is a Facebook group, but I myself haven't used it since a long time. AP uses Discord (earlier Gitter) and has a forum as well, similar with PX4. Maybe after there is more activity and members. Having something like this would definitely reduce the number of issues also, since more confirmed issues will be created rather than duplicate and support ones

[AOS55]

Yes I agree thats a good idea, the documentation for AirSim would need to reflect this too as there is not an obvious point of contact when onboarding new users/developers with the application.

Also after commenting out the linking errors in the application I have managed to compile through to the blocks environment 😀.

[rajat2004]

Awesome, now comes the probably more difficult part of modelling, physics and making sure everything works! Having a more direct method of communication will also increase community interaction a lot. Which platform to select, etc will have to be decided by the maintainers though, might require collecting some opinions from the users

[AOS55]

Yes I agree, would you like to open up a communication issue, I'm not sure if you can do a poll on github?

[rajat2004]

Don't think I'm the best person to open such a issue, a maintainer should do this probably. They'll also have to setup the platform and maybe maintain it (no other word is coming to my mind right now) Github doesn't have polls AFAIK

[AOS55]

Yes that makes sense, a moderator or project manager sort of thing. Perhaps if you know anyone you could @ them into this issue so they can set it up. Ok sure you could always make the proposed solution as a comment and get people to vote 👍 or 👎 on the comment.

[rajat2004]

@madratman @saihv @ironclownfish @jonyMarino Do you think having another (more chat & voice type) communication platform will be worthwhile? This will definitely increase community interaction, and will probably reduce support issues. Just wanted to know your opinions, and definitely don't want to increase your (already heavy) workload too much. Looking forward to hearing from you!

[rajat2004]

@AOS55 AirLib is actually not compiling for me, getting some undeclared identifier errors for the function declarations which have been commented out. Here's a Travis build - https://travis-ci.com/github/rajat2004/AirSim/builds/181419292, even the Windows build fails. Did some files not get committed when commenting out the API methods?

Would also recommend activating Travis on your Github repositories (atleast Airsim), that has saved me a few times when making some changes which don't compile on Windows, or breaking something else. (Side note, I did break the Windows build in my first contribution to Airsim (and others later on also), that was a big motivation to get Travis setup properly on Airsim)

[AOS55]

@rajat2004 hmm ok, this is interesting could you please send me the identifier errors you are getting I will cross-check them on my local machine, I just ran update_to_git.bat and had no further changes and I also ran update_from_git.bat in Blocks and it still compiles ok on my end. A couple of thoughts:

• Are you using UE4.25.3 or UE4.24.3? I have been using 4.25.3 I don't know if this would effect the build or travis.
• Did you try rebuilding from the root directory as that seemed to be what caused mine to compile in the first instance?

Nice Idea, I have setup Travis with my branch, and it also appears to fail. Looking at the job log it has the following on line 1457. I wonder if this is unity related or just what the .travis.yml script should print to the command line.

The command "if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then ./setup.sh; ./build.sh || travis_terminate 1; echo "Starting Unity Build!"; (cd Unity && ./build.sh || travis_terminate 1); elif [[ "$TRAVIS_OS_NAME" == "windows" ]]; then export PATH=$MSBUILD_PATH:$PATH_FIX:$PATH; cmd.exe //C 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat' amd64 '&&' build.cmd '&&' cd Unity '&&' build.cmd; elif [[ "$TRAVIS_OS_NAME" == "osx" ]]; then ./setup.sh; ./build.sh || travis_terminate 1; echo "Starting Unity Build!"; (cd Unity && ./build.sh || travis_terminate 1); fi" exited with 1.

[rajat2004]

Very strange that it works on your machine, these are the errors I'm getting -

/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/fixedwing/api/FixedWingApiBase.cpp:514:9: error: use of undeclared identifier 'safetyCheckVelocity'
    if (safetyCheckVelocity(Vector3r(vx, vy, vz)))
        ^
/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/fixedwing/api/FixedWingApiBase.cpp:526:9: error: use of undeclared identifier 'safetyCheckDestination'
    if (safetyCheckDestination(dest))
        ^
/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/fixedwing/api/FixedWingApiBase.cpp:538:9: error: use of undeclared identifier 'safetyCheckVelocity'
    if (safetyCheckVelocity(getVelocity()))
        ^
/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/fixedwing/api/FixedWingApiBase.cpp:562:9: error: use of undeclared identifier 'safetyCheckVelocity'
    if (safetyCheckVelocity(getVelocity()))
        ^
[ 80%] Building CXX object AirLib/CMakeFiles/AirLib.dir/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/multirotor/api/MultirotorApiBase.cpp.o
[ 81%] Building CXX object AirLib/CMakeFiles/AirLib.dir/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/multirotor/api/MultirotorRpcLibClient.cpp.o
4 errors generated.
AirLib/CMakeFiles/AirLib.dir/build.make:230: recipe for target 'AirLib/CMakeFiles/AirLib.dir/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/fixedwing/api/FixedWingApiBase.cpp.o' failed
make[2]: *** [AirLib/CMakeFiles/AirLib.dir/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/fixedwing/api/FixedWingApiBase.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 83%] Linking CXX executable ../../output/bin/MavLinkTest
[ 83%] Built target MavLinkTest
/home/rajat/Github/AirSim-FixedWing/AirLib/src/vehicles/fixedwing/api/FixedWingRpcLibServer.cpp:116:45: error: no member named 'moveByManual' in 'msr::airlib::FixedWingApiBase'
        return getVehicleApi(vehicle_name)->moveByManual(vx_max, vy_max, z_min, duration);
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~  ^
1 error generated.

AirLib compilation doesn't depend on UE, Travis also doesn't have UE, it just tests whether AirLib, ROS wrapper compiles, and there's the recently added Azure Pipelines CI which compiles the entire Blocks project on Ubuntu and Windows and packages it.

Yup, I'm rebuilding from the root directory, ./clean.sh & ./build.sh.

That's just Travis printing the entire command which failed to run, the errors can be seen above it, starting from here

[AOS55]

Ok yeah, I was just looking at the code on my machine and I have those if statements at the lines the compiler called out still there, given I commented them out in the header file I would have expected an undeclared header error. I have commented them out now in the CPP file. Still waiting for Travis.

[AOS55]

Got some further travis build errors in FixedWingRpcLibServer.cpp commented out the moveByManual Function. This now succeeds for macOS and windows but fails for ubuntu 16.04 LTS and 18.04 LTS.

[rajat2004]

@AOS55 this was a problem with the CMake config in ROS wrapper, I've opened #2978 which fixes this. The reason's described there, and as to why it was missed, looks like not many people have named the folder something other than AirSim :)