Aircraft Model

[ZF113120]

Could you please tell me if QPlane can freely select aircraft models from XPlane or JSBSim for training? If so, could you please provide some guidance?

[JDatPNW]

Hello! In XPlane you can simply change the aircraft through the UI when you launch the game. Simply start a new flight, pick the plane, start and then connect to it. For JSBSim you can change the plane via

self.fdm.load_model('c172r')  # loading cassna 172

in /src/environments/jsbsim/JSBSimEnv.py and then if you want to render with FlightGear you should change the command too.

--fdm=null --native-fdm=socket,in,60,localhost,5550,udp --aircraft=c172r --airport=RKJJ

Note, however, that you might need to change some other values as well! Such as starting velocity, terminating AoA, etc.

On a more general note, feel free to also look through my thesis on this topic, where QPlane is described in more detail: Link to Thesis

Hope this helps, best, David

[ZF113120]

Hello! In XPlane you can simply change the aircraft through the UI when you launch the game. Simply start a new flight, pick the plane, start and then connect to it. For JSBSim you can change the plane via

self.fdm.load_model('c172r') # loading cassna 172 in /src/environments/jsbsim/JSBSimEnv.py and then if you want to render with FlightGear you should change the command too.

--fdm=null --native-fdm=socket,in,60,localhost,5550,udp --aircraft=c172r --airport=RKJJ Note, however, that you might need to change some other values as well! Such as starting velocity, terminating AoA, etc.

On a more general note, feel free to also look through my thesis on this topic, where QPlane is described in more detail: Link to Thesis

Hope this helps, best, David

Hello! Thank you for your very helpful reply. I will read your paper in detail to learn about the use of QPlane.

[JDatPNW]

You are welcome!

[ZF113120]

Hello! After switching the training environment to XPlane, I encountered the following errors when running QPlaneTest.py. Could you please advise on the causes and solutions? I eagerly await your response.

Traceback (most recent call last):
  File "QPlaneTest_xplane.py", line 278, in <module>
    epoch(i_epoch)
  File "QPlaneTest_xplane.py", line 261, in epoch
Traceback (most recent call last):
  File "QPlaneTest_xplane.py", line 278, in <module>
    epoch(i_epoch)
  File "QPlaneTest_xplane.py", line 261, in epoch
  File "QPlaneTest_xplane.py", line 278, in <module>
    epoch(i_epoch)
  File "QPlaneTest_xplane.py", line 261, in epoch
    epoch(i_epoch)
  File "QPlaneTest_xplane.py", line 261, in epoch
  File "QPlaneTest_xplane.py", line 261, in epoch
    done, reward, logList, oldState = step(i_step, done, reward, oldState)
  File "QPlaneTest_xplane.py", line 196, in step
    newState, reward, done, info = env.step(action)
  File "E:\RL-plane\QPlane\src\environments\xplane\XPlaneEnv.py", line 190, in step
    reward, done = self.rewardFunction(action, position)
  File "E:\RL-plane\QPlane\src\environments\xplane\XPlaneEnv.py", line 159, in rewardFunction
    crash = client.getDREF("sim/flightmodel2/misc/has_crashed")[0]
IndexError: tuple index out of range

[JDatPNW]

Hello, did you make sure that XPlane and XPlaneConnect are both running and that they are communicating with each other? Also, if you updated the state vector during training, then you would need to update it for testing too, same goes for if you updated the state space in JSBSim, then you would have to update it for XPlane too, which means you would have to find the correct corresponding value and perhaps convert it to be the same unit, orientation, etc. Hope this helps, David

[ZF113120]

Hello, I have identified the cause of the error. The issue might be due to my use of XPlane-10 instead of XPlane-11.5. The error occurs in the XPlaneEnv.py file, specifically at line 158:

crash = client.getDREF("sim/flightmodel2/misc/has_crashed")[0]

After commenting out line 158 and the subsequent statements that reference crash , the program runs normally. This is because I noticed that XPlane-10 's DataRefs.txt does not include "sim/flightmodel2/misc/has_crashed". Could you please explain the purpose of crash in this context? ( I apologize for not fully understanding your code.) Additionally, will commenting out these lines affect the program's normal operation? If so, I would greatly appreciate your guidance on how to modify the code accordingly. Thank you! I hope you can reply at your convenience. I look forward to hearing from you!

[JDatPNW]

Great find! I did not know that XPlane 10 did not have the crash DREF, thanks for letting me know! If I remember correctly, sorry it has been a while since I actively worked on this project, then crash is not too important because I already implemented a certain altitude as terminal (meaning if the plane was lower than a certain altitude the done flag would be set to true and the episode would be over and a new one would start). Crash where you found it technically does the same, it checks if the plane has crashed and if it did, then it would reset the episode and start over (and also give it bad rewards). I think if the altitude terminal state works for you, then maybe you can do without the crash DREF.

In this file you can see the functions getTermination and rewardFunction. Here you can see:

 def getTermination(self, alt, alpha):

        # checks if plane is less than x feet off the ground, if not it will count as a crash
        if (alt < 1000):
            terminate = True
        elif(alpha >= 16):
            terminate = True
        else:
            terminate = False
        return terminate

    def rewardFunction(self, action, newObservation, alt, alpha):
        observation = newObservation[:]
        if(self.randomDesiredState):
            observation[self.dictObservation["pitch"]] = -(self.desiredPosition["pitch"] - observation[self.dictObservation["pitch"]])
            observation[self.dictObservation["roll"]] = -(self.desiredPosition["roll"] - observation[self.dictObservation["roll"]])

        roll = float(abs(observation[self.dictObservation["roll"]] / 180))
        pitch = float(abs(observation[self.dictObservation["pitch"]] / 180))
        reward = pow(float((2 - (roll + pitch)) / 2), 2)

        if(abs(observation[self.dictObservation["roll"]]) > 40 or abs(observation[self.dictObservation["pitch"]]) > 40):
            reward = reward * 0.1
        elif(abs(observation[self.dictObservation["roll"]]) > 20 or abs(observation[self.dictObservation["pitch"]]) > 20):
            reward = reward * 0.25
        elif(abs(observation[self.dictObservation["roll"]]) > 10 or abs(observation[self.dictObservation["pitch"]]) > 10):
            reward = reward * 0.5
        elif(abs(observation[self.dictObservation["roll"]]) > 5 or abs(observation[self.dictObservation["pitch"]]) > 5):
            reward = reward * 0.75
        elif(abs(observation[self.dictObservation["roll"]]) > 1 or abs(observation[self.dictObservation["pitch"]]) > 1):
            reward = reward * 0.9

        if(observation[self.dictObservation["alt"]] <= 1000):
            reward = reward * 0.1

        done = False
        if(self.getTermination(alt, alpha)):  # Would be used for end parameter - for example, if plane crahsed done, or if plane reached end done
            done = True
            reward = -1

        return reward, done

That the code checks if the agent is lower than altitude of 1000, and if it is it will set done to true and the episode will end and a new one will start. Meaning this should prevent crash from ever happening. I think (but I still recommend you test it out), that you can just comment crash and everything should be ok.

In my thesis you can also read a bit about it on pages 74, 75, 94, and 95.

Hope this helps, David

[ZF113120]

Thank you for your extremely helpful reply; I truly appreciate it! Wishing you happiness every day!

[JDatPNW]

You are very welcome @ZF113120 ! I hope your experiments will go well! If you need any further help let me know, and I hope I can help. Have a great day, best, David