Aerial Robotics Testbed » telekyb3 » maneuver-genom3

While we looked into detail into those crash. We are now pretty sure that it comes from maneuver and the kdtp trajectory generation that cannot follow multiple velocity sudden variation.

As shown in the figure angular_velocity.fig, we noticed as well a non-continuation into the yaw velocity conversion caused by ** if (fabs(dyaw) < 0.25) {
dqw = 1 - dyaw2/8; / cos(dyaw/2) ± 1e-5 /
dqz = (0.5 - dyaw2/48) * dyaw; / sin(dyaw/2) ± 1e-6 */
}. Though we do not think it is the cause of our main issue, we did not understand what is done here ?

I know that you do not use maneuver::velocity at Laas but could you look into this to give your opinion about this issue please ?

Thank you very much

On Thursday 25 Oct 2018, at 19:15, Pol MORDEL wrote:

As shown in the figure angular_velocity.fig, we noticed as well a
non-continuation into the yaw velocity conversion caused by ** if
(fabs(dyaw) < 0.25) {
dqw = 1 - dyaw2/8; / cos(dyaw/2) ± 1e-5 /
dqz = (0.5 - dyaw2/48) * dyaw; / sin(dyaw/2) ± 1e-6 */
}. Though we do not think it is the cause of our main issue, we
did not understand what is done here ?

I don't see a discontinuity in the logs, the yaw, wz, dwz and ddwz
seem consistent to me at first sight.
What are you sending for wz and duration during this phase?

What is done in the code excerpt above is the computation of the Δq
induced by the wz, dwz and ddwz of the trajectory, for small angle
approximation. The Δq is then added to the current yaw. As the comment
suggest, this is precise up to 1e-5 and should not introduce big
errors compared to using the proper trigonometric functions. It can be
confirmed by looking at the resulting yaw in the plot, which is
visually not "crazy".

On Thursday 25 Oct 2018, at 19:12, Pol MORDEL wrote:

While we looked into detail into those crash. We are now pretty sure
that it comes from maneuver and the kdtp trajectory generation that
cannot follow multiple velocity sudden variation.

Do you have a log file of the velocity commands you send ?
Or a small sequence that triggers the crash?

Anthony Mallet wrote:

On Thursday 25 Oct 2018, at 19:15, Pol MORDEL wrote:

As shown in the figure angular_velocity.fig, we noticed as well a
non-continuation into the yaw velocity conversion caused by ** if
(fabs(dyaw) < 0.25) {
dqw = 1 - dyaw2/8; / cos(dyaw/2) ± 1e-5 /
dqz = (0.5 - dyaw2/48) * dyaw; / sin(dyaw/2) ± 1e-6 */
}. Though we do not think it is the cause of our main issue, we
did not understand what is done here ?

I don't see a discontinuity in the logs, the yaw, wz, dwz and ddwz
seem consistent to me at first sight.
What are you sending for wz and duration during this phase?

We send to wz a value between -0.62 and 0.62 (duration is 1)

what prompted us about this would be the comment since if cos(dyaw/2) ± 1e-5, sin(dyaw/2) should be 1

What is done in the code excerpt above is the computation of the Δq
induced by the wz, dwz and ddwz of the trajectory, for small angle
approximation. The Δq is then added to the current yaw. As the comment
suggest, this is precise up to 1e-5 and should not introduce big
errors compared to using the proper trigonometric functions. It can be
confirmed by looking at the resulting yaw in the plot, which is
visually not "crazy".

Ok my bad, I did check the math and there is no discontinuity indeed. Thank you for your explanation

Anthony Mallet wrote:

On Thursday 25 Oct 2018, at 19:12, Pol MORDEL wrote:

While we looked into detail into those crash. We are now pretty sure
that it comes from maneuver and the kdtp trajectory generation that
cannot follow multiple velocity sudden variation.

Do you have a log file of the velocity commands you send ?
Or a small sequence that triggers the crash?

Here is a second set of data with the log and figure of the velocity command : https://filesender.renater.fr/?s=download&token=a7846323-cb15-eb00-3651-d97283601f14 (do not use the mk2 data I copied those value by error from a previous test). The command remains between the bounds (vx, vy, vz goes between -0.45 and +0.45). In order to repeat the crash we then switch the command repeatedly between cmd_vel_max and cmd_vel_min until the drone start to diverge. Then maneuver does not respond to the command that is 0,0,0,0 (I can send you the video of the simulation as well if you want)

On Friday 26 Oct 2018, at 15:18, Pol MORDEL wrote:

The command remains between the bounds (vx, vy, vz goes between
-0.45 and +0.45). In order to repeat the crash we then switch the
command repeatedly between cmd_vel_max and cmd_vel_min until the
drone start to diverge. Then maneuver does not respond to the
command that is 0,0,0,0 (I can send you the video of the simulation
as well if you want)

So far I could not reproduce any similar crash (in simulation),
sending alternating -0.5 and +0.5 x,y velocities at various
frequencies, up to 500Hz.

So I have a few additional questions/remarks:

• When you say "command that is 0,0,0,0", what does it mean?
  maneuveur::velocity expects vx, vy, vz, wz, ax, ay, az, duration.
  I imagine that 0,0,0,0 mean 0 velocity, but what about acceleration
  and duration?

• In general, when you use a duration > 0, maneuver::velocity may
  reply with an error that this is not feasable given current state
  and velocity (and acceleration, jerk etc.) limits.
  Using a duration of 0 is safer from this point of view, and also
  gives you the "minimum time trajectory" to reach the desired
  velocity/acceleration.

In your last logs, the last maneuver acceleration is not 0. This
means that a maneuver::velocity command with 0 is not taken into
account. Did you check that the 'duration' is OK with the limits (or
0) ?

• What do you mean by "maneuver does not respond"? Does it crash or
  block? Any error message?

• Do you have nhfc error messages about "emergency descent" ?

• I just meant that it does not repond to the input of maneuver::velocity anymore. Maneuver does not crash nor give any error messages

• I used to set the duration to 1. Passing this parameter to 0 did indeed improve the behavior and i was not able to reproduce the crash in simulation.

• Howerver, while testing this setup using three quadrotors (and duration to 0) in the same simulation, after about one minute the simulation stopped and I get the following error message :

...
samples 202
8.7e-05
samples 202
0.000219
16:39:17.118333:genomixd: sock17cbc90: reply status 400: {"ex":"::genom::mwerr","detail":{"what":"S_csLib_TOO_MANY_RQST_IDS"}}
16:39:17.804482:genomixd: sock1819cb0: reply status 400: {"ex":"::genom::mwerr","detail":{"what":"S_csLib_TOO_MANY_RQST_IDS"}}
samples 56
7.5e-05
samples 56
6.9e-05

"samples" comes form printf in maneuver (maybe we could remove it since it was meant for debug I guess ?) but the TOO_MANY_RQST_IDS error is new.

The TOO_MANY_RQST_IDS error is produced by the genomix request of maneuver/velocity :

An error occurred while running the simulation and the simulation was terminated
Caused by:
Error reported by S-function 'genomix_block' in 'velocity_maneuver_2017a/mkQuadro1/genomix request':
{"ex":"::genom::mwerr","detail":{"what":"S_csLib_TOO_MANY_RQST_IDS"}}

• in one of the test, before getting the previous error, I received the following message as well but the program was able to recover :
  maneuver-pocolibs: kdtp::Spline::a_b: unknown case 2200 (vC = -nan)

And thank very much you for your time !

I noticed that the TOO_MANY_RQST_IDS error occurs eveytime i send a yaw velocity (wz) input (I am trying to understand what is going on)

--> after about 10 tests using 1 uav, I can reproduce the bug every time I try to send wz a value at 0.62 for more than 1-2 seconds
--> I did test it with wz max input at 0.3, same issue
--> the behavior is that the quadrotor spins well around z axis, then stop suddently, then spins fast around z axis
--> I need to restart the maneuver-pocoilbs component every time

On Monday 29 Oct 2018, at 16:51, Pol MORDEL wrote:

{"ex":"::genom::mwerr","detail":{"what":"S_csLib_TOO_MANY_RQST_IDS"}}
16:39:17.804482:genomixd: sock1819cb0: reply status 400: {"ex":"::genom::mwerr","detail":{"what":"S_csLib_TOO_MANY_RQST_IDS"}}
samples 56 7.5e-05 samples 56 6.9e-05

This is a completly different issue: this is because genomixd does
not process the replies to previous requests quickly enough to empty
its mailbox.

[ in a few words: the requests sent via simulink are sent in "oneway"
mode, meaning that the request reply is not sent back to simulink, to
save time - however, genomixd still receives and drops it, not quickly
enough in your case ]

There is not easy workaround right now. Your best bet would be to run
different genomixd for different quads and/or (if it makes sense)
lower the frequency at which you send velocities requests to
maneuver. I will try to think of something more clever.

"samples" comes form printf in maneuver (maybe we could remove it
since it was meant for debug I guess ?) but the TOO_MANY_RQST_IDS
error is new.

Yes, you can definitely remove the printf, it slipped in by mistake,
it's was just for debug/development.

maneuver-pocolibs: kdtp::Spline::a_b: unknown case 2200 (vC = -nan)

That is a bug in libkdtp, then. If it happens only once in a while,
it's not too bad I guess, but this should be investigated. It's most
probably due to numerical roundings when your new target velocity is
too close from the current one or something like this.