Image processing software » realsense-genom3

Is there a way I can publish a unified IMU also from
realsense-genom3?

It's not possible right now, but I can put this item on the TODO list
:)

I was asking because I am planning to use the IMU topic as input to OpenVINS https://docs.openvins.com/, which supports only a united IMU message.

Requiring the presence of a 6DOF IMU sounds like a surprising
shortcoming of openvins. Given the way Kalman filters work, it does
not really matter what you feed to the filter as measurements, and a
system might very well just have an accelerometer, for instance, or
like in the present situation different data rates for gyro and
accelerometer. Fiddling with the data that have different rates is not
that clean, as this requires a model of the motion which is just an
approximation. This is also redundant with the filter in pom that is
just meant for this. So even if there could be a small kalman in the
realsense component, I still find it better to have it export just the
raw data and let pom handle this. pom has no problem with sensors at
different rates, and that would just work.

Also, I only quickly went through the documentation of openvins, but
you're using the "feed_imu" API call, right? Indeed, I did not find
any other way to pass just the gyro or accelerometer data. But I did
not look that carefully yet.

In any case, I think openvins needs that data for the prediction of the
features, so it does not matter where the inertial data comes from.
And actually, it sounds better in terms of architecture to feed it
with the estimated state (angular vel. and acceleration) of pom, since
this would abstract where the inertial data comes from (it could be
the fusion of all other sensors, so not only IMU but also mocap or
GPS, or anything like multiple IMUs, a telemeter measuring the height
etc.).
You would then feed back the estimated velocity from the visual
odometry back to pom to further improve the state estimation.

This might look like a loop in the data, but I think in practice this
would work. Now, regarding the estimation theory, I'm not sure how to
formalize this.

Maybe this message is not that clear. I can suggest you to deal with
that topic during the next ART meeting (Thu 12). If you're interested,
I will add this to the agenda. Let me know.

Thanks, I think I understand what you mean but it could be good to discuss this in the meeting.
What time is it?

Ok, we can still discuss it here or in the chat anyway.
The meeting is on Thu 12 at 11am, on https://jitsi.laas.fr/art

OK, I will participate.

In the meanwhile, I have already started using POM to unify gyro and accel, but I get the warning:
state covariance is not positive definite ... fixing
which is probably due to avel_cov and acc_cov being identically zero.
Realsense SDK was instead publishing a 0.01*Identity matrix for both types of measurements. Does this choice make sense for the driver or shall I post-process the covariance? Shall we discuss this as well in the meeting?

realsense-genom3 is using
get_motion_intrinsics()
to retrieve the IMU variances.

It seems indeed to be all zeroes also for me. The following issue does
not make me confident on the ability of the above API to actually
report something useful:
https://github.com/IntelRealSense/librealsense/issues/8899

Note the hilarious response « variances found by other calibration
tools have "e-" something, so they can be considered as 0 ».
So ... I guess I have to add a service for setting them manually.

In the meantime, for testing further, you can change those lines

port->acc_cov._value.cov[0] = i.noise_variances[0];
port->acc_cov._value.cov[2] = i.noise_variances[1];
port->acc_cov._value.cov[5] = i.noise_variances[2];

port->acc_cov._value.cov[0] = 0.01;
port->acc_cov._value.cov[2] = 0.01;
port->acc_cov._value.cov[5] = 0.01;

Do the same for the gyro just below the above.