ICM20948 in HERE 2 is not detected

[krevne]

Describe the bug
The ICM20948 in the HERE 2 from ProfiCNC is not detected. WHOAMI value 0x00 is read.

To Reproduce
Steps to reproduce the behavior:

1. Connect HERE 2 to Pixhawk 2.1
2. Flash PX4 master
3. Boot flightcontroller
4. See warning WARN [mpu9250] no device on bus 2

I've added a few lines of debug code and have figured out that the WHOAMI value is returned as 0x00

/*
 * If it's not an MPU it must be an ICM
 * Make sure register bank 0 is selected - whoami is only present on bank 0, and that is
 * not sure e.g. if the device has rebooted without repowering the sensor
 */
write(ICMREG_20948_BANK_SEL, &register_select, 1);
read(ICMREG_20948_WHOAMI, &whoami, 1);
warnx("whoami returned %u", (unsigned)whoami);

FMUv2 ver 0xE : Rev 0 V30
[boot] Fault Log info File No 4 Length 3177 flags:0x01 state:1
[boot] Fault Log is Armed
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4_FMU_V3
HW type: V30
HW version: 0x0009000E
HW revision: 0x00000000
FW git-hash: 8a313ffc26985f1c81ea494dbcf20760ff3be59a
FW version: 1.9.0 0 (17367040)
FW git-branch: master
OS: NuttX
OS version: Release 7.22.0 (118882559)
OS git-hash: c225d4a691ef4572839b19635a84f2d9bf07ea74
Build datetime: Jan 15 2019 08:34:24
Build uri: localhost
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204]
PX4GUID: 0001000000003334353731375103003e0045
MCU: STM32F42x, rev. 3
[hardfault_log] Fault Log is Armed
INFO  [tune_control] Publishing standard tune 1
INFO  [dataman] Unknown restart, data manager file '/fs/microsd/dataman' size is 350216 bytes
INFO  [param] selected parameter default file /fs/mtd_params
rgbled on I2C bus 1 at 0x55 (bus: 100 KHz, max: 100 KHz)
WARN  [rgbled_ncp5623c] no RGB led on bus #2
nsh: rgbled_pwm: command not found
Board init: /etc/init.d/rc.board
WARN  [hmc5883] no device on bus 1 (type: 2)
WARN  [lis3mdl] no device on bus 2
INFO  [ist8310] no device on bus 1
INFO  [ist8310] no device on bus 5
INFO  [mpu9250] Bus probed: 2
WARN  [mpu9250] whoami returned 0
WARN  [mpu9250] no device on bus 2, init not OK
WARN  [hmc5883] no device on bus 2 (type: 1)
WARN  [mpu6000] no device on bus #3 (SPI1)
WARN  [mpu6000] no device on bus #5 (SPI4)
INFO  [mpu9250] Bus probed: 5
MPU9250 on SPI bus 4 at 4 (1000 KHz)
INFO  [mpu9250] accel cutoff set to 30.00 Hz
INFO  [mpu9250] gyro cutoff set to 80.00 Hz
L3GD20 on SPI bus 4 at 1 (11000 KHz)
LSM303D on SPI bus 4 at 2 (11000 KHz)
INFO  [mpu9250] Bus probed: 3
MPU9250 on SPI bus 1 at 4 (1000 KHz)
INFO  [mpu9250] accel cutoff set to 30.00 Hz
INFO  [mpu9250] gyro cutoff set to 80.00 Hz
MS5611_SPI on SPI bus 4 at 3 (20000 KHz)
MS5611_SPI on SPI bus 1 at 3 (20000 KHz)
WARN  [bst] no devices found
INFO  [mpu9250] Bus probed: 2
WARN  [mpu9250] whoami returned 0
WARN  [mpu9250] no device on bus 2, init not OK
px4flow [69:100]
INFO  [px4flow] scanning I2C buses for device..
INFO  [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
Starting Main GPS on /dev/ttyS3
Starting MAVLink on /dev/ttyS1
INFO  [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
No autostart ID found
INFO  [logger] logger started (mode=all)

NuttShell (NSH)
nsh>

[krevne]

I've also tried flashing Ardupilot firmware and there it's working.

AP_I2C_12 on I2C bus 1 at 0x0c (bus: 100 KHz, max: 100 KHz)
Found a AK09916 on 0xc09 as compass 0

[mhkabir]

@flochir

[flochir]

@krevne I don't have a Here 2 GNSS at hand so I can't reproduce.
Just a hunch - the ICM20948 can use 2 different I2C addresses. Can you try to change
PX4_I2C_EXT_ICM20948_1
to
PX4_I2C_EXT_ICM20948_0

in main.cpp line 139?

[krevne]

@flochir Unfortunately this didn't work.

[flochir]

@krevne @mhkabir Right now I have no guess why it wouldn't work. Took a glance at the Ardupilot driver, but I don't see what's different there up to probing the WHOAMI.

[krevne]

@proficnc

[flochir]

@krevne @mhkabir
I (temporarily) got my hands on a Here 2 GNSS.
ICM20948 itself does not respond on the I2C bus on any of its addresses, but a complete scan of the bus shows a "naked" magnetometer of the type built into ICM20948 (AK09916, Address 0x0C, WHOAMI1 0x72 on address 0x00, WHOAMI2 0x09 on address 0x01):

...
WARN  [mpu9250] Address: b
WARN  [mpu9250] Address: c
WARN  [mpu9250] Address 0 returned 72
WARN  [mpu9250] Address 1 returned 9
WARN  [mpu9250] Address 8 returned 72
WARN  [mpu9250] Address f returned
WARN  [mpu9250] Address 19 returned 9
WARN  [mpu9250] Address 1f returned 72
WARN  [mpu9250] Address 26 returned 9
WARN  [mpu9250] Address 2c returned 72
WARN  [mpu9250] Address: d
WARN  [mpu9250] Address: e
...

This explains why the Ardupilot driver works - it's only meant for the magnetometer and only configures ICM20948 to put the mag through on I2C. And it's able to work with a standalone AK09916, while the MPU9250 driver as of now is only meant to talk to a whole ICM20948 (but also support the whole IMU).

My guess is that the main I2C interface of ICM20948 in Here 2 is connected to the integrated microcontroller (which will handle the CAN interface later), while on the external I2C of the Pixhawk only the AK09916 is attached, connected through AUX_CL and AUX_DA of ICM20948.
@proficnc can you confirm?

This means that support for a standalone AK09916 is needed. Can you give some deeper PX4 driver philosophy on how best to do this @dagar @DanielePettenuzzo?
Personally I think making the AK09916 (and subsequently AK8963) driver independent of MPU9250 (a bit like in my original proposal for ICM20948 integration, #9345), then "remotely" use it in the MPU9250 driver might be the way to go.
Maybe AK09916 only support could be built into the MPU9250 driver on top of everything else. I'll see if its reasonable, but I don't quite like it.

Also not sure when I'll be able to pick that up, could take a bit. If anyone else does, please give me a note.

[jschall]

The Here 2 MCU acts as an I2C slave and emulates an AK09916 and a Toshiba LED.

So functionally yes, it shows up as a "naked" ak09916 on the I2C bus.

[flochir]

@jschall Thanks for the information!

[flochir]

@jschall Do you maybe have a link or document with more background information on the Here 2?

[jzazbert]

@flochir I found your PR adding ICM20948 support to PX4(#10116. Does the "mag only" mode allow communication to the HERE2's emulated AK09916 mag? Or is there still an addressing problem?

Do you know if there are plans to support this? We'd like to use the HERE2 and HERE2+ but not having the external mag is a problem.

Thanks!

[flochir]

@jzazbert The mag only does not support a standalone AK09916. It just does not register the accel/gyro sensors of an ICM20948 if you just want to use it as an external magnetometer, e.g. in the Here GNSS.

I want to try to create a standalone driver for AK09916 eventually, but I'm short on time for this right now. I don't know of anyone else working on this, either.

If you want you can take a look at #9345. I implemented a standalone AK09916 driver there that was mostly working, but not yet finished. We implemented ICM20948 support in the MPU9250 driver instead.

[AndreasAntener]

Anybody working on this? @flochir If you can point me to your previous impl for AK09916 I'll reserve some time to look into it.

[dagar]

Does anyone know the current state of CAN support on the Here 2? Going straight to CAN might be more worthwhile.

[jzazbert]

We have a working driver that Proficnc helped us write for HERE2, on a private repo. I'll share but I don't currently have time to do a PR for it. I'd be happy to provide to someone to add.

As for CAN support, there is engineering firmware available that has been provided to us that did work with PX4 though UAVCAN and the other cable supplied with HERE2. You'd need to contact proficnc to get it. We had to use an external CAN tool to update the firmware but then it worked with PX4. This was not extensively tested.

[proficnc]

What Jzazbert said...
The CAN driver will be available via mission planner shortly, at firmware.cubepilot.org
You can update it via a cube running ardupilot, then switch back to PX4

Please give a couple of days for the firmware to show up.

A PR for the i2c driver would be awesome!

[mhkabir]

@proficnc Does the Here2 not support the standard UAVCAN firmware update protocol?

[flochir]

@AndreasAntener
Looks like I removed the branch at some point. I re-pushed, this should have been the last version:
icm20948_old_approach_separate_mag_driver.

[AndreasAntener]

@flochir thanks!
@jzazbert sure, if you share it I planned to do this in 1.5 to 2 weeks. Maybe it makes sense to compare with @flochir's impl, he had the structure that I believe was first rejected but then wanted after all, am I correct? ;)

[jzazbert]

@AndreasAntener

I've uploaded our driver as the following commit, based off PX4 1.8.2 stable tag. We have probably 100 flight hours on this driver on 5 different aircraft and haven't had any issues to date.

jzazbert@b6498b6

Some caveats:

• I do not know if this passes the build tests for PX4
• I do not know if this is implemented in the "standard" way from an integration with the rest of the ecosystem. The startup for it was added to rc.sensors for our aircraft.
• It has some dummy IOCTL functions that simply return true, such as "MAGIOCSELFTEST", which should probably be implemented properly
• I am unsure of the orientation requirements. Our Cube has a YAW_270 orientation, and experimentally we had to set HERE2 to YAW_180 to make it correct.
• We do occasionally see spikes in the telemetry log similar to what we see on the Cube's mag0. Unsure why, but both are similar flavor mags (MPU9250 vs ICM20948)
• This driver works for HERE2 in i2c mode, but has not been tested with any other AK09916 sensors. The HERE2 is "emulating" the ak09916 in software, and is internally connected to the ICM20948, so there is a possibility that there might be some subtle differences in their implementation vs a standalone sensor.
• I didn't compile this specific commit, I copied it over from our local repos to share. It should work though...

Thanks for the support upstreaming this!

[dagar]

The other part of CAN support to check is the safety button and led.

[flochir]

@AndreasAntener looks like it ;)
Be aware that the old driver concept was working and delivering mag data, but we fixed quite a bit of problems and issues on the subsequent work on the mpu9250 driver. However most of them won't matter to the (much simpler) mag only driver. But the best way to go (to not loose ICM20948 full IMU support) would be to give the mpu9250 a standalone mag driver for AK09916 - which would then need to also support AK8963, or a separate driver for that, too.

@dagar Do you think it would make sense to port the mpu9250 driver to the new driver model? Would that potentially reduce the boilerplate code enough to make it worthwhile tearing it apart into an mpu9250, icm20948, mpu6500, ak09916 and ak8963 driver?

[proficnc]

@bugoblitorator
Can you please put in a PR for the patch for Here2?

[bugobliterator]

@flochir here is a working branch on top of current stable for your reference, https://github.com/bugobliterator/PX4Firmware/tree/pr-v1.8.2-ak09916 .

[AndreasAntener]

Thanks guys, on this now.

[AndreasAntener]

@proficnc what is the emulated orientation on the Here2?

[bys1123]

I'm using Pixhawk4, also get this message #11748

[dagar]

Support for the compass in the Here 2 GPS is now in master. 95a0803

[proficnc]

Thanks @AndreasAntener
Thanks @dagar
Thanks @bugobliterator

[jzazbert]

Was compass calibration tested without a HERE2 plugged in, but the driver enabled? I had some trouble on the earlier version of this driver where QGC would spit out an error about magnetometer 3081 not being found and aborting compass cal.

[hamishwillee]

As an aside ...

@proficnc If Here 2 is now fully supported, can we perhaps have a page like the one for Here+ - and is Here+ still available?