New nodeMCU ChipID already in use since 2018

[JanBosNL]

For the project https://www.arnhemspeil.nl/acties/luchtdata-project.html in Arnhem the Netherlands we are in the proces of rolling out 200 new luftdaten sensors.

During one of the workshops we encountered a problem.
Brand new freshly bought NodeMCU,s are being used.
While trying to connect the chipID of this new Sensor to Meine Luftdaten, we discovered that the ChipID from this new NodeMCU is already in use since 2018. It seems there are 2x esp8266-1342840

Is this an isue that happened before? And how can we still use this NodeMCU for this project? Has espressif manufactured so many ESP8266 chips that they have started re-using the chipid?

The chip ID in question is 1342840

https://www.madavi.de/sensor/graph.php?sensor=esp8266-1342840-sds011

Hope somebody has an idear so we can stilluse the NodeMCU.

Best Regards.

[RikDrabs]

For the project https://www.luchtpijp.be in Brussels - Belgium we are building & following up the installation problems from about 2500 sensors in the Brussels & Flanders region of Belgium. A similar problem appeared on at least 10 sensors up to now, and can be understood by users registering a faulty chip-id on meine.luftdaten.info. If you register a chip-id you don't own yourself, then the user in possession of that chip-id cannot register it for himself, but receives the message "This sensor is already registered". The registering portal has no protection against this sort of mishaps. And this also protects against intentionally flooding the registration database with blocking, but non-existing chip-id's. In the past I already proposed to double-check the registering process, to check the sensor with that chip-id, first that it be non-registered, and at the same time must be sending data, for example by looking at madavi.de. This way you can't register a chip-id of a "future" sensor still in the box at another user. As long as this is not added to the registering process, these kind of mishaps will continue to happen, and a gaping hole for abuse will continue to exist. I already proposed to do the modification myself, but the source program for the registering portal is not my thing. Groetjes van Rik aan de Hollandse collega's ! Op di 14 jan. 2020 om 12:13 schreef JanBosNL <[email protected]>:

…

For the project https://www.arnhemspeil.nl/acties/luchtdata-project.html in Arnhem the Netherlands we are in the proces of rolling out 200 new luftdaten sensors. During one of the workshops we encountered a problem. Brand new freshly bought NodeMCU,s are being used. While trying to connect the chipID of this new Sensor to Meine Luftdaten, we discovered that the ChipID from this new NodeMCU is already in use since 2018. It seems there are 2x esp8266-1342840 Is this an isue that happened before? And how can we still use this NodeMCU for this project? Has espressif manufactured so many ESP8266 chips that they have started re-using the chipid? The chip ID in question is 1342840 https://www.madavi.de/sensor/graph.php?sensor=esp8266-1342840-sds011 Hope somebody has an idear so we can stilluse the NodeMCU. Best Regards. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#633?email_source=notifications&email_token=AJLIM7DJFHM2QYLFJZ3CX43Q5WM55A5CNFSM4KGRG5OKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IGAVHXQ>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJLIM7CWE5UQV37DDPZHS4LQ5WM55ANCNFSM4KGRG5OA> .

-- Rik Drabs Vrijwilliger Luchtpijp project van CM / beweging.net

[JanBosNL]

Hello @RikDrabs ,

The ChipID and esp https://www.madavi.de/sensor/graph.php?sensor=esp8266-1342840-sds011 has produced data.

It seems we have the same chipID , and only thing I can think of is that the producer of the ESP8266 might have started using the same numbers for a second run. They might have produced 10 milion esp8266 units and restarted at 0000001 etc....

If that is so more people will have the same problem in near future. (lets hope not) but I cant realy think of any other cause of this broblem. Since the ChipID has been producing data since 2018.

[dirkmueller]

@JanBosNL can you share a screenshot of the device status page? The espid is derived from the lower bytes of the mac address.

It is possible that we get duplicates because the prefix is actually changed

[dirkmueller]

We might have to change the espid calculation then

[JanBosNL]

[dirkmueller]

Yep, so that's a different mac vendor prefix than the usual one. It is registered in espressif inc so we have a valid collision simply because we discard the first 3 bytes of the mac address for the espid calculation.

So we need to get to a different calculation scheme that is less prone to collisions

[dirkmueller]

This could be fixed server side by grandfathering in the old ids somehow into a larger id space. Gonna be painful to get this correct though.

[dirkmueller]

@JanBosNL where did you get such an old firmware version? It's not relevant for this issue but current is 2020 not 2018 (which is two years old)

[JanBosNL]

I use this luftdaten flashingtool. And flash the dutch version to the nodeMCU, maybe the latest dutch version is that old?

[JanBosNL]

@dirkmueller Glad I could help clarify and find the bug. Lets hope we dont get to many collisions in the other nodeMCU's we are still building and rolling out. We are working on 200+ in the coming weeks and rolled out 80+ so far and this is the first that collided.

I hope the rest will just evade this bug.

[dirkmueller]

No, all language versions are build and updated at the same time. Note you're running the flashing tool from 2018!

I would really appreciate if we could flash be sensors with a current version of firmware. It just avoids a few bugs that have been fixed a long time ago (plus allowing more features)

[JanBosNL]

Will do, Autoupdate will fix that when the sensors reach the web I hope? Will download new flashing tool.

[JanBosNL]

The download link on https://luftdaten.info/feinstaubsensor-bauen/#firmware-einspielen links to the old 2018 firmware flashing tool. So thats why I still had that one running.

[JanBosNL]

Seems there is a bug in the flashing tool? The latest from https://github.com/opendata-stuttgart/airrohr-firmware-flasher is from 2018... And is what I used. Seems it flashes old firmware when using the dutch option? Maybe you can verify by flashing a nodeMCU with the

latest_DHT22_nl.bin

[ricki-z]

@JanBosNL Use the latest_nl.bin . This is preconfigured for SDS011 and DHT22 and it's the latest release.

[dirkmueller]

This is being discussed in Arduino Core: esp8266/Arduino#2309

[JanBosNL]

Last Weekend we rolled out another 25 sensors. And we found another chipID problem
We have another double chipID for: 8013503

This one Seems more like the problem @RikDrabs mentioned about people entering the wrong chipID nr on the luftdaten website?
The ChipID can be found in the MADAVI database, but I cant find any generated data for that specific chipID. Have a look for yourself through MADAVI https://www.madavi.de/sensor/graph.php?sensor=esp8266-8013503-sds011#l_year

Is there a way to get this ChipID reset in the database? So we can still use this nodeMCU?

For the rest @dirkmueller thanks for pointing out the root cause of the problem. I now see where the bug originates. Lets hope someone can come up with a solution that wont interfere with the already running nodeMCU's and functions based on this chipID calculated from the MACaddress. If I understand correctly the problem is how the chipID is calculated from the MACaddress. In the current situation The espressif mac adress prefix is skipped and only the latter part of the mac adress is used. Seems Espressif has baked so many esp8266 chips that they needed a new prefix, and started rerunning the rest of the macaddresses.

Best regards,

[ricki-z]

The chipID 8013503 was registered on Jan. 18 2020 for an account in the Netherlands (Arnhem).
Are you sure that it wasn't just a "double click" (second time sending the form because of a slow page load) that caused the message?

[RikDrabs]

Thanks for all the effort spent in upgrading and debugging the firmware of the the sensor. I'm receiving all your messages, and i'm realising the amount of work that is done ... Up until now i only saw chip-id's with 6, 7 or 8 digits, in a batch of over 3000 MCU's. As a solution to the calculation of the chip-id I propose the following: with the "normal" prefix, the calculation should produce the same resulting chip-id as before, with 6, 7 or 8 digits; with a different prefix the number of digits must be 5 or less, or 9 or more. The calculation for the other prefixes must of course take into account the full MACaddress. In that way the "normal" prefix chip-id's don't change, while for the other prefixes the resulting chip-id does not conflict with the existing ones ... May I insist on the solution of the problem I mentioned before: users entering by accident the wrong chip-id in the registration database -> the solution there is to check the entered chip-id, if it is not yet registered (already done), and at the same time sending data to madavi (not yet implemented), before it is allowed to be registered ? Best regards. Op ma 20 jan. 2020 om 12:58 schreef JanBosNL <[email protected]>:

…

Last Weekend we rolled out another 25 sensors. And we found another chipID problem We have another double chipID for: 8013503 This one Seems more like the problem @RikDrabs <https://github.com/RikDrabs> mentioned about people entering the wrong chipID nr on the luftdaten website? The ChipID can be found in the MADAVI database, but I cant find any generated data for that specific chipID. Have a look for yourself through MADAVI https://www.madavi.de/sensor/graph.php?sensor=esp8266-8013503-sds011#l_year Is there a way to get this ChipID reset in the database? So we can still use this nodeMCU? For the rest @dirkmueller <https://github.com/dirkmueller> thanks for pointing out the root cause of the problem. I now see where the bug originates. Lets hope someone can come up with a solution that wont interfere with the already running nodeMCU's and functions based on this chipID calculated from the MACaddress. If I understand correctly the problem is how the chipID is calculated from the MACaddress. In the current situation The espressif mac adress prefix is skipped and only the latter part of the mac adress is used. Seems Espressif has baked so many esp8266 chips that they needed a new prefix, and started rerunning the rest of the macaddresses. Best regards, — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#633?email_source=notifications&email_token=AJLIM7A25FKATIEHYX2P5OLQ6WGV3A5CNFSM4KGRG5OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJMMLPY#issuecomment-576243135>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJLIM7CWJPDAY4TTGPGFFZTQ6WGV3ANCNFSM4KGRG5OA> .

-- Rik Drabs Vrijwilliger Luchtpijp project van CM / beweging.net

[JanBosNL]

Hello @ricki-z it seems that someone tried to register the NodeMCU 2 times during our workshop, Thanks for clarifying. So we took this Sensor back. We have a new problem then... Seems that sensor has been connected to someone their personal account during our workshop.

But no idear who, I will ask the people that actually did the workshop if they kept track of whom origionally received this sensor and if we can mail them to hand over the sensor to us.. Lets hope we can trace it back internally.

Thanks for the help and clarification.

[JanBosNL]

The chipID 8013503 was registered on Jan. 18 2020 for an account in the Netherlands (Arnhem).
Are you sure that it wasn't just a "double click" (second time sending the form because of a slow page load) that caused the message?

@ricki-z Is there a madavi link to check the date of first registration that we can use during the workshop to check this the next workshops may we encounter people trying to double register their sensor?

[ricki-z]

There is no link to check this. We could try to insert this in the error message. But this may need some days. We haven't worked on the source code for my.luftdaten.info for some days ...