DTCO 4.0 Structure ?

[cbrede]

Hi,
from 14 JUN 2019 new Tacho will be install in the trucks. There also will
new drivercard. Has some already some test data? How will be the implemtation?

BR

Carsten

[Bobertin89]

Hi,
I try to use the program with a new V1B file but I've got this error :
"First unrecognised region with magic 0 at 0x0000"
"There were 26765 unmatched regions (magics) in the file."
I attach the file in this post.

newDTCO.zip

Can you tell me if you have an issue?

BR
Sylvain

[jugglingcats]

I assume V1B is a different file format, ie. not DDD. If it's a different format entirely (as opposed to just a DDD format with a different extension in the filename), this project doesn't currently support it.

[Bobertin89]

Hi,
thank you for your reply.
I thought it was the same format with different extension for different country.
With my tachygraph, I can take the file with TGD, DDD or V1B format.
I take the files with the three extensions and they are strictely identical.

I have some trucks with the old file format (DTCO3) and some trucks with the new file format (DTCO4, since june 2019).
When I use your application with the V1B or DDD extensions, it work with the DTCO3 and I've the same result.

I attach the file with the three format in dtco3 in a zip and with the three format in dtco4 in other zip.

dtco3.zip
dtco4.zip

Thank you in advance for your Reply!
BR

[jugglingcats]

Hi @Bobertin89, I understand thank you. Unfortunately I don't know anything about the new standard or the effort required to implement it. Perhaps it's something @davispuh has looked at.

[davispuh]

Haven't really looked into it, but most structures in gen2 is different with extra fields and signature validation is completely different.

[Bobertin89]

Hi all,
thanks for reply.
In your project, the two xml configuration file appears to contain the file structure. In order to read the new file, if I make an other xml config file with the new additionnal field, it should' be work?
I don't see anything for the identifier...
I find this documentation https://dtc.jrc.ec.europa.eu/iot_doc/EU%202016-799-EN.pdf that seems to contain the structure of the new file, but not user friendly...
Do you know if an other "user friendly" documentation exist?
Sorry for my poor english ^^

[davispuh]

if I make an other xml config file with the new additionnal field, it should' be work?

Most likely it won't be that easy, but you can try.

I find this documentation https://dtc.jrc.ec.europa.eu/iot_doc/EU%202016-799-EN.pdf
Do you know if an other "user friendly" documentation exist?

That's the most user friendly you'll find, I've been going over it for ages...
Also it's not even latest, here's latest one with amendments http://data.europa.eu/eli/reg_impl/2016/799/2018-04-17

[jugglingcats]

Unfortunately these specs are not written to be developer friendly, which was my motivation for publishing this project in the first place (because the XML config file provides a good interpretation of the first digital spec!).

[Bobertin89]

Hi,
thank's for your reply.
I'm trying to make the xml config file for the gen2.
I think I understand the purpose.
However, I don't understand why some data are with the "global value" attributes and other are not.
Can you explain me why, and in addition, can you tell me more about the validation signature?

Best regards and thank you in advance!

[Sh0ckwav3]

Hey Guys,
Im trying to implemet your solution in my Office.
But my Problem is simmilar to @Bobertin89.
Some of our cars are Driving with the new TachoGraph and new DriverCards named G2 with the format 1C on it.
Someone of you already created a new XML Config file for those new Versions?
Or has another way, how to read the new Generation DDD-Files?

Ty verry much und Best regards.
Dominic

[jugglingcats]

Hi @Bobertin89, I don't really understand your comment about "global value". The XML config for a card (driver or vehicle) contains definitions for the elementary files defined by the gen 1 spec. These elementary files (think of them as sections) follow each other in the ddd file. Each one starts with a magic, and this selects which bit of the config will be used to interpret that section. For example, here is the config for the driver card "CardIccIdentification" elementary file. It has a magic of 0x0002.

	<ElementaryFile Name="CardIccIdentification" Identifier="0x0002" Unsigned="true">
		<UInt8 Name="ClockStop" Length="1"/>
		<ExtendedSerialNumber Name="CardExtendedSerialNumber"/>
		<SimpleString Name="CardApprovalNumber" Length="8"/>
		<UInt8 Name="CardPersonaliserId" Length="1"/>
		<Object Name="EmbedderIcAssemblerId">
			<SimpleString Name="CountryCode" Length="2" />
			<BCDString Name="ModuleEmbedder" Size="2" />
			<UInt8 Name="ManufacturerInformation" Length="1" />
		</Object>
		<UInt16 Name="IcIdentifier" Length="2"/>
	</ElementaryFile>

It is the responsibility of the config above to consume the entire elementary file. This is very important, because when this section is consumed we expect to put the read head at the exact start of the next elementary file, otherwise we will fail to read the correct magic for the next section and things will not work.

Within the config you can consume primitive values, such as UInt8 or UInt16 but there are also helpers to consume more complex structs in the spec, such as ExtendedSerialNumber. These helpers know how to read and output (in the XML) these specific bits of structure.

You can introduce further nesting in the XML using Object, which is just a convenient way of grouping properties together.

There are also facilities for reading repeated data from the elementary file given a count already encountered. If you need I can find examples of this. There are more esoteric facilities available too, such as reading from a cyclic buffer within the elementary file. Hopefully the gen 2 spec doesn't have these, but the feature is there!

I hope this gives a reasonable overview of how the config files work. I have no idea whether the capabilities provided are sophisticated enough to process the gen 2 spec. Does it even have the concept of elementary files with magics, because this is fundamental to the current way of processing.

And finally, @davispuh wrote the signature checking and I believe it is switched off by default.

[Sh0ckwav3]

I created a new ConfigFile, which contains now the C1 or Gen2 structure.
But the main problem is, that the Driver-DDD file contains some informations twice. One time in Gen1-Format and a second time in Gen2-Format, but with the extenden Informations.
So when i will read the Magic Identifier="0x0501" i will get more than one result in my XML file, this also happens by other Magics.
I hope i can modify the code, that this will not happen anymore.
If someone is familiar with this problem and wants to help me pls write me a message.

[jugglingcats]

The code to pick the region which is going to be used for processing (based on magic) is here:

https://github.com/jugglingcats/tachograph-reader/blob/master/src/DataFile.cs#L100

Clearly if the new standard has the same magic used more than once in the file there must be a way choose the correct region XML in the config to be used. I am not sure how this could/should be determined.

[nick-tn]

Does anybody knows why Gen2 files have Objects ID's like 0x7621, 0x7622, etc. Old files have 0x7601, 0x7602..?

I understand that it could specify Gen2 file but I can't find any document proving this idea. Moreover the document https://dtc.jrc.ec.europa.eu/iot_doc/EU%202016-799-EN.pdf describes that it must be 0x7601, 0x7602 because there's no difference for TREP in messages sending to old or new DTCO4.0 tachographs.

[WallHackUp01]

Greetings,

There is an other problem that i met. Gen 2 have new Elementary Files nammed DIR, ATR/INFO and EXTENDED_LENGHT. When the documentation redirect the user to TCS_145, TCS_146 and TCS_147 in order to complete the EF, I don't undersatnd what we really have to put in. Which type of data did i have to put in the EF ?

Thanks in advance.

[jugglingcats]

@WallHackUp01 it looks like the information needed is in this ISO spec: https://www.iso.org/standard/54550.html. Unfortunately it is not free 👎. And it is annoying because the EF is variable length so in order to skip it we need to know the structure.

[andreasringdal]

There is some information regarding the ISO spec available her
https://gnupg.org/ftp/specs/openpgp-card-2.1.pdf
Don't know if it is of any help.

[davispuh]

Does anybody knows why Gen2 files have Objects ID's like 0x7621, 0x7622, etc. Old files have 0x7601, 0x7602..?

I understand that it could specify Gen2 file but I can't find any document proving this idea. Moreover the document https://dtc.jrc.ec.europa.eu/iot_doc/EU%202016-799-EN.pdf describes that it must be 0x7601, 0x7602 because there's no difference for TREP in messages sending to old or new DTCO4.0 tachographs.

That's not true, it just looks like this isn't included in document you're looking at but if you look at latest one, there are described those TREP IDs.

http://data.europa.eu/eli/reg_impl/2016/799/2018-04-17

See

Appendix  7
DATA  DOWNLOADING  PROTOCOLS
2.2.2 Message types
2.2.2.9 Transfer Data Request (SID 36)

[nick-tn]

Thank you for pointing to the information! Now it is clear.

[patientzerouk]

I'm about to dive into this myself, and it would be really handy if others have something to start with.

Once/if I get this working, I intend to submit this back into the project. Hopefully others will as well!

Does anyone have a working example for GEN2 stuff?

[patientzerouk]

Hi All, Just posting a note to say I haven't forgotten about my GEN2 Stuff, but the holidays and work overtook me. I'll be back on it in the new year. Hope all is well with everyone.

[mpi-wl]

@WallHackUp01 it looks like the information needed is in this ISO spec: https://www.iso.org/standard/54550.html. Unfortunately it is not free 👎. And it is annoying because the EF is variable length so in order to skip it we need to know the structure.

It's not true.
All the information you need is here: https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1468399756621&uri=CELEX:32016R0799
amending: https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1524124831090&uri=CELEX:32018R0502

and here: https://dtc.jrc.ec.europa.eu/

The length of all EF is known. Also DIR, ATR / INFO (these EF do not appear in ddd files).

Other documentation needed is ASN.1, Elliptic curve cryptography (Brainpool, NIST), etc.

Does anyone have a working example for GEN2 stuff?

Yes :) (app is not completed): https://esmreader.tachosfera.eu/

[Sh0ckwav3]

@mpi-wl are you going to share the code for gen2 stuff?

[mpi-wl]

@Sh0ckwav3 my library is not available as open source.
But if you have a problem with 2G (or 1G) I will answer your questions.

[Sh0ckwav3]

Well so i will give it a second try...
I got the specification for Driver File:
https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1468399756621&uri=CELEX:32016R0799

but i cant find Specifications for Vehile, anyone got one for Vehicle DDD Files?

[mpi-wl]

The file structure of vu is in the same documentation. Search for "DDP_029".
This documentation has changes: https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1524124831090&uri=CELEX:32018R0502
e.g.
(ii) | the heading ‘Data structure generation 1’ is replaced by the following:
‘Data structure generation 1 (TREP 01 Hex)’;
(iii) | the heading “Data structure generation 2” is replaced by the following:
‘Data structure generation 2 (TREP 21 Hex)’;

The file structure from vu is differently described than the file structure from the driver card.
Here, to know the length of the fields you must search for specific data types in Appendix 1

[Sh0ckwav3]

@mpi-wl is our library as .net as well?
i added and edited now all settings for the new format for the driver cards and added regions to parse the data.
but i only get the gen 1 data all the gen 2 will not be read.
why is this, how is it possible to have 2 times an Identifier with the same code but different data behind?
how can i solve this, that i can get only the g1 or only the g2?

Ty for your help

[mpi-wl]

@mpi-wl is our library as .net as well?

JavaScript

why is this, how is it possible to have 2 times an Identifier with the same code but different data behind?

The structure of the ddd file for the driver card

AA BB CC DD EE XX XX ... XX XX

AA (or BB, CC, DD ... XX.) is 1 byte

AA BB - EF file id
CC (third byte) - 00: content of EF file for gen 1, 01: signature for gen 1, 02: content of EF file for gen 2, 03: EF signature for gen 2.

DD EE - length (data or signature)

XX XX ... XX XX - data or signature

Not all EF are signed (e.g. certificates)
The correct ddd file for the 2G driver card contains gen 1 and gen 2 data

Ddd file for the VU
76 01, 76 02, 76 03, 76 04, 76 05 - gen 1
76 21, 76 22, ​​76 23, 76 24, 76 25 - gen 2

The VU ddd file contains the data of gen 1 or gen 2

[Sh0ckwav3]

Well i have to ask again sorry, maybe im dumb or whatever.

So when i have a look at my DDD File in HexEdit, then i can see that Gen1 "CardVehiclesUsed" starts somewhere at "00004620" but the Identifier is "0505" (as meantioned in documentation), this would not match with the data i can see in this image

the same for gen2 ... it starts somewhere at 0000b380 and not 0505, there are only 2 entries but where can i find the correct identifier to jump to this point and read all the data ...

AA BB - EF file id
CC (third byte) - 00: content of EF file for gen 1, 01: signature for gen 1, 02: content of EF file for gen 2, 03: EF signature for gen 2.

I understand in this example that the gen 1 and gen 2 information are directly in line, unfortunately it does not look like this according to these pictures.

so if u have a tip for me how i can find only the gen 2 data or booth at the same time and pare it with the given code from this project ... it would help me a lot.
i understand what the code from this repo is doing and how it works, but i dont get it how to identify the correct offset for each starting region.

[mpi-wl]

Well i have to ask again sorry, maybe im dumb or whatever.

Don't worry :) it's not easy.

Everything is ok:
05 04 - Driver Activity Data
01 - means there is still a signature (gen 1)
00 80 - 0x0080 = 128 dec

After 128 bytes (containing the signature for 05 04) there is 05 05 - so it's ok

You can only find data for gen 2 by checking 3 byte.
You could also check the length or content of a given EF file and in case of an error it means that it is gen 2 but it is not an optimal solution.

Unfortunately, I don't know C# well and I can't help you with the code from this project. If I find some time, I can look at this code, but I don't know if I can help you.

[mpi-wl]

@mpi-wl
Hello, any chance that you can find your old java(android) code.
I have small problems with "performHashFile".

What is your problem?

[RazviPatcas]

@mpi-wl

Do you know if this converter works for the new tachograph files (SMART tachograph) ?

[mpi-wl]

@RazviPatcas
This project doesn't support 2G structure.

@AskmeKolaric
My android program is incomplete. When I was able to connect to the driver card and read one EF file, it stopped working.
I read the entire driver card in my other program (c++). Describe exactly what your problem is - maybe I can help.

[AskmeKolaric]

@mpi-wl
When the app red file on "05 04", when I have to make PERFORM HASH on that file, the app brake down on some driver cards. For some cards, work perfectly. I am trying to find a solution how to make an app to work on every card:) I am not exactly sure, what I have to test to find a solution for it. Thank you for your help.

[mpi-wl]

Do you read a 1G or 2G driver card?
For 2G, an algorithm selection has been added (SHA-1, SHA-256, SHA-384 and SHA-512).

Do you do it in the following order?

1. Select file
2. Perform Hash of File
3. Read Binary
4. Compute Digital Signature

Are readable cards and non-readable cards issued by the same authority?

What is the response of the Perform Hash of File (SW1, SW2) command when the error occurs?

[AskmeKolaric]

@mpi-wl
At the moment 1G. We use the same order in the app.
Non-readable cards are not issued by the same authority.
When I selcet file "0504" i get '9000' , after that goes, perfomHasha and i get "bulk response failed with status 80 and error: 1'

[mpi-wl]

Do you send commands to the card in a similar way?

I don't know if I can help you. Send me an email: [email protected]

[jugglingcats]

@mpi-wl, @AskmeKolaric, it seems you are having a conversation about different software? Let's keep the conversation here about tachograph-reader

[mpi-wl]

@jugglingcats
You're right.
The conversation is about reading a driver card to create a ddd file.
This conversation will continue via email.

Sorry.

[WallHackUp01]

Hi !
Did someone succeed to make a VehicleUnitData.config for G2 Vehicle file in here, or succeed to read a G2 vehicle file ? I tried, but impossible to have a decent documentation to clearly explain it, like the driver cards. Can someone send me this structure ? (or a perfect documentation)
Thanks in advance !

[martinlowinski]

@WallHackUp01 a colleague and I did some experiments and could read gen2 vehicle unit files to some degree. But we used Kaitai Struct for that, it's a declarative language that compiles to source files of many programming languages (JavaScript, C#, Python etc.).

We have used the tachograph-reader for quite some time and are very happy about it (thanks @jugglingcats!). So we tried to add support for gen2, as many of you tried as well, but haven't been successful. This is why we took a different direction and Kaitai Struct seems promising so far.

[rimutaka]

@jugglingcats Alfie, would you allow a fork of this repo as AGPL3 for v.2 development? I will be adding v.2 support and want to make sure it stays in FOSS.

[jugglingcats]

Hi @rimutaka, I think the AGPL is same as or perhaps more restrictive than the vanilla GPL. I imagine that most people using tachograph-reader are static linking it with or compiling it into their proprietary code of some form. Wouldn't making it AGPL discouraged almost all potential developers from using it?

Having said that you are perfectly at liberty to fork the code if you wish. Or we could think about changing the licence on this repo to LGPL which is more permissive...

[rimutaka]

@jugglingcats , let me clarify. I want to change the license for the fork only. However, any code created before that will be under the original license. That's complicated. What I'm asking is your permission to replace the original license in my v.2 fork with (A)GPL while retaining the reference to the original project + your copyright. Then any changes made to the fork will be automatically under (A)GPL without having to delineate which part of the v.2 code is under which license.

No pressure. I totally understand if you want to keep it as-is.

[jugglingcats]

Understood. I'm exploring whether you'd be happy to have a more permissive licence on this repo and make your changes here, in the interests of keeping everything in one place and avoid a fork. I'd be happy to make you a contributor so you have the ability to merge pull requests, etc. I'm not quite clear what your rationale is for wanting your changes to be AGPL, given it will prevent most people from using the code...

[rimutaka]

@jugglingcats ,

1. a fork is because you said in PR .Net5.0 upgrade #58 you are no longer involved. More than happy to keep it all in one repo, though.

2. AGPL is to make sure this type of conversation does not happen ...

What I see there is they piggy-backed on the open source and are not willing to contribute. Maybe I'm reading too much into it.

[davispuh]

I'm strongly against any *GPL and always prefer the least restrictive license as possible and MIT is very good :). That is true that some people may wish to build their own stuff on top of it and don't contribute anything back but that's their own choice maintaining their own fork which over time might be worse than this because it's a lot of work to merge/rebase changes especially if forks have diverged a lot.
Also this library is very tiny part of whole stack - download .ddd file from vehicle, parse/process .ddd file, save to DB, analytics/user friendly graphs/UI.
I can say that if this library was AGPL the company I worked, we just wouldn't have used it at all and there wouldn't be any contributions from our side

28 commits
26 files changed, 2562 insertions(+), 1405 deletions(-)

The problem with AGPL is that this is a library to be integrated with wider software stack and that would require open sourcing those other parts which is core business of company. I would say this library alone isn't that useful without rest of functionality. For example for .ddd file downloading from different vehicle manufacturers there weren't any library at all so we built everything on own from scratch and all proprietary. The default choice for most companies is implement from scratch themselves and keep it proprietary which is kinda stupid but that's how it usually is. So 10 different companies waste time on implementing same things for no reason. If there is some library there is chance some companies will use same library and some might even contribute back thus everyone benefits :P
So anyway current license does allow you to fork with more restrictive license but then you'll be pretty much on your own as I don't think if any company will even touch it.

[mpi-wl]

What I see there is they piggy-backed on the open source and are not willing to contribute. Maybe I'm reading too much into it.

@rimutaka

That's not true.
Read my post carefully. My code is in JavaScript and has nothing to do with this project.
I didn't use this project when I was creating my application.
Additionally, I support 2nd generation ddd files (also cryptography), and this project doesn't have that.

Even though I don't share my (and only mine) library as open source, I helped other people with their problems (2nd generation structure).

Your opinion is unfair and harmful.

[jugglingcats]

Let's calm this discussion down folks... ;)

@mpi-wl, I read your post and understood your code was not related to tachograph-reader in any way, and am sure people appreciate your contribution to the discussion / problem solving. @rimutaka can be forgiven for not picking up on this detail.

@rimutaka's concern is valid especially if he is planning to contribute significant enhancements.

@davispuh articulated my concern with (A)GPL perfectly. This code is going to end up embedded in bigger system and GPL licence doesn't work for any commercial company wanting to use this code in their closed source product.

Perhaps we could compromise on LGPL which would be more permissive. The company I work for is generally comfortable with LGPL code. Open to suggestions / further thoughts.

[rimutaka]

Thanks everyone for your input. I tend to agree that no license change is needed.

[jugglingcats]

Thanks @rimutaka if you are happy to contribute your changes to this repo I will give you contributor access. Perhaps you could work on a v2.x branch until it becomes stable

[jugglingcats]

@rimutaka I have made you collaborator - welcome!

I am going to close this issue because it's getting rather long and covers a few different threads of conversation. I'll create a new pinned issue giving high level on DTCO 4.0 support.

[rimutaka]

Thanks Alfie. I'll start by posting new issues for .net5 and other changes I made in that PR to make sure I'm not breaking it for everyone else. I won't be merging anything in a hurry.

[Phoryn]

Hello

I want to summarize and consult all information which I need to write update to gen2.

-The structures of bytes for the segments is the same like in gen1.
-In the segment is only one change, in new generation type can be also 0x02(content) and 0x03(sygnature).
-On the one card might be also types of gen1 and gen2

On the beggining I thought that I can compare length of data "application_Identification" becouse it is exacly 10 byte in gen1(pic.2) and 15 byte in gen2(pic.3) . After select generation, choosen config. But I found information that in the card can be mixed information (gen1 and gen2 segments).

*"segment" mean one block of data like "application_Identification"

*@Dilemma725 wrote a program which to be able to check wchich generation card is. In his code he checking first and second byte of all card byte data.
-if first 0x76 and second 0x06 - G1
-if first 0x00 and second 0x02 - G2
Why? First block of data is the same for both generation(pic1)

My question is:
Did the best way do update this application to generation 2 is getting each segment one by one and checking his type?

[tcepxxx]

@mpi-wl Is there somehow possible to contact you ? I want to ask few questions about project you were making,

[MRDTONY]

Hi！

I have some questions regarding the public key used internal authentication process.

1.How can I retrieve the public key from the card?
2.What is the relationship between the public key stored in the card and the RSA encryption algorithm?

I would greatly appreciate it if anyone could provide insights, guidance, or resources on these topics.