-
Notifications
You must be signed in to change notification settings - Fork 0
Reverse Engineering Structure
File structures are very important; they give ability to parse TDU files and read/write contents. We'll see below how structures do look like and how to create/improve them.
This JSON file describes known structure of files used by TDU, here is an adapted sample:
{
"name": "TDU BTRQ (BinaryTorque?)",
"fileNamePattern": "*.btrq",
"comment": "A global comment about file structure...",
"littleEndian": true,
"cryptoMode": 1,
"fields": [
{
"name": "entryCount",
"type": "INTEGER",
"signed": true,
"size": 4
},
{
"name": "gap1",
"type": "GAP",
"size": 4
},
{
"name" : "graph",
"type" : "REPEATER",
"size" : "?entryCount?-1",
"subFields" : [
{
"name": "torqueNm",
"type": "FPOINT",
"size": 4
},
{
"name": "engineSpeed",
"type": "FPOINT",
"size": 4
}
]
},
{
"name": "gap2",
"type": "GAP",
"size": 4
},
{
"name": "optional",
"comment": "only present when 10 entries...",
"type": "UNKNOWN",
"size": 8,
"condition": "?entryCount?=10"
},
{
"name": "rest",
"type": "UNKNOWN"
}
]
}- name : to identify file you want to decode
- (optional, TDUF 2.0+) fileNamePattern : used to determine automatically to which file this structure applies to
- littleEndian : true/false, indicates byte order - most of TDU files are little endian, btw
-
cryptoMode : in case of encrypted file, indicates the encryption mode used
- 0=Savegames,
- 1=Others (db, btrq...)
- fields : list of fields contained in this TDU file (see Field reference below)
- (optional, TDUF 2.0+) comment: free text (surrounded by double quotes ofc), to help with understanding, debugging, etc. Does not perform any particular operation.
Note that cryptoMode set to 0 or 1 will make file to be automatically encrypted/decrypted by FileTool, so you won't have to worry about it.
- name : give clearest meaning as possible - must be unique !
- type : one of predefined values indicating data kind (see Field types below)
- signed : true if value should be signed, false otherwise. When absent, value is considered as unsigned.
- size : number of bytes used by this field / items count in a REPEATER (can be formula - see Size tricks below)
- contentsSize : number of bytes used by all fields in a REPEATER (can be formula - see Size tricks below)
- subFields : list of repeated fields (used when field type is a REPEATER)
- constantValue : value of this field (will always be the same) - with hex format 0x[AA AA ...]
- (optional, TDUF 2.0+) comment: free text like top-level section above
- (optional, TDUF 2.0+) condition: determines if current field should appear or not (see Conditions below)
- (optional, TDUF 2.0+) linkSource: true meaning that current field value is address to another field (see References below), only applicable to an INTEGER
- (optional, TDUF 2.0+) linkTarget: true meaning that each repeated value is pointed by a link source (see References below), only applicable to a REPEATER.
- TEXT : clear text data (e.g.: MAP4)
- INTEGER : an integer, numeric value (e.g.: 53)
- FPOINT : a floating-point, numeric value (e.g.: 235,6547)
- GAP : an area filled with 0's
- CONSTANT : a value which will always be the same
- UNKNOWN : to be discovered still
- REPEATER : embeds a list of complex values (with their own layout)
Here is example of CONSTANT field def:
{
"name": "constant",
"type": "CONSTANT",
"constantValue": "0x[10 BE AD 86]",
},
...is basically the number of bytes used by corresponding field, but with some exceptions:
-
When UNKNOWN field type, it is not mandatory - if omitted, the field is expected to occupy all remaining bytes in file.
-
When REPEATER, its meaning is the count of repeated items. It's not mandatory either - if omitted, the sub-items are expected to be repeated till the end of the file.
-
When INTEGER field type, only supported values are 1 or 2 or 4.
-
When FPOINT field type, values 2 or 4 are supported.
-
When CONSTANT field type, value is not used, thus relying only on number of hex items in constantValue information.
-
Value of this attribute may also be a formula:
"size": "=(2+3)*6"
and will be interpreted as 30.
Moreover, a formula may contain a reference to one (or more) previously read fields:
"size": "=5*?fieldName?"
will be interpreted as 30, as long a field named fieldName has been read before, having a value equal to 6.
Please note:
-
Field reference writes itself as field name declared earlier in structure, surrounded by question marks characters (?)
-
Concerning repeated fields, they get unique name each, following:
repeaterFieldName[itemRank].repeatedFieldName
e.g
?itemList[5].speed?
will reference speed field, repeated under itemList field, at rank 6 (as first item is always at rank 0).
-
Also, it is possible to reference a repeated field with the same rank without giving the full name:
?speed?
will reference speed field, repeated under the same field as current one and as same rank.
... is reserved to REPEATER fields only, its meaning is the size of repeated items, in bytes. It may be a formula, as size attribute above.
TDUF will stop parsing sub-items of a given repeater when given amount of bytes is reached.
Note that size and contentsSize attributes may be used altogether, processing will check conditions in following order, size reached, otherwise contentsSize reached.
Available from TDUF 2.0
Allows to consider some fields only when given condition is satisfied. It's particularly useful when file structure is dynamic, depending on value of one or many fields.
Alot like formulas seen above, condition value contains a logical operator between 2 numerical values, which can be field references as well. For example, if we know that some field is present only when a previously read field is set to 1:
...
{
"name": "aFlag",
"type": "INTEGER",
"size": 1,
},
{
"name": "aConditionedField",
"type": "INTEGER",
"size": 4,
"condition": "?aFlag?=1",
},
...Following operators are supported:
- a < b: strictly lower than
- a > b: strictly higher than
- a = b: equal.
Available from TDUF 2.0
Handles address pointers in files: some fields may refer to particular location in same file, expressed with numeric value. That's the address of another item, in bytes.
In this article, we will identify those particular fields this way:
- link source is a field pointing to another field, at a particular address in bytes (from beginning of parsed file)
- link target is field pointed by link source. It's assumed to be an item within a repeater.
For now, TDUF supports only link sources as INTEGER fields (providing address to targets) and link targets as REPEATER fields.
Given the following structure extract:
{
"fields": [
{
"name": "linkSource",
"type": "INTEGER",
"size": 4,
"isLinkSource": true
},
...
{
"name": "linkedEntries",
"type": "REPEATER",
"size": 2,
"isLinkTarget": true,
"subFields": [
{
"name": "linkTarget",
"type": "INTEGER",
"size": 4
},
...
]
}
]
}When parsing file matching above structure, TDUF will create associations between isLinkSource tagged field and isLinkTarget tagged repeater field, when adresses actually match.
Once parsing done and JSON file generated, following contents will be observed:
{
"linkSource": 10,
"linkedEntries": [
{
"linkTarget": 100
},
{
"linkTarget": 200
}
],
"#links#": [
{
"sourceKey": "linkSource1",
"targetKey": "linkedEntries[0]."
},
...
]
}A special section, #links#, will appear in generated file. It lists all matched links while parsing. In above example, targetKey points to any repeater item which has been targeted.
To manually go to a targeted item, you have to find in same JSON file the repeater array at targetKey location. Here, the target for linkSource1 is in the first item (0) from linkedEntries array. To help you doing so, each repeater item has a #meta# section with accessKeyPrefix value; this particular field gives key associated to current item.