Merge pull request #42 from xoofx/elf-refactoring

ELF refactoring with breaking changes

doc/readme.md

@@ -2,44 +2,48 @@
 
 This is the manual of LibObjectFile with the following API covered:
 
-- [ELF Object File Format](#elf-object-file-format) via the `ElfObjectFile` API
+- [ELF Object File Format](#elf-object-file-format) via the `ElfFile` API
 - [Archive ar File Format](#archive-ar-file-format) via the `ArArchiveFile` API
+- [PE File Format](#pe-object-file-format) via the `PEFile` API
 
 ## ELF Object File Format
 
 ### Overview
 
-The main entry-point for reading/writing ELF Object file is the [`ElfObjectFile`](https://github.com/xoofx/LibObjectFile/blob/master/src/LibObjectFile/Elf/ElfObjectFile.cs) class.
+The main entry-point for reading/writing ELF Object file is the [`ElfFile`](https://github.com/xoofx/LibObjectFile/blob/master/src/LibObjectFile/Elf/ElfFile.cs) class.
 
 This class is the equivalent of the ELF Header and contains also sections and segments (program headers)
 
 ![ELF class diagram](elf_class_diagram.png)
 
 ### ELF Reading
 
-The ELF API allows to read from a `System.IO.Stream` via the method `ElfObjectFile.Read`:
+The ELF API allows to read from a `System.IO.Stream` via the method `ElfFile.Read`:
 
 ```C#
-ElfObjectFile elf = ElfObjectFile.Read(inputStream);
+ElfFile elf = ElfFile.Read(inputStream);
 ``` 
 
 ### ELF Writing
 
 You can create an ELF object in memory and save it to the disk:
 
 ```c#
-var elf = new ElfObjectFile();
+var elf = new ElfFile();
 
 var codeStream = new MemoryStream();
 codeStream.Write(Encoding.UTF8.GetBytes("This is a text"));
 codeStream.Position = 0;
 
 // Create a .text code section
 var codeSection = new ElfCustomSection(codeStream).ConfigureAs(ElfSectionSpecialType.Text);
-elf.AddSection(codeSection);
+elf.Add(codeSection);
 
 // Always required if sections are added
-elf.AddSection(new ElfSectionHeaderStringTable());
+elf.Add(new ElfSectionHeaderStringTable());
+
+// Always required if sections are added
+elf.Add(new ElfSectionHeaderTable());
 
 using var outputStream = File.OpenWrite("test.out");
 elf.Write(outputStream);
@@ -49,7 +53,7 @@ elf.Write(outputStream);
 
 #### Print
 
-You can print an object to a similar textual format than `readelf` by using the extension method `ElfObjectFile.Print(TextWriter)`:
+You can print an object to a similar textual format than `readelf` by using the extension method `ElfFile.Print(TextWriter)`:
 
 ```c#
 // Using the previous code to create an ELF with a code section
@@ -108,13 +112,13 @@ The `Print` is trying to follow `readelf` from as compiled on `Ubuntu 18.04`. It
 
 #### Section Header String Table
 
-When sections are added to an `ElfObjectFile.Sections`, it is required to store their names in a Section Header String Table (`.shstrtab`)
+When sections are added to an `ElfFile.Sections`, it is required to store their names in a Section Header String Table (`.shstrtab`)
 
 In that case, you need to add explicitly an `ElfSectionHeaderStringTable` to the sections:
 
 ```C#
 // Always required if sections are added
-elf.AddSection(new ElfSectionHeaderStringTable());
+elf.Add(new ElfSectionHeaderStringTable());
 ```
 
 This section can be put at any places in the sections, but is usually put at the end.
@@ -125,43 +129,43 @@ There is a type of section called `ElfShadowSection` which are only valid at run
 
 A shadow section is used by an `ElfSegment` for which a region of data might not be associated with an existing section. In that case, you still want to associate data with the segment.
 
-This is specially required when working with executable that don't have any sections but have only segments/program headers. In that case, `ElfObjectFile.Read` will create `ElfCustomShadowSection` for each part of the file that are being referenced by an `ElfSegment`.
+This is specially required when working with executable that don't have any sections but have only segments/program headers. In that case, `ElfFile.Read` will create `ElfCustomShadowSection` for each part of the file that are being referenced by an `ElfSegment`.
 
 #### Null section and Program Header Table
 
-The null section `ElfNullSection` must be put as the first section of an `ElfObjectFile`. It is the default when creating an `ElfObjectFile`.
+The null section `ElfNullSection` must be put as the first section of an `ElfFile`. It is the default when creating an `ElfFile`.
 
 The Program Header Table is implemented as the `ElfProgramHeaderTable` shadow section and is added right after the `NullSection`. This is required because a segment of type `PHDR` will reference it while it is not an actual section in the original ELF file.
 
 #### ELF Layout
 
-An `ElfObjectFile` represents an ELF Object File in memory that can be freely modified. Unlike its serialized version on the disk, the offsets and size of the sections and segments references can be changed dynamically.
+An `ElfFile` represents an ELF Object File in memory that can be freely modified. Unlike its serialized version on the disk, the offsets and size of the sections and segments references can be changed dynamically.
 
-You can force the computation of the layout of an ELF object file before saving it to the disk by using the method `ElfObjectFile.UpdateLayout`:
+You can force the computation of the layout of an ELF object file before saving it to the disk by using the method `ElfFile.UpdateLayout`:
 
 ```C#
-ElfObjectFile elf = ...;
+ElfFile elf = ...;
 
 // Update layout (ObjectFile.Layout, all offsets of Sections and Segments)
 elf.UpdateLayout();
 
 foreach(var section in elf.Sections)
 {
-    // Section.Offset is now calculated as it was on the disk
-    Console.WriteLine($"Section {section} Offset: 0x{section.Offset:x16}");
+    // Section.Position is now calculated as it was on the disk
+    Console.WriteLine($"Section {section} Offset: 0x{section.Position:x16}");
 }
 ```
 
 #### Diagnostics and verification
 
-An `ElfObjectFile` can be created in memory with an invalid configuration (e.g missing a link between a symbol table and a string table).
+An `ElfFile` can be created in memory with an invalid configuration (e.g missing a link between a symbol table and a string table).
 
-You can verify the validity of an `ElfObjectFile` by calling `ElfObjectFile.Verify`
+You can verify the validity of an `ElfFile` by calling `ElfFile.Verify`
 
 ```C#
-ElfObjectFile elf = ...;
+ElfFile elf = ...;
 
-// Verify the validity of an ElfObjectFile instance
+// Verify the validity of an ElfFile instance
 var diagnostics = elf.Verify();
 
 // If we have any errors, we can iterate on diagnostic messages
@@ -215,7 +219,7 @@ arFile.Write(outputStream);
 
 Although the example above is storing a text, one of the main usage of an `ar` archive is to store object-file format (e.g `ELF`)
 
-If you want to store direct an `ElfObjectFile` you can use the `ArElfFile` to add an ELF object-file directly to an archive.
+If you want to store direct an `ElfFile` you can use the `ArElfFile` to add an ELF object-file directly to an archive.
 
 ### Symbol Table
 
@@ -231,7 +235,7 @@ var arFile = new ArArchiveFile();
 var symbolTable = new ArSymbolTable();
 arFile.AddFile(symbolTable);
 // Create an ELF
-var elf = new ElfObjectFile();
+var elf = new ElfFile();
 // ... fill elf, add symbols
 arFile.AddFile(elf);
 // Add a symbol entry

readme.md

@@ -19,27 +19,29 @@ LibObjectFile is a .NET library to read, manipulate and write linker and executa
 ```C#
 // Reads an ELF file
 using var inStream = File.OpenRead("helloworld");
-var elf = ElfObjectFile.Read(inStream);
+var elf = ElfFile.Read(inStream);
 foreach(var section in elf.Sections)
 {
     Console.WriteLine(section.Name);
 }
 // Print the content of the ELF as readelf output
 elf.Print(Console.Out);
-// Write the ElfObjectFile to another file on the disk
+// Write the ElfFile to another file on the disk
 using var outStream = File.OpenWrite("helloworld2");
 elf.Write(outStream);
 ```
 
 ## Features
 - Full support of **Archive `ar` file format** including Common, GNU and BSD variants.
 - Full support for the **PE file format**
+  - Support byte-to-byte roundtrip
   - Read and write from/to a `System.IO.Stream`
   - All PE Directories are supported
   - `PEFile.Relocate` to relocate the image base of a PE file
   - `PEFile.Print` to print the content of a PE file to a textual representation
   - Support for calculating the checksum of a PE file
 - - Good support for the **ELF file format**:
+  - Support byte-to-byte roundtrip
   - Read and write from/to a `System.IO.Stream`
   - Handling of LSB/MSB
   - Support the following sections: 

src/Directory.Packages.props

@@ -6,6 +6,7 @@
   <ItemGroup>
     <PackageVersion Include="MinVer" Version="6.0.0" />
     <PackageVersion Include="MSTest" Version="3.6.0" />
+    <PackageVersion Include="SuperluminalPerf" Version="1.3.0" />
     <PackageVersion Include="System.Text.Encoding.CodePages" Version="8.0.0" />
     <PackageVersion Include="Verify.DiffPlex" Version="3.1.0" />
     <PackageVersion Include="Verify.MSTest" Version="26.6.0" />

src/LibObjectFile.Bench/LibObjectFile.Bench.csproj

@@ -0,0 +1,19 @@
+<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <OutputType>Exe</OutputType>
+    <TargetFramework>net8.0</TargetFramework>
+    <ImplicitUsings>enable</ImplicitUsings>
+    <Nullable>enable</Nullable>
+    <IsPackable>false</IsPackable>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <PackageReference Include="SuperluminalPerf" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <ProjectReference Include="..\LibObjectFile\LibObjectFile.csproj" />
+  </ItemGroup>
+
+</Project>

src/LibObjectFile.Bench/Program.cs

@@ -0,0 +1,86 @@
+// Copyright (c) Alexandre Mutel. All rights reserved.
+// This file is licensed under the BSD-Clause 2 license.
+// See the license.txt file in the project root for more information.
+
+using System.Diagnostics;
+using LibObjectFile.Elf;
+
+namespace LibObjectFile.Bench;
+
+internal class Program
+{
+    static void Main(string[] args)
+    {
+        Console.WriteLine("Loading files into memory");
+        var clock = Stopwatch.StartNew();
+        var streams = new List<MemoryStream>();
+        int biggestCapacity = 0;
+        foreach (var file in GetLinuxBins())
+        {
+            using var stream = File.OpenRead((string)file[0]);
+            if (ElfFile.IsElf(stream))
+            {
+                stream.Position = 0;
+                var localStream = new MemoryStream((int)stream.Length);
+                stream.CopyTo(localStream);
+                localStream.Position = 0;
+                streams.Add(localStream);
+                if (localStream.Capacity > biggestCapacity)
+                {
+                    biggestCapacity = localStream.Capacity;
+                }
+            }
+        }
+
+        clock.Stop();
+        Console.WriteLine($"End reading in {clock.Elapsed.TotalMilliseconds}ms");
+        Console.ReadLine();
+
+        Console.WriteLine("Processing");
+        var memoryStream = new MemoryStream(biggestCapacity);
+        clock.Restart();
+        //SuperluminalPerf.Initialize();
+        for (int i = 0; i < 10; i++)
+        {
+            //SuperluminalPerf.BeginEvent($"Round{i}");
+            foreach (var stream in streams)
+            {
+                stream.Position = 0;
+                var elf = ElfFile.Read(stream);
+                memoryStream.SetLength(0);
+                elf.Write(memoryStream);
+            }
+            //SuperluminalPerf.EndEvent();
+        }
+        clock.Stop();
+        Console.WriteLine($"{clock.Elapsed.TotalMilliseconds}ms");
+    }
+
+    public static IEnumerable<object[]> GetLinuxBins()
+    {
+        var wslDirectory = @"\\wsl$\Ubuntu\usr\bin";
+        if (OperatingSystem.IsLinux())
+        {
+            foreach (var file in Directory.EnumerateFiles(@"/usr/bin"))
+            {
+                yield return new object[] { file };
+            }
+        }
+        else if (OperatingSystem.IsWindows() && Directory.Exists(wslDirectory))
+        {
+            foreach (var file in Directory.EnumerateFiles(wslDirectory))
+            {
+                var fileInfo = new FileInfo(file);
+                // Skip symbolic links as loading them will fail
+                if ((fileInfo.Attributes & FileAttributes.ReparsePoint) == 0)
+                {
+                    yield return new object[] { file };
+                }
+            }
+        }
+        else
+        {
+            yield return new object[] { string.Empty };
+        }
+    }
+}

src/LibObjectFile.CodeGen/Program.cs

@@ -1,4 +1,4 @@
-// Copyright (c) Alexandre Mutel. All rights reserved.
+// Copyright (c) Alexandre Mutel. All rights reserved.
 // This file is licensed under the BSD-Clause 2 license.
 // See the license.txt file in the project root for more information.
 
@@ -215,6 +215,15 @@ private static void ProcessEnum(CSharpConverterOptions cppOptions, CSharpCompila
                     case "960":
                         csFieldName = "I960";
                         break;
+                    case "8051":
+                        csFieldName = "I8051";
+                        break;
+                    case "78KOR":
+                        csFieldName = "R78KOR";
+                        break;
+                    case "56800EX":
+                        csFieldName = "F56800EX";
+                        break;
                     default:
                         // assume Motorola
                         if (csFieldName.StartsWith("68"))