Trajectory information with custom files to be exported as Parquet files instead of CSVs

[ChristopherRabotin]

High level description

CSVs store only data, without any metadata or compression. This causes loss of information and makes it difficult to reproduce runs.

Requirements

• All data exports from Nyx shall be in Parquet format instead of CSVs as is current implemented. (If CSVs are needed, users can convert required columns or the full file to CSVs using an external tool of their choice.)
• Export shall support different parameters and frames as is currently the case with CSVs
• Spacecraft states, orbits, and navigation trajectories shall be exportable in Parquet (navigation trajectories are part of Search through Navigation Trajectories #87 ).
• Trajectory data shall include instantaneous events as well as appending a column to specify the state of an Event for each entry in the trajectory.

Test plans

• Verify that all data exports from Nyx are in Parquet format, and not in CSV format.
• Test that the export supports the same parameters and frames as the current CSV export functionality.
• Ensure that spacecraft states, orbits, and navigation trajectories are correctly exportable in Parquet format.
• Check that instantaneous events are properly included in the Parquet export, and that the appended column specifying the state of each event is accurate.
• Test that the exported Parquet files can be read and processed by other tools and frameworks, such as Apache Arrow or Pandas.
• Verify that the Parquet export maintains the same precision and accuracy as the current CSV export.
• Check that the Parquet export produces files of reasonable size, compared to the current CSV export.
• Ensure that the Parquet export can be performed efficiently, and that it does not introduce significant overhead or slowdowns.

Design

Succinctly explain the idea behind the design.

Algorithm demonstration

N/A

API definition

Once a trajectory is generated, a user can initialize a trajectory output implementation which will then serialize the data to parquet with all of the fields requested.

High level architecture

sequenceDiagram
    participant A as Thread A
    participant B as Thread B
    participant C as Thread C

    A->>B: State data
    loop for each state
        B->>B: Compute requested parameters
        B->>C: Formatted state data
    end
    C->>C: Serialize data
    C->>C: Close file

Loading

Detailed design

The detailed design *will be used in the documentation of how Nyx works.

[ChristopherRabotin]

Limitations:

1. By default, all possible fields are exported to simplify setup. But the file size may be large (2.5 MB for a 1.5 day spacecraft propagation).
2. Exporting into other frames is not directly supported: one must convert the trajectory into another frame first.