You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be useful to be able to define different file names for input and output files. This will allow for some more flexibility when running large sequences of cases.
I propose adding an optional argument to all input loading and output writing functions with the current filenames as the default values. We'll also write a short log of where the outputs were saved to, to help automated parsing of the outputs later.
The text was updated successfully, but these errors were encountered:
@lbonaldo and I have talked about some related improvements like allowing .gz or parquet input files to reduce file size, or letting users build a database with tables that can be reused across cases. Some input files can get large and/or might be duplicated across cases (and even periods).
I recently wrote some python functions that use DuckDB to load data from either a data file (.csv or .parquet) or a database table (.sqlite or .duckdb). The top-level function (load_data) takes arguments of the data location (folder or database path), the file or table name, and an optional sql query. Something similar could be implemented in GenX, letting users define input file names/locations and use different file types.
Update 7/12 -- Added default input names and file paths so that users can specify names of their input files in a YAML file called "input_settings.yml" in the settings folder of their case. I added an example (10_three_zones_define_input) with this file in it. I tested this for run_genx_simple and will test on multistage soon. I'm working on using DuckDB so that users can input files of types other than CSV (currently input files still need to be CSV).
It would be useful to be able to define different file names for input and output files. This will allow for some more flexibility when running large sequences of cases.
I propose adding an optional argument to all input loading and output writing functions with the current filenames as the default values. We'll also write a short log of where the outputs were saved to, to help automated parsing of the outputs later.
The text was updated successfully, but these errors were encountered: