Record log into file #71

Betristor · 2022-10-04T05:02:11Z

To redirect log into log file, there're three things to be done:

record basic output from println into log
record complex output from println into log file
record gurobi log into log file (or other solvers)

Temporarily julia has nothing stored in stdout for the whole computation process. So, we need another io stream other than stdout to record log into file while keeping console output. This needs Logging and Loggingextras modules. And I set a flag called Log in global_model_settings.yml to turn this function on. Solver log is temporarily separate with main log.

^{Created from VS Code using CodeStream}

Betristor · 2022-10-04T06:21:08Z

Complex output stream in time_domain_reduction.jl is not processed. Many println functions in this file prints much complicated message which is beyond the processing scope of print_and_log function.

Betristor · 2022-10-04T06:34:58Z

Example log looks like this

┌ Info: Time Series Data Already Clustered.
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Configuring Solver
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Reading Input CSV Files
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Warning: thread = 1 warning: only found 14 / 15 columns around data row: 4. Filling remaining columns with `missing`
└ @ CSV /home/zyh/.julia/packages/CSV/Zl2ww/src/file.jl:612
┌ Info: Network.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Load_data.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Fuels_data.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Generators_data.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Generators_variability.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: CO2_cap.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: CSV Files Successfully Read In From /home/zyh/DOLPHYN-dev/Example_Systems/SmallNewEngland/ThreeZones/
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Reading H2 Input CSV Files
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: HSC_generation.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: HSC_load_data.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: HSC_generators_variability.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: HSC_pipelines.csv Successfully Read!
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: HSC Input CSV Files Successfully Read In From /home/zyh/DOLPHYN-dev/Example_Systems/SmallNewEngland/ThreeZones/
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Discharge Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Non-served Energy Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Investment Discharge Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Emissions Module for CO2 Policy modularization
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Transmission Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Dispatchable Resources Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Storage Resources Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Storage Investment Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Storage Core Resources Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Storage Resources with Symmetric Charge/Discharge Capacity Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Thermal (No Unit Commitment) Resources Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Hydrogen Generation and Storage Discharge Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Hydrogen Investment Discharge Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Hydrogen Production Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: H2 Production Core Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: H2 Production (Unit Commitment) Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: H2 Production (No Unit Commitment) Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: H2 Emissions Module for CO2 Policy modularization
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Hydrogen Non-served Energy Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Hydrogen Storage Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: H2 Storage Energy Investment Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: H2 Storage Charging Investment Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: H2 Storage Core Resources Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: H2 Flexible Demand Resources Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Hydrogen Pipeline Module
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: C02 Policies Module for power and hydrogen system combined
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Solving Model
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: LP solved for primal
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Writing Output
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing costs is 0.054800755
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing storage is 0.237652613
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing nse is 1.209355993
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing power balance is 1.207144466
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing transmission flows is 0.261108722
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing transmission losses is 0.042963978
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing network expansion is 0.78526882
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing emissions is 1.149132102
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing reliability is 0.515915773
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing storage duals is 0.247215648
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing time weights is 0.181174432
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Time elapsed for writing net revenue is 1.829231964
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Wrote outputs to /home/zyh/DOLPHYN-dev/Example_Systems/SmallNewEngland/ThreeZones/Results_1/
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29
┌ Info: Wrote outputs HSC outputs to /home/zyh/DOLPHYN-dev/Example_Systems/SmallNewEngland/ThreeZones/Results_1/Results_HSC/
└ @ DOLPHYN /home/zyh/DOLPHYN-dev/src/print_and_log.jl:29

Betristor · 2022-10-04T06:37:46Z

Gurobi.log (or other solvers) looks like this

Gurobi 9.1.2 (linux64) logging started Tue Oct  4 14:33:34 2022


Gurobi 9.1.2 (linux64) logging started Tue Oct  4 14:33:34 2022

Gurobi Optimizer version 9.1.2 build v9.1.2rc0 (linux64)
Thread count: 28 physical cores, 56 logical processors, using up to 28 threads
Optimize a model with 34308 rows, 17195 columns and 113176 nonzeros
Model fingerprint: 0x2212f6e5
Coefficient statistics:
  Matrix range     [2e-05, 2e+02]
  Objective range  [1e-04, 3e+03]
  Bounds range     [0e+00, 0e+00]
  RHS range        [3e-03, 2e+02]

Deterministic concurrent LP optimizer: primal simplex, dual simplex, and barrier
Showing barrier log only...

Presolve removed 12106 rows and 7195 columns
Presolve time: 0.06s
Presolved: 22202 rows, 10000 columns, 80957 nonzeros

Ordering time: 0.02s

Barrier statistics:
 Dense cols : 30
 AA' NZ     : 1.610e+05
 Factor NZ  : 6.927e+05 (roughly 20 MBytes of memory)
 Factor Ops : 1.365e+08 (less than 1 second per iteration)
 Threads    : 26

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0   7.95185502e+07 -6.74541883e+06  3.04e+02 8.94e+02  4.04e+05     0s
   1   3.91591187e+07 -5.18153368e+07  1.65e+02 4.16e+03  2.15e+05     0s
   2   3.62472427e+07 -1.23739537e+08  1.38e+02 2.17e+03  1.56e+05     0s
   3   4.69250928e+06 -1.34621520e+08  2.94e+01 5.01e+02  3.78e+04     0s
   4   2.61829660e+06 -9.36535119e+07  1.53e+01 1.54e+02  1.55e+04     0s
   5   1.82773707e+06 -5.35692318e+07  9.65e+00 4.57e+01  6.47e+03     0s
   6   3.43480042e+05 -3.85723066e+07  2.89e-01 2.50e+01  2.61e+03     0s
   7   2.04190315e+05 -4.61162689e+06  9.10e-06 2.58e+00  2.89e+02     0s
   8   1.06654878e+05 -4.03761427e+05  8.87e-09 2.08e-01  2.71e+01     0s
   9   4.46741308e+04 -3.75850377e+04  2.94e-10 1.56e-02  3.30e+00     0s
  10   1.67646554e+04 -9.03725780e+03  1.61e-10 3.41e-03  9.56e-01     0s
  11   1.05369766e+04 -5.35219348e+02  1.17e-10 1.13e-03  3.90e-01     0s
  12   8.11117863e+03  3.17226117e+03  7.62e-11 4.24e-04  1.68e-01     0s
  13   6.29928621e+03  4.35043904e+03  3.95e-11 1.13e-04  6.49e-02     0s
  14   5.80494867e+03  4.60740612e+03  2.24e-11 5.75e-05  3.96e-02     0s
  15   5.72595221e+03  4.68947605e+03  2.25e-11 3.44e-05  3.41e-02     0s
  16   5.39830607e+03  4.81690048e+03  3.11e-11 1.21e-05  1.91e-02     0s
  17   5.20059920e+03  4.85685631e+03  1.25e-10 8.19e-06  1.13e-02     0s
  18   5.11043717e+03  4.88396262e+03  1.75e-10 6.74e-06  7.45e-03     0s
  19   5.06832819e+03  4.90311051e+03  3.36e-10 4.91e-06  5.44e-03     0s
  20   5.04664271e+03  4.91315475e+03  4.91e-10 4.07e-06  4.39e-03     0s
  21   5.02675045e+03  4.92695482e+03  7.00e-10 3.12e-06  3.28e-03     0s
  22   5.00727991e+03  4.94095937e+03  2.52e-09 1.55e-06  2.18e-03     1s
  23   4.98532746e+03  4.94758762e+03  1.84e-08 9.05e-07  1.24e-03     1s
  24   4.97875259e+03  4.95076567e+03  2.27e-08 5.82e-07  9.18e-04     1s
  25   4.97729746e+03  4.95096263e+03  2.05e-08 5.68e-07  8.64e-04     1s
  26   4.97617239e+03  4.95164667e+03  1.91e-08 5.16e-07  8.05e-04     1s
  27   4.97450483e+03  4.95194747e+03  1.79e-08 4.95e-07  7.40e-04     1s
  28   4.97195533e+03  4.95297577e+03  1.89e-08 4.10e-07  6.23e-04     1s
  29   4.96945605e+03  4.95368426e+03  2.46e-08 3.59e-07  5.18e-04     1s
  30   4.96568420e+03  4.95556864e+03  2.18e-08 1.92e-07  3.32e-04     1s
  31   4.96394068e+03  4.95630616e+03  1.48e-08 1.35e-07  2.50e-04     1s
  32   4.96218717e+03  4.95679639e+03  1.26e-08 8.82e-08  1.77e-04     1s
  33   4.96099618e+03  4.95686536e+03  2.06e-08 7.99e-08  1.36e-04     1s
  34   4.96027472e+03  4.95711264e+03  1.53e-08 6.25e-08  1.04e-04     1s
  35   4.95967114e+03  4.95734704e+03  2.53e-08 4.50e-08  7.63e-05     1s
  36   4.95924993e+03  4.95758680e+03  2.72e-08 2.28e-08  5.45e-05     1s
  37   4.95858134e+03  4.95779760e+03  4.68e-08 8.31e-09  2.57e-05     1s
  38   4.95845520e+03  4.95789123e+03  3.52e-08 4.23e-09  1.84e-05     1s
  39   4.95836563e+03  4.95792383e+03  2.70e-08 3.05e-09  1.44e-05     1s
  40   4.95835190e+03  4.95794485e+03  2.53e-08 2.44e-09  1.33e-05     1s
  41   4.95823202e+03  4.95795789e+03  1.60e-08 1.88e-09  8.96e-06     1s
  42   4.95820678e+03  4.95797487e+03  1.38e-08 1.34e-09  7.57e-06     1s
  43   4.95817887e+03  4.95799220e+03  1.13e-08 8.84e-10  6.09e-06     1s
  44   4.95810247e+03  4.95802311e+03  5.00e-09 2.96e-11  2.59e-06     1s
  45   4.95805252e+03  4.95803354e+03  8.88e-10 1.82e-12  6.19e-07     1s
  46   4.95804053e+03  4.95803641e+03  1.88e-10 1.82e-12  1.34e-07     1s
  47   4.95803830e+03  4.95803695e+03  6.55e-10 1.82e-12  4.39e-08     1s
  48   4.95803830e+03  4.95803695e+03  6.37e-06 3.64e-12  4.39e-08     1s
  49   4.95803830e+03  4.95803695e+03  6.37e-06 3.64e-12  4.39e-08     1s
  50   4.95803830e+03  4.95803695e+03  6.37e-06 3.64e-12  4.39e-08     1s
  51   4.95803830e+03  4.95803695e+03  6.37e-06 3.64e-12  4.39e-08     1s
  52   4.95803830e+03  4.95803695e+03  5.47e-06 3.64e-12  4.39e-08     1s
  53   4.95803830e+03  4.95803695e+03  5.47e-06 3.64e-12  4.39e-08     1s
  54   4.95803830e+03  4.95803695e+03  5.40e-06 3.64e-12  4.39e-08     1s
  55   4.95803830e+03  4.95803695e+03  5.40e-06 3.64e-12  4.39e-08     1s
  56   4.95803830e+03  4.95803695e+03  5.52e-06 3.64e-12  4.39e-08     1s
  57   4.95803830e+03  4.95803695e+03  5.51e-06 3.64e-12  4.38e-08     1s

Barrier performed 57 iterations in 1.16 seconds
Sub-optimal termination - objective 4.95804053e+03

Crossover log...

   11230 DPushes remaining with DInf 0.0000000e+00                 1s
       0 DPushes remaining with DInf 0.0000000e+00                 2s

    5753 PPushes remaining with PInf 1.6801803e-03                 2s
       0 PPushes remaining with PInf 0.0000000e+00                 2s

  Push phase complete: Pinf 0.0000000e+00, Dinf 1.6777277e+01      2s

Iteration    Objective       Primal Inf.    Dual Inf.      Time
   10736    4.9580370e+03   0.000000e+00   1.677728e+01      2s
   10858    4.9580370e+03   0.000000e+00   0.000000e+00      2s
Concurrent spin time: 0.00s

Solved with barrier
Solved in 10858 iterations and 1.71 seconds
Optimal objective  4.958036996e+03

User-callback calls 17601, time in user-callback 0.22 sec

Gurobi 9.1.2 (linux64) logging started Tue Oct  4 14:36:11 2022


Gurobi 9.1.2 (linux64) logging started Tue Oct  4 14:36:45 2022

Gurobi Optimizer version 9.1.2 build v9.1.2rc0 (linux64)
Thread count: 28 physical cores, 56 logical processors, using up to 28 threads
Optimize a model with 34308 rows, 17195 columns and 113176 nonzeros
Model fingerprint: 0x2212f6e5
Coefficient statistics:
  Matrix range     [2e-05, 2e+02]
  Objective range  [1e-04, 3e+03]
  Bounds range     [0e+00, 0e+00]
  RHS range        [3e-03, 2e+02]

Deterministic concurrent LP optimizer: primal simplex, dual simplex, and barrier
Showing barrier log only...

Presolve removed 12106 rows and 7195 columns
Presolve time: 0.05s
Presolved: 22202 rows, 10000 columns, 80957 nonzeros

Ordering time: 0.02s

Barrier statistics:
 Dense cols : 30
 AA' NZ     : 1.610e+05
 Factor NZ  : 6.927e+05 (roughly 20 MBytes of memory)
 Factor Ops : 1.365e+08 (less than 1 second per iteration)
 Threads    : 26

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0   7.95185502e+07 -6.74541883e+06  3.04e+02 8.94e+02  4.04e+05     0s
   1   3.91591187e+07 -5.18153368e+07  1.65e+02 4.16e+03  2.15e+05     0s
   2   3.62472427e+07 -1.23739537e+08  1.38e+02 2.17e+03  1.56e+05     0s
   3   4.69250928e+06 -1.34621520e+08  2.94e+01 5.01e+02  3.78e+04     0s
   4   2.61829660e+06 -9.36535119e+07  1.53e+01 1.54e+02  1.55e+04     0s
   5   1.82773707e+06 -5.35692318e+07  9.65e+00 4.57e+01  6.47e+03     0s
   6   3.43480042e+05 -3.85723066e+07  2.89e-01 2.50e+01  2.61e+03     0s
   7   2.04190315e+05 -4.61162689e+06  9.10e-06 2.58e+00  2.89e+02     0s
   8   1.06654878e+05 -4.03761427e+05  8.87e-09 2.08e-01  2.71e+01     0s
   9   4.46741308e+04 -3.75850377e+04  2.94e-10 1.56e-02  3.30e+00     0s
  10   1.67646554e+04 -9.03725780e+03  1.61e-10 3.41e-03  9.56e-01     0s
  11   1.05369766e+04 -5.35219348e+02  1.17e-10 1.13e-03  3.90e-01     0s
  12   8.11117863e+03  3.17226117e+03  7.62e-11 4.24e-04  1.68e-01     0s
  13   6.29928621e+03  4.35043904e+03  3.95e-11 1.13e-04  6.49e-02     0s
  14   5.80494867e+03  4.60740612e+03  2.24e-11 5.75e-05  3.96e-02     0s
  15   5.72595221e+03  4.68947605e+03  2.25e-11 3.44e-05  3.41e-02     0s
  16   5.39830607e+03  4.81690048e+03  3.11e-11 1.21e-05  1.91e-02     0s
  17   5.20059920e+03  4.85685631e+03  1.25e-10 8.19e-06  1.13e-02     0s
  18   5.11043717e+03  4.88396262e+03  1.75e-10 6.74e-06  7.45e-03     0s
  19   5.06832819e+03  4.90311051e+03  3.36e-10 4.91e-06  5.44e-03     0s
  20   5.04664271e+03  4.91315475e+03  4.91e-10 4.07e-06  4.39e-03     0s
  21   5.02675045e+03  4.92695482e+03  7.00e-10 3.12e-06  3.28e-03     0s
  22   5.00727991e+03  4.94095937e+03  2.52e-09 1.55e-06  2.18e-03     0s
  23   4.98532746e+03  4.94758762e+03  1.84e-08 9.05e-07  1.24e-03     1s
  24   4.97875259e+03  4.95076567e+03  2.27e-08 5.82e-07  9.18e-04     1s
  25   4.97729746e+03  4.95096263e+03  2.05e-08 5.68e-07  8.64e-04     1s
  26   4.97617239e+03  4.95164667e+03  1.91e-08 5.16e-07  8.05e-04     1s
  27   4.97450483e+03  4.95194747e+03  1.79e-08 4.95e-07  7.40e-04     1s
  28   4.97195533e+03  4.95297577e+03  1.89e-08 4.10e-07  6.23e-04     1s
  29   4.96945605e+03  4.95368426e+03  2.46e-08 3.59e-07  5.18e-04     1s
  30   4.96568420e+03  4.95556864e+03  2.18e-08 1.92e-07  3.32e-04     1s
  31   4.96394068e+03  4.95630616e+03  1.48e-08 1.35e-07  2.50e-04     1s
  32   4.96218717e+03  4.95679639e+03  1.26e-08 8.82e-08  1.77e-04     1s
  33   4.96099618e+03  4.95686536e+03  2.06e-08 7.99e-08  1.36e-04     1s
  34   4.96027472e+03  4.95711264e+03  1.53e-08 6.25e-08  1.04e-04     1s
  35   4.95967114e+03  4.95734704e+03  2.53e-08 4.50e-08  7.63e-05     1s
  36   4.95924993e+03  4.95758680e+03  2.72e-08 2.28e-08  5.45e-05     1s
  37   4.95858134e+03  4.95779760e+03  4.68e-08 8.31e-09  2.57e-05     1s
  38   4.95845520e+03  4.95789123e+03  3.52e-08 4.23e-09  1.84e-05     1s
  39   4.95836563e+03  4.95792383e+03  2.70e-08 3.05e-09  1.44e-05     1s
  40   4.95835190e+03  4.95794485e+03  2.53e-08 2.44e-09  1.33e-05     1s
  41   4.95823202e+03  4.95795789e+03  1.60e-08 1.88e-09  8.96e-06     1s
  42   4.95820678e+03  4.95797487e+03  1.38e-08 1.34e-09  7.57e-06     1s
  43   4.95817887e+03  4.95799220e+03  1.13e-08 8.84e-10  6.09e-06     1s
  44   4.95810247e+03  4.95802311e+03  5.00e-09 2.96e-11  2.59e-06     1s
  45   4.95805252e+03  4.95803354e+03  8.88e-10 1.82e-12  6.19e-07     1s
  46   4.95804053e+03  4.95803641e+03  1.88e-10 1.82e-12  1.34e-07     1s
  47   4.95803830e+03  4.95803695e+03  6.55e-10 1.82e-12  4.39e-08     1s
  48   4.95803830e+03  4.95803695e+03  6.37e-06 3.64e-12  4.39e-08     1s
  49   4.95803830e+03  4.95803695e+03  6.37e-06 3.64e-12  4.39e-08     1s
  50   4.95803830e+03  4.95803695e+03  6.37e-06 3.64e-12  4.39e-08     1s
  51   4.95803830e+03  4.95803695e+03  6.37e-06 3.64e-12  4.39e-08     1s
  52   4.95803830e+03  4.95803695e+03  5.47e-06 3.64e-12  4.39e-08     1s
  53   4.95803830e+03  4.95803695e+03  5.47e-06 3.64e-12  4.39e-08     1s
  54   4.95803830e+03  4.95803695e+03  5.40e-06 3.64e-12  4.39e-08     1s
  55   4.95803830e+03  4.95803695e+03  5.40e-06 3.64e-12  4.39e-08     1s
  56   4.95803830e+03  4.95803695e+03  5.52e-06 3.64e-12  4.39e-08     1s
  57   4.95803830e+03  4.95803695e+03  5.51e-06 3.64e-12  4.38e-08     1s

Barrier performed 57 iterations in 1.14 seconds
Sub-optimal termination - objective 4.95804053e+03

Crossover log...

   11230 DPushes remaining with DInf 0.0000000e+00                 1s
       0 DPushes remaining with DInf 0.0000000e+00                 1s

    5753 PPushes remaining with PInf 1.6801803e-03                 1s
       0 PPushes remaining with PInf 0.0000000e+00                 1s

  Push phase complete: Pinf 0.0000000e+00, Dinf 1.6777277e+01      1s

Iteration    Objective       Primal Inf.    Dual Inf.      Time
   10736    4.9580370e+03   0.000000e+00   1.677728e+01      1s
   10858    4.9580370e+03   0.000000e+00   0.000000e+00      2s
Concurrent spin time: 0.00s

Solved with barrier
Solved in 10858 iterations and 1.53 seconds
Optimal objective  4.958036996e+03

User-callback calls 17617, time in user-callback 0.01 sec

Betristor · 2022-10-04T06:40:12Z

It's enabled by specific setting in gurobi_settings.yml called GurobiLog to control whether a log file will be generated and other setting called GurobiLogFile to control the filename if GurobiLog is true. Other solvers may have the same setting but I don't implement them for now.

anna-nadia

This looks good, thank you for implementing this!
Just wondering, why did you implement 2 parameters in the settings file instead of using just the one "LogFile" one, and then if it's blank, there is no log? Wouldn't that be simpler to have one parameter?

Betristor · 2022-10-05T02:34:44Z

@anna-nadia Yes, it's simpler to have one parameter GurobiLogFile. If it's blank, default situation is that gurobi log will be directed to console.
I used two parameters to keep consistent with global settings and hold the consideration once we don't want gurobi log to go into console, LogToConsole in gurobi could decide gurobi log behaviors in console. But I didn't implement this for temporarily console log is necssary.

RuaridhMacd · 2022-10-05T15:05:22Z

This looks good and it's great to be able to save the log.

If we update this in the future, I think it would be better to avoid using a global and instead pass the setting into the function. That should have better performance and make it clearer where the setting it coming from.

As an intermediate fix, you could define the type on the fly:

function print_and_log(message::AbstractString)
    # Log is set as global variable
    if Log::Bool
        println(message)
        @info(message)
    else
        println(message)
    end
end

log setttings

26c159f

Betristor linked an issue Oct 4, 2022 that may be closed by this pull request

Write log to text file #68

Closed

Betristor self-assigned this Oct 4, 2022

Betristor added enhancement New feature or request wip labels Oct 4, 2022

Betristor added 3 commits October 4, 2022 13:37

modify env add logging and loggingextras

3fa095a

auxiliary print and log function and test in run

57c46aa

replace print_and_log in src files

d42f0f1

Betristor requested review from dharik13 and anna-nadia October 4, 2022 06:04

handle complex output

5e7f79d

enable gurobi log

ad15924

Betristor changed the title ~~[!WIP] record log into log file~~ Record log into file Oct 4, 2022

Betristor removed the wip label Oct 4, 2022

anna-nadia approved these changes Oct 5, 2022

View reviewed changes

RuaridhMacd self-requested a review October 5, 2022 15:02

RuaridhMacd approved these changes Oct 5, 2022

View reviewed changes

Betristor merged commit c2d0906 into main Oct 6, 2022

Betristor deleted the 68-write-log-to-text-file branch October 6, 2022 02:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Record log into file #71

Record log into file #71

Betristor commented Oct 4, 2022 •

edited

Loading

Betristor commented Oct 4, 2022

Betristor commented Oct 4, 2022

Betristor commented Oct 4, 2022

Betristor commented Oct 4, 2022

anna-nadia left a comment

Betristor commented Oct 5, 2022

RuaridhMacd commented Oct 5, 2022

Record log into file #71

Record log into file #71

Conversation

Betristor commented Oct 4, 2022 • edited Loading

Betristor commented Oct 4, 2022

Betristor commented Oct 4, 2022

Betristor commented Oct 4, 2022

Betristor commented Oct 4, 2022

anna-nadia left a comment

Choose a reason for hiding this comment

Betristor commented Oct 5, 2022

RuaridhMacd commented Oct 5, 2022

Betristor commented Oct 4, 2022 •

edited

Loading