"Model is infeasible" using CPLEX as solver

[mlozano-rmi]

Hello, I've run into an error where it claims there are zero solutions to the model and I'm not sure why as I am using example data (ERCOT_2021_Annual_1GW). I will post the full error below. If anyone has any suggestions (especially if you've successfully run the model using CPLEX), please reach out with suggestions. Thank you.

Full error:

[ Info: MILP solved for primal
ERROR: LoadError: Result index of attribute MathOptInterface.VariablePrimal(1) out of bounds. There are currently 0 solution(s) in the model.
Stacktrace:
[1] check_result_index_bounds
@ ~/.julia/packages/MathOptInterface/wrorD/src/attributes.jl:207 [inlined]
[2] get(model::CPLEX.Optimizer, attr::MathOptInterface.VariablePrimal, x::MathOptInterface.VariableIndex)
@ CPLEX ~/.julia/packages/CPLEX/Vd1Gd/src/MOI/MOI_wrapper.jl:2974
[3] get(b::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, attr::MathOptInterface.VariablePrimal, index::MathOptInterface.VariableIndex)
@ MathOptInterface.Bridges ~/.julia/packages/MathOptInterface/wrorD/src/Bridges/bridge_optimizer.jl:1195
[4] get(model::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, attr::MathOptInterface.VariablePrimal, index::MathOptInterface.VariableIndex)
@ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/wrorD/src/Utilities/cachingoptimizer.jl:911
[5] _moi_get_result(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, ::MathOptInterface.VariablePrimal, ::Vararg{Any})
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:640
[6] get(model::JuMP.Model, attr::MathOptInterface.VariablePrimal, v::JuMP.VariableRef)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:680
[7] value(v::JuMP.VariableRef; result::Int64)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/variables.jl:1080
[8] value
@ ~/.julia/packages/JuMP/ptoff/src/variables.jl:1079 [inlined]
[9] #1563
@ ./none:0 [inlined]
[10] iterate
@ ./generator.jl:47 [inlined]
[11] _all(f::Base.var"#343#345", itr::Base.Generator{Vector{JuMP.VariableRef}, DOLPHYN.var"#1563#1564"}, #unused#::Colon)
@ Base ./reduce.jl:1250
[12] all
@ ./reduce.jl:1246 [inlined]
[13] Dict(kv::Base.Generator{Vector{JuMP.VariableRef}, DOLPHYN.var"#1563#1564"})
@ Base ./dict.jl:131
[14] fix_integers(jump_model::JuMP.Model)
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/solve_model.jl:40
[15] solve_model(EP::JuMP.Model, setup::Dict{Any, Any})
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/solve_model.jl:89
[16] top-level scope
@ ~/Library/CloudStorage/OneDrive-RMI/Documents/45V/Dolphyn_Examples/Time_matching/DOLPHYN-LB-Annual/DOLPHYN-ERCOT/ERCOT_2021_Annual_1GW/Run.jl:86
in expression starting at /Users/mlozano/Library/CloudStorage/OneDrive-RMI/Documents/45V/Dolphyn_Examples/Time_matching/DOLPHYN-LB-Annual/DOLPHYN-ERCOT/ERCOT_2021_Annual_1GW/Run.jl:86

caused by: Result index of attribute MathOptInterface.VariablePrimal(1) out of bounds. There are currently 0 solution(s) in the model.
Stacktrace:
[1] check_result_index_bounds
@ ~/.julia/packages/MathOptInterface/wrorD/src/attributes.jl:207 [inlined]
[2] get(model::CPLEX.Optimizer, attr::MathOptInterface.VariablePrimal, x::MathOptInterface.VariableIndex)
@ CPLEX ~/.julia/packages/CPLEX/Vd1Gd/src/MOI/MOI_wrapper.jl:2974
[3] get(b::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, attr::MathOptInterface.VariablePrimal, index::MathOptInterface.VariableIndex)
@ MathOptInterface.Bridges ~/.julia/packages/MathOptInterface/wrorD/src/Bridges/bridge_optimizer.jl:1195
[4] get(model::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, attr::MathOptInterface.VariablePrimal, index::MathOptInterface.VariableIndex)
@ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/wrorD/src/Utilities/cachingoptimizer.jl:911
[5] _moi_get_result(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, ::MathOptInterface.VariablePrimal, ::Vararg{Any})
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:640
[6] get(model::JuMP.Model, attr::MathOptInterface.VariablePrimal, v::JuMP.VariableRef)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:680
[7] value(v::JuMP.VariableRef; result::Int64)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/variables.jl:1080
[8] value
@ ~/.julia/packages/JuMP/ptoff/src/variables.jl:1079 [inlined]
[9] #1563
@ ./none:0 [inlined]
[10] iterate
@ ./generator.jl:47 [inlined]
[11] Dict{JuMP.VariableRef, Float64}(kv::Base.Generator{Vector{JuMP.VariableRef}, DOLPHYN.var"#1563#1564"})
@ Base ./dict.jl:103
[12] dict_with_eltype
@ ./abstractdict.jl:575 [inlined]
[13] dict_with_eltype
@ ./abstractdict.jl:582 [inlined]
[14] Dict(kv::Base.Generator{Vector{JuMP.VariableRef}, DOLPHYN.var"#1563#1564"})
@ Base ./dict.jl:129
[15] fix_integers(jump_model::JuMP.Model)
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/solve_model.jl:40
[16] solve_model(EP::JuMP.Model, setup::Dict{Any, Any})
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/solve_model.jl:89
[17] top-level scope
@ ~/Library/CloudStorage/OneDrive-RMI/Documents/45V/Dolphyn_Examples/Time_matching/DOLPHYN-LB-Annual/DOLPHYN-ERCOT/ERCOT_2021_Annual_1GW/Run.jl:86

[dharik13]

As context, this example solves without any issues using Gurobi. @RuaridhMacd @Betristor @gn-he any thoughts on what might be the cause? I heard from another user that DOLPHYN fails on CPLEX

[Betristor]

I didn't have access to CPLEX using my official school email account. So my tests are all based on Gurobi. Basically the reason for that model is infeasible is caused by unstatisfied constraints. I suggest to check 1) whether in the model all resources you want are added into the balance, this could be done by accessing the balance constraints; 2) whether the total generation could satisfy all demand including enabling resource expansion and if total demand is extremely large; 3) check the minimum power constraint and compare it to the average capacity factor for hydro resources if any since this is quite hard to pay attention to. My experience mainly concentrates on these cases.

But the wierd part is that CPLEX and Gurobi disagree with the same model we used in the example. Is the method option for Gurobi and CPLEX same? Solver-level difference may introduce different results for the model.

[RuaridhMacd]

Hi @mlozano-rmi

Could you link a copy of your model, run file, and settings for me to look at? If it's not too big, it would be great if you could also save a copy of the model to an .LP file so we can compare it to the Gurobi version. This function should do the trick if run after you've generated the model: write_to_file(EP, "rmi_model")

I took a look around for common issues and CPLEX may be struggling with an overly large coefficient range. What is the range when solving your problem with CPLEX?

[mlozano-rmi]

@Betristor Thank you for the reply! A few follow up comments:

1. Can you share where I'd access the balance constraints?
2. I haven't changed the demand so I don't know why this would become an issue.
3. By minimum power constraint do you mean the value in minimum_capacity_requirement.csv? If so, then it cannot be met purely by hydro, but is met through existing wind and solar

[mlozano-rmi]

@RuaridhMacd Thank you also for replying. I am still figuring out how to save a .LP file, but in the meantime, I've attached .zip files of the DOLPHYN files I'm using as well as the specific case data. Those can be found here (https://drive.google.com/drive/folders/105exrfun3oR1vMTAl4RW6i1PT6AHOtwx?usp=sharing)

Here are the changes I made to get the model running up to this point.

--

Deleted the following:
Manifest.toml in DOLPHYN folder and case-specific folder:
[[Gurobi]]
deps = ["LazyArtifacts", "Libdl", "MathOptInterface"]
git-tree-sha1 = "82a44a86f4dc4fa4510c9d49b0a74d3d73914d5c"
uuid = "2e9cd046-0924-5485-92f1-d5272153d98b"
version = "0.11.5"

Project.toml in DOLPHYN folder and case-specific folder:
Gurobi = "2e9cd046-0924-5485-92f1-d5272153d98b"
Gurobi = "0.11"
In case folder, add: CPLEX = "a076750e-1247-5638-91d2-ce28b192dca0"

DOLPHYN.jl:
Comment out
#using Gurobi
ADD using CPLEX

--

Other changes:

global_model_settings.yml - Change Solver to CPLEX
genx_settings.yml - Change Solver to CPLEX

Error: Load data asks for “Load_MW_1” not “Load_MW_z1”
In “load_load_data.jl” in src->GenX->load_inputs folder, add z to lines 84 and 89, so that: "Load_MW_z$z" for z in Zones
“load_h2_demand.jl” in src->HSC->load_inputs, add z to line 41 "Load_H2_tonne_per_hr_z$z" for z in Zones

In DOLPHYN/src/GenX/load_inputs/load_inputs.jl , comment out lines 78-80

In minimum_capacity_requirement.jl
Change line 42 to Symbol("MinCapTag") (not “MinCapTag_$mincap”)

Used updated generator file, change Biomass THERM value to 2.

[RuaridhMacd]

@mlozano-rmi Thanks very much. I'll take a look at these.

As a quick check, could you try setting WriteShadowPrices: 0 in the global_model_settings.yml file? I still expect it to fail, but some outputs should be written, which will help the diagnosis.

[mlozano-rmi]

@RuaridhMacd Thank you for looking into to. I made the requested change, please find the generated error below:

[ Info: MILP solved for primal
Writing Output
ERROR: LoadError: Result index of attribute MathOptInterface.ObjectiveValue(1) out of bounds. There are currently 0 solution(s) in the model.
Stacktrace:
[1] check_result_index_bounds
@ ~/.julia/packages/MathOptInterface/wrorD/src/attributes.jl:207 [inlined]
[2] get(model::CPLEX.Optimizer, attr::MathOptInterface.ObjectiveValue)
@ CPLEX ~/.julia/packages/CPLEX/Vd1Gd/src/MOI/MOI_wrapper.jl:3254
[3] get(b::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, attr::MathOptInterface.ObjectiveValue)
@ MathOptInterface.Bridges ~/.julia/packages/MathOptInterface/wrorD/src/Bridges/bridge_optimizer.jl:1067
[4] _get_model_attribute(model::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, attr::MathOptInterface.ObjectiveValue)
@ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/wrorD/src/Utilities/cachingoptimizer.jl:828
[5] get
@ ~/.julia/packages/MathOptInterface/wrorD/src/Utilities/cachingoptimizer.jl:876 [inlined]
[6] _moi_get_result(model::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, args::MathOptInterface.ObjectiveValue)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:640
[7] get(model::JuMP.Model, attr::MathOptInterface.ObjectiveValue)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:660
[8] objective_value(model::JuMP.Model; result::Int64)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/objective.jl:54
[9] objective_value
@ ~/.julia/packages/JuMP/ptoff/src/objective.jl:50 [inlined]
[10] write_status(path::String, sep::String, inputs::Dict{Any, Any}, setup::Dict{Any, Any}, EP::JuMP.Model)
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/GenX/write_outputs/write_status.jl:29
[11] write_outputs(EP::JuMP.Model, path::String, setup::Dict{Any, Any}, inputs::Dict{Any, Any})
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/GenX/write_outputs/write_outputs.jl:51
[12] top-level scope
@ ~/Library/CloudStorage/OneDrive-RMI/Documents/45V/Dolphyn_Examples/Time_matching/DOLPHYN-LB-Annual/DOLPHYN-ERCOT/ERCOT_2021_Annual_1GW/Run.jl:93
in expression starting at /Users/mlozano/Library/CloudStorage/OneDrive-RMI/Documents/45V/Dolphyn_Examples/Time_matching/DOLPHYN-LB-Annual/DOLPHYN-ERCOT/ERCOT_2021_Annual_1GW/Run.jl:93

[RuaridhMacd]

Thanks @mlozano-rmi

Looking at your settings, it might be good to try turning barrier crossover on. To do that, set SolutionType: 1 in your CPLEX settings.

Would it be possible for you to save the entire console output? You should be able to do so by going to your DOLPHYN directory in a regular terminal (not Julia REPL) and running your example like this:
julia --project=DOLPHYNJulEnv /path/to/Run.jl > outputs_log.txt

[mlozano-rmi]

@RuaridhMacd I tried setting SolutionType: 1 and it didn't work either. I've attached the outputs_log for only one version as they are identical
Thank you again for helping me debug
outputs_log.txt

[RuaridhMacd]

Thanks @mlozano-rmi

There is a warning in your log file about an infeasible constraint: Infeasibility row 'c1944724': 0 >= 5550.
If you can print the model, we might be able to identify that constraint.

One last suggestion for today: try tightening Optimal_Tol: 1e-6 in your CPLEX settings. The Gurobi version has a tighter tolerance.

[Betristor]

@Betristor Thank you for the reply! A few follow up comments:

1. Can you share where I'd access the balance constraints?
2. I haven't changed the demand so I don't know why this would become an issue.
3. By minimum power constraint do you mean the value in minimum_capacity_requirement.csv? If so, then it cannot be met purely by hydro, but is met through existing wind and solar

Sorry that I didn't make myself clear. @mlozano-rmi

1. After each run, the console could be used to access the model and inputs. With reference to the balance constraint name, like EP[:ePowerBalance];
2. The DOLPHYN model is established based on the match between supply and demand. So if total demand is greater than maximum generation (capacity times availability) that could be generated, the model is definitely infeasible;
3. Similar to 2, but this type of error is mainly caused by desired minimum power output that could not be realized due to. low availability of renewables and hydro resources if Min_power is set to some non-zeros.

[mlozano-rmi]

@RuaridhMacd I've attached a .lp file but I'm not sure whether it has what you're asking for.

Re: infeasible constraint, it's coming from the value in Minimum_capacity_requirement.csv (Min_MW = 5500). Does this mean the model isn't loading the generators file properly?
output.lp.zip

[dharik13]

@mlozano-rmi I see that under genx_settings.yml, you have MinCapReq = 1 , which means that the minimum capacity requirement is being considered. Can you set that to zero and see if the model solves? The reason the model is infeasible is because you have not specified which generators are eligible to meet this capacity requirement constraint. The way you do that is: 1) by adding a column MinCapTag_1 for the 1st minimum capacity requirement constraint and 2) then assigning values of 1 for those generators that can be eligible to meet this constraint. Mike ran your example by deactivating MinCapReq and therefore did not run into issues.

[RuaridhMacd]

@mlozano-rmi @dharik13
Good catch. It looks like the model is requiring that you have at least 5500MW of "renewables" based on the input in Minimum_capacity_requirement.csv but the set of "renewable" resources isn't defined in the MinCapTag column of Generators_data.csv

[dharik13]

@mlozano-rmi if you want to see how specify the set of generators for a min/max capacity requirement constraint, take a look at generators_data.csv file in https://github.com/macroenergy/DOLPHYN/blob/H2_PTC_hourly_modeling/Example_Systems/FRCC_2021_hourly_1GW_base/Generators_data.csv

[mlozano-rmi]

@dharik13 @RuaridhMacd Well it's running now! Thank you for helping me find it, I didn't realize I had to deactivate the MinCapReq

[mlozano-rmi]

@mlozano-rmi if you want to see how specify the set of generators for a min/max capacity requirement constraint, take a look at generators_data.csv file in https://github.com/macroenergy/DOLPHYN/blob/H2_PTC_hourly_modeling/Example_Systems/FRCC_2021_hourly_1GW_base/Generators_data.csv

I will take a look now, thank you so much!