Remove unnecessary reordering of gens in ext2int().

CHANGES.md

@@ -12,6 +12,13 @@ For change history for [MOST][3], see [most/CHANGES.md](most/CHANGES.md).
 Since last release
 ------------------
 
+#### 2/26/19
+  - **INCOMPATIBLE CHANGE:** Eliminate unnecessary reordering of on-line
+    generators (sorted by increasing bus index) from `ext2int`. The
+    order is now left unmodified by `ext2int`. This change should only
+    affect user code that explicitly depends on the order of generators with
+    internal numbering (hopefully quite rare).
+
 #### 2/22/19
   - Clarify in manual and `printpf` output that units on branch current
     limits and outputs (i.e. when `opf.flow_lim` is `'I'`) are kA * basekV.

docs/src/MATPOWER-manual/MATPOWER-manual.tex

@@ -729,7 +729,7 @@ \subsection{Documentation}
 \section{Modeling}
 \label{sec:modeling}
 
-\matpower{} employs all of the standard steady-state models typically used for power flow analysis. The AC models are described first, then the simplified DC models. Internally, the magnitudes of all values are expressed in per unit and angles of complex quantities are expressed in radians. Internally, all off-line generators and branches are removed before forming the models used to solve the power flow or optimal power flow problem. All buses are numbered consecutively, beginning at~1, and generators are reordered by bus number. Conversions to and from this internal indexing is done by the functions \code{ext2int} and \code{int2ext}. The notation in this section, as well as Sections~\ref{sec:pf} and~\ref{sec:opf}, is based on this internal numbering, with all generators and branches assumed to be in-service. Due to the strengths of the \matlab{} programming language in handling matrices and vectors, the models and equations are presented here in matrix and vector form.
+\matpower{} employs all of the standard steady-state models typically used for power flow analysis. The AC models are described first, then the simplified DC models. Internally, the magnitudes of all values are expressed in per unit and angles of complex quantities are expressed in radians. Internally, all off-line generators and branches are removed before forming the models used to solve the power flow, continuation power flow or optimal power flow problem, and all buses are numbered consecutively, beginning at~1. Conversions to and from this internal indexing is done by the functions \code{ext2int} and \code{int2ext}. The notation in this section, as well as Sections~\ref{sec:pf} and~\ref{sec:opf}, is based on this internal numbering, with all generators and branches assumed to be in-service. Due to the strengths of the \matlab{} programming language in handling matrices and vectors, the models and equations are presented here in matrix and vector form.
 
 \subsection{Data Formats}
 \label{sec:data}
@@ -1476,7 +1476,7 @@ \subsection{{\tt runpf}}
 \end{threeparttable}
 \end{table}
 
-Internally, the \code{runpf} function does a number of conversions to the problem data before calling the appropriate solver routine for the selected power flow algorithm. This external-to-internal format conversion is performed by the \code{ext2int} function, described in more detail in Section~\ref{sec:ext2int_callback}, and includes the elimination of out-of-service equipment, the consecutive renumbering of buses and the reordering of generators by increasing bus number. All computations are done using this internal indexing. When the simulation has completed, the data is converted back to external format by \code{int2ext} before the results are printed and returned.
+Internally, the \code{runpf} function does a number of conversions to the problem data before calling the appropriate solver routine for the selected power flow algorithm. This external-to-internal format conversion is performed by the \code{ext2int} function, described in more detail in Section~\ref{sec:ext2int_callback}, and includes the elimination of out-of-service equipment and the consecutive renumbering of buses. All computations are done using this internal indexing. When the simulation has completed, the data is converted back to external format by \code{int2ext} before the results are printed and returned.
 
 \subsection{Linear Shift Factors}
 \label{sec:lsf}
@@ -2525,7 +2525,7 @@ \subsection{{\tt runopf}}
 \end{table}
 
 
-Internally, the \code{runopf} function does a number of conversions to the problem data before calling the appropriate solver routine for the selected OPF algorithm. This external-to-internal format conversion is performed by the \code{ext2int} function, described in more detail in Section~\ref{sec:ext2int_callback}, and includes the elimination of out-of-service equipment, the consecutive renumbering of buses and the reordering of generators by increasing bus number. All computations are done using this internal indexing. When the simulation has completed, the data is converted back to external format by \code{int2ext} before the results are printed and returned. In addition, both \code{ext2int} and \code{int2ext} can be customized via user-supplied callback routines to convert data needed by user-supplied variables, constraints or costs into internal indexing.
+Internally, the \code{runopf} function does a number of conversions to the problem data before calling the appropriate solver routine for the selected OPF algorithm. This external-to-internal format conversion is performed by the \code{ext2int} function, described in more detail in Section~\ref{sec:ext2int_callback}, and includes the elimination of out-of-service equipment and the consecutive renumbering of buses. All computations are done using this internal indexing. When the simulation has completed, the data is converted back to external format by \code{int2ext} before the results are printed and returned. In addition, both \code{ext2int} and \code{int2ext} can be customized via user-supplied callback routines to convert data needed by user-supplied variables, constraints or costs into internal indexing.
 
 
 
@@ -2756,9 +2756,9 @@ \subsubsection{{\tt ext2int} Callback}
 \begin{Code}
 mpc = ext2int(mpc, mpopt);
 \end{Code}
-All isolated buses, out-of-service generators and branches are removed, along with any generators or branches connected to isolated buses. The buses are renumbered consecutively, beginning at 1, and the in-service generators are sorted by increasing bus number. All of the related indexing information and the original data matrices are stored in an \code{order} field in the case struct to be used later by \code{int2ext} to perform the reverse conversions when the simulation is complete.
+All isolated buses, out-of-service generators and branches are removed, along with any generators or branches connected to isolated buses and the buses are renumbered consecutively, beginning at 1. All of the related indexing information and the original data matrices are stored in an \code{order} field in the case struct to be used later by \code{int2ext} to perform the reverse conversions when the simulation is complete.
 
-The first stage callback is invoked from within the \code{ext2int} function immediately after the case data has been converted. Inputs are a \matpower{} case struct (\mpc{}) freshly converted to internal indexing, a \matpower{} options struct \code{mpopt},\footnote{The \code{mpopt} may be empty if \code{ext2int()} is called manually without providing it.} and any (optional) \code{args} value supplied when the callback was registered via \code{add\_userfcn}. Output is the (presumably updated) \mpc{}. This is typically used to reorder any input arguments that may be needed in internal ordering by the formulation stage. The example shows how \code{e2i\_field} can also be used, with a case struct that has already been converted to internal indexing, to convert other data structures by passing in 2 or 3 extra parameters in addition to the case struct. In this case, it automatically converts the input data in the \code{qty}, \code{cost} and \code{zones} fields of \code{mpc.reserves} to be consistent with the internal generator ordering, where off-line generators have been eliminated and the on-line generators are sorted in order of increasing bus number. Notice that it is the second dimension (columns) of \code{mpc.reserves.zones} that is being re-ordered. See the on-line help for \code{e2i\_field} and \code{e2i\_data} for more details on what all they can do.
+The first stage callback is invoked from within the \code{ext2int} function immediately after the case data has been converted. Inputs are a \matpower{} case struct (\mpc{}) freshly converted to internal indexing, a \matpower{} options struct \code{mpopt},\footnote{The \code{mpopt} may be empty if \code{ext2int()} is called manually without providing it.} and any (optional) \code{args} value supplied when the callback was registered via \code{add\_userfcn}. Output is the (presumably updated) \mpc{}. This is typically used to reorder any input arguments that may be needed in internal ordering by the formulation stage. The example shows how \code{e2i\_field} can also be used, with a case struct that has already been converted to internal indexing, to convert other data structures by passing in 2 or 3 extra parameters in addition to the case struct. In this case, it automatically converts the input data in the \code{qty}, \code{cost} and \code{zones} fields of \code{mpc.reserves} to be consistent with the internal generator ordering, where off-line generators have been eliminated. Notice that it is the second dimension (columns) of \code{mpc.reserves.zones} that is being re-ordered. See the on-line help for \code{e2i\_field} and \code{e2i\_data} for more details on what all they can do.
 
 \begin{Code}
 function mpc = userfcn_reserves_ext2int(mpc, mpopt, args)
@@ -2828,7 +2828,7 @@ \subsubsection{{\tt int2ext} Callback}
 \begin{Code}
 results = int2ext(results, mpopt);
 \end{Code}
-This conversion essentially undoes everything that was done by \code{ext2int}. Generators are restored to their original ordering, buses to their original numbering and all out-of-service or isolated generators, branches and buses are restored.
+This conversion essentially undoes everything that was done by \code{ext2int}. Buses are restored to their original numbering and all out-of-service or isolated generators, branches and buses are restored.
 
 This callback is invoked from \code{int2ext} immediately before the resulting case is converted from internal back to external indexing. At this point, the simulation has been completed and the \results{} struct, a superset of the original \matpower{} case struct passed to the OPF, contains all of the results. This \results{} struct is passed to the callback, along with the \matpower{} options struct \code{mpopt},\footnote{The \code{mpopt} may be empty if \code{int2ext()} is called manually without providing it.} and any (optional) \code{args} supplied when the callback was registered via \code{add\_userfcn}. The output of the callback is the updated \results{} struct. This is typically used to convert any results to external indexing and populate any corresponding fields in the \results{} struct.
 
@@ -3994,7 +3994,7 @@ \subsubsection{{\tt ext2int}, {\tt int2ext}}
   mpc_ext = int2ext(mpc_int, mpopt)
 \end{Code}
 
-These functions convert a \matpower{} case struct from external to internal, and from internal to external numbering, respectively. \code{ext2int} first removes all isolated buses, off-line generators and branches, and any generators or branches connected to isolated buses. Then the buses are renumbered consecutively, beginning at 1, and the generators are sorted by increasing bus number. Any \codeq{ext2int} callback routines registered in the case are also invoked automatically. All of the related indexing information and the original data matrices are stored in an \codeq{order} field in the struct to be used later by code{int2ext} to perform the reverse conversions. If the case is already using internal numbering it is returned unchanged. The optional \matpower{} options struct (\code{mpopt}) input argument is only needed in conjunction with callback routines that depend on it, e.g. when \code{toggle\_softlims} is on.
+These functions convert a \matpower{} case struct from external to internal, and from internal to external numbering, respectively. \code{ext2int} first removes all isolated buses, off-line generators and branches, and any generators or branches connected to isolated buses. Then the buses are renumbered consecutively, beginning at 1.\footnote{In \matpower{} versions 4 through 7.0b1, \code{ext2int} also sorted the generators by increasing internal bus number, but beginning with version 7.0, the order of on-line generators is left unmodified.} Any \codeq{ext2int} callback routines registered in the case are also invoked automatically. All of the related indexing information and the original data matrices are stored in an \codeq{order} field in the struct to be used later by \code{int2ext} to perform the reverse conversions. If the case is already using internal numbering it is returned unchanged. The optional \matpower{} options struct (\code{mpopt}) input argument is only needed in conjunction with callback routines that depend on it, e.g. when \code{toggle\_softlims} is on.
 
 \subsubsection{{\tt e2i\_data}, {\tt i2e\_data}}
 
@@ -8176,6 +8176,7 @@ \subsubsection*{Incompatible Changes}
 \item Remove \code{nln.mu.l.<name>} and \code{nln.mu.u.<name>} fields from OPF \code{results} struct. Use \code{nle.lambda.<name>} and \code{nli.mu.<name>} fields instead for nonlinear constraint multipliers.
 \item Modify order of default output arguments of \code{opt\_model/get\_idx()}.
 \item Add \code{mpopt} to input args for OPF \codeq{ext2int}, \codeq{formulation}, and \codeq{int2ext} callbacks.
+\item Eliminate unnecessary reordering of on-line generators (sorted by increasing bus index) from \code{ext2int()}. The order is now left unmodified by \code{ext2int()}. This change should only affect user code that explicitly depends on the order of generators with internal numbering (hopefully quite rare).
 \end{itemize}
 
 

lib/ext2int.m

@@ -28,9 +28,8 @@
 %   by a MATOWER options struct, then all isolated buses, off-line
 %   generators and branches are removed along with any generators or
 %   branches connected to isolated buses. Then the buses are renumbered
-%   consecutively, beginning at 1, and the generators are sorted by
-%   increasing bus number. Any 'ext2int' callback routines registered in
-%   the case are also invoked automatically. All of the related indexing
+%   consecutively, beginning at 1. Any 'ext2int' callback routines registered
+%   in the case are also invoked automatically. All of the related indexing
 %   information and the original data matrices are stored in an 'order'
 %   field in the struct to be used by INT2EXT to perform the reverse
 %   conversions. If the case is already using internal numbering it is
@@ -72,8 +71,14 @@
 %
 %   See also INT2EXT, E2I_FIELD, E2I_DATA.
 
+%   UNDOCUMENTED OPTION:
+%       MPC = EXT2INT(MPC, MPOPT, 1)
+%       Calling EXT2INT in the second form with a 3rd argument of 1 will
+%       cause it to sort the generators by increasing bus index as in
+%       MATPOWER v4 through v7.0b1.
+
 %   MATPOWER
-%   Copyright (c) 1996-2018, Power Systems Engineering Research Center (PSERC)
+%   Copyright (c) 1996-2019, Power Systems Engineering Research Center (PSERC)
 %   by Ray Zimmerman, PSERC Cornell
 %
 %   This file is part of MATPOWER.
@@ -82,7 +87,7 @@
 
 if isstruct(bus)
     mpc = bus;
-    if nargin < 3
+    if nargin < 3 || isscalar(gen)
         first = ~isfield(mpc, 'order');
         if first || mpc.order.state == 'e'
             %% define names for columns to data matrices
@@ -126,6 +131,11 @@
             else
                 dc = 0;
             end
+            if nargin == 3
+                reorder_gens = gen;
+            else
+                reorder_gens = 0;
+            end
 
             %% save data matrices with external ordering
             o.ext.bus    = mpc.bus;
@@ -165,8 +175,9 @@
                 mpc.gen(o.gen.status.off, :) = [];
             end
 
-            %% update size
+            %% update sizes
             nb = size(mpc.bus, 1);
+            ng = size(mpc.gen, 1);
 
             %% apply consecutive bus numbering
             o.bus.i2e = mpc.bus(:, BUS_I);
@@ -178,10 +189,17 @@
                 mpc.branch(:, T_BUS)    = o.bus.e2i( mpc.branch(:, T_BUS)   );
             end
 
-            %% reorder gens in order of increasing bus number
-            [tmp, o.gen.i2e] = sort(mpc.gen(:, GEN_BUS));
-            [tmp, o.gen.e2i] = sort(o.gen.i2e);
-            mpc.gen = mpc.gen(o.gen.i2e, :);
+            if reorder_gens
+                %% reorder gens in order of increasing bus number
+                [tmp, o.gen.i2e] = sort(mpc.gen(:, GEN_BUS));
+                [tmp, o.gen.e2i] = sort(o.gen.i2e);
+                mpc.gen = mpc.gen(o.gen.i2e, :);
+            else
+                %% don't reorder gens in order of increasing bus number, but
+                %% keep the mappings in place for backward compatibility
+                o.gen.i2e = (1:ng)';
+                o.gen.e2i = o.gen.i2e;
+            end
 
             if isfield(o, 'int')
                 o = rmfield(o, 'int');
@@ -241,10 +259,10 @@
         end
         if ischar(gen) || iscell(gen)   %% field
             warning('Calls of the form MPC = EXT2INT(MPC, ''FIELD_NAME'', ...) have been deprecated. Please replace EXT2INT with E2I_FIELD.');
-            i2e = e2i_field(mpc, gen, branch, dim);
+            i2e = e2i_field(mpc, gen, ordering, dim);
         else                            %% value
             warning('Calls of the form VAL = EXT2INT(MPC, VAL, ...) have been deprecated. Please replace EXT2INT with E2I_DATA.');
-            i2e = e2i_data(mpc, gen, branch, dim);
+            i2e = e2i_data(mpc, gen, ordering, dim);
         end
     end
 else            %% old form

lib/t/opf_nle_fcn1.m

@@ -1,7 +1,6 @@
 function [g, dg] = opf_nle_fcn1(x)
 Pg = x{1};
-g = Pg(1)*Pg(2) - Pg(3);
+g = Pg(1)*Pg(2) - Pg(6);
 if nargout == 2
-    dg = sparse(1, length(Pg));
-    dg(1:3) = [Pg(2) Pg(1) -1];
+    dg = [Pg(2) Pg(1) 0 0 0 -1];
 end

lib/t/t_case_int.m

@@ -27,9 +27,9 @@
 %% generator data
 %	bus	Pg	Qg	Qmax	Qmin	Vg	mBase	status	Pmax	Pmin	Pc1	Pc2	Qc1min	Qc1max	Qc2min	Qc2max	ramp_agc	ramp_10	ramp_30	ramp_q	apf
 mpc.gen = [
-	1	0	0	300	-300	1	100	1	250	90	0	0	0	0	0	0	0	0	0	0	0;
-	2	163	0	300	-300	1	100	1	300	10	0	0	0	0	0	0	0	0	0	0	0;
 	3	85	0	300	-300	1	100	1	270	10	0	0	0	0	0	0	0	0	0	0	0;
+	2	163	0	300	-300	1	100	1	300	10	0	0	0	0	0	0	0	0	0	0	0;
+	1	0	0	300	-300	1	100	1	250	90	0	0	0	0	0	0	0	0	0	0	0;
 ];
 
 %% branch data
@@ -51,9 +51,9 @@
 %	1	startup	shutdown	n	x1	y1	...	xn	yn
 %	2	startup	shutdown	n	c(n-1)	...	c0
 mpc.gencost = [
-	1	0	0	4	0	0	100	2000	200	4403.5	270	6363.5;
-	1	0	0	4	0	0	100	2500	200	5500	250	7250;
 	2	0	0	2	15	0	0	0	0	0	0	0;
+	1	0	0	4	0	0	100	2500	200	5500	250	7250;
+	1	0	0	4	0	0	100	2000	200	4403.5	270	6363.5;
 ];
 
 %% bus names
@@ -71,24 +71,24 @@
 
 %% generator unit type (see GENTYPES)
 mpc.gentype = {
-	'HY';
-	'GT';
 	'ST';
+	'GT';
+	'HY';
 };
 
 %% generator fuel type (see GENFUELS)
 mpc.genfuel = {
-	'hydro';
-	'ng';
 	'coal';
+	'ng';
+	'hydro';
 };
 
 mpc.A = [
-	1	2	3	4	5	7	8	9	10	11	12	13	14	15	17	18	19	20	24	22	21	28	26	25	29	30;
-	2	4	6	8	10	14	16	18	20	22	24	26	28	30	34	36	38	40	48	44	42	56	52	50	58	60;
+	1	2	3	4	5	7	8	9	10	11	12	13	14	15	17	18	19	20	21	22	24	25	26	28	29	30;
+	2	4	6	8	10	14	16	18	20	22	24	26	28	30	34	36	38	40	42	44	48	50	52	56	58	60;
 ];
 
 mpc.N = [
-	30	29	28	27	26	24	23	22	21	20	19	18	17	16	14	13	12	11	7	9	10	3	5	6	2	1;
-	60	58	56	54	52	48	46	44	42	40	38	36	34	32	28	26	24	22	14	18	20	6	10	12	4	2;
+	30	29	28	27	26	24	23	22	21	20	19	18	17	16	14	13	12	11	10	9	7	6	5	3	2	1;
+	60	58	56	54	52	48	46	44	42	40	38	36	34	32	28	26	24	22	20	18	14	12	10	6	4	2;
 ];