Implementation of first MACRO changes

[lukeggert]

Hello all,

The pull request is about the implementation of an alternative version of the macroeconomic MACRO model. This new model version has not been finalised yet. However, I wanted to demonstrate the changes I have made so far. I have tried to calibrate the new model version to the MESSAGE model. The calibration process uses scen.add_macro(), but some errors occur as important MESSAGE variables are dropped due to infeasibilities.

I think these infeasibilities are mostly related to the fact that the model is not finalised yet. The new energy accounting equations in particular need to be checked for feasibility and mathematical plausibility. Nevertheless, I would like to share my progress and perhaps gain some insight into how to solve the infeasibilities in the calibration process (for instance, by changing the start values for specific parameters). Currently, the most influential parameter to be changed for calibration purposes is the newly implemented exogenous wage rate in the capital constraint.

Description of changes:

In order to understand the changes made to the parameters, variables and equations in macro_core.gms, I have uploaded a PDF file containing documentation describing the alternative MACRO implementation: "new_model_description.pdf".

The following files have been changed:

• macro_core.gms
• macro_data_load.gms
• macro_calibration.gms
• macro_reporting.gms
• macro_solve.gms
• As some parameters and variables of the old model have been deleted and some new have been added, I also made changes to the macro python scripts init.py and calibrate.py .

I created a new folder "run script and output" to see the latest calibration run and related errors and infeasibilities:

• A script of loading the baseline scenario and adding the MACRO model (scen.add_macro()) -> See GAMS Error Warnings
• The excel sheet containing the data for the MACRO calibration process
• The output of the MESSAGE-MACRO_run log and lst files

How to review

• The preview build of the documentation allows to view a 'diff' (removals in red, additions/replacements in green) between the existing MACRO formulation docs and the version on this branch.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Lukas Eggert seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[khaeru]

Excellent, thanks @lukeggert for sharing this here! The code changes in the Python and GAMS file seem fairly clean and easy to trace. I will make an attempt to read them together with the PDF and notebooks and reproduce any issues/exceptions described there.

[khaeru]

Per:

The calibration process uses scen.add_macro(), but some errors occur as important MESSAGE variables are dropped due to infeasibilities. […] I think these infeasibilities are mostly related to the fact that the model is not finalised yet.

I looked at the added "MESSAGE-MACRO_run_log.ipynb", I see the following at the first attempt to solve MACRO:

--- Executing CONOPT (Solvelink=2): elapsed 0:04:22.804

CONOPT 4         47.6.0 c2de9d6d Sep 12, 2024          LEG x86 64bit/Linux    

--- *** This solver runs with a demo license. No commercial use.

Reading parameter(s) from "/home/lukas/environments/ssp2/lib/python3.10/site-packages/message_ix/model/conopt.opt"
>>  LFILOG 50
>>  LFILOS 50
Finished reading from "/home/lukas/environments/ssp2/lib/python3.10/site-packages/message_ix/model/conopt.opt"
 
    C O N O P T   version 4.35
    Copyright (C) GAMS Software GmbH
                  GAMS Development Corporation
 
    The user model has 266 constraints and 279 variables
    with 627 Jacobian elements, 86 of which are nonlinear.
    The Hessian of the Lagrangian has 84 elements on the diagonal,
    181 elements below the diagonal, and 84 nonlinear variables.
 
    Iter Phase   Ninf   Infeasibility   RGmax      NSB   Step  InItr MX OK
       0   0          5.2296682234E+03 (Input point)
 
 ** An equation in the pre-triangular part of the model cannot
    be solved because the critical variable is at a bound.
  
    The pre-triangular part of the model has 25 constraints and 3 variables.
 
 ** Infeasible solution. The Preprocessor has determined that the model
    is infeasible.

So this indeed indicates the MACRO MCP is not feasible, probably due to the changes to the GAMS code.

Some points of information:

• The Scenario.add_macro() method solves MESSAGE-MACRO to derive/check the values of certain free parameters.
• In solving MESSAGE-MACRO, there is an iteration between the solution of MESSAGE (with CPLEX) and MACRO (with CONOPT).
• If either of these fails, then subsequent solutions/outputs of either do not contain valid or meaningful solution data.
• In some cases, the iteration between the two models will be halted automatically and propagated to an exception raised by the Python code. But in other cases (as it seems here), the failure of the MACRO solve does not terminate the iteration. This is why we see further iterations in the above-mentioned log file.
• The infeasibility causes further downstream errors (like the "ERROR at.ac.iiasa.ixmp.objects.Scenario:1691 - variable 'ACT' not found in gdx!") and the process eventually fails. The target should not be to debug these, but rather to understand and fix the infeasibility, after which the downstream errors should disappear.

As mentioned in Slack, there may be a way to get CONOPT to print out information on the offending variables and equations.