340 re organise do files health wellbeing

[andrewbaxter439]

What

• Partially addresses Re-organise do files #340 by completing this for Health and Wellbeing files to new pattern.
• Updates statistics output files also

To address before merging:

• This uses the UC interacting with hours worked to create a 'gradient' of UC effects on mental health.
• Should I aim to do this for financial distress before merging?

[Copilot]

🟡 Changes recommended

The refactored Stata do-files contain verified correctness/syntax issues (wrong dependent variables, duplicated regressors, inconsistent variable naming, and likely-invalid option syntax) that would break estimation and/or generate incorrect coefficient sheets.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR advances Issue #340 by refactoring the Stata regression-estimation “do” files for health/mental-health/wellbeing (and financial distress) into the newer standardized pattern (shared variable update + shared export programs + shared if-condition globals), and wires the Java side to load the resulting coefficient sheets via validated reads. It also updates the integration-test expected CSV outputs to match the new model behaviour/parameterization.

Changes:

• Reworked Stata regression pipelines for mental health and wellbeing to use shared variable_update.do, shared programs.do export helpers, and centralized *_if_condition globals.
• Extended Java regressor/parameter plumbing to support new/renamed lag variables and to load the new regression sheet names via safeReadExcel(...) validation.
• Updated integration-test expected statistics outputs (Wellbeing/Health/Labour/Demographic/Alignment/Wealth-Income).

File summaries

File	Description
input/InitialPopulations/compile/RegressionEstimates/00_master_conditions.do	Adds global if-conditions used by the refactored health/wellbeing/financial-distress regressions.
input/InitialPopulations/compile/RegressionEstimates/variable_update.do	Adds/adjusts derived variables and lags used by the refactored regression specifications.
input/InitialPopulations/compile/RegressionEstimates/12_reg_health_mental.do	Refactors mental-health regressions to the shared workflow + new specification structure.
input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do	Refactors wellbeing regressions to the shared workflow + new specification structure (MCS/PCS/Life satisfaction).
input/InitialPopulations/compile/RegressionEstimates/11_reg_financial_distress.do	Refactors financial-distress regression to the shared workflow and new regressor naming.
src/main/java/simpaths/data/Parameters.java	Switches coefficient loading for these processes to safeReadExcel(...) (validated regressor names) and updates sheet names.
src/main/java/simpaths/model/Person.java	Adds new enum entries/aliases so coefficient keys can be resolved to model variables.
src/test/java/simpaths/integrationtest/expected/WellbeingByGender.csv	Updates expected wellbeing-by-gender outputs to match the new model/regression setup.
src/test/java/simpaths/integrationtest/expected/WealthIncomeStatistics.csv	Updates expected wealth/income statistics outputs.
src/test/java/simpaths/integrationtest/expected/LabourStatistics.csv	Updates expected labour statistics outputs.
src/test/java/simpaths/integrationtest/expected/HealthStatistics.csv	Updates expected health statistics outputs.
src/test/java/simpaths/integrationtest/expected/DemographicStatistics.csv	Updates expected demographic statistics outputs.
src/test/java/simpaths/integrationtest/expected/AlignmentStatistics.csv	Updates expected alignment statistics outputs.

Review details

Suppressed comments (9)

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:117

• In DHE_MCS2_Females, L_Dhe_pcs is included twice (in L_Dhe_mcs L_Dhe_pcs and again at the end of the household-controls line). This will cause Stata to fail with a repeated-variable varlist error.

	Ded Dgn Dag Dag_sq ///
	L_Dhe_mcs L_Dhe_pcs ///
	i.Deh_c4_Medium i.Deh_c4_Low i.Deh_c4_Na ///
	Dhh_owned_L1 Dcpst_Single_L1 Dnc_L1 L_Dhe_pcs ///
	L_Ydses_c5_Q2 L_Ydses_c5_Q3 L_Ydses_c5_Q4 L_Ydses_c5_Q5 ///

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:142

• In DHE_MCS2_Males, L_Dhe_pcs is included twice in the regressor varlist. Stata does not allow duplicated regressors in a varlist.

	Ded Dgn Dag Dag_sq ///
	L_Dhe_mcs L_Dhe_pcs ///
	i.Deh_c4_Medium i.Deh_c4_Low i.Deh_c4_Na ///
	Dhh_owned_L1 Dcpst_Single_L1 Dnc_L1 L_Dhe_pcs ///
	L_Ydses_c5_Q2 L_Ydses_c5_Q3 L_Ydses_c5_Q4 L_Ydses_c5_Q5 ///

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:186

• DHE_PCS2_Females is a PCS stage-2 model, but it currently runs reghdfe dhe_mcs. This will estimate the wrong dependent variable and corrupt the PCS2 coefficient sheet.

reghdfe dhe_mcs ///

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:196

• In DHE_PCS2_Females, L_Dhe_pcs is duplicated in the regressor list (once on the lag line and again on the household-controls line). This will fail in Stata with a repeated-variable varlist error.

	Ded Dgn Dag Dag_sq ///
	L_Dhe_mcs L_Dhe_pcs ///
	i.Deh_c4_Medium i.Deh_c4_Low i.Deh_c4_Na ///
	Dhh_owned_L1 Dcpst_Single_L1 Dnc_L1 L_Dhe_pcs ///
	L_Ydses_c5_Q2 L_Ydses_c5_Q3 L_Ydses_c5_Q4 L_Ydses_c5_Q5 ///

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:221

• In DHE_PCS2_Males, L_Dhe_pcs is included twice in the regressor varlist, which will fail in Stata due to repeated variables.

	Ded Dgn Dag Dag_sq ///
	L_Dhe_mcs L_Dhe_pcs ///
	i.Deh_c4_Medium i.Deh_c4_Low i.Deh_c4_Na ///
	Dhh_owned_L1 Dcpst_Single_L1 Dnc_L1 L_Dhe_pcs ///
	L_Ydses_c5_Q2 L_Ydses_c5_Q3 L_Ydses_c5_Q4 L_Ydses_c5_Q5 ///

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:240

• In DLS1, L_Dhe_pcs is duplicated in the regressor list (Dls_L1 L_Dhe_pcs and again at the end of the household-controls line). This will fail with a repeated-variable varlist error.

	Ded Dgn Dag Dag_sq ///
	Dls_L1 L_Dhe_pcs ///
	i.Deh_c4_Medium i.Deh_c4_Low i.Deh_c4_Na ///
	Dhh_owned_L1 Dcpst_Single_L1 Dnc_L1 L_Dhe_pcs ///
	L_Ydses_c5_Q2 L_Ydses_c5_Q3 L_Ydses_c5_Q4 L_Ydses_c5_Q5 ///

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:275

• In DLS2_Females, L_Dhe_pcs is duplicated in the regressor list (once on the lag line and again on the household-controls line). Stata will error on repeated variables.

	Ded Dgn Dag Dag_sq ///
	Dls_L1 L_Dhe_pcs ///
	i.Deh_c4_Medium i.Deh_c4_Low i.Deh_c4_Na ///
	Dhh_owned_L1 Dcpst_Single_L1 Dnc_L1 L_Dhe_pcs ///
	L_Ydses_c5_Q2 L_Ydses_c5_Q3 L_Ydses_c5_Q4 L_Ydses_c5_Q5 ///

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:300

• In DLS2_Males, L_Dhe_pcs is duplicated in the regressor varlist. This will cause a Stata error for repeated variables.

	Ded Dgn Dag Dag_sq ///
	Dls_L1 L_Dhe_pcs ///
	i.Deh_c4_Medium i.Deh_c4_Low i.Deh_c4_Na ///
	Dhh_owned_L1 Dcpst_Single_L1 Dnc_L1 L_Dhe_pcs ///
	L_Ydses_c5_Q2 L_Ydses_c5_Q3 L_Ydses_c5_Q4 L_Ydses_c5_Q5 ///

input/InitialPopulations/compile/RegressionEstimates/13_reg_health_wellbeing.do:211

• DHE_PCS2_Males is a PCS stage-2 model, but it currently runs reghdfe dhe_mcs. This estimates the wrong outcome and will populate the DHE_PCS2_Males sheet with MCS parameters.

reghdfe dhe_mcs ///

• Files reviewed: 13/17 changed files
• Comments generated: 8
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[andrewbaxter439]

Dont merge yet - we're having trouble with the directions of effects of the employment transitions bits!

[andrewbaxter439]

cheers for running co-pilot on this @dav-sonn - rather a lot of typos to fix! will get round to those asap

[dav-sonn]

cheers for running co-pilot on this @dav-sonn - rather a lot of typos to fix! will get round to those asap

Thank you, @andrewbaxter439! I'll wait for your green light to proceed with the merge. Regardless, my main concern is that you opened the PR directly into main, while this should pass through develop first. Is there a reason for this or was it simply an oversight?

[andrewbaxter439]

Yup just an oversight! I'm usually a bit more aware of doing that 😄. Will correct that before getting final approval