Hyperparameter tuning#69
Conversation
|
@meiertgrootes and @rogerkuou this PR adds support for hyper parameter tuning using ray. I added an example notebook that can be run locally using cpu. In meantime, I'm testing the workflow in levante. See #40 for details on tuning strategy. |
|
Hi @SarahAlidoost for now I will only review and test this on my local. Will wait for your signal about the HPC test |
rogerkuou
left a comment
rogerkuou
left a comment
There was a problem hiding this comment.
Hi @SarahAlidoost , thanks! Just some small comments. After fixing them please feel free to merge!
| filename_pattern="*_hr_ERA5dc_masked_tos.nc", | ||
| train_range=(2018, 2020), | ||
| validation_range=(2021, 2021), | ||
| test_range=(2022, 2022), |
There was a problem hiding this comment.
Just a small opinion here: maybe these default arguments are too strong constraints.
There was a problem hiding this comment.
Comment for the last cell, the experiment_path changes per run. I propose it can be changed to:
check_best_model(Path(best_result.path).parent, test_dataset, run_dir)so two changes:
- I think
check_best_modelcan acceptpathlib.Pathso no need to convert to str? - Use
Path(best_result.path).parenttorefer the experiment path
| return results | ||
|
|
||
|
|
||
| def check_best_model(experiment_path: str, test_dataset: STDataset, run_dir): |
There was a problem hiding this comment.
I think both paths here can also accept pathlib.Path?
| def check_best_model(experiment_path: str, test_dataset: STDataset, run_dir): | |
| def check_best_model(experiment_path: [str | Path], test_dataset: STDataset, run_dir: [str | Path]): |
| input_data = xr.open_mfdataset(data_config["input_filenames"]) | ||
| monthly_data = xr.open_mfdataset(data_config["monthly_filenames"]) | ||
| lsm_mask = xr.open_dataset(data_config["landmask_filename"]) |
There was a problem hiding this comment.
I have some small concerns of wrapping this into a function, since this hides API of xr.open_mfdataset
With our current data, for Python<=3.13 this works. While I am using Python=3.14, where the CF concention changed. Then loading input_data and monthly_data will cause kernel error if not specifying chunksizes in open_mfdataset. But this also relates to the Python version where nc files are created.
Maybe we should either expose the args of open_mfdataset, or mention Python<=3.13 in pyproject.toml ?
2 participants
closes #40