43 read support log file

[rogerkuou]

fix #43

• add timestamp suffix
• add a logfile reader utility functin using tbparse -> there seems no need?
• add example of reading logfiles in dailexample notebook

[CLAassistant]

All committers have signed the CLA.

[rogerkuou]

Hi @SarahAlidoost , I added the timestamp to logfile suffix. However I found tbparse already provides very nice interface reading logs as DataFrames, see these lines of usages.
For now, I do not see the need to create a utility functions to read log files. Unless you thin the logfile should be read as a specific format? Or maybe I can add the use of tbparse to the example notebook?

[SarahAlidoost]

Hi @SarahAlidoost , I added the timestamp to logfile suffix. However I found tbparse already provides very nice interface reading logs as DataFrames, see these lines of usages. For now, I do not see the need to create a utility functions to read log files. Unless you thin the logfile should be read as a specific format? Or maybe I can add the use of tbparse to the example notebook?

Hi @rogerkuou Thanks! There are a information in train.py and predict.py that are written by SummaryWriter, for example Loss/best. Can you please add an example to the notebook on how to read those information? This is useful when verbose=False.

[rogerkuou]

Hi @SarahAlidoost , I updated the variable names in train to solve the duplication of the training loss. Also added an example of reading log and plotted the history of training/validation loss.
Can you give this another look? Thanks!

[rogerkuou]

@SarahAlidoost and just solved the conflict. Feel free to review!

[SarahAlidoost]

@rogerkuou thanks for the implementation 👍 In addition to what I suggested on changed code, I have two more comments about the notebook:

1. You added a plotting code (train vs validation loss) to the notebook as below. Can we have the plotting as a util function? also can we use a better axis labels so later we can put the plots in presentations/publications? Here suggestions: replace "step" with "epoch", in "ylabel" replace "Loss" with "Average loss per epoch (K)". It would be nice if the unit i.e. K can be adjusted based on the input data.

# Load saved training status with tbparse.SummaryReader
reader = SummaryReader(run_dir)

# plot training and validation loss
plt.figure(figsize=(10, 5))
train_loss = reader.scalars[reader.scalars["tag"] == "Loss/train"]
validation_loss = reader.scalars[reader.scalars["tag"] == "Loss/validation"]
plt.plot(train_loss["step"], train_loss["value"], label="Train Loss")
plt.plot(validation_loss["step"], validation_loss["value"], label="Validation Loss")
plt.xlabel("Step")
plt.ylabel("Loss")
plt.legend()

2. in the line below, why chunk and compute() is needed? I can tun the notebook without them:

monthly_data = xr.open_mfdataset(file_names, chunks={'time': 1, 'lat': 240, 'lon': 240}).compute()

[rogerkuou]

Thanks @SarahAlidoost, I maked the plot_loss function, and removed the chunking and compute (related to our discussion here). Can you have another look?

[SarahAlidoost]

@rogerkuou thanks 👍 . Let's merge this.