Hello,
I am using the pre-trained weights from RadImageNet (DenseNet121) for my model training in PyTorch, but I have encountered an issue with the running_var values in certain layers. Specifically, the running_var values for layers such as denseblock4.denselayer13.norm1.running_var are significantly high: Min: 14.656, Max: 56296, Mean: 3797.477.
These extreme values in the RadImageNet weights seem unusual and are affecting the stability of the model during validation (i.e., producing extreme logits for certain batches).
I would like to understand the following:
- Is this behavior expected, or could it be related to a specific issue in the training or weight initialization process?
- How were the images in RadImageNet normalized prior to training? There seems to be inconsistencies for both PyTorch (image = (image-127.5)*2 / 255) and Tensorflow (rescale=1./255) implementations and I would like to know exactly what needs to be employed.
Has anyone experienced this with DenseNet121 in PyTorch?
Any insights or guidance on this would be greatly appreciated!
Thank you!
Hello,
I am using the pre-trained weights from RadImageNet (DenseNet121) for my model training in PyTorch, but I have encountered an issue with the running_var values in certain layers. Specifically, the running_var values for layers such as denseblock4.denselayer13.norm1.running_var are significantly high: Min: 14.656, Max: 56296, Mean: 3797.477.
These extreme values in the RadImageNet weights seem unusual and are affecting the stability of the model during validation (i.e., producing extreme logits for certain batches).
I would like to understand the following:
Has anyone experienced this with DenseNet121 in PyTorch?
Any insights or guidance on this would be greatly appreciated!
Thank you!