PyTorchZeroToAll
PyTorchZeroToAll copied to clipboard
hunkim
poor performance of diabetes prediction
Why is the performance of using 07_diabetes_logistic.py to predict diabetes pretty poor?
I guess it's due to the optimizer setting, since I modified the update rule to:
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
the loss decreases dramatically.
@kopxiong It makes sense to change the hyperparameters. I've tried your suggestion, but the loss didn't drop to a satisfying level. Have you checked the y_pred of the final model?
@kopxiong Any thoughts? Since the goal of these tutorials is understanding the concept, I did not pay attention to the accuracy.
@Andy-jqa @hunkim Sorry for the late reply. I think we can improve the model's performance from the following aspects:
- add more layers or increase the number of nodes in each layer, since more units mean more powerful representation capability, like
super(Model, self).__init__()self.l1 = torch.nn.Linear(8, 32) #(8, 6)self.l2 = torch.nn.Linear(32, 16) #(6, 4)self.l3 = torch.nn.Linear(16, 1) #(4, 1) - use Adam optimizer to RMSProp instead of SGD
- use more epochs for training (like 2000 or more)
- sigmoid activation function maybe not good, try ReLU or LeakyReLU (I tried ReLU, but didn't work)
maybe we also should add some regularization terms to avoid overfitting.
also made more nodes in the hidden layer so i set my layers to torch.nn.Linear(8, 30) torch.nn.Linear(30, 10) torch.nn.Linear(10, 1)
