Fixed error when first inplace ReLU from first middle ResBlock appears on the main data propagation path

[Apich238]

First, I tried to create IResNet 50, save it in ONNX and view in Netron. This is what I saw: According to paper, ReLUs on the main information propagation path have to appear only at the end of stages. But in fact, they also appear at the first middle ResBlock.

The problem source I found is the following lines: self.relu = nn.ReLU(inplace=True) in __init__ of BasicBlock and Bottleneck, and out = self.relu(x) in following forward method. Inplace ReLU modifies tensor in place, and after second line out and x refer to the same object and both equal ReLU(x). As the result, identity equals to ReLU(x) instead of x.

To fix issue, I deleted inplace=True in classes where line out = self.relu(x) appears. Now Netron diagram for ResNet50 more corresponds paper: