neuraloperators-TL-scaling icon indicating copy to clipboard operation
neuraloperators-TL-scaling copied to clipboard

Scales for solution function not used?

Open rohan-mehta-1024 opened this issue 2 years ago • 2 comments

The median value for the solution functions is computed in get_scales.ipynb, but is never used in the following normalization code:

if self.scales is not None:
            f_norm = np.linalg.norm(X[0]) * self.measure
            f_scaling = f_norm / self.scales[0]
            X = X / f_scaling # ensures that 10f and 10k for example, have the same input
            # scale the tensors
            X[self.in_channels:] = X[self.in_channels:] / self.scales[self.in_channels:(self.in_channels + self.tensor_shape), None, None]

Is this intentional, or accidental?

rohan-mehta-1024 avatar Aug 07 '23 12:08 rohan-mehta-1024

As an update, training only works for me when I add in solution normalization as well (otherwise I get an inf loss) so it seems like solution normalization should be present (unless I'm missing something?)

rohan-mehta-1024 avatar Aug 07 '23 14:08 rohan-mehta-1024

Hi, sorry for the delay, I missed this. Could you please provide some additional information on what test/config you are trying to run? The solution scales are usually not too large (as opposed to the scales of the source and coefficients) and hence were not used. But, as we noted in the paper, we did find the normalization to be quite important to getting good performance.

ShashankSubramanian avatar Sep 15 '23 17:09 ShashankSubramanian