Very HIgh MSE Loss

[GittyHarsha]

Epoch [95/100], Loss: 15795335555002138624.0000
Epoch [96/100], Loss: 15947571735960748032.0000
Epoch [97/100], Loss: 22230783709444833280.0000
Epoch [98/100], Loss: 24243408957763223552.0000
Epoch [99/100], Loss: 20029352523428003840.0000
Epoch [100/100], Loss: 26956084463393570816.0000
Mean Squared Error: 20261864048330539008.0000

I trained a simple neural network with 3 linear layers, relu and a dropout and Adam optimizer
class Net(nn.Module):
def init(self):
super(Net, self).init()
self.fc1 = Linear(12, 128)
self.fc2 = Linear(128, 64)
self.fc3 = Linear(64, 32)
self.fc4 = Linear(32, 1)
self.dropout = nn.Dropout(0.2)

def forward(self, x):
    x = torch.relu(self.fc1(x))
    x = self.dropout(x)
    x = torch.relu(self.fc2(x))
    x = self.dropout(x)
    x = torch.relu(self.fc3(x))
    x = self.dropout(x)
    x = self.fc4(x) 
    return x

The error seems to be very high.

Something is unique about the dataset.
A custom model tailored for this dataset is required

[HridayM25]

You need to find the minima of the function given in the dataset. What you have done, you are trying to reduce the MSE loss between the fit NN and the target values. You have changed the objective function altogether here. Plus, scaling plays a big role in NN convergence. Try to scale the values

[GittyHarsha]

ah ok

[GittyHarsha]

hey, is it okay to estimate the function and then find the minima? but it may not correspond to the actual minima. Can you give hints on this? To get a better estimate, maybe i'll use gradient free black optimization: ZO-AdaMM, which eliminates the need of finding the gradient of the function with respect to $x_1, x_2 ,... x_{12}$ via chain rule through the gradients wrt the model weights

[HridayM25]

Yes you can use any method.
Start with a simple approach : try to fit some curve that you think can best describe the data.

HINT : Use numpy.polynomial.polynomial, there is an easy method to find f'(x) with this

Next, look at other easy methods that will not require curve fitting.