[QUESTION] Chapter 4, how do I roll back model parameters? (Early Stopping)

[serchinnho]

Hi,

Please be patient, I am very new to machine learning. The Early Stopping section introduces an implementation. I ran the code and everything went smooth. My question is how I use the epoch and model that I just found. Here's the code:

from copy import deepcopy

#Creating the data
np.random.seed(42)
m = 100
X = 6 * np.random.rand(m, 1) - 3
y = 2 + X + 0.5 * X**2 + np.random.randn(m, 1)

X_train, X_val, y_train, y_val = train_test_split(X[:50], y[:50].ravel(), test_size=0.5, random_state=10)

# Preparing the data
poly_scaler = Pipeline([
        ("poly_features", PolynomialFeatures(degree=90, include_bias=False)),
        ("std_scaler", StandardScaler())
])
X_train_poly_scaled = poly_scaler.fit_transform(X_train)
X_val_poly_scaled = poly_scaler.transform(X_val)

sgd_reg = SGDRegressor(max_iter=1, tol=-np.infty, warm_start=True, penalty=None, learning_rate="constant", eta0=0.0005)

minimum_val_error = float("inf")
best_epoch = None
best_model = None
for epoch in range(1000):
    sgd_reg.fit(X_train_poly_scaled, y_train) #this continues where it left off
    y_val_predict = sgd_reg.predict(X_val_poly_scaled)
    val_error = mean_squared_error(y_val, y_val_predict)
    if val_error < minimum_val_error:
        minimum_val_error = val_error
        best_epoch = epoch
        best_model = deepcopy(sgd_reg)

Versions

• OS: Windows 10.0.19044 Build 19044
• Python: 3.9.7
• Scikit-Learn: 0.24.2

The best epoch is around 230 ish. I just don't understand how I can use best_model to make predictions on X. Thank you so much in advance.

[ageron]

Hi @serchinnho ,

Thanks for your question, and my sincere apologies for the late response, I somehow missed your message. The best_model object is a clone of the best SGDRegressor model, so it can be used completely normally, there's no need to roll back model parameters, since this object contains the best ones. So once early stopping is finished, just use best_model normally, for example:

y_pred = best_model.predict(X)

Hope this helps.

[serchinnho]

Hi Aurelien!

Don't worry about the late response, it happens. I forgot to update this entry, but I checked the exercises on this and other chapters and got my answer. I've tried online courses with videos and only text and your book is by far the tool that has worked the best for me! I'm already at Chapter 8 because I had to stop and learn some statistical methods but I plan on finishing reading. Really great work!

[ageron]

Thanks a lot for your very kind words, @serchinnho , and I'm really glad you found the solution!