Evaluating performance of data model

[meljsingh]

This post explains the performance parameters to be plotted for the data forecasting model.

Inputs

• observed data: from zensie_data table, temperature from sensors 16B1, 16B2 and 16B4

observed_temps

• temperature forecast: data from the last temperature forecast of sensors 16B1, 16B2 and 16B4

forecast_temps

Performance

1. Isolate timestamps required:

which_timestamps = forecast_temps[timestamp]

2. Calculate performance metrics for each sensor and forecast

# function which calculates and outputs three forecast statistics based on mean observed values and mean forecast values

def performance_stats(obs,forecast_mean):
    RMSE = sqrt(mean((forecast_mean - obs)^2)) # root mean square error 
    MAE = mean(abs(forecast_mean - obs)) # mean average error
    MBE = mean(forecast_mean - obs) # mean bias error
    stats = vector(RMSE,MAE,MBE)

# for sensor 16B1
obs= observed_temps[timestamp=which_timestamps, Temperature_Farm_16B1]
forecast_mean = forecast_temps[,"Forecast_mean_16B1"]

3. Plot the performance

e.g. Plot the MBE for the past 14 forecasts for each model at location 16B4 (past 7 days)