[FEATURE] *Validity Check and Bad Data Handling in State Estimation*

[scud-soptim]

The new feature should introduce a function to calculate the expected value $E$ and the actual value of the minimization function $J(x)$. If $J(x)$ lies within the 3-sigma band, the result of the state estimation is considered valid. Additionally, bad data should not be included in the determination of $J(x)$, or should only be considered with an adjusted weight in the calculation of $J(x)$.

Detailed Description:

1. Calculation of Expected Value $E$ and Variance of $E$:

   • The function should calculate the expected value $E$ as $E = m - n$, where:
     • $m$ is the number of measurements.
     • $n$ is the number of state variables ( = 2 * Number of Busses - Slack )
   • The variance $\sigma_J^2$ should be calculated as $\sigma_J^2 = 2(m - n)$. so: $\sigma_J = sqrt(2*(m-n))$

2. Calculation of Minimization Function $J(x)$:

   • The function should compute ( J(x) ) as the sum of the weighted squared residuals:

     $J(x) = \sum_{i=1}^{m} \left( \frac{r_i}{\sigma_i} \right)^2$

   • The residual $r_i$ is defined as $r_i = z_i - h_i(x)$, where $z_i$ is the measurement and $h_i(x)$ is the predicted value based on the state $x$.

3. Validity Check within the 3-Sigma Band:

   • Check if $J(x)$ lies within the 3-sigma band: $E - 3\sigma_J < J(x) < E + 3\sigma_J$.
   • If $J(x)$ lies within this range, the result of the state estimation is considered valid.

4. Handling Bad Data:

   • Bad data can be detected by evaluating the residual $r_i$. If $|r_i|$ exceeds a threshold, the data is considered bad. The threshold can be defined as follows: $|r_i| > \alpha \cdot \sigma_i $, where $\alpha$ is typically in the range of 3 to 5.

   • Bad data should be excluded from the calculation of $J(x)$ or included with an adjusted weight $w$. The adjusted weight can be calculated as:

     $w_i^{\text{new}} = \frac{w_i}{1 + \alpha \cdot \frac{r_i^2}{\sigma_i^2}}$

• $w_i = \frac{1}{\sigma_i^2}$