[Bug]: PyBaMM Unable to accurately re-create Ecker 2015 figures.

[Tazarus]

PyBaMM Version

22.5

Python Version

3.9.7

Describe the bug

In recreating figures 4a), 7a), and 7b), from Ecker et al's 2015 paper "Parameterization of a Physico-Chemical Model of a Lithium-Ion Battery" I was unable to accurately replicate the groups simulated results using PyBaMM.

I recreated figures 4a), 7a), and 7b). Including digitizing the original measurement data and overlaying it with the PyBaMM results to compare against the simulations used by the original authors.

The PyBaMM DFN and SPMe results were noticeably less accurate then the discharge curve simulations done by Ecker et al in the original paper for most figures, while still doing a reasonable job at normal operating temperatures.

However, at higher charge rates and lower temperatures the discrepancies got quite bad. Particularly for figures 7b) at discharge 1.3C , and figure 7a) at -10 celcius.

Considering there is already a premade Ecker 2015 parameter set for this very paper, I was surprised that I was unable to reproduce the simulation results, even using a full DFN model.

I have included both the original figures and the figures I produced using PyBaMM with mirrored styles for easily comparison. I also included a simplified .py file with the basics for reproducing the most troublesome discharge curve from figure 7b).

I also included my full .ipynb file for the entire experiment, and the digitized measurement data from the figures from the paper (digitized using grapher) if anyone wants to have access to the experimental data for comparison purposes.

The full .ipynb file is included primarily for completeness, it is much longer, and not really intended for troubleshooting, as it is mostly just looping through lists of parameters with slight experimental or temperature variations to reproduce the ~4 discharge curves on each figure. But it is included in case anyone is interested.

PyBaMM_Ecker.zip Ecker_figures.zip

Steps to Reproduce

import pybamm
import matplotlib.pyplot as plt

model = pybamm.lithium_ion.DFN(name="DFN")

chemistry = pybamm.parameter_sets.Ecker2015

K = 273.15 # Celsius to Kelvin conversion factor
params = pybamm.ParameterValues(chemistry=chemistry)
params["Ambient temperature [K]"] = -10 + K
params["Initial temperature [K]"] = -10 + K
params["Reference temperature [K]"] = -10 + K

experiment = pybamm.Experiment([("Discharge at 1.3C until 2.85V (1 second period)")])

sim = pybamm.Simulation(model, experiment = experiment, parameter_values=params)

sol = sim.solve()

pybamm.plot_voltage_components(sol)

plt.plot(sol["Discharge capacity [A.h]"].data*48, 
            sol["Terminal voltage [V]"].data)

# Optional figure formatting
plt.grid(True)
plt.xticks([0,2,4,6,8])
plt.xlim(0,8)
plt.ylim(2.6,4.2)

plt.xlabel("Discharge Capacity [Ah]")
plt.ylabel("Voltage [V]")
name = model.name
temp = params['Ambient temperature [K]'] - C_to_K

plt.title(rf"Fig 7b at 1.3C using {name} model at {temp}$^\circ$C")
plt.show()

Relevant log output

No response

[valentinsulzer]

Would you be able and willing to double check all the parameters in the Ecker paper and the ones in the PyBaMM dataset to see where the discrepancy is?

[Tazarus]

**TLDR: Willing yes.

Able? I am unsure. If so, likely not until August**

1. I may lack the knowledge to do so. I am only a senior undergraduate student who was asked to test run this software for the lab who hired me.

2. The Ecker paper reproductions I made were just a test run, and we are moving on to trying to implement a parameter set based on the cells we are using/developing here in the lab.

Until I have made significant progress on the project I was hired for, I won't have additional time to work on side contributions. I likely won't know how much time I'll have to dedicate to the PyBaMM project itself until August.

[valentinsulzer]

For 1, you would basically need to look for the files where the Ecker parameters are defined, and compare each parameter with the value given in the paper. Either there is an error in the transcribing of the parameters, or the pybamm simulations are wrong, or Ecker's simulations were wrong.

For 2, understandable, we are all in the same boat :)

[TomTranter]

Thanks for the comprehensive analysis and raising the issue. As this is an important paper and one of the few very well defined parameter sets I think we should attempt to fix this and in the next month or so I can take a closer look. Quickly glancing at the -10[deg] it could be an issue with the particle diffusivity and possibly the mesh resolution as the big drop in the middle of discharge looks quite similar to some funny results I was seeing using the ORegan dataset which needs a finer mesh.

[MustafaDragonborne]

The nominal cell capacity is specified to be 7.5Ah in the paper. Why is it defined as 0.15625 Ah in the PyBaMM dataset?