SambaMixer: State of Health Prediction of Li-ion Batteries using Mamba State Space Models

🔼 The SambaMixer architecture takes multi-variate time series data (current, voltage, temperature, and sample time) as input. The sample time is first resampled using an anchor-based method to ensure consistent length across different cycles. The resampled sample time is then fed into a positional encoding layer, along with the time difference between consecutive discharge cycles (in hours), which is also positionally encoded. The current, voltage, and temperature data undergoes an input projection layer before being combined with the positional embeddings. A CLS token (optional) can be added. This combined data feeds into the SambaMixer encoder, which consists of multiple stacked SambaMixer encoder blocks. The encoder output is finally passed to the head, which predicts the state of health (SOH) for a given cycle of a specific battery.

read the caption

Figure 2: SambaMixer architecture. We input a multi-variate time series of current, voltage, temperature and sample time. We first first resample the time signals using our anchor-based resampling technique. We then feed the resampled sample time into the sample time positional encoding layer. We further feed the time difference between two discharge cycles in hours into the cycle time difference positional encoding layer. The other signals, i.e. current, voltage and temperature are fed into the input projection. The projected signals are added to the sample time embeddings and the cycle time difference embeddings. Optionally, a CLS token can be inserted at any position. The embedded tokens are then fed into the SambaMixer Encoder. The SambaMixer Encoder consists of M𝑀Mitalic_M stacked SambaMixer Encoder blocks. The output of the encoder is finally fed into the head, which predicts the state of health of the current cycle k𝑘kitalic_k for battery bψsubscript𝑏𝜓b_{\psi}italic_b start_POSTSUBSCRIPT italic_ψ end_POSTSUBSCRIPT.

🔼 Figure 3 illustrates four different resampling techniques applied to a sample time sequence. The original sequence is shown with its actual, variable number of samples (represented as L_k^ψ). Three resampling methods are then compared to the original: linear resampling creates a new sequence with an equal number of equidistant samples; random resampling generates a new sequence with the same number of samples randomly selected from a uniform distribution across the range of the original data; finally, anchor-based resampling begins with equidistant samples (like linear resampling) but adds random noise to each sample, creating slight variations around the original equidistant anchors.

read the caption

Figure 3: Resample techniques. Original: The original sample time sequence with Lkψsuperscriptsubscript𝐿𝑘𝜓L_{k}^{\psi}italic_L start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_ψ end_POSTSUPERSCRIPT samples. Linear: linear resampling with L𝐿Litalic_L equidistant samples. Random: random resampling with L𝐿Litalic_L samples drawn from a uniform distribution. Anchor: anchor-based resampling with random uniform noise z𝑧zitalic_z added to L𝐿Litalic_L equidistant samples.

🔼 The figure visualizes the capacity degradation patterns observed across several lithium-ion batteries over their lifespan. The x-axis represents the cycle number (number of charge-discharge cycles), while the y-axis denotes the state of health (SOH) expressed as a percentage. Each line corresponds to a different battery, illustrating how the SOH diminishes over time. This graph highlights the variability in battery degradation rates and provides a visual representation of the data used to train and validate the models described in the paper.

read the caption

Figure 4: Capacity degradation for all selected batteries.

🔼 This figure displays the predicted state of health (SOH) for Battery #06 over its lifespan, alongside the actual measured SOH values. The plot showcases the model’s ability to accurately predict the battery’s degradation over time, with the predicted SOH values closely tracking the ground truth. It also shows the prediction error, highlighting the accuracy of the model’s predictions throughout the battery’s lifetime. Additionally, the plot indicates the predicted and actual end of life (EOL) of the battery, demonstrating the model’s capacity to foresee the point at which the battery reaches the end of its usable lifespan.

read the caption

Figure 5: SOH prediction for Battery #06

🔼 This figure showcases the predicted State of Health (SOH) values for Battery #07 over its lifespan, compared against the actual measured SOH. It provides a visual representation of the model’s accuracy in predicting SOH degradation over time, indicating both the predicted SOH and the prediction error. The plot also highlights the End of Life (EOL) prediction from the model and compares it to the actual EOL point for this specific battery.

read the caption

Figure 6: SOH prediction for Battery #07

🔼 This figure displays the predicted state of health (SOH) for battery #47 over its lifespan, comparing the model’s prediction to the actual measured SOH. It visualizes the prediction accuracy by showing the difference between the predicted and actual SOH values over a series of discharge cycles. The plot also indicates the predicted end-of-life (EOL) point, comparing it with the actual EOL of the battery. The prediction error is also presented, visually representing the model’s performance in SOH estimation.

read the caption

Figure 7: SOH prediction for Battery #47

🔼 This figure presents a histogram visualizing the distribution of State of Health (SOH) values from the NASA-L dataset, which is used to train and evaluate a deep learning model for Li-ion battery health prediction. The histogram compares the SOH value distributions for the training and evaluation subsets of the NASA-L dataset, showing how frequently certain SOH ranges appear in each subset. A total of 50 bins were used to create this histogram. The purpose is to illustrate the data’s characteristics and how it might influence the model’s training and evaluation performance. Differences between the training and evaluation distributions might point to potential overfitting or insufficient data representation issues.

read the caption

Figure 8: Histogram of SOH value counts. Comparison of train and eval split of the NASA-L dataset. Number of bins: 50.

🔼 This figure visualizes the results of a model scaling experiment. It shows how the mean absolute error (MAE) in state-of-health (SOH) estimation changes based on different model sizes (S, M, L, XL) and datasets (NASA-S, NASA-M, NASA-L). Each bar represents the MAE achieved by a specific model on a specific dataset. This allows for a direct comparison of performance across different model complexities and data amounts, helping to determine the optimal combination for accurate SOH prediction.

read the caption

Figure 9: Model scaling experiment. MAE metric for the SOH estimation task for different model sizes and datasets. Values are reported in Table VI

🔼 This table details the characteristics of various NASA Lithium-ion batteries used in the experiments. For each battery, it provides the discharge profile (constant current or pulse width modulation), the ambient temperature during the discharge tests, the cut-off voltage at which the discharge cycle ends, and the battery’s initial capacity at the start of the measurement campaign.

read the caption

TABLE II: Discharge specifications for various NASA Li-ion batteries. For the profile we report the discharge current signal form and the discharge amplitude. Ta⁢m⁢bsubscript𝑇𝑎𝑚𝑏T_{amb}italic_T start_POSTSUBSCRIPT italic_a italic_m italic_b end_POSTSUBSCRIPT is the ambient temperature, VC⁢Osubscript𝑉𝐶𝑂V_{CO}italic_V start_POSTSUBSCRIPT italic_C italic_O end_POSTSUBSCRIPT is the cut-off voltage and Initial Capacity is the initial capacity of the battery at the beginning of the measurement campaign.

🔼 This table details the different training and evaluation splits used for the NASA Li-ion battery datasets in the experiments and ablations of the paper. Each row represents a specific battery ID from the NASA dataset, indicating whether that battery’s data was used for training or evaluation in the various experiments and ablations. The table helps to clarify which datasets were used for model training, validation, and testing purposes, enabling readers to better understand and interpret the results presented in the paper.

read the caption

TABLE III: Different Training and Evaluation splits for the NASA Li-ion batteries used throughout our experiments and ablations.

🔼 This table compares the performance of the SambaMixer models (introduced in this paper) against the state-of-the-art Mazzi et al. (2024) model for predicting the state-of-health (SOH) of Lithium-ion batteries using the NASA dataset. The comparison uses three common metrics for evaluating regression models: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE). The results for each metric are provided for several individual batteries from the NASA dataset, allowing for a battery-by-battery comparison of model accuracy. The best performing model for each battery is indicated in bold.

read the caption

TABLE IV: Comparing our SambaMixer models with the state-of-the-art Mazzi et al. (2024) on the NASA Li-ion batteries. We report the MAE, RMSE and MAPE for each battery. The best results are highlighted in bold.

🔼 This table presents a comparison of the SambaMixer model’s performance when trained on different datasets. The model was trained on three variations of the NASA Li-ion battery dataset: NASA-S, NASA-M, and NASA-L, each representing different sizes of data. The evaluation sets remain consistent across all training sets. The table displays the MAE (Mean Absolute Error), RMSE (Root Mean Squared Error), and MAPE (Mean Absolute Percentage Error) metrics for each training set. This allows for a direct comparison of the model’s accuracy and generalization capabilities when trained on datasets with varying data sizes.

read the caption

TABLE V: Performance of our SambaMixer model when trained on different training sets. Evaluation sets are the same for all datasets.

🔼 This table presents the results of an experiment assessing the impact of model size and dataset size on the accuracy of State-of-Health (SOH) prediction for lithium-ion batteries. Different sized SambaMixer models (S, M, L, XL) were trained on three datasets (NASA-S, NASA-M, NASA-L) of varying sizes. The table reports the Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE) for each model-dataset combination, providing a comprehensive view of the model’s scalability and performance across different data conditions.

read the caption

TABLE VI: Model scaling experiment. We report the metrics MAE, RMSE and MAPE for the SOH estimation task for different model sizes and datasets.

🔼 Table VII presents a detailed comparison of State-of-Health (SOH) estimation performance across different starting points within the battery discharge cycles for multiple batteries. The evaluation utilizes the same evaluation set across all scenarios. The table compares the performance of the SambaMixer model against results reported by Mazzi et al., offering a comprehensive assessment of predictive accuracy for various stages of battery life. Metrics included are Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Absolute End-of-Life Error (AEOLE). The ‘Start’ column indicates the cycle at which the SOH prediction begins, where capital letters within parentheses correspond to scenario labels used by Mazzi et al. ‘N/R’ indicates that Mazzi et al. did not report results for that specific starting point.

read the caption

TABLE VII: SOH estimation performance on the evaluation batteries starting at different cycle IDs. We report the metrics MAE, RMSE and MAPE for the SOH estimation task and the AEOLE for EOL indication. Capital letters in brackets for the start column represent Mazzi et al. notation for those scenarios. N/R=Not Reported.

🔼 This table presents the results of an ablation study on the impact of using a class token in the SambaMixer model. The study examines different positions for the class token (tail, middle, head) and the effect of omitting it entirely. The table shows the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) for each class token configuration and the ’none’ (average) condition. The results help assess the optimal strategy for incorporating class tokens in the model architecture to improve its performance. The results are important for understanding and optimizing the model’s architecture.

read the caption

TABLE VIII: Ablation of inserting a class token into the input token sequence and at which positions.

🔼 This table presents an ablation study comparing the performance of two different backbone architectures: a vanilla Mamba model and the SambaMixer model proposed in the paper. The comparison is done using the MAE, RMSE, and MAPE metrics, providing a quantitative assessment of the impact of the SambaMixer architecture on the model’s accuracy in predicting the state of health of lithium-ion batteries.

read the caption

TABLE IX: Ablation of different backbone architectures.

🔼 This table presents the results of an ablation study comparing different resampling methods used in the SambaMixer model for predicting the State of Health (SOH) of Li-ion batteries. The methods compared are linear resampling, random resampling, and the proposed anchor-based resampling. The table shows the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) for each resampling technique, allowing for a quantitative comparison of their effectiveness. The results highlight the relative performance of different methods for handling variations in sample lengths across different discharge cycles of batteries.

read the caption

TABLE X: Ablation of various resampling methods.

🔼 This table presents an ablation study on the impact of different positional encoding methods on the performance of the SambaMixer model for predicting the state-of-health of Li-ion batteries. The study compares three methods: no positional encoding, sample time positional encoding, and combined sample time and cycle time difference positional encoding. The results show the MAE, RMSE, and MAPE for each method, demonstrating the effectiveness of incorporating both sample time and cycle time difference for improved prediction accuracy.

read the caption

TABLE XI: Ablation for various positional encoding methods.