Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteClean time-series data by correcting verified errors, treating missing values according to why and how long they are missing, and preserving unusual observations until you have evidence they are wrong. Keep the original measurements, record every transformation, and check that the cleaned series still reflects the trend, seasonality, peaks, and abrupt changes that matter to your task.
Decide what “clean” means for your task
Data repair, prediction, and historical reconstruction have different goals. Repair means correcting known measurement or recording errors. Prediction means preparing inputs without leaking future information or discarding useful cases. Reconstruction means estimating missing historical values as credibly as possible. A transformation suitable for one goal may be unsuitable for another; scikit-learn’s imputation guide distinguishes prediction from reconstructing the data itself.
As an Amazon Associate I earn from qualifying purchases.
Keep an immutable copy of the raw series. Store cleaned or estimated values separately, along with a reason and method for each change. That lets you revisit a decision instead of silently replacing evidence.
Check timestamps and cadence before values
A time series is an ordered sequence, and many methods assume a meaningful interval structure. Parse timestamps consistently, confirm the expected time zone, sort observations, and verify the intended cadence. Identify duplicate timestamps and determine whether they are duplicate ingestion, separate events, or values that should be aggregated. Check units and convert sentinel values such as special numeric codes into genuine missing values where appropriate. Forecasting: Principles and Practice notes that its regular-interval discussion does not cover every irregularly spaced series; do not force irregular observations onto a regular grid without a reason.
#1 Best Overall
Profile the raw series before changing it
Plot the measurements over time. Look for gaps, repeated values, shifts in level, seasonal structure, and extremes. Compare suspicious periods with operational context, sensor logs, related series, or source records when available. A numeric cutoff can flag candidates, but it cannot establish that a measurement is wrong.
For series with trend and seasonality, Hyndman and Athanasopoulos demonstrate screening the remainder after robust STL decomposition in their discussion of outliers and missing values. In that example, they use a threshold of 3 interquartile ranges (IQR) from the central 50 percent as a stricter flagging rule. The book contrasts this with a 1.5-IQR fence, which under a normally distributed remainder would label about 7 in every 1,000 observations; its 3-IQR rule would label about 1 in 500,000 under that same assumption. These are textbook illustrations for a particular assumption, not universal error rates or guarantees for real-world series.
Investigate outliers; do not automatically delete them
An extreme observation may be a recording mistake, a genuine rare event, or evidence that the model you are using does not describe the data well. NIST distinguishes flagging an outlier for investigation from deciding it is bad data, and notes that an unusual point may be scientifically interesting or reveal an unsuitable distributional model. NIST’s outlier guidance supports removal when the observation can be determined to be erroneous, not simply because it is far from the rest.
Recommended Free Tools
- Verified error: Correct it from a reliable source value if possible. If the correct value cannot be established, mark it as missing and decide whether an estimate is appropriate.
- Plausible real event: Retain the measurement and add context, such as an event flag or intervention variable, if it helps explain the series.
- Uncertain candidate: Keep the original value, flag it for review, and assess whether conclusions change under a robust treatment.
As Hyndman and Athanasopoulos put it, “Simply replacing outliers without thinking about why they have occurred is a dangerous practice.”
Rank #3
Reduce influence without rewriting history
When extreme values distort a model or visualization, use a method suited to the analysis rather than pretending the observations never occurred. For example, scikit-learn’s preprocessing guide describes robust scaling as preferable to mean-and-variance scaling when many outliers are present. Scaling changes the representation used by a model; it does not correct the original measurements. Preserve the raw values and make the transformation explicit.
Handle missing values by cause and gap length
First ask why observations are missing and whether the timing relates to the outcome. A store’s missing sales on a public holiday may reflect a closure and a different pattern on the following day; randomly missed sensor readings have another cause. Missingness can itself carry information, so record its pattern and consider a context variable where relevant.
Choose a treatment based on the series’ cadence and behavior, not just the fact that a cell is empty:
- Brief gap in a smooth series: Linear or time-aware interpolation may be reasonable. Interpolation estimates values; it does not recover ground truth. Pandas documents multiple methods, including linear and time-index-aware interpolation, and supports a
limiton consecutive missing values filled. See the pandas missing-data guide. - Long gap or likely regime change: Avoid drawing a smooth bridge across a period where the process may have changed. Consider a model that accounts for trend, seasonality, and known drivers, or leave values missing if the downstream method can handle them.
- Gap at the beginning or end: Be cautious about extrapolation: there may be no observations on one side to support an interpolation estimate.
- Structured or outcome-related missingness: Investigate the mechanism and use relevant context rather than treating the gaps as random.
Different interpolation methods can give materially different estimates. Keep an indicator identifying which values were observed and which were estimated, and set a maximum gap length based on the cadence and the process rather than filling every run of missing values by default. For longer gaps, Forecasting: Principles and Practice demonstrates ARIMA-based interpolation for a series containing missing observations; it is an example, not a universal recommendation.
For prediction, do not discard incomplete rows by reflex
Dropping every row with a missing value can discard useful cases and introduce bias unless missingness is completely at random, according to scikit-learn’s imputation guide. Depending on the estimator and data, options include simple statistical imputation, KNN or iterative imputation, a missingness indicator, or an estimator that natively accepts missing values. Fit any learned imputation or scaling step using training data only; using future or held-out observations to prepare earlier inputs can leak information into evaluation.
For predictive work, start with a simple defensible treatment and test whether a more complex imputer improves time-ordered validation. For historical reconstruction, spend more effort judging the estimates themselves because the values—not merely the prediction—are the output of interest.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Validate that cleaning preserved the signal
Compare the raw and transformed series, marking every changed or estimated observation. Check whether the features that matter to the task remain visible:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Trend and seasonal shape
- Peak size and timing
- Abrupt level changes and unusual events
- Summary statistics before and after transformation
For prediction, evaluate with time-ordered validation appropriate to the use case. If a result improves only after difficult periods are removed, that is a change in the evaluation set, not proof that cleaning improved the model. Keep the observed-versus-imputed mask so later users do not mistake estimates for measurements.
Quick Recap
A practical decision checklist
- State the goal: repair, prediction, or historical reconstruction.
- Preserve the source: retain raw timestamps and values, and log each edit or estimate.
- Verify the time axis: check time zone, order, cadence, duplicates, units, and missing-value sentinels.
- Inspect before filtering: plot the series and investigate suspicious periods against domain context.
- Classify extremes: correct verified errors, retain plausible events, and flag unresolved candidates.
- Diagnose gaps: identify their cause and duration before selecting a method.
- Constrain estimation: choose an imputer that fits the cadence and process, set an appropriate gap limit, and mark estimated values.
- Validate the result: check whether the cleaned series and downstream conclusions still represent the signal you need.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
