Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsUse groupby(...).agg(...) when you want one summary row per group; use groupby(...).transform(...) when you want a group calculation attached to every original row. The distinction explains why a grouped result is shorter than its source—and how to avoid losing track of which output belongs to which record.
Choose a GroupBy operation by the shape you need
A pandas GroupBy operation splits data into groups, applies a calculation, and combines the results. The method you choose determines whether the result summarizes groups, stays aligned to the input rows, or selects original records.
As an Amazon Associate I earn from qualifying purchases.
| Need | Use | Typical result |
|---|---|---|
| One or more calculated values for each group | agg, aggregate, or a built-in reduction such as sum() |
One row per group |
| A group statistic repeated for each source row | transform |
Same index and row alignment as the grouped object |
| Keep or select original rows according to a group condition | Filtration methods such as filter, head, tail, or nth |
A selection of original rows |
| Custom logic or output shape that specific methods do not cover | apply |
Depends on what the function returns |
The pandas GroupBy reference and user guide describe these operations and their result shapes.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchSummarize groups with agg
Aggregation intentionally reduces the data: the output has a row for each group rather than a row for each input record. For example, this groups players by team and calculates the average points and number of non-missing player names:
#1 Best Overall
import pandas as pd
df = pd.DataFrame({
"team": ["A", "B", "A", None],
"player": ["Mina", "Noah", "Omar", "Pia"],
"points": [10, 7, 14, 8],
})
summary = (
df.groupby("team", as_index=False, dropna=False)
.agg(points_mean=("points", "mean"), player_count=("player", "count"))
)The summary DataFrame has one row per team group, including a group for the missing team value. The named aggregation syntax assigns clear output column names to calculations. See the pandas aggregation reference for supported forms.
Keep group keys in columns or move them out of the index
By default, grouped DataFrame aggregations put group labels in the result index. When grouping by multiple keys, those labels are normally a MultiIndex. Use as_index=False for SQL-style output with group keys as regular columns, as in the example. Alternatively, aggregate with the default and call reset_index() to turn index levels into columns.
as_index controls aggregation output; it does not make a transformation return a different number of rows.
Recommended Free Tools
Keep every input row with transform
When each record needs a value calculated from its group, use transform. It returns a result aligned to the original grouped object, so the values can be added as a new column:
df["team_mean"] = df.groupby("team", dropna=False)["points"].transform("mean")Here, each row receives its team’s mean points, and the DataFrame retains its original rows. In contrast, summary is deliberately shorter because it has one row per group. The transform reference documents its same-indexed output and alignment behavior.
Do not assign a shorter aggregation result to the full original DataFrame as if its rows still matched one-to-one. If you need to attach a separate summary, join it using explicit grouping keys and check index alignment; use transform when the value is naturally a per-row group calculation.
Rank #4
Understand missing keys, group order, and categorical values
Missing grouping keys are excluded by default
With the default dropna=True, rows whose grouping key is missing do not participate in a group. That can make the grouped result appear to have lost records. Set dropna=False when missing-key rows should contribute to an NA-key group. The example uses this option for both aggregation and transformation so the row with no team is retained in a group.
Group-key sorting is separate from row order within groups
By default, sort=True sorts group keys. Set sort=False to use the order in which group keys first appear in the data. pandas preserves the observations’ order within each group either way; the sort option governs group-key ordering, not the order of records inside a group. See the GroupBy sorting guide.
Specify categorical behavior when version consistency matters
For categorical group keys, pandas 3.0 changed the default for observed to True, which includes only categories that occur in the data. Older versions had a different default. If your output must be consistent across versions—or must include unobserved categories—set observed= explicitly according to the output you need. The DataFrame.groupby reference documents this option and its version history.
Use apply only when a specific method does not fit
apply can combine results from a function that returns a DataFrame, Series, or scalar, so its output shape depends on that function. Prefer a specific GroupBy operation such as agg or transform when it expresses the task: pandas notes that apply can be slower than these methods. The apply reference describes its flexibility and performance caveat.
If you do use apply, pay attention to group_keys, which affects index construction. Its default became True in pandas 2.0; for an apply result with a like-indexed shape, inspect the resulting index rather than assuming it matches the input.
Version notes for transformations
For a DataFrame-returning transform, pandas 2.0 changed behavior so the result index aligns with the input index. If you intentionally need raw positional behavior instead of index alignment, the transform documentation describes using .to_numpy() for that purpose.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
