PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchFor logits shaped (batch, classes), PyTorch uses dim=1 to turn each example’s class scores into probabilities. For classification training, pass the raw logits—not those probabilities—to CrossEntropyLoss. Use log_softmax when you specifically need log probabilities.
What does dim mean in PyTorch softmax?
The dim argument selects the tensor axis whose values are normalized together. PyTorch applies softmax independently to each slice along that axis: it exponentiates the values and divides each by the sum of exponentials in the slice. The resulting values are between 0 and 1 and sum to 1 along the selected dimension. See the PyTorch softmax documentation.
As an Amazon Associate I earn from qualifying purchases.
For a tensor shaped (N, C), where N is the batch size and C is the number of classes, dim=1 normalizes the class scores separately for each example. The axis is determined by your tensor layout; select the one that indexes the mutually exclusive classes.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Choosing the class axis
(N, C): usedim=1for per-example class probabilities.(N, C, H, W): classes are on dimension 1, sodim=1produces a class distribution for each spatial location.
Applying softmax on the wrong axis still produces normalized values, but they will sum to 1 across the wrong set of elements. Check the shape and meaning of every axis rather than choosing a dimension by habit.
#1 Best Overall
What is the difference between log_softmax and softmax?
torch.softmax(input, dim=...) returns probabilities. torch.nn.functional.log_softmax(input, dim=...) returns their logarithms, or log probabilities. Use the first when probabilities themselves are needed, such as for reporting or inference; use the second when the next operation expects log probabilities.
When log probabilities are needed, call log_softmax directly instead of computing softmax and then taking a logarithm. The PyTorch log_softmax documentation notes that the separate operations are slower and numerically unstable; log_softmax uses an alternative formulation to compute the output and gradient correctly.
Rank #2
Should I apply softmax before CrossEntropyLoss?
No. PyTorch CrossEntropyLoss expects unnormalized logits, so give it the model’s raw class scores. For class-index targets, the loss is equivalent to applying LogSoftmax and then NLLLoss, as described in the functional cross-entropy documentation.
Recommended Free Tools
# logits: (batch, classes); targets: class IDs, shape (batch)
loss_fn = torch.nn.CrossEntropyLoss()
loss = loss_fn(logits, targets)
# Convert to probabilities only when needed for reporting or inference
probabilities = torch.softmax(logits, dim=1)This example assumes classes occupy dimension 1. For another layout, keep the model output and target shapes consistent with the class axis expected by the loss.
Rank #3
Which target format should I use?
CrossEntropyLoss accepts either class-index targets or class-probability targets. Choose the form that matches the labels you actually have.
| Target form | Target shape | When to use it |
|---|---|---|
| Class indices | Input shape with the class axis omitted. For logits (N, C), targets have shape (N); for spatial logits, targets match the non-class dimensions. |
Ordinary classification with one class ID per example or spatial location. |
| Class probabilities | Same shape as the logits. | Soft labels or blended labels represented as distributions over classes. |
Class-index targets
For logits with C classes, each target is a class ID in [0, C), except for an ignore_index value when that option is configured. This is usually the appropriate and more computationally optimized form when each example has one correct class.
Rank #4
Probability targets
Each target should be a valid probability distribution over classes, with the same shape as the logits. PyTorch does not strictly validate that these targets satisfy probability constraints, so invalid values can produce misleading losses and unstable gradients. Use this format when soft or blended labels are intentional, not as a substitute for ordinary class IDs.
What other CrossEntropyLoss options affect the result?
The loss accepts an unbatched class vector (C), a batch (N, C), or higher-dimensional input (N, C, d1, …, dK), where dimension 1 is the class axis for the higher-dimensional forms. Its reduction option can be 'none', 'mean', or 'sum'; the documented default is 'mean'. It also supports class weights and label_smoothing. ignore_index applies to class-index targets.
The mean reduction is not identical for both target forms: for class indices, the documented mean accounts for class weights and ignored targets; for probability targets, it divides the summed element losses by the number of loss elements. Consult the loss documentation when the reduction details matter to your calculation.
Common mistakes to avoid
- Applying softmax before the loss: pass logits directly to
CrossEntropyLoss. - Normalizing the wrong axis: set
dimto the axis that represents classes in your tensor. - Taking
log(softmax(...)): uselog_softmaxdirectly when log probabilities are required. - Passing one-hot or soft labels with an incompatible shape: probability targets must match the logits’ shape and represent valid distributions.
- Assuming index and probability targets behave identically: they have different shapes and documented mean-reduction behavior.
Version note
The API references here are PyTorch’s main functional documentation and stable documentation labeled 2.14 for CrossEntropyLoss and functional cross_entropy. Documentation and API behavior can change; check the documentation for the PyTorch release used by your project.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problems
