Evaluate a photovoltaic defect segmentation model with per-class IoU or Dice, precision and recall, and a separate breakdown for small targets—not a single overall score. State the task, dataset, split, labels, class support, probability threshold and aggregation method. This makes it possible to see whether a model actually finds scarce defects, localizes their boundaries, or merely performs well on the much larger background.
Why rare and small defects need a different evaluation lens
In pixel segmentation, every pixel contributes to the confusion counts. When defect pixels are scarce, a model can classify most of an image correctly by predicting background almost everywhere, while still missing the defect that matters. An overall score can therefore look reassuring without demonstrating useful defect segmentation.
As an Amazon Associate I earn from qualifying purchases.
The MSPS project illustrates the scale of the issue for its particular task: binary segmentation of monocrystalline silicon cell electroluminescence (EL) images. Its project materials report a mean background-to-defect pixel ratio of 2020:1, or approximately 0.05% defect pixels. These are characteristics of that dataset, not universal statistics for photovoltaic imagery.
Small masks create a second problem: a modest boundary displacement may account for a large fraction of a tiny object. A model can identify the general location yet produce poor overlap, or miss a small defect entirely while scoring well on large image regions. The 2023 EL benchmark authors reported higher mIoU for large image features than for the smaller defects they evaluated. That result is a reason to inspect target size explicitly, not a universal performance rule.
#1 Best Overall
- 900-1G136-2505-000
What a good evaluation should answer
- Does the model find each relevant defect class, including rare classes?
- When it predicts a defect, how much of that prediction overlaps the annotated mask?
- How many predictions are false alarms, and how many annotated defects are missed?
- Does performance change for the smallest targets or the most difficult image conditions?
- Can another team reproduce the result from the stated data split, labels, threshold and metric definitions?
Define the prediction task before comparing scores
“PV defect segmentation” can describe materially different tasks. A score only has meaning beside its task definition and label set. Identify whether the model does binary semantic segmentation, multi-class semantic segmentation, instance segmentation, or anomaly segmentation. State whether labels cover defects only or also include non-defect features, and document what annotators were instructed to mark as a defect.
| Task | What the output represents | What to state |
|---|---|---|
| Binary semantic segmentation | Each pixel is assigned to defect or background. | Which defect types are grouped together, the background definition, and any ignored or uncertain pixels. |
| Multi-class semantic segmentation | Each pixel is assigned to one of several defect or feature classes. | Every class name, whether feature classes are included, and how overlapping or ambiguous annotations are handled. |
| Instance segmentation | The model returns separate masks for individual objects or defects. | How instances are defined, how predictions are matched to annotations, and which instance-level metric and IoU threshold are used. |
| Anomaly segmentation | The model marks regions considered anomalous, often without a conventional supervised defect-class label set. | How anomaly scores become masks, which examples are normal or anomalous, and how the operating threshold is selected. |
Do not treat results from these tasks as interchangeable. MSPS is a binary semantic task on monocrystalline silicon cell EL images. The 2023 benchmark uses pixel labels for 24 defect and feature classes, though its abstract reports benchmark metrics for a subset of five. The 2025 multi-class study evaluates a 29-class PV module EL dataset. The 2026 benchmark reports instance-segmentation metrics on UCF EL Defect. Those label schemes and metric definitions are not one common leaderboard.
Choose metrics that expose both overlap and errors
Use a set of complementary measures. IoU or Dice describes mask overlap; precision and recall expose different error costs. Report them per class rather than relying on one aggregate.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Intersection over Union and Dice
For a given class, let TP be the number of pixels correctly predicted as that class, FP the pixels predicted as that class but not annotated as it, and FN the annotated pixels the model missed. Then:
Rank #2
- Item Package Dimension - 15.0L x 12.25W x 4.25H inches
- Item Package Weight - 6.0 Pounds
- Item Package Quantity - 1
- Product Type - VIDEO CARD
- IoU = TP / (TP + FP + FN)
- Dice = 2TP / (2TP + FP + FN)
Both summarize overlap, penalizing false-positive and false-negative pixels. Dice and IoU are related but not numerically equal, so comparisons must use the same metric. State whether the score is calculated per image and then averaged, or calculated from pooled pixel counts; these aggregation methods answer different questions, especially when image sizes or defect supports differ.
Precision and recall
- Precision = TP / (TP + FP). Low precision means many pixels labeled as a defect were not annotated as that defect.
- Recall = TP / (TP + FN). Low recall means the model failed to label a substantial share of the annotated defect pixels.
Neither is a substitute for overlap. Recall can be high even if the predicted mask spills beyond the target, and precision can be high when the model marks only a small, safe fragment of the defect. In the 2023 EL benchmark, the authors reported a DeepLabv3+ custom-weight result with 86% recall for cracks alongside 25% IoU for cracks. This is a specific model-and-dataset result, not an expected benchmark level; it demonstrates why recall alone cannot establish localization quality.
Class-level and aggregate scores
For multi-class work, publish each defect class separately and show the support behind it, such as the number of test images containing the class and the number of annotated instances where instance labels exist. A score calculated from many common examples can conceal poor performance on a class represented by very few examples.
If you report mean IoU (mIoU), specify which classes enter the mean, whether absent classes are excluded, and whether the mean weights classes equally. If using a median, name it explicitly and say which units are aggregated. For example, a median of per-class IoUs is not the same as a mean of per-image IoUs. Do not use an overall background-plus-defect score as the only headline result when the scientific or operational question is whether defects are found.
| Measure | Best use | What it does not establish by itself |
|---|---|---|
| Per-class IoU or Dice | How well the predicted and annotated masks overlap for a class. | Whether the chosen false-alarm/miss trade-off is acceptable in use. |
| Per-class precision | How much predicted defect area is supported by annotations. | Whether the model misses other defects. |
| Per-class recall | How much annotated defect area the model captures. | Whether predicted boundaries are tight or false alarms are rare. |
| Size-stratified metrics | Whether small targets behave differently from larger targets. | Performance outside the size and image conditions represented by the test set. |
| Instance mAP at a stated IoU threshold | Instance-detection and mask performance under the stated matching rule. | A directly comparable semantic-segmentation result unless task and definitions align. |
Make small-target performance visible
Add a size-stratified evaluation alongside class-wise metrics. Define size bins from the dataset and task, and explain whether size means annotated mask area in pixels, physical area, or another measure. There is no universal size cutoff established by the cited PV sources; choose bins that reflect the target distribution and intended use, then keep them fixed for every model in the comparison.
For each size bin, report the number of examples and at least the same overlap and detection measures used in the headline results. A bin containing only a few examples should be visibly identified as low-support rather than treated as equally conclusive. Consider showing representative image crops with ground-truth and predicted masks for small, low-contrast, irregular, and difficult examples. Images explain failure modes that a scalar score cannot.
The MSPS project reports a mean defect footprint of 128 pixels², a median of 114 pixels², and a reported size range of 1–821 pixels². It also describes minute targets, low contrast, size variation and irregular boundaries. These figures explain why tiny-mask behavior is worth separating for that dataset; they should not be used as thresholds for unrelated data.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11When contour quality matters, consider adding a boundary-sensitive measure, but name its definition and tolerance. The cited PV source summaries do not establish one universally preferred boundary metric or a standard tolerance. Keep such a measure supplementary: it should clarify contour behavior, not replace class-wise overlap, precision and recall.
Rank #4
- Item Package Quantity - 1
- Product Type - VIDEO CARD
- This pre-owned product has been professionally inspected, tested and cleaned by Amazon qualified vendors.
- Accessories may not be original, but will be compatible and fully functional. Product may come in generic box.
Build a credible test protocol
- Freeze the task and labels. Write down the segmentation type, class map, annotation rules, ignored pixels and treatment of ambiguous areas before evaluating candidate models.
- Identify the test data. Name the dataset version, modality, cell or module type, image conditions and official split where one exists. For a custom split, state how it was created.
- Prevent related-image leakage. Keep crops and closely related images from the same source from crossing between training and test when that would make the test set non-independent. The exact grouping rule depends on how the data were collected; the cited dataset summaries do not establish a universal rule.
- Keep the final test set out of model selection. Use training and validation data to choose architecture, class weights, threshold and other settings; reserve the test set for the final reported evaluation. Document any deviations.
- Lock prediction and scoring rules. State how class probabilities become masks, including the threshold or argmax rule, any post-processing, and the treatment of empty predictions or absent classes.
- Calculate the complete metric set. Report per-class IoU or Dice, precision and recall, then clearly defined aggregates. Add size strata and example masks for small targets.
- Record support and variability. Give per-class image or instance counts. Where feasible, report variability across repeated runs or confidence intervals, and explain how they were computed. The cited summaries do not establish a shared statistical protocol for this literature.
- Publish implementation details needed to reproduce scoring. Record the evaluation code or metric implementation, image resizing and tiling rules, inference settings, hardware used for throughput, and any post-processing.
The MSPS dataset record describes 1,500 images at 640 × 590 pixels, with 1,000 training and 500 test images, expert annotations and a published split. The project materials describe labels for busbar breaks, black spots and cracks; two experts annotated each image and a third reviewed disagreements. These details make it possible to identify the scope of that benchmark, but they do not make its result directly comparable with another dataset’s score.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Compare imbalance treatments fairly
Class weighting, resampling and semi-supervised learning can change the rare-class/false-alarm trade-off. Compare such approaches on the same split, class map, threshold-selection procedure and scoring code. Report the change for each defect class as well as the aggregate: a gain in an overall measure may coincide with a loss for a rare class, or a recall gain may come with more false positives.
The 2023 benchmark evaluated four model families and three class-weight configurations on 593 cell images, with 50 test images; its abstract presents benchmark metrics for a subset of five classes. The 2025 multi-class study evaluated six U-Net/SegNet architectures with VGG16, MobileNet or ResNet50 encoders on a 29-class PV module EL dataset using custom class weights. The authors report mean IoU of 0.663 for feature classes and 0.326 across defect classes, plus an IoU improvement of 0.129 for rare dead-cell defects in the reported comparison. Preserve those class and comparison contexts; they are not a generic score or a direct result for other datasets.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →PV-S3 is a semi-supervised semantic-segmentation approach using labeled and unlabeled images and a proposed semi-cross-entropy loss for imbalance. Its authors report an absolute 9.7 percentage-point mIoU improvement using 20% labeled samples versus a supervised comparison using 100% labeled samples on UCF-EL. Treat this as the authors’ stated comparison under that setup, not a general guarantee that fewer labels will outperform full supervision elsewhere.
Keep semantic and instance results separate
Semantic metrics summarize labeled pixels. Instance metrics also depend on matching each predicted object to a ground-truth instance and applying a specified overlap criterion. A model can therefore have an instance mAP and a semantic mIoU that look like comparable percentages but answer different questions.
The 2026 instance-segmentation benchmark authors report ConvNeXt-V2 segmentation mAP at IoU 0.50 of 0.602 and segmentation mIoU of 0.650 on UCF EL Defect. Those are results from that instance-segmentation benchmark and should not be ranked directly against binary or multi-class semantic segmentation scores unless task definitions, labels and metric definitions genuinely align. The benchmark compares Mask R-CNN, ConvNeXt, ConvNeXt-V2 and Mask2Former; the reported values here are specifically for ConvNeXt-V2.
Report operating conditions and limits on generalization
Segmentation results depend on the image source and conditions. State whether images are EL or another modality, whether they show cells or modules, and any relevant acquisition conditions represented in the test set. A result on EL images does not by itself demonstrate performance on outdoor RGB inspection, a different PV product, or a production line with different capture conditions.
Free tools Windows power users keep installed
One-click scans. No signup required.
For probability-based models, disclose the threshold used to form defect masks and how it was selected. If a threshold is tuned for recall, make clear that precision and overlap are measured at that operating point. Where the application requires a particular balance between missed defects and false alarms, report performance at the relevant operating point rather than implying a single metric resolves that trade-off.
If reporting inference speed, give the hardware, image size, batch size, preprocessing and whether timing includes post-processing. The MSPS project reports throughput measured on an NVIDIA RTX 3090 Ti in its benchmark context. That is reproducibility context, not evidence that the GPU is required or a current purchasing recommendation.
A practical comparison table for model reports
Use one row per model and keep each evaluation axis consistent. If a value is not measured, label it as not reported rather than silently omitting a difficult dimension.
| Report item | What to include |
|---|---|
| Task and labels | Semantic, instance or anomaly segmentation; full class list; defect definition; treatment of feature classes. |
| Dataset and split | Dataset/version, modality, image type and conditions, train/validation/test split, and grouping method used to avoid leakage. |
| Per-class performance | IoU or Dice, precision, recall, and support for every defect class; include features separately where applicable. |
| Small-target performance | Dataset-defined size bins, support per bin, and the same key metrics for each bin. |
| Error examples | Representative true masks, predictions, false alarms and misses, with particular attention to small or difficult targets. |
| Aggregation | Per-image versus pooled counts; mean or median; class inclusion and absent-class handling. |
| Operating point | Probability threshold or decision rule, post-processing, and method used to select it. |
| Reproducibility and speed | Evaluation implementation, run variability where feasible, and throughput with hardware and timing conditions. |
How to read an evaluation without overclaiming
A strong result is not simply the largest mIoU in a table. First verify that the compared models use the same task, labels, split, aggregation and operating point. Then inspect defect classes individually, check sample support, and examine the small-target breakdown. Finally, compare precision and recall to understand the cost of missed defects versus false alarms. If those conditions differ, report each result in its own context instead of presenting a cross-study ranking.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
