Freedom251 › Statistical Analysis Software
autorank
by autorank
What autorank is
autorank is a free Python package for comparing and ranking multiple paired populations, such as machine-learning algorithms measured across datasets. It aims to simplify statistical analysis for people without specialist statistics knowledge. The package supports frequentist analysis based on 2006 guidelines and Bayesian analysis based on 2017 guidelines. It checks normality and homogeneity, then selects statistical tests and reporting measures for the supplied data. Results can include plots, result tables, and LaTeX reports, including a complete LaTeX document. Input must be a Pandas DataFrame, with populations in columns and paired measurements in rows. Analysis requires at least five measurements, rejects NaN values, and is described as reliable only for fewer than 5,000 measurements. The documented tests use SciPy, statsmodels, and baycomp. Bayesian analysis may take seconds or minutes when many populations are included. autorank supports scripting, Bayesian analysis, and statistical tests but not regression models. It runs as a desktop package on Linux through API or self-hosted use and is licensed under Apache 2.0.
Who it suits
autorank suits Python users who need paired-population comparisons and ranking, especially those less familiar with statistics. It is useful for analysts able to provide Pandas DataFrames and work within its input limits.
What it does well
- Free Python package
- Selects tests from normality and homogeneity checks
- Generates plots, tables, and LaTeX reports
- Supports frequentist and Bayesian analysis
- Available under Apache 2.0 License
What to know first
- Requires at least five measurements
- Rejects NaN values
- Reliability is described below 5,000 measurements
- No regression model support
Plans
| Plan | Billed and limits | Price |
|---|---|---|
| Autorank | Python package for comparing paired populations | Free |
As sherbold.github.io lists them, read 8 Oct 2026. The maker's own page has the final word.
Verdict
autorank provides guided frequentist and Bayesian comparisons from Pandas DataFrames, with report-generation options. Its documented limits on measurements and missing values should be checked before analysis.
Facts from autorank's own pages
- Analysis approaches
- It supports frequentist analysis based on 2006 guidelines and Bayesian analysis based on 2017 guidelines. · sherbold.github.io, 8 Oct 2026
- Analysis methods
- It supports frequentist analysis following 2006 guidelines and Bayesian analysis following 2017 guidelines. · sherbold.github.io, 8 Oct 2026
- API documentation
- The site links to online API documentation for the current master version. · sherbold.github.io, 8 Oct 2026
- Automatic test selection
- It checks normality and homogeneity to select statistical tests and reporting measures for the data. · sherbold.github.io, 8 Oct 2026
- Bayesian analysis
- Its Bayesian approach uses a Bayesian Signed Rank test and a Region of Practical Equivalence (ROPE). · sherbold.github.io, 8 Oct 2026
- Bayesian performance
- The site notes that Bayesian analysis can take seconds or, with many populations, minutes because it is computationally expensive. · sherbold.github.io, 8 Oct 2026
- Data import formats
- pandas DataFrame · sherbold.github.io, 23 Sept 2026
- Dependencies
- The implementation uses SciPy, statsmodels, and baycomp for statistical tests. · sherbold.github.io, 8 Oct 2026
- Dependencies used
- The documentation says it uses SciPy, statsmodels, and baycomp for statistical tests. · sherbold.github.io, 8 Oct 2026
- Deployment
- desktop · sherbold.github.io, 23 Sept 2026
- Founded
- 2020 · sherbold.github.io, 19 Oct 2026
- Free plan
- Yes · sherbold.github.io, 23 Sept 2026
- Input format
- The package expects population data in a Pandas DataFrame, with each column representing a population and each row a paired measurement. · sherbold.github.io, 8 Oct 2026
- Input limits
- The documented analysis requires at least five measurements, does not accept NaNs, and is described as reliable only for fewer than 5,000 measurements. · sherbold.github.io, 8 Oct 2026
- Installation
- The package can be installed from PyPI with pip or from the GitHub repository. · sherbold.github.io, 8 Oct 2026
- Intended users
- The project aims to simplify statistical analysis for people who are not statistics experts. · sherbold.github.io, 8 Oct 2026
- License
- Autorank is published under the Apache 2.0 License. · sherbold.github.io, 8 Oct 2026
- Purpose
- Autorank is a Python package for comparing and ranking multiple paired populations, such as machine learning algorithms evaluated on multiple data sets. · sherbold.github.io, 8 Oct 2026
- Regression models
- No · sherbold.github.io, 23 Sept 2026
- Reporting
- It can generate plots, result tables, and LaTeX reports, including a complete LaTeX document. · sherbold.github.io, 8 Oct 2026
- Scripting support
- Yes · sherbold.github.io, 23 Sept 2026
- Statistical tests
- For frequentist analysis, it selects tests based on normality, homogeneity, and the number of populations. · sherbold.github.io, 8 Oct 2026
- Support
- The project invites questions, bug reports, and feature requests through GitHub issues, and welcomes pull requests. · sherbold.github.io, 8 Oct 2026
- Support and contributions
- The project invites questions, bug reports, and feature requests through GitHub issues and welcomes pull requests. · sherbold.github.io, 8 Oct 2026
Where autorank stands on Freedom251
- Statistical Analysis Software#25 of 106