Experimently

Experimently combines A/B testing, feature flags, and rollout controls for teams that can operate their own infrastructure. Its broad testing and analysis toolkit is a strong fit for product and engineering groups; teams looking for a managed service should look elsewhere.

Overview

Apache-2.0 licensing and a free Free Preview make the platform available without a subscription charge. The trade-off is operational: there is no hosted tier, and teams deploy it themselves and install its SDKs using the product’s guides rather than package registries. Assignment data stays on the user's infrastructure, which suits organizations that want to keep that data in-house.

The testing range covers A/B and multivariate tests, split URL testing, sequential testing, CUPED, Bayesian testing, multi-armed bandits, and LLM or model evaluation. That breadth can serve teams with varied experimentation programs, but may be unnecessary overhead for those needing only a basic hosted test tool.

Key features

Experiments and rollouts

Sticky assignment keeps users in consistent experiment groups, while staged rollouts support gradual releases. Mutual-exclusion groups and a global holdout help control overlap and preserve a baseline. Automatic rollback based on error-rate and latency thresholds adds a release safeguard for teams that want experiments and rollout controls in the same system.

Targeting and analysis

More than 20 targeting operators include semantic version comparisons, geographic distance, time windows, JSON paths, and array operations. This gives teams several ways to define audiences beyond basic user attributes. Analysis includes frequentist and Bayesian methods, sequential mSPRT testing, multi-armed bandits, and segment breakdowns with Bonferroni correction. The range is useful for teams applying different statistical approaches, though it calls for people able to choose and interpret them appropriately.

SDKs, integrations, and controls

SDKs cover JavaScript, React, React Native, Edge, Python, Go, Java, Android, iOS, Flutter, Ruby, PHP, .NET, and Elixir; OpenFeature providers are listed for JavaScript and Python. Since the SDKs are not yet on package registries such as npm or PyPI, installation takes a guide-based step rather than a standard registry install. Connections include Snowflake, BigQuery, Redshift, Databricks, ClickHouse, MySQL, Jira, Salesforce, GitHub, Slack, and email. Listed security features include HMAC-SHA256 audit logging, role-based access control, scoped API keys, SAML and OIDC/OAuth2 SSO, and SOC 2, ISO 27001, and HIPAA compliance features.

Pricing

The Free Preview costs nothing and is self-hosted on your AWS account with all features included. It is a practical fit for teams able to take responsibility for deployment and infrastructure, rather than a no-setup hosted free tier. The maker says build profiles have no price, and the platform is Apache-2.0 licensed. No free trial is offered; teams wanting a managed hosted option will need another product.

Platforms

Experimently supports web and mobile use, with listed platforms spanning Android, iOS, Linux, macOS, Windows, API, web, and self-hosted deployment. Visual editing, server-side testing, mobile SDKs, audience targeting, and feature flags are supported. Deployment options are Docker Compose, Kubernetes via Helm, or an AWS CDK app targeting ECS Fargate, Aurora PostgreSQL, and ElastiCache in the user's AWS account.

Who it's for

Experimently is best suited to product and engineering teams that want a broad experimentation toolkit, need rollout safeguards, and can run software on their own infrastructure. Keeping assignment data in-house is a meaningful advantage for teams with that requirement. It is a weaker fit for teams that prioritize a hosted service, standard package-registry installation, or minimal operational work.

Pros and cons

• Pros: Broad test methods, targeting operators, and statistical approaches support varied experimentation needs.
• Pros: Staged rollouts, holdouts, mutual exclusion, and threshold-based rollback put release controls alongside testing.
• Pros: Assignment data remains on the user's infrastructure, with an Apache-2.0 license and a free all-features preview.
• Cons: Self-hosting means teams must deploy and operate the platform themselves; there is no hosted tier.
• Cons: SDKs are not yet on package registries, so installation requires following individual guides.

Alternatives

A/B Testing Tools is a useful starting point for comparing products in the category. Consider Confidence by Spotify if a free plan with 75,000 credits per month, unlimited feature flags and seats, and up to five surfaces better matches your usage; its free plan also includes Google SSO, EU or US hosting, and a one-month audit log.

DevCycle is another option with a free plan and support across web and mobile platforms. Firebase Cloud Messaging is a no-cost alternative for readers considering that product. GrowthBook may suit teams that want a self-hosted open-source option with one project, unlimited feature flags, experiments, and traffic at no cost. LaunchDarkly Experimentation offers a free Developer plan billed forever with 100K experimentation MAU per month, one project, and three environments.

Mida has a no-time-limit Sandbox plan with no credit card required, 100,000 monthly tested users, 50,000 daily tracked events, 60 days of data retention, and two active tests. Omniconvert Explore offers a free plan for 50,000 website visitors with its full Explore feature set and no credit card required. Statsig may appeal to teams needing a free Developer plan with two million events per month, unlimited flag and config checks, 50,000 session replays per month, one-year analytics retention, and unlimited seats.

Verdict

Choose Experimently if your team wants extensive testing and rollout controls, values keeping assignment data on its own infrastructure, and can handle self-hosting. Its breadth and all-features free preview are compelling, but teams that want a managed service or registry-based SDK installation should choose elsewhere.