Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsSome links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
Markowitz mean-variance theory turns estimates of asset returns and co-movement into portfolio weights. Its central insight is that portfolio risk depends not only on each asset’s volatility, but also on how its returns move with the others. That makes it a useful, interpretable framework for allocation—but the output is only as reliable as the data, assumptions, constraints, and validation behind it.
What Markowitz optimization does
Portfolio optimization answers a conditional question: given a defined set of investable assets, estimates of their expected returns and covariance, and a set of constraints, which allocation best meets a specified objective? “Optimal” therefore means optimal under those inputs and rules—not guaranteed to perform best in the future.
Harry Markowitz formalized the risk-and-return trade-off in his 1952 paper, “Portfolio Selection”. The contribution was to evaluate securities as parts of a portfolio rather than in isolation. Modern portfolio theory is the broader framework; mean-variance optimization is one method within it. The Capital Asset Pricing Model is a later asset-pricing theory, not another name for the optimizer.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Suppose a portfolio holds n assets. Let w be the vector of portfolio weights, μ the vector of expected returns, and Σ the covariance matrix of returns. Then:
#1 Best Overall
Expected portfolio return: E(Rp) = wᵀμ
Portfolio variance: σp² = wᵀΣw
Portfolio volatility: σp = √(wᵀΣw)Weights usually sum to one for a fully invested, unlevered portfolio. A common long-only problem is to minimize variance while requiring a minimum expected return:
Minimize: wᵀΣw
Subject to: wᵀμ ≥ target return
1ᵀw = 1
wi ≥ 0Changing the target return generates portfolios with different estimated risk-return trade-offs. This standard formulation is a convex quadratic optimization problem under common assumptions; see the PyPortfolioOpt guide to the efficient frontier.
Why covariance creates diversification
Variance measures dispersion of returns; volatility is its square root and is usually easier to interpret because it is in return units. Covariance measures whether two assets tend to move together. Correlation is standardized covariance, ranging from -1 to +1, and is useful for comparing co-movement across pairs with different volatilities.
Because portfolio variance includes cross-asset covariance terms, combining assets that do not move in lockstep can reduce total risk. The number of holdings alone does not establish diversification: ten highly correlated technology shares may share much of the same risk, while a smaller mix of assets with different risk drivers may behave differently. Correlations, factor exposures, concentration, and contributions to portfolio risk all matter.
Objectives and the efficient frontier
- Global minimum-variance portfolio: The feasible portfolio with the lowest estimated variance, without requiring a particular return.
- Target-return portfolio: The lowest-variance portfolio that meets a specified expected-return threshold.
- Target-risk portfolio: The highest estimated expected return subject to a volatility limit.
- Maximum-Sharpe portfolio: The portfolio with the highest estimated excess return per unit of volatility, where the Sharpe ratio is (E(Rp) − Rf) / σp.
- Efficient frontier: The boundary of feasible portfolios that are not dominated by another portfolio with higher expected return at the same risk or lower risk at the same return.
On a typical chart, estimated annualized volatility is on the horizontal axis and estimated annualized return on the vertical axis. The frontier forms the upper-left boundary. The global minimum-variance portfolio is at its leftmost point; a maximum-Sharpe portfolio depends on the chosen risk-free rate and assumptions. Equal weighting is a useful comparison, not usually an optimized point. Every point is based on estimates, so a frontier is not a promise about realized returns.
Rank #2
The inputs: data before optimization
A usable workflow starts with adjusted prices or total-return series, asset identifiers and classifications, a defined estimation window, and a stated rebalancing schedule. If optimizing a Sharpe ratio, specify a risk-free rate in the same currency and on a compatible time basis. Also define constraints and realistic assumptions about trading costs, liquidity, and execution.
Raw closing prices may omit dividends or mishandle splits and other corporate actions. Use consistently adjusted prices or total returns, and check the chosen provider’s adjustment methodology, coverage, licensing, and treatment of delisted assets. Missing observations and assets trading in different time zones can complicate return alignment; do not silently fill gaps in a way that creates artificial returns.
Estimating expected returns
A simple historical arithmetic estimate for asset i is the average of its periodic returns:
μ̂i = (1/T) Σt ri,tFor regular periodic data, annualizing a mean is commonly approximated by multiplying by the number of periods per year. A geometric return describes compounded historical growth and is not interchangeable with the arithmetic mean used in many one-period optimization formulations. State which convention you use.
Historical averages are not forecasts. Expected returns may instead come from CAPM or multifactor models, analyst forecasts, dividend-growth assumptions, equilibrium-implied returns, or Black-Litterman views. Whatever method is chosen, the optimizer does not discover expected returns; it uses the estimates supplied to it. This is especially important for maximum-Sharpe optimization, which can react sharply to small changes in those estimates.
Estimating covariance
The sample covariance between assets i and j is estimated from their paired return observations. For regular periodic observations, annual covariance is commonly approximated as periodic covariance multiplied by the number of periods per year. The scaling convention must match the return frequency and reporting horizon.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchSample covariance is noisy, particularly when the history is short relative to the number of assets, when assets are highly correlated, or when the market structure changes. A covariance matrix should be positive semidefinite for standard quadratic optimization; numerical or data problems can violate that condition. Shrinkage methods pull noisy sample estimates toward a more structured target and may improve stability or conditioning, though they do not guarantee better realized returns. PyPortfolioOpt documents shrinkage-based risk models as alternatives to raw sample covariance in its project documentation.
A Python starting point
The following example assumes a CSV of adjusted prices with one column per asset and dates in the index. It estimates annualized historical returns and sample covariance, then solves a long-only maximum-Sharpe problem with a 30% per-asset cap. The 2% risk-free rate is illustrative; choose a rate consistent with the portfolio’s currency, period, and evaluation date. It is not an investment recommendation.
import pandas as pd
from pypfopt import expected_returns, risk_models
from pypfopt.efficient_frontier import EfficientFrontier
prices = pd.read_csv(
"adjusted_prices.csv", index_col=0, parse_dates=True
)
mu = expected_returns.mean_historical_return(prices)
S = risk_models.sample_cov(prices)
ef = EfficientFrontier(mu, S, weight_bounds=(0, 0.30))
weights = ef.max_sharpe(risk_free_rate=0.02)
cleaned_weights = ef.clean_weights()
performance = ef.portfolio_performance(
verbose=True, risk_free_rate=0.02
)
print(cleaned_weights)PyPortfolioOpt also supports minimum volatility and target-return objectives, weight bounds, and performance reporting; consult its user guide and mean-variance API documentation for the installed version. Do not assume a library’s defaults match your intended return convention, frequency, constraints, or current API.
Regularization and trading costs
Constraints themselves can stabilize a result. A weight cap prevents a single asset from dominating. L2 regularization adds a penalty related to the squared size of weights, discouraging some extreme solutions. For example, with the documented API:
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →from pypfopt import objective_functions
ef = EfficientFrontier(mu, S, weight_bounds=(0, 0.30))
ef.add_objective(objective_functions.L2_reg, gamma=0.1)
weights = ef.min_volatility()The value of gamma is a modeling choice to select using training and validation data, not the final test set. A transaction-cost penalty can discourage unnecessary turnover. PyPortfolioOpt documents an objective using previous weights and a cost parameter:
previous_weights = {ticker: 0.10 for ticker in prices.columns}
ef = EfficientFrontier(mu, S, weight_bounds=(0, 0.30))
ef.add_objective(
objective_functions.transaction_cost,
w_prev=previous_weights,
k=0.001
)
weights = ef.min_volatility()Check the installed package documentation because interfaces can change. A zero commission does not mean zero implementation cost: spreads, slippage, exchange and regulatory fees, market impact, borrow costs, taxes, and execution delays may all matter.
Model the optimization directly with CVXPY
For a custom model or to learn the optimization formulation, CVXPY exposes the variables, objective, and constraints directly. This example minimizes variance for a target return with long-only weights and a 30% cap:
import cvxpy as cp
import numpy as np
n = len(mu)
w = cp.Variable(n)
mu_array = mu.to_numpy()
cov_array = S.to_numpy()
target_return = 0.08
max_weight = 0.30
problem = cp.Problem(
cp.Minimize(cp.quad_form(w, cov_array)),
[
cp.sum(w) == 1,
mu_array @ w >= target_return,
w >= 0,
w <= max_weight,
],
)
problem.solve()
optimized_weights = np.asarray(w.value).ravel()The target may be infeasible under the supplied estimates and bounds; always check solver status and whether a solution was returned before using weights. CVXPY’s quadratic-programming example and optimization examples explain the modeling approach.
Make the portfolio investable
Real portfolios need constraints that reflect the investor and market, not just the mathematics. Common choices include:
- Long-only and position bounds: Set lower and upper bounds, such as 0 ≤ wi ≤ 0.30.
- Sector or asset-class bands: Bound the sum of weights in each group to avoid unintended exposures.
- Turnover limits: Restrict the sum of absolute changes from current weights, for example Σ|wi − wi,prev| ≤ τ.
- Leverage and gross exposure: Control total borrowing or long-short exposure where shorting is allowed.
- Liquidity limits: Relate proposed trades to average daily volume, spreads, and estimated market impact.
- Tracking error: Limit benchmark-relative risk when managing against an index.
- Cardinality and minimum positions: Limit holding count or require meaningful positions; discrete choices can make the problem nonconvex or mixed-integer.
A cost-aware objective might minimize variance plus a turnover penalty, or minimize cost-weighted trading changes while meeting a return target. Taxes and cash-flow or liability needs may also matter. Explicitly state the signal date, execution date, assumed trade price, rebalance frequency, and treatment of cash; an optimizer’s assumptions must match the strategy that is actually evaluated and implemented.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Why a mathematically valid answer may be a poor portfolio
- Noisy return estimates: Small forecast changes can produce large weight changes, especially in maximum-Sharpe portfolios.
- Unstable covariance: Too many assets relative to observations, highly correlated holdings, missing data, or regime shifts can create ill-conditioned estimates.
- Extreme weights: The optimizer may concentrate in assets whose estimated statistics look unusually favorable, despite a fragile result.
- Shorting and leverage: An unconstrained solution can include large positive and negative positions that are costly, unavailable, or outside the investor’s risk limits.
- Incomplete risk description: Variance does not fully describe skewed or fat-tailed returns, liquidity risk, or losses in a specific crisis scenario.
- Regime change: Historical volatility and correlation may not persist through crises, inflation shocks, interest-rate shifts, or structural market changes.
Useful responses include long-only bounds and position caps, turnover penalties, covariance shrinkage, factor covariance models, minimum-variance objectives, bootstrap or resampling analysis, and stress scenarios. Black-Litterman combines equilibrium-implied returns with views and confidence levels; hierarchical risk parity uses a different allocation structure; robust optimization explicitly addresses parameter uncertainty, but can be conservative and requires choices about uncertainty sets. Mean-semivariance and CVaR objectives focus more directly on downside risk. These approaches are alternatives or complements, not guarantees. PyPortfolioOpt’s project overview describes several of these methods, and its alternative frontier documentation covers semivariance methods.
Validate with a walk-forward backtest
Do not judge a portfolio by its in-sample frontier or the highest Sharpe ratio found after trying many variants. Use data only as it would have been available at each historical decision point:
- Define the investable universe and its membership as of each date, including delisted assets where appropriate.
- Choose an initial training window and estimate returns and covariance using only that window.
- Optimize under stated constraints, then trade at a realistic subsequent execution time and price.
- Hold the portfolio for the declared rebalance interval; deduct plausible costs and account for cash, missing data, and corporate actions.
- Move the window forward and repeat, without using future observations to revise past weights.
- Compare against equal weight, market-cap weight, minimum variance, risk parity, or a policy portfolio relevant to the use case.
Keep model selection disciplined: use training data to fit inputs, validation data to choose settings, and a final test period only for a final evaluation. Leakage can arise from future index membership, survivorship-biased constituents, corporate-action information applied incorrectly, fitting estimates beyond the rebalance date, or repeatedly tuning the lookback, constraints, and rebalance frequency on the test period.
Report more than cumulative return: annualized return and volatility, Sharpe ratio with its risk-free-rate convention, maximum drawdown, turnover and cost drag, concentration, downside deviation, worst month or rolling period, weight stability, and performance across market regimes. A credible backtest protocol states the data source and adjustment method, universe rules, estimation window, rebalance and execution assumptions, costs, benchmarks, and tuning procedure. A backtest is evidence about a model under those assumptions, not proof of future success.
Choosing a method
| Method | Expected returns needed? | Useful when | Main limitation |
|---|---|---|---|
| Equal weight | No | Simple baseline and transparent comparison | Ignores risk differences and may create unintended exposures |
| Minimum variance | Usually no | Reducing dependence on return forecasts | Still depends heavily on covariance estimates |
| Maximum Sharpe | Yes | Seeking estimated excess return per unit of volatility | Highly sensitive to expected-return inputs |
| Risk parity | No or limited | Allocating by risk contribution | May require leverage or produce an unsuitable return profile |
| Black-Litterman | Structured estimates and views | Combining equilibrium assumptions with explicit views | Adds assumptions about views and confidence |
| Hierarchical risk parity | No traditional return vector | Using a clustering-based alternative allocation structure | Less direct interpretation as a return-risk frontier |
| Robust optimization | Yes, with uncertainty modeling | Making parameter uncertainty explicit | Requires uncertainty-set choices and can be conservative |
| CVaR or semivariance | Usually some return input or target | Focusing on downside or tail losses | More dependent on scenario choices and modeling |
Mean-variance optimization is a sound baseline when the universe is investable, objectives and constraints are clear, and out-of-sample validation is possible. Be cautious when forecasts are guesses, assets are illiquid, liabilities or cash flows dominate, tail risk is central, or implementation limits are not modeled. “Data science” here is not synonymous with machine learning: cleaning, estimation, optimization, backtesting, sensitivity analysis, and production monitoring are the core workflow. Machine learning can be added to forecasting or risk estimation, but it is not required by Markowitz theory.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
