Deflated Sharpe ratio

The Deflated Sharpe Ratio (DSR) is a statistical method used to determine whether the Sharpe ratio of an investment strategy is statistically significant, developed in 2014 by Marcos López de Prado at Guggenheim Partners and Cornell University, and David H. Bailey at Lawrence Berkeley National Laboratory. It corrects for selection bias, backtest overfitting, sample length, and non-normality in return distributions, providing a more reliable test of financial performance, especially when many trials are evaluated. The application of the DSR, helps practitioners to detect false investment strategies.

DSR offers a more precise and robust adjustment for multiple testing compared to traditional methods like the Šidák correction because it explicitly models both the selection bias arising from choosing the best among many trials and the estimation uncertainty inherent in Sharpe ratios. Unlike Šidák, which assumes independence and adjusts p-values based only on the number of tests, the DSR accounts for the variance of Sharpe estimates, the number of trials, and their effective independence, often estimated through clustering. This leads to a more realistic threshold for statistical significance that reflects the true probability of a false discovery in data-mined environments. As a result, the DSR is particularly well-suited for finance, where researchers often conduct large-scale, correlated searches for profitable strategies without strong prior hypotheses.

Relation to the Sharpe Ratio

One of the most important statistics for assessing the performance of an investment strategy is the Sharpe Ratio (SR). The Sharpe ratio was developed by William F. Sharpe and is a widely used measure of risk-adjusted return, calculated as the annualized ratio of excess return over the risk-free rate to the standard deviation of returns. While useful, the Sharpe Ratio has important limitations, especially when applied to multiple strategy evaluations. Issues such as selection bias, where the best-performing strategy is chosen from a large set, and backtest overfitting, where a strategy is tailored to past data, can inflate the Sharpe Ratio, leading to misleading conclusions about a strategy's effectiveness. Additionally, the Sharpe Ratio assumes normally distributed returns, an assumption often violated in practice, and it does not take into account sample length.

Applying the Deflated Sharpe Ratio in Practice

1. Get a record of all the trials.

In order to apply the DSR, researchers need to record the investment performance in returns (%), for every backtest that they ran. This is in relation to the development of a single specific strategy. For example: when building a momentum based strategy that trades at the end-of-day, 100 historical simulations were run to evaluate the performance and the best set of parameters were selected for the final strategy. Here all 100 simulations need to be recorded, with the strategies daily returns in %.

2. Estimating the Effective Number of Trials N.

In practice, many trials are not independent due to overlapping features. To estimate the effective number of independent trials N, López de Prado (2018) proposes 3 techniques to clustering similar strategies using unsupervised learning techniques:

1. The Optimal Number of Clusters (ONC) algorithm.
2. Hierarchical clustering could be used to get a conservative lower bound for N.
3. Alternatively, spectral methods (e.g. eigenvalue distribution of the correlation matrix) can also provide estimates of N.

Tip:

• Multiple testing exercises should be carefully planned in advance, so as to avoid running an unnecessary large number of trials. Investment theory, not computational power, should motivate what experiments are worth conducting.

Steps to estimate N:

2.1. Convert the correlation matrix to a distance matrix.

In order to apply a clustering algorithm to the returns data, we need make use of a statistical association measure (such as a correlation matrix) and we need to transform it into a distance matrix (such as angular distance) so that elements that are very similar to each other will be close together in their higher-dimensional space.

2.2. Apply a clustering algorithm to estimate the number of independent trials.

The number of clusters N, are an estimate of the number of independent trials.

2.3 Plot the Block Correlation Matrix

In the figure below we can see a correlation matrix before and after clustering has been applied. Note how we can see blocks down the diagonal, each block corresponds to a cluster.

Clustering of 6,385 trials, typical of multiple testing of a group of strategies, before and after clustering.

Tip: If you don't use the ONC algorithm to cluster, then you can have blocks with trials that don't match very closely. The ONC algorithm uses silhouette scores to make sure each trial is in the best cluster, at the expense of high computational complexity and longer run times.

3. Compute the Sharpe ratio variance, across clusters.

3.1 Calculate the Sharpe ratio for each cluster.

Each cluster will now form a collection of time series returns (in%), for each cluster you need to create a new time series which represents that cluster using the Inverse Variance Portfolio (IVP) and then compute the Sharpe Ratio for each IVP portfolio. One doesn't need to use the IVP - the goal is to form an aggregate cluster return time series. For this a weighting scheme needs to be used, another alternative could be the minimum variance portfolio.

3.2 Compute the variance of these Sharpe Ratios

$\mathbf {V} [{\hat {SR}}_{n}]$ is used in the next step, where we apply the False Strategy Theorem to determine the Expected Maximum Sharpe ratio.

4. Compute the Expected Maximum Sharpe ratio using the False Strategy Theorem.

Using the equation from the False Strategy Theorem (FST) we can compute $SR_{0}$, which is the threshold Sharpe Ratio that reflects the highest Sharpe Ratio expected from $N$ unskilled strategies.

$SR_{0}={\sqrt {\mathbf {V} [{\hat {SR}}_{n}]}}\left((1-\gamma )\Phi ^{-1}\left[1-{\frac {1}{N}}\right]+\gamma \Phi ^{-1}\left[1-{\frac {1}{Ne}}\right]\right)$

Where:

• $\mathbf {V} [{\hat {SR}}_{n}]$ is the cross-sectional variance of Sharpe Ratios across trials,
• $\gamma$ is the Euler-Mascheroni constant (approx. 0.5772),
• $e$ is Euler's number,
• $\Phi ^{-1}$ is the inverse standard normal CDF,
• $N$ is the number of independent strategy trials.

Note:

The FST highlights that the optimal outcome of an unknown number of historical simulations is right-unbounded, with enough trials, there is no Sharpe ratio sufficiently large enough to reject the hypothesis that a strategy is false, i.e., that it is over-fit and wont generalize in the out-of-sample data.

The figure shows how the $Max\{SR\}$ increases as a function of purely the number of trials.

5. Compute the DSR for each cluster.

You now have all the variables you need to compute the DSR.

${\text{DSR}}=\Phi \left({\frac {({\hat {SR}}^{*}-SR_{0})\cdot {\sqrt {T-1}}}{\sqrt {1-{\hat {\gamma }}_{3}SR_{0}+{\frac {{\hat {\gamma }}_{4}-1}{4}}{SR_{0}}^{2}}}}\right)$

Where:

• ${\hat {SR}}^{*}$ is the observed Sharpe Ratio (not annualized),
• $SR_{0}$ is the threshold Sharpe Ratio that reflects the highest Sharpe Ratio expected from $N$ unskilled strategies,
• ${\hat {\gamma }}_{3}$ is the skewnewss of the returns,
• ${\hat {\gamma }}_{4}$ is the kurtosis of the returns,
• $T$ is the returns' sample length.
• $\Phi$ is the standard normal cumulative distribution function.

Notes:

• Readers may recognize that the DSR is the Probabilistic Sharpe Ratio (PSR), where $SR_{0}$ is the maximum expected Sharpe Ratio (estimated using the False Strategy Theorem) instead of a simple threshold SR (often 0).
• The PSR assumes that only 1 trial was run and is often used to determine that the observed SR is greater than 0.
• To account for multiple-testing, use the DSR.
• The DSR will increase with:
  • Greater observed SRs.
  • Longer track records.
  • Positively skewed returns.
• The DSR decreases with:
  • Fatter tails (Kurtosis).

6. Complete the Template for Disclosing Multiple Tests.

6.1 Aggregate statistics into a table.

Several peer reviewed papers recommend to aggregate the cluster statistics into a table format.

The table below is Exhibit 7 from "A Practitioner’s Guide to the Optimal Number of Clusters Algorithm".

Template for Disclosing Multiple Tests

Where:

• Cluster is the index of the cluster; there are N clusters.
• Strat Count is the number of strategies included in that cluster.
• aSR is the annualized Sharpe Ratio of that cluster's inverse variance portfolio (IVP).
• SR is the non-annualized Sharpe Ratio of that cluster's IVP.
• Skew is the skew of the returns of that cluster's IVP.
• Kurt is the kurtosis of the returns of that cluster's IVP.
• T is the number of observations in the cluster's IVP.
• sqrt(V[SR]) is the square root of the variance of Sharpe Ratios that was computed in step 3.
• E[max SR] is the Expected Maximum Sharpe ratio ($SR_{0}$), computed in step 4.
• DSR is the Deflated Sharpe Ratio for that cluster's IVP.

6.2 Plot the Sharpe Ratios, for each cluster.

Bar plot of Deflated Sharpe Ratios, for each cluster.

In the figure above, we can see a collection of non-annualized Sharpe ratios for the 26 independent trials that were tested in the development of this investment strategy. The bars are highlighted based on if they passed the DSR at a 95% confidence level.

Note that this bar chart doesn't correspond to table above in Exhibit 7 but shares the result that only 1 cluster passed the DSR. The goal with this analysis is to show that for all clusters, except 1 - all of them failed the DSR. This would indicate that the strategy is over-fit and is likely to be a false investment strategy.

6.3 Plot the cumulative returns of the strategies.

ICumulative Returns of Trials

In the figure above the cumulative returns are plotted. On the y axis is the total return in% and the x axis are the time indexes. Do you see the very straight line (the strategy with an outlier performance)?

7. Derive a conclusion from these results.

As seen in the plot of cumulative returns, there is one outlier strategy which is likely a false investment strategy as this outlier has very high performance relative to its own cluster and others.

We can see in the bar plots that all the cluster portfolios failed to pass the DSR at a 95% confidence level, except for the one that included this outlier strategy.

Mathematical Definitions

The Deflated Sharpe Ratio (DSR)

${\text{DSR}}=\Phi \left({\frac {({\hat {SR}}^{*}-SR_{0})\cdot {\sqrt {T-1}}}{\sqrt {1-{\hat {\gamma }}_{3}SR_{0}+{\frac {{\hat {\gamma }}_{4}-1}{4}}{SR_{0}}^{2}}}}\right)$

Where:

• ${\hat {SR}}^{*}$ is the observed Sharpe Ratio (not annualized),
• $SR_{0}$ is the threshold Sharpe Ratio that reflects the highest Sharpe Ratio expected from $N$ unskilled strategies,
• ${\hat {\gamma }}_{3}$ is the skewnewss of the returns,
• ${\hat {\gamma }}_{4}$ is the kurtosis of the returns,
• $T$ is the returns' sample length.
• $\Phi$ is the standard normal cumulative distribution function.

The threshold $SR_{0}$ is approximated by:

$SR_{0}={\sqrt {\mathbf {V} [{\hat {SR}}_{n}]}}\left((1-\gamma )\Phi ^{-1}\left[1-{\frac {1}{N}}\right]+\gamma \Phi ^{-1}\left[1-{\frac {1}{Ne}}\right]\right)$

Where:

• $\mathbf {V} [{\hat {SR}}_{n}]$ is the cross-sectional variance of Sharpe Ratios across trials,
• $\gamma$ is the Euler-Mascheroni constant (approx. 0.5772),
• $e$ is Euler's number,
• $\Phi ^{-1}$ is the inverse standard normal CDF,
• $N$ is the number of independent strategy trials.

False Strategy Theorem: Statement and Proof

The False Strategy Theorem provides the theoretical foundation for the Deflated Sharpe Ratio (DSR) by quantifying how much the best Sharpe Ratio among many unskilled strategies is expected to exceed zero purely due to chance. Even if all tested strategies have true Sharpe Ratios of zero, the highest observed Sharpe Ratio will typically be positive and statistically significant—unless corrected. The DSR corrects for this inflation.

Statement

Let $\{{\hat {SR}}_{1},{\hat {SR}}_{2},\dots ,{\hat {SR}}_{N}\}$ be $N$ Sharpe Ratios independently drawn from a normal distribution with mean zero and variance $\sigma ^{2}$. Then the expected maximum Sharpe Ratio among these $N$ trials is approximately:

$SR_{0}={\sqrt {\sigma ^{2}}}\cdot \left((1-\gamma )\Phi ^{-1}\left(1-{\frac {1}{N}}\right)+\gamma \Phi ^{-1}\left(1-{\frac {1}{Ne}}\right)\right)$

Where:

• $\Phi ^{-1}$ is the quantile function (inverse CDF) of the standard normal distribution,
• $\gamma \approx 0.5772$ is the Euler–Mascheroni constant,
• $e\approx 2.718$ is Euler’s number,
• $N$ is the number of independent trials.

This value $SR_{0}$ is the **expected maximum Sharpe Ratio** under the null hypothesis of no skill, $H_{0}:SR=SR_{0}$. It represents a benchmark that any observed Sharpe Ratio must exceed in order to be considered statistically significant.

Proof Sketch

Let $X_{1},X_{2},\dots ,X_{N}\sim {\mathcal {N}}(0,1)$ be independent standard normal variables. The expected maximum of $N$ such variables is approximated by:

$\mathbb {E} [\max(X_{1},\dots ,X_{N})]\approx (1-\gamma )\Phi ^{-1}\left(1-{\frac {1}{N}}\right)+\gamma \Phi ^{-1}\left(1-{\frac {1}{Ne}}\right)$

Now let ${\hat {SR}}_{i}\sim {\mathcal {N}}(0,\sigma ^{2})$ for each $i$. Then:

$\mathbb {E} \left[\max({\hat {SR}}_{1},\dots ,{\hat {SR}}_{N})\right]=\sigma \cdot \mathbb {E} [\max(X_{1},\dots ,X_{N})]$

Combining the two expressions gives:

$SR_{0}=\sigma \cdot \left((1-\gamma )\Phi ^{-1}\left(1-{\frac {1}{N}}\right)+\gamma \Phi ^{-1}\left(1-{\frac {1}{Ne}}\right)\right)$

If $\sigma ^{2}$ is estimated as the cross-sectional variance of Sharpe Ratios $\mathbf {V} [{\hat {SR}}_{n}]$, then:

$SR_{0}={\sqrt {\mathbf {V} [{\hat {SR}}_{n}]}}\cdot \left((1-\gamma )\Phi ^{-1}\left(1-{\frac {1}{N}}\right)+\gamma \Phi ^{-1}\left(1-{\frac {1}{Ne}}\right)\right)$

This completes the derivation.

Implication for the DSR

The False Strategy Theorem shows that in large-scale testing, even unskilled strategies will produce apparently "significant" Sharpe Ratios. To correct for this, the DSR adjusts the observed Sharpe Ratio ${\hat {SR}}^{*}$ by subtracting the expected maximum from noise $SR_{0}$, and scaling by the standard error around the null hypothesis:

${\text{DSR}}=\Phi \left({\frac {{\hat {SR}}^{*}-SR_{0}}{\sigma _{\hat {SR_{0}}}}}\right)$

This yields the probability that the observed Sharpe Ratio reflects true skill, not selection bias or overfitting. DSR is more accurate than methods based on Šidák correction, because DSR takes into account the dispersion across trials, $\mathbf {V} [{\hat {SR}}_{n}]$.

Confidence and Power of the Sharpe Ratio under Multiple Testing

To assess the significance of Sharpe Ratios under multiple testing, López de Prado (2018) derives closed-form expressions for the Type I and Type II errors.

Confidence

DSR is the probability of observing a Sharpe ratio less extreme than the estimated ${\hat {SR}}^{*}$, subject to $H_{0}:SR\leq 0$ being true, where the multiple testing-adjusted baseline is $SR_{0}$. This can also be interpreted as the maximum confidence with which the null hypothesis can be rejected after observing ${\hat {SR}}^{*}$:

${\text{DSR}}=P({\hat {SR}}<{\hat {SR}}^{*}|H_{0})=\Phi \left({\frac {{\hat {SR}}^{*}-SR_{0}}{\sigma _{\hat {SR_{0}}}}}\right)$

where the standard deviation around the null hypothesis is:

$\sigma _{\hat {SR_{0}}}={\sqrt {\frac {1-{\hat {\gamma }}_{3}SR_{0}+{\frac {{\hat {\gamma }}_{4}-1}{4}}SR_{0}^{2}}{T-1}}}$

Power

The power of a test is the proportion of positives that are correctly identified. This is also known in machine learning as the test's true positive rate or recall, and sensitivity in medicine. Let $SR_{1}$ be the expected value of the alternative hypothesis, $H_{1}:SR>0$. For instance, this may be the average Sharpe ratio observed among strategies that have yielded positive excess returns. Then, the false negative rate ($\beta$, type II error) is defined as the probability of not rejecting $H_{0}$ given that $H_{1}$ is true,

$\beta =P({\hat {SR}}<SR_{c}|H_{1})=\Phi \left({\frac {SR_{c}-SR_{1}}{\sigma _{\hat {SR_{1}}}}}\right)$

where $\alpha$ is the false positive rate (type I error), and:

$SR_{c}=SR_{0}+\sigma _{\hat {SR_{0}}}\Phi ^{-1}(1-\alpha )$

$\sigma _{\hat {SR_{1}}}={\sqrt {\frac {1-{\hat {\gamma }}_{3}SR_{1}+{\frac {{\hat {\gamma }}_{4}-1}{4}}SR_{1}^{2}}{T-1}}}$

Finallly, power is the probability of rejecting the null hypothesis when it is false, namely:

$P({\hat {SR}}\geq SR_{c}|H_{1})=1-\beta$

The above equations reveal that power $(1-\beta )$ decreases the number of trials $K$, through the effect that $SR_{0}$ has on $SR_{c}$. These equations quantify the reliability of observed Sharpe Ratios under multiple testing and return non-normality. They can be used to assess the sample size needed to reject $H_{0}$ with a given power $1-\beta$.

Minimum Track Record Length

A related concept is the Minimum Track Record Length (MinTRL), which computes the minimum sample size $T$ needed such that a null hypothesis $SR_{0}$ is rejected with confidence $DSR^{*}$, given an observed ${\hat {SR}}^{*}$. Formally, the problem can be stated as

$MinTRL=min_{T}\{P({\hat {SR}}<{\hat {SR}}^{*}|H_{0})=DSR^{*}\}$

with solution

$MinTRL=1+{\Bigl (}1-{\hat {\gamma }}_{3}SR_{0}+{\frac {{\hat {\gamma }}_{4}-1}{4}}SR_{0}^{2}{\Bigr )}{\Bigl (}{\frac {\Phi (DSR^{*})}{{\hat {SR}}^{*}-SR_{0}}}{\Bigr )}^{2}$

For example, given an observed annualized ${\hat {SR}}^{*}=0.95$, we need approximately 3 years worth of daily strategy returns in order to reject the null hypothesis $H_{0}:SR_{0}=0$ with confidence 95%. This provides mathematical support to the common expectation among investors that a hedge fund must produce track records with a minimum length of 3 years, which may be reduced to 2 years for Sharpe ratios above 1.15. It is important to understand MinTRL as a minimum requirement, since this assumes a single trial (more trials will require longer track records).

See also

• Sharpe Ratio
• Backtesting
• Overfitting
• Selection bias
• Multiple comparisons problem