Systematic Backtesting & Analysis of Breakout Setups

Introduction

The primary objective of this report is to present a systematic approach to studying and analyzing stock market movements, rather than providing a definitive guide on how to trade breakouts. By examining historical market data, we aim to develop a framework for understanding the various factors that influence trade performance and provide insights to inform future research.

The study is structured as follows: we first outline the methodology employed, detailing the data sources, entry and exit strategies, and other parameters used to analyze the market. Next, we delve into the analysis section, where we explore different dimensions such as seasonality, price, volume, consolidation, and Average Daily Range percentage (ADR%). In each subsection, we present the data and offer in-depth interpretations to uncover trends and relationships.

Following the analysis, we present the key findings, summarizing the most significant insights gleaned from the study. We then discuss areas for further research, highlighting unanswered questions and potential avenues for future exploration.

Through this report, we hope to provide a valuable framework for examining and analyzing past moves of the stock market, offering insights that can inform future research and decision-making processes. It is important to note that this report is not intended to provide financial advice or specific trading recommendations. Rather, it aims to contribute to the broader knowledge base around stock market dynamics and foster a deeper understanding of the factors that drive market performance.

Table of Contents

1. Introduction
2. Table of Contents
3. Methodology
   • Data Collection
   • Entry Criteria
   • Partial Target and Exit Adjustment
   • Exit Criteria
4. Analysis
   • Overview
   • Over the Years
   • Seasonality
   • Sectors
   • Price
   • Liquidity
   • Consolidation
   • ADR%
5. Conclusion
   • Key Findings
   • Further Research
   • Invitation to Collaborate
6. Changelog

Methodology

In this section, the Average Daily Range (ADR) plays a crucial role in determining the entry and exit criteria for the backtesting of systematic breakout setups on the daily timeframe. ADR is the measure of the average price movement of a security over a specified period, often used to gauge volatility and potential price movements. By using the ADR percentage of the past 20 trading days as a key factor in the entry and exit criteria, the strategy aims to capture breakout setups with substantial price movements and to avoid securities with low volatility. The ADR not only helps to establish the consolidation range, but also serves as the basis for setting partial targets and identifying breakouts with higher than average price movement. By incorporating ADR into multiple aspects of the methodology, the strategy is designed to identify and capitalize on significant price moves while managing risk and mitigating potential losses.

Data Collection

Historical daily prices, up to and including March 21, 2023, were collected from tickers listed on base.report. The data, originally sourced from Financial Modeling Prep, includes tickers from the NASDAQ, NYSE, and AMEX exchanges in the U.S. equity market.

Though a small number of recently delisted stocks, such as TWTR, may be present, the majority of historically delisted stocks are not included. Thus, be aware of potential survivorship bias.

Backtesting begins on the first trading day for each ticker and iterates until the most recent trading day (2023-03-21), identifying trades that fit the entry and exit criteria. The following fields are collected for each trade:

• Ticker (symbol)
• Sector (may not be available for funds)
• Industry (may not be available for funds)
• Entry date
• Entry price (always the close price of the entry day to avoid intraday price data limitations)
• Days of consolidation (number of consecutive trading days leading up to the entry day with prices within a certain range)
• ADR% (average daily range percentage for the past 20 trading days, excluding the entry day)
• Partial target reached?
• Partial sell price
• Partial sell date
• Exit date
• Exit price
• Exit reason (try to sell at LOD of entry day, try to sell at breakeven, or when price crosses down SMA10)

For more information on these fields, refer to the following sections.

Entry Criteria

Each recorded trade must meet the following requirements:

1. A move of at least 50% within the past 50 trading days (the high must be at least 50% higher than the low, with the high date occurring after the low date)
2. An ADR for the past 20 trading days of at least 3%
3. An average volume above 100,000 for the past 20 trading days (dollar volume was considered but rejected due to issues like reverse splits)
4. An open price on the potential entry day of at least 50% of the range of the past 50 trading days (a stock that has lost over 50% from the high could mean that it has lost too much momentum)
5. A consolidation phase (at least 5 trading days, but no more than 40) leading up to the entry, with prices staying within a certain range (calculated as double the ADR% of the past 20 trading days)
6. Breakout, determined by:
   1. Crossing the trend line drawn using the highest price of each day in the consolidation phase and the Theil-Sen estimator
   2. The entry (closing) price being at least a certain amount higher than the trend line interception point (calculated as 0.75 times the ADR% of the past 20 trading days)

A trade starts when these conditions are met, with the following hard stops (full position sell) added:

1. Whenever the price crosses below the low of the entry day.
2. When the closing price is lower than the SMA10.

Partial Target and Exit Adjustment

Partial Target

The partial target, set as the entry price plus the ADR% of the past 20 trading days prior to the entry day, aims to identify a continuation of strength.

Partial Sell Window

During the partial sell window, which spans from day 3 to day 5 (with day 1 being the entry day), one of three outcomes can occur:

1. If the hard stop is triggered at any time, the trade ends immediately. A hard stop occurs when the price crosses below the LOD or if the closing price crosses below SMA10.
2. If the price reaches the partial target at any time, a partial sell is executed.
3. If neither of the above events occurs by the end of the sell window, a partial sell is executed at the close of day 5.

In cases 2 or 3, when a partial sell is made, the hard stop is moved up to the entry price to minimize loss and reduce trade risk. A field is also recorded for each trade to indicate whether the partial target was reached.

Exit Criteria

A trade is exited under any of the following conditions:

1. The price closes below the SMA10. In this case, the exit sell price is set to the closing price of the day, and the exit reason is recorded as SMA .
2. If a partial sell has not been made and the price crosses below the entry LOD. In this case, the exit reason is recorded as TRY_LOD (attempting to sell at the low of the entry day).
3. If a partial sell has been made and the price crosses below the entry price. In this case, the exit reason is recorded as TRY_BREAKEVEN (attempting to sell at the entry price for a breakeven trade).

For cases 2 and 3, if a gap down occurs (when the price opens significantly lower than the previous close), the exit sell price is set to the opening price. Otherwise, the exit sell price is set to the entry LOD or the entry price, respectively.

Along with the exit date, sell price, and reason, the number of days held (trading days from entry to exit) is also recorded.

This concludes the trade process.

With the methodology for backtesting systematic breakout setups on the daily timeframe now established, the next section delves into the analysis of the gathered data. Various aspects such as sector performance, yearly trends, seasonality, liquidity, consolidation, and ADR will be examined to identify patterns and derive insights.

Analysis

Overview

Before diving deep into the data from various perspectives, a general overview of the data was conducted. One of the initial comparisons made was to assess the average performance of full sells versus partial sells, where 50% of the position was sold during the partial sell window. This comparison was crucial in understanding the potential benefits of each approach and determining the most effective strategy for maximizing gains in the systematic breakout setups.

The backtesting results reveal that making full sells at the exit provides significantly better performance compared to selling 50% of the position during the partial sell window. This observation can be attributed to the fact that selling the entire position at the exit allows the trade to capitalize on the full extent of the price movement, maximizing the potential gains from the momentum. On the other hand, selling 50% during the partial sell window reduces the exposure to the remaining upward price movement and could lead to missed opportunities for larger profits.

In both cases, if a partial sell is made, the hard stop is moved up from the low of the entry day (LOD) to breakeven (the entry price). This adjustment effectively reduces the risk and minimizes potential losses for the trade. However, the superior performance of full sells at the exit suggests that the strategy benefits more from allowing the position to capture the entirety of the momentum, rather than attempting to secure partial gains during the specified sell window.

It is important to consider that the partial taking strategy might work better for variations of the core strategy being tested. For instance, if the entry was made earlier in the day instead of always using the closing price, it could lead to different outcomes for the partial selling approach. The effectiveness of partial selling in such scenarios might be significantly different and could result in better overall performance. Therefore, it is essential to explore various alternatives and nuances of the strategy to identify the most optimal approach for different trading scenarios and market conditions.

A few outliers were removed as their excessive gains skewed the data. Given the small number of cases, their exclusion from the final study seemed reasonable. For transparency, the outliers are as follows:

• GFAI, 2023-01-26
• LRMR, 2020-05-22
• SBFM, 2021-01-11
• ALPP, 2019-10-10

The dataset can be broken down into different gain categories, which reveals the following insights:

Over the Years

In this subsection, we will analyze the data from a historical perspective, focusing on the performance of the strategies on a year-by-year basis. This approach helps to uncover potential trends and patterns that may have emerged over time. Please note that the number of trades prior to 1990 is relatively sparse. Therefore, to provide a more accurate analysis, we have consolidated the data from those years into a single category. Let's examine the yearly performance of the breakout strategies, taking into consideration both the no partial sells and partial sells approaches.

The data provides a year-by-year analysis of average gains with and without partial sells. Some key observations from the data are:

1. In general, the average gains without partial sells tend to be higher than with partial sells across most years. This trend indicates that holding a position until the full sell signal might lead to better performance compared to taking partial profits along the way.
2. The number of trades varies significantly from year to year. For instance, the number of trades in 2009 was exceptionally high at 2434, while the number of trades in 1994 was considerably low at 107. This variation could be attributed to market conditions, economic events, or changes in stock market behavior over the years.
3. Certain years, such as 1991, 1995, 1997, 2003, 2016, and 2021, show relatively strong performance in terms of average gains. This could be due to favorable market conditions or the effectiveness of the strategy during those periods.
4. Some years, like 2008 and 2022, exhibit negative average gains for both scenarios (with and without partial sells). These years might have been impacted by challenging market conditions, financial crises, or other economic events that influenced stock prices negatively.
5. In recent years, such as 2020 and 2021, there has been a noticeable increase in the number of trades. This could be related to the increased market volatility and trading activity during these years.

Overall, the data suggests that the effectiveness of the systematic breakout strategy varies over time, with some years exhibiting better performance than others. Moreover, the trend of higher average gains without partial sells seems to persist across the years.

Given the clear indication that the strategy performs better without partial sells, we will analyze the data in the following sections without partial sells. However, please keep in mind that the results may vary depending on the approach used.

Seasonality

In this subsection, we will explore the seasonality of the systematic breakout strategy by analyzing the average gains and number of trades per month. Understanding the seasonal patterns can provide insights into how the strategy performs throughout the year and help identify potential opportunities or challenges associated with certain months.

The data reveals some interesting trends regarding the performance of the systematic breakout strategy across different months:

1. January and December stand out as strong months in terms of average gains, with 3.18% and 2.59% respectively. These months also have a relatively high number of trades, suggesting that the strategy might be more effective and active during the beginning and end of the year.
2. Months such as February, April, and November also demonstrate decent average gains, ranging from 2.01% to 2.25%. These months might also be considered favorable for implementing the strategy.
3. On the other hand, months like March, June, and September exhibit lower average gains, with values below 1%. This indicates that the strategy might be less effective during these months.
4. The number of trades varies across months, with the highest number observed in June (2976) and the lowest in October (1514). This variation could be attributed to market conditions, economic events, or other factors that influence stock prices and trading activity throughout the year.

In summary, the seasonality analysis highlights that the performance of the systematic breakout strategy can vary across different months.

Sectors

The data reveals interesting patterns in the performance of different sectors. Communication Services, despite having a relatively lower number of trades (1,500), outperformed all other sectors in terms of average gains at 2.68%. This indicates that the sector might have more successful breakout setups, or the underlying stocks in this sector tend to have stronger price movements during breakouts.

On the other hand, Real Estate had the lowest average gain at -0.03%, with 546 trades, suggesting that breakout strategies might not be as effective in this sector. The negative average gain could be attributed to factors such as lower volatility or different market dynamics in the Real Estate sector compared to other sectors.

The Healthcare sector had the highest number of trades (6,340), which may point to more frequent breakout opportunities or higher volatility in this sector. The average gain of 2.10% is also among the top performers, indicating that Healthcare could be an area worth exploring for systematic breakout strategies.

Sectors with fewer trades, such as Utilities (193 trades) and Consumer Defensive (751 trades), might not provide as many opportunities for breakout trading, but they still show positive average gains of 0.98% and 1.14%, respectively.

The analysis reveals a potential inclination towards sectors with higher average gains or more frequent breakout opportunities, such as Communication Services, Healthcare, and Consumer Cyclical. At the same time, it suggests a cautious approach to sectors like Real Estate, where the strategy appears to underperform.

Taking a closer look at the performance distributions, some trends emerge across various sectors:

1. The majority of sectors exhibit a majority of trades in the "Loss" category, with Real Estate showing the highest percentage of losses at 58.61%. Financial Services and the Other sectors also have a notably high proportion of losses at 55.30% and 55.82%, respectively.
2. The 0-5% Gain category is consistently the second most common outcome across all sectors, with percentages ranging from 26.56% (Other) to 35.75% (Utilities). Technology and Energy have particularly high percentages in this category at 30.04% and 30.29%, respectively.
3. Sectors with higher percentages in the 5-10% Gain category include Energy (7.57%), Technology (5.89%), and Industrials (6.14%).
4. The 10-20% Gain category sees relatively consistent percentages across most sectors, ranging from 4.58% (Real Estate) to 6.71% (Energy).
5. The 20%+ Gain category demonstrates greater variability across sectors. Communication Services and Healthcare stand out with higher percentages at 7.47% and 7.48%, respectively, while Real Estate has the lowest percentage at 3.66%.

This data analysis suggests that the performance of systematic breakout strategies varies across sectors, with different sectors showing distinctive trends in their gain categories.

Price

In this subsection, we analyze the performance of breakout strategies based on different price groups. It's important to note that the data used for this analysis is split-adjusted, which means that historical prices have been retroactively altered to account for stock splits. As a result, the prices in these groups may not accurately represent the actual trading prices at the time of the trades. Despite this limitation, examining price groups can still provide valuable insights into the influence of stock prices on the performance of breakout strategies.

Here's a breakdown of the average gains for each price group:

1. For stocks priced below $3, the average gain was 4.82%, with 4,321 trades observed.
2. In the $3 - $6 price range, the average gain was 2.17%, with 4,308 trades observed.
3. For stocks priced between $6 - $10, the average gain was 1.84%, with 4,277 trades observed.
4. In the $10 - $15 price range, the average gain was 1.22%, with 3,663 trades observed.
5. For stocks priced between $15 - $25, the average gain was 0.79%, with 4,362 trades observed.
6. In the $25 - $50 price range, the average gain was 0.31%, with 3,821 trades observed.
7. For stocks priced above $50, the average gain was 0.45%, with 2,748 trades observed.

From this data, it appears that lower-priced stocks (below $3) had the highest average gains, while the gains generally decreased as the stock price increased. This trend could be a result of various factors, such as higher volatility or lower liquidity in lower-priced stocks, which might lead to larger price movements. However, it's crucial to keep in mind the potential limitations of using split-adjusted data when interpreting these findings, as it may not accurately represent the trading prices at the time of the trades.

Liquidity

In this section, we analyze the performance of breakout strategies based on different volume groups. The volume of a stock can provide insights into the level of interest, liquidity, and potential volatility of a security. Examining volume groups allows us to understand how these factors may impact the performance of breakout strategies.

Here's a breakdown of the average gains for each volume group:

1. For stocks with a volume between 100K - 150K, the average gain was 2.06%, with 2,935 trades observed.
2. In the 150K - 225K volume range, the average gain was 1.46%, with 3,056 trades observed.
3. For stocks with a volume between 225K - 325K, the average gain was 1.83%, with 3,014 trades observed.
4. In the 325K - 500K volume range, the average gain was 1.71%, with 3,486 trades observed.
5. For stocks with a volume between 500K - 750K, the average gain was 1.56%, with 3,049 trades observed.
6. In the 750K - 1.25M volume range, the average gain was 1.75%, with 3,230 trades observed.
7. For stocks with a volume between 1.25M - 2.25M, the average gain was 1.76%, with 3,036 trades observed.
8. In the 2.25M - 5M volume range, the average gain was 1.96%, with 2,727 trades observed.
9. For stocks with a volume above 5M, the average gain was 1.77%, with 2,967 trades observed.

The data indicates that the highest average gains were observed in stocks with a volume between 100K - 150K and 2.25M - 5M. However, the differences in average gains between the volume groups are relatively small. This suggests that the relationship between volume and the performance of breakout strategies might not be as strong as other factors such as stock price or sector.

Consolidation

In this section, we analyze the performance of breakout strategies based on the length of consolidation periods. Consolidation refers to a period where a stock's price is moving within a relatively tight range before a potential breakout. Understanding the relationship between the duration of consolidation and the performance of breakout strategies can help in identifying the optimal consolidation periods for executing trades. Please note that the consolidation calculation used in this study is relatively simple, and a more sophisticated determination of consolidation may provide better results.

Here's a breakdown of the average gains for each consolidation group:

1. For stocks with a 5-day consolidation period, the average gain was 1.24%, with 9,837 trades observed.
2. In the 6-day consolidation group, the average gain was 1.88%, with 5,204 trades observed.
3. For stocks with a 7-day consolidation period, the average gain was 2.20%, with 4,158 trades observed.
4. In the 8-9 day consolidation range, the average gain was 1.94%, with 4,781 trades observed.
5. For stocks with a 10-day+ consolidation period, the average gain was 2.27%, with 3,520 trades observed.

The data suggests that longer consolidation periods generally result in higher average gains, with the highest gains observed in the 10-day consolidation group. However, please note that the number of trades tends to decrease as the consolidation period increases.

ADR% (Average Daily Range Percentage)

Last but certainly not least, we delve into the impact of ADR% on the performance of breakout strategies. ADR% represents the average daily range percentage of a stock, calculated as the average difference between the high and low prices over a specified period, divided by the stock's price. This metric can provide insight into a stock's volatility and the potential risk associated with trading it. ADR% proved to be instrumental in the data collection phase, as it allowed for the identification and categorization of stocks based on their volatility, providing valuable insights into the relationship between volatility and the performance of breakout strategies. In this section, we examine how different ADR% groups affect the average gains of breakout strategies.

Here's an overview of the average gains for each ADR% group:

1. For stocks with an ADR% of 3-4, the average gain was 0.71%, with 3,362 trades observed.
2. In the 4-5 ADR% group, the average gain was 1.23%, with 4,756 trades observed.
3. For stocks with an ADR% of 5-6, the average gain was 1.02%, with 4,763 trades observed.
4. In the 6-7 ADR% group, the average gain was 1.34%, with 4,052 trades observed.
5. For stocks with an ADR% of 7-8, the average gain was 1.75%, with 2,920 trades observed.
6. In the 8-10 ADR% group, the average gain was 2.95%, with 3,652 trades observed.
7. For stocks with an ADR% of 10-15, the average gain was 2.96%, with 2,991 trades observed.
8. In the 16+ ADR% group, the average gain was 5.10%, with 1,004 trades observed.

The data indicates that higher ADR% values generally correspond to higher average gains. This suggests that breakout strategies may perform better when trading more volatile stocks, as the increased volatility provides more room for price movements.

Upon closer examination, it becomes apparent that the relative gain seems to map well to the ADR%. By dividing the ADR% by the average gain in each group, it can be observed that the numbers are relatively consistent, suggesting a strong correlation between the two and emphasizing the importance of considering ADR% when evaluating breakout strategies. However, it is essential to remember that higher volatility also carries higher risks.

Conclusion

Key Findings

Throughout our analysis of breakout strategies, we have uncovered several critical insights that can help provide better insight into the underlying dynamics of these trading approaches. The key findings of our study include:

1. Partial Profit Taking? : Full sells at the exit outperform partial sells, as they capture the entire price movement, maximizing gains from momentum. Partial sells minimize risk, but the strategy benefits more from capturing full momentum rather than securing partial gains.
2. Sector Performance : Certain sectors, such as Healthcare, Technology, and Consumer Cyclical, tended to yield better results on average in comparison to other sectors.
3. Yearly Performance : The performance of breakout strategies varied across different years, indicating the importance of considering broader market trends and conditions when utilizing these strategies.
4. Seasonality : Our analysis revealed that some months, such as January and April, showed stronger performance for breakout strategies, suggesting a potential seasonal influence on trade outcomes.
5. Stock Prices : While we observed a general trend of higher average gains for lower-priced stocks, it is important to note that our analysis was based on split-adjusted data, which may impact the interpretation of these results.
6. Trading Volume : Our study found no clear relationship between trading volume and average gains, suggesting that volume may not be a critical factor in determining the success of breakout strategies.
7. Consolidation Periods : We discovered that longer consolidation periods tended to yield better results, although our simple calculation method for consolidation may warrant further refinement for more accurate insights.
8. Average Daily Range (ADR%) : Stocks with higher ADR% generally produced higher gains, and the relative gain mapped well to the ADR% across different groups, indicating a potential relationship between volatility and strategy performance.

By examining these factors, we were able to gain a deeper understanding of the underlying mechanisms driving breakout strategies and their effectiveness across different market conditions.

Further Research

This is certainly one of the studies of all time. Bad puns aside, it is important to emphasize that the primary objective of this research is to present a framework for examining and analyzing historical market movements.

The numerous entry and exit parameters at our disposal allow for virtually endless variations in the study, which can lead to vastly different outcomes. There are several intriguing questions that remain unanswered, such as:

• How would the results change if we employed a different hard exit strategy than the LOD? For instance, using a multiplier of the ADR% for the initial stop at entry close?
• We used the closing price crossing down the SMA10 as an exit condition. How would the results differ if we utilized the SMA20 or another indicator instead?
• Is there any correlation between the relative volume on the entry day and the performance of the trade?

This study relies solely on historical daily data. Access to intra-day data would offer greater flexibility for testing various entry strategies, such as using a 5-minute candle for breakout confirmation.

The consolidation and breakout calculations could also be adjusted. For example, the consolidation rules might be made more lenient by permitting minor deviations outside the range as long as the overall consolidation remains intact. Alternatively, the rules could be made more stringent by incorporating requirements for "higher lows" and other criteria.

Another promising avenue for exploration involves the use of machine learning (ML) techniques to better understand the data. For instance, stock charts prior to the entry day for each trade can be compiled and analyzed using ML algorithms. By clustering these charts into similar groups based on various features, such as price patterns or technical indicators, we may be able to identify correlations between these groupings and trade performance. This approach could lead to the discovery of new insights and potentially unveil previously unrecognized patterns in the market data. Combining traditional analysis methods with advanced ML techniques has the potential to significantly enhance our understanding of the factors that influence stock market performance and refine our trading strategies.

In summary, this research provides a foundation for further exploration of the stock market's dynamics and the factors influencing trading outcomes. By considering various entry and exit strategies and adjusting consolidation and breakout calculations, we can continue to refine our understanding of market behavior and the performance of different trading approaches. Future research can build on this study's framework to address the remaining questions and dive deeper into the art of studying breakout setups.

Invitation to Collaborate

If you have any thoughts, feedback, or ideas for collaboration regarding this study, we would love to hear what you have to say. We invite you to join our communities on discord and reddit and contribute to the ongoing discussion and exploration of this approach.

For those who are extra ambitious, don't forget that you can access the full dataset (as well as the PDF of this study) here.

As a student of Qullamaggie , we strongly believe in openness and collaboration. We'd like to give a shoutout to Kristjan and encourage you to join the Qullamaggies discord server. It's an open, yet strictly moderated, community of traders who aim to learn Kristjan's methodology and help each other improve. As the author of this study, I ( e0 ) have learned a great deal from this server, not only about trading but also about the importance of persistence and discipline.

Finally, if you enjoyed this study and would like to directly support us, please consider paying an amount that you are comfortable with for the PDF + Dataset. Alternatively, if you're not already a subscriber of base.report, please take a look at our offerings, including a stock screener with unique filters that works excellently for scanning for Qullamaggie style breakouts . Our aim with this study is to inspire further exploration and refinement of systematic approaches and frameworks in market analysis, empowering traders with valuable insights. Thank you for your support!

Changelog

• 2023-04-02 - Initial release.

• 2023-04-18 - In the “Exit Criteria” section, the exit reasons for 2 and 3 were mixed up. This has now been fixed. Please note that the exit reasons were recorded correctly during data collection and only the text in the article was mixed up.