Part III · Models · Chapter 3A

What a trading model is

Regression, classification and honest testing

Quantitative trading can be distilled to one sentence: create a statistical edge and execute it to produce risk-adjusted returns. Both halves matter equally. A great model with poor execution makes no money, while great execution without a model is just clean losses.

The industry divides along the same line, into the alpha researchers who excel at the first half and the execution engines who excel at the second. The execution engines include firms such as Jane Street, Citadel Securities and Tower Research. The most profitable firms in the world are those that have built unified pipelines from research to execution, with minimal slippage between them.

Part III walks through the five components in order:

1. The model: what it predicts and how to test it honestly (this chapter).
2. The most basic model class worth studying, the AR(1) model (Chapter 3B).
3. Fitting: how a model's parameters are found from data (Chapter 3C).
4. Execution: how predictions become orders, which Chapter 2C covers in Part II.
5. Sizing and frequency: how big each position is and how often you trade (Chapter 3D).

The treatment is deliberately introductory, since entire books exist on each of these topics. The goal is to give you enough mechanics to read the dealer chapters that follow, from Part VI on, without confusion.

1 · What a model is

A trading model is a mathematical object that takes inputs and produces a prediction. The inputs are features: numbers observed at time $t$, such as past prices, volumes, technical indicators, macro variables or news sentiment scores. The output is a prediction of some quantity at time $t + h$, where $h$ is the prediction horizon. If you write the features as $x_t$ and the quantity to be predicted as $y_{t+h}$, a model is a function $f$ of the features:

The hat marks a prediction, as distinct from the value that later occurs. Two basic flavors of model dominate the literature: one predicts a number and the other a category.

Regression models predict a number

A regression model predicts a continuous quantity. Four examples from trading show the range:

• The next bar's return, as a log return over the next minute, hour or day.
• The price $h$ minutes from now, as a price level.
• Realized volatility over the next 30 days, a non-negative number often log-transformed to make it more Gaussian-shaped, that is, closer to the normal curve.
• The cost to execute a 1,000-share buy in the next 30 seconds, as the implementation shortfall in basis points: the gap between the price when you decided and the price you paid.

In principle the output of a regression model is unbounded. In practice it is bounded by what is physically possible: a single-day simple return below −100% is impossible, because you cannot lose more than you have.

Classification models predict a category

A classification model predicts which of a finite set of categories the next observation falls into, usually with a probability for each. Three examples from trading:

• Next bar direction, as the probability of "up" against "down".
• Next event type, as probabilities over {breakout, mean reversion, range-bound}.
• Whether a position ends as a winner, as the probability of profit given the current setup and execution path.

Classification is useful when you care about direction rather than magnitude. It is also useful when the underlying signal is structurally bounded, as a tick can only go up or down (or stay flat, in a three-class version). And it is useful when the output feeds a downstream sizing function, since many of the sizing rules in Chapter 3D take a probability, or a signed signal between −1 and +1, as their input.

Judge a classifier against its base rate, as you would a diagnostic test against the prevalence of the disease. If the market rose on 53% of the days in a test period, a "model" that always says "up" scores 53% while knowing nothing. A classifier with 55% accuracy has then shown 2 points of skill, not 55.

2 · Features, targets and lookahead bias

A model is fitted on training data made of (feature, target) pairs:

• Features $x$ are the inputs available at time $t$, before the future is revealed. They are typically a vector, a list of numbers such as past returns, indicator values and volume.
• The target $y$ is the quantity to be predicted at time $t + h$: a number for regression, or a label for classification.

A core principle of model building is that the feature must be available at the time of prediction. If your features include tomorrow's open while you try to predict tomorrow's close, you have lookahead bias, and your backtest is meaningless. This sounds obvious, yet in practice lookahead bias is the most common backtest error in retail strategy development. Three checks catch most of it:

• Check every joined dataset, so that when two tables are merged by date, each row holds only what was known on that date.
• Check every indicator that uses smoothing or filtering, to verify that it uses only past data.
• Lag every macro variable to reflect when it was published, not the period it describes.

Suppose a monthly economic figure for March is published on April 15, while a dataset stamps it March 31, the end of the period it measures. A backtest that joins on that date lets a trade on March 31 use a number that nobody could see for another two weeks. Stamp it with its publication date so that the model can use it only from then on.

A centered moving average makes the same mistake more quietly. The average of the 10 days before a date and the 10 days after it looks like a smooth indicator, even though half of it is the future.

3 · Start simple

Univariate and multivariate models

The most basic model is univariate, with one feature and one target. Next come multivariate models, with many features and one target. The most complex are multi-target models, with many features and many targets at once.

Univariate models are easy to interpret, easy to debug and surprisingly competitive. The AR(1) model of Chapter 3B is univariate. A linear regression of the next bar's return on the past bar's return has just two parameters, a slope and an intercept, yet it is often hard to beat for short-horizon prediction.

Multivariate models can capture interactions between features at a price in interpretability and in the risk of overfitting. With more features than you have independent observations, even a linear regression can fit noise perfectly. The reason is counting: a model with $n$ free parameters can pass exactly through $n$ data points, whatever those points are. Two points always lie on a straight line, just as 100 points always lie on some curve with 100 parameters. The danger compounds when the features are correlated with each other, because the fit can then give them large, offsetting weights that chase noise.

The Occam tax

There is a strong empirical prior in quant trading that simpler is better. This is not just folklore, because it has explicit statistical foundations.

The bias-variance tradeoff comes first. Every estimator has two sources of error. Bias is being consistently wrong in some direction, while variance is the noise in an estimate that comes from a finite sample. Measured by the mean squared error (MSE), the average squared distance between an estimate and the truth, the two add:

Complex models reduce bias but increase variance, while simple models do the opposite. Finance combines very low signal-to-noise ratios with sample sizes effectively bounded by the length of the historical record. So the optimal point on the tradeoff tends to favor simple models.

Shrinking an estimate toward zero trades a little bias for a large cut in variance, and Chapter 3C builds that trade into the fitting itself.

Overfitting is the next foundation. Given enough parameters, any model fits any in-sample data perfectly, including the noise, so the model "learns" patterns that do not exist in the process that generated the data. In financial time series the signal-to-noise ratio is roughly 1:100 to 1:1000 on short horizons. Overfitting is therefore the default outcome, not the exception.

Consider a neural network with 10,000 parameters trained on 5 years of daily data, which is 1,260 observations. It is extremely prone to overfit without regularization (Chapter 3C), dropout, early stopping or other ways of controlling complexity. Dropout switches off random parts of the network during training, while early stopping halts the fit before it has finished. With aggressive regularization and large auxiliary datasets it is possible to fit such a model usefully. But the burden of proof, that the fit reflects signal rather than noise, sits firmly on the modeler.

The "no free lunch" theorem of Wolpert and Macready (1997) proved that, averaged over all possible problems, no single algorithm outperforms any other. In specific problem domains some algorithms work better than others, although choosing them takes strong prior knowledge. In finance that prior knowledge is hard to come by, so robust simple methods tend to outperform exotic complex ones once you account for transaction costs and out-of-sample degradation.

López de Prado's principle, from Advances in Financial Machine Learning (2018), is that most published trading strategies are statistical artifacts. The probability that a backtested Sharpe ratio survives to live deployment is roughly inverse in the number of trials conducted to find it. If you tested 100 strategies and the best one has a Sharpe ratio of 1.5, its deflated Sharpe ratio, corrected for multiple testing, is probably indistinguishable from zero. Chapter 1D derives the correction and shows how fast the bar rises with the number of trials.

Linear models are competitive baselines. A univariate autoregressive model predicts the next value from the last one, with a single weight and bias. Models like it often outperform deep neural networks on noisy financial data, because the deep network finds patterns that do not generalize. This is the central empirical finding of three decades of academic finance: the linear baseline is hard to beat once you control for overfitting and transaction costs.

Earning the right to add complexity means four things:

1. Show that the simpler model leaves systematic residuals, deviations from its predictions, that the more complex model can capture.
2. Verify that the improvement survives walk-forward testing (below) on data the more complex model never saw.
3. Account for the additional degrees of freedom, the extra fitted parameters, in your performance evaluation.
4. Stress-test that the more complex model is robust to noise in its features.

Most retail content on machine learning (ML) for trading skips every one of these steps.

4 · The model and the strategy

The model is one half of the trading system. The other half is the strategy, the execution layer that turns predictions into orders. A perfect predictor with no execution policy is useless. Without one you would not know how big to trade, when to enter, when to exit, or what to do when the model contradicts a position you already hold.

Model     →  produces edge (a prediction with positive E[outcome])
Strategy  →  executes edge (turns predictions into sized orders
             with entry/exit rules)

Card counting in blackjack makes the split concrete. The counting system is the model, which estimates how many favorable cards remain in the deck. The betting strategy is everything else: bet size as a function of the count, table choice, session length and when to walk away. A perfect counter who bets the same amount on every hand loses, while a perfect bettor with no counting system loses faster. You need both, designed together.

Chapter 3B builds the most basic model class, Chapter 2C builds the execution layer, and Chapter 3D builds the sizing.

5 · Testing on the future

Retail strategy developers routinely get one subtle but critical point wrong: standard k-fold cross-validation does not work on time series. The standard method splits the data randomly into $k$ buckets, trains on $k - 1$ of them and tests on the remaining one, rotating through all $k$. In finance this produces meaningless results, because it mixes future data into the training set. A model fitted to 2019 and tested on 2018 was fitted with knowledge of what followed its test period.

The correct method is walk-forward analysis or, more rigorously, purged k-fold cross-validation (López de Prado, 2018). Both keep the test data out of the training data. Walk-forward analysis runs in four steps:

1. Split the data into time-ordered chunks.
2. For each chunk, train on data strictly before the chunk starts, possibly leaving a purge gap to avoid contamination from the labeling of data near the boundary.
3. Test the model on that chunk.
4. Aggregate the test-set performance across all the chunks.

Performance measured on the data a model was fitted to is in-sample, while performance on data it was not fitted to is out-of-sample. Walk-forward analysis costs more computation than standard cross-validation. Yet it is the only honest way to estimate out-of-sample performance on time series data. Any backtest that does not use it is overestimating performance.

The figure below runs walk-forward testing on ten years of simulated daily returns, 2,520 days, with a fixed true mean and fixed noise. It cuts them into equal windows. Each fold trains on every window before a test window and is then scored on that window alone, so five windows make four folds. You set four controls:

• Windows, from 3 to 10, starting at 5, which makes each window two years long.
• True annualized μ, the mean return, from −5% to 30%, starting at 6%.
• Noise σ, the annualized volatility, from 5% to 50%, starting at 15%.
• Seed, which redraws the returns.

The bars give each fold's annualized Sharpe ratio in sample (train) and out of sample (test). The readouts average them and report the out-of-sample degradation, one minus the ratio of the test average to the train average. Notice what the figure leaves out: there is no model in it. The true Sharpe ratio is fixed at $0.06/0.15 = 0.40$, so every difference between the bars is sampling noise. Step the seed from 1 to 3, then try 10 windows.

windows

5

true annualized μ

6.0%

noise σ

15%

seed

1

avg train Sharpe = 0.81

avg test Sharpe = 0.16

OOS degradation = 80%

At the starting settings the train bars read 1.43, 1.02, 0.51 and 0.30, while the test bars read 0.61, −0.48, −0.31 and 0.84. The readouts give an average train Sharpe ratio of 0.81, an average test Sharpe ratio of 0.16 and a degradation of 80%. At seed 3 the degradation is −18%, with the test folds beating the training folds. Averaged over many seeds, both averages come out near the true 0.40.

The lesson is about noise before it is about overfitting. By the formula in Chapter 1D, a two-year test window measures an annualized Sharpe ratio with a standard error of about $1/\sqrt{2} = 0.71$. That is larger than the true 0.40 itself, so one fold proves almost nothing. Real overfitting adds a systematic gap on top of this scatter, which walk-forward testing can reveal only when enough folds over enough years average the noise away.

6 · Check yourself