Subject: 10 | Easy Money...?

Hey everyone

I know Ryan’s been in touch a couple of times since my last update, so you already know the bot’s quite a bit happier since we made a few changes. In this post, I’m going to delve into the changes we made, explain the rationale behind them and talk about the effect they (so far!) seem to have had.

Hopefully it will give you a feel for the thought process of creating a trading strategy, and signpost a couple of examples of things you might want to consider each time you develop one.

Back To Basics

The first and most obvious decision we made when talking about how to get this train back on the tracks was to drop to one strategy. Even at the end of my last post, I was talking about trying to balance the variance of this lay strategy out with a solid back strategy but honestly, why complicate things and ADD risk? One of the first things we tell people when they join is to focus on one thing, so we should follow our own process.

We were running two lay strategies and the other wasn’t doing anything exceptional in either direction. The majority of the swings in both directions came from the strategy we’ve been writing most about so far, so we ditched the second one and accepted the challenge of rescuing this one.

The remainder of the work centred on the strategy rules themselves. We needed to look at the rules that were in there and figure out which ones were meaningful and should be kept, and which should go. Plus, were there other restrictions that we should also add?

Based on several of the key figures, I strongly suspected that at least a few of the rules were not statistically significant and had either been chosen almost at random or had just been backfitted so they looked productive at the time, but now their true colours had been shown. This started life as one of our presets in the horse racing software and it was basically added to that section to demonstrate one specific feature of the software, so this wouldn’t be that much of a surprise.

Where To Start?

But where to start? Horse racing has so many variables and you can quickly end up down a rabbit hole of changing one thing, changing another thing, observing that some change downstream has now affected the first thing you changed and changing that again!

If the profit trend of this strategy had been consistently good for some time and then gradually fallen off, I would have started by looking at the price restrictions, in case we just weren’t getting as good value as we had previously.

But in this case, the figures were all over the place, despite achieving roughly the same average price consistently over time. It seemed to be our strike rate that was the problem, so this points more to an issue with the horse selection rules.

Generally when developing a strategy, I like to start from the broadest possible filters with the fewest categories and gradually get more fine-grained. So, for example, one of the first rules I will always test is handicap vs non-handicap. There are only two possible categories and whether the race is a handicap or not tends to have a big effect on the way the race is run.

In this case, we already had a set of rules that also needed evaluating, so I wasn’t starting from scratch. So, should I just clear all the existing rules and start from scratch, or should I look at the existing set of rules in context and try to refine them to make a winning strategy?

Why Not Both?

We decided on a two-pronged approach, where we would look at a couple of different versions of the strategy: one where we stripped it right back to the core principle of the strategy and almost started from scratch; and one where we looked at each of the existing rules in context and weighed them up to see if they were actually meaningful. We would then see if we could merge them into one cohesive whole.

If I were doing this in a spreadsheet, I’d be keeping the raw race data on one sheet, creating sheets for each variation I was testing and probably using lots of VLOOKUPs. In this case, I was glad to have our software with its version control, so we can easily switch back and forth between different versions of a strategy and not faff around with spreadsheets.

We’d be here all day if I described every step we took to analyse the existing rules, but I’ll outline the process and pick one or two examples. Here’s what we did to test each rule:

Remove it from the list of rules
Group the results by that rule
Run the strategy again and analyse the results data, specifically looking at that rule
Decide whether to keep or remove it
Save a new version of the strategy and repeat for the other rules

By removing the rule from the list of restrictions and adding it to the results grouping, we can see the split of the results by each category that rules is comprised of, without actually filtering on it.

Age Is Just A Number?

The original strategy had a restriction on horse age (excluding horses older than 9 years).

Obviously it’s reasonable to have some suspicions about how a horse’s age may affect its performance, but there are many confounding factors:

Weight allowance: A younger horse may be in its prime, but it may also be given a weight allowance to even the playing field
Distance: A younger horse might excel in shorter, “sprintier” races while older horses might be better suited to longer, endurance-based runs
Surface type: Younger horses might find it harder to adapt to synthetic surfaces compared to more experienced horses

If we run the strategy with an age breakdown, the results don’t really support the idea of age being a meaningful factor, at least not on its own:

We don’t really see a steady transition from profit to loss based on the age of the horse. It’s more haphazard than that.

It does seem like the youngest horses are where the bulk of our profits come from, but I can’t think of a reason why a heavy loss would be incurred on 6-year-old runners and then a profit made on 7-year-olds, so it feels like something else is going on here.

Maybe the restriction on horses older than 9 was added because the sample size is so small beyond that point (look at the number of trades), but I prefer not to apply a restriction unless I have positive data to back it up.

We could just narrow the restriction to 4 years and younger and have done with it. Doing so would definitely improve the bottom line. But why? This would be a classic example of backfitting: Looking at the results and applying a filter just because it improves the number.

We removed the age filter completely and made a note to evaluate the other correlated factors I mentioned earlier in my “start from scratch” version.

Par For The Course

The original strategy also had restrictions on the course characteristics (e.g. “sharp”, “galloping”, etc) and also the individual course.

Obviously these two rules are very strongly correlated, because the course characteristics are static, e.g. Bangor is always a left-handed, sharp, flat course with a long run-in. So already it feels a bit suspicious that both rules have been included here.

Also, there’s some conflict between the selected options. “Galloping” courses have been excluded from the course characteristics, but courses known to have a galloping kind of profile have then been included in the course list, so this is now looking fishier than a dolphin’s dump (sorry 😂)!

Deciding which of these to focus on, if any, is a good example of why I start as “zoomed-out” as possible and then get more granular. It’s quite common to see different strategies performing differently at different courses, and there certainly are “course specialist” strategies, but often the “true” factor will actually be some shared attribute that we don't know about yet.

There are 88 courses in our database, but only 9 categories of course characteristics. It’s a lot easier to spot patterns with fewer categories (and a bigger sample size in each category), so we got rid of the course filter and broke the results down by course characteristics.

There’s some evidence of a pattern here, although seemingly not the pattern the original author of this strategy saw (they filtered out “galloping” and “galloping/undulating” courses)!

It would be nice to have a more even distribution of trades in each category, but beggars can’t be choosers.

One thing that stands out is that there are a couple of categories that seem more statistically significant (based on the Archie score), but the category with the biggest sample size (”galloping”) seems to be the least significant.

Back To The Drawing Board

By this point, we had managed to cast doubt over pretty much every single one of the original rules in the strategy, which made it easier in a way, as we decided to now focus more on creating a new version almost from scratch, but drawing on what we’d learned so far.

We took as wide a view as we could and gradually drilled down, repeating the process of trying to isolate different rules, check the group breakdowns and find meaningful correlations. Again, describing all of this in full would take forever, but what follows are the highlights.

Handicaps

There was a decisive split between handicaps and non-handicaps, in favour of handicaps:

Location, Location, Location

I took a look at the split of GB vs Irish races too. I think the reason I did this was because all of my trading has been suffering recently on Irish races!

I put a post on our forum to ask if anyone else was seeing the same thing. John, one of our super users, said his DOB and TROB strategies had been doing pretty well in Ireland, so evidently the country thing is just me. Sadly, I had to suspend his membership until he repaid the funds he has evidently won from me in recent months.

Just kidding, don’t get too excited, trolls 😘

Anyway, although there’s clearly more profit on the GB races, there’s also a much bigger sample size. If anything, the strike rate in Ireland is actually slightly higher, so they’re staying in!

Race Type

One of the interesting things to test was race type. The original strategy excluded the all-weather and hurdles.

Excluding hurdles looked like a good shout, as it was the only race type that showed a heavy loss. Although the low Archie score suggests there might be something else going on there:

Eliminating A/W seemed questionable as well. Despite what’s shown in the screenshot above, I think when I first started looking at it, it did actually also show a loss, but the Archie score was very low.

This meant some A/W races were profitable and some weren’t, and there was no statistical significance to excluding A/W, so back in it went.

Blown Off Course

We were getting a bit more fine-grained now, so we added the course filter back in to see if we could spot any trends.

Here’s the breakdown, sorted in order of worst loss first:

Imagine the temptation to just filter all the “losing” courses out and instantly improve the overall profit by about 500 points!

But why? What’s the connection? I haven’t found it yet. I will probably do some analysis of just those worst-performing courses at some point to see if there really are any patterns, but for now they stay in. It looks like a couple of them are verging on being statistically significant, but if we restrict the results to this year only, we get a completely different set of “worst” courses, so this just feels random.

Other Additions

We added distance, jockey allowance and classifications filters (excluding graded races) based on a bit more research of the same kind that I’ve already gone into extensively here.

We also widened the entry price criteria a little.

The Bottom Line

We’ve managed to significantly improve the headline profit through all this work. Note also the significant increase in the Archie score, suggesting we’re using far more statistically relevant rules now.

We’ve also increased the number of trades significantly, which won’t be for everyone, but we’ve achieved a 7.12 ratio of new profit to old profit, off the back of a 2.68 ratio of new bets to old bets, which we're happy with.

The margin is still very tight though with a low ROI and we're actually averaging higher odds, leading to a higher breakeven strike rate.

The month-on-month is looking more stable now though, with the exception of last September, which I also plan on isolating to see if we can see what happened there at some point:

Conclusions

As you can see, automated trading isn’t as simple as “plug it in and profit”. You still need to know what you want your bot to do. And you need to be able to look at things from different angles and question your decisions and beliefs.

The downside of all this tweaking is that we now need to chill a bit, take a step back and observe it again before scaling it up.

But the great thing is, now we’ve done the upfront work, the qualifiers will be automatically generated every day and fed into our bot software for it to do its thing without us having to sit here placing the bets. So we can start looking for the next opportunity!

With automated trading, the emphasis shifts from execution of the bets to upfront programming and then reviewing it regularly. I check weekly to make sure the strategy still looks sound and that what we’re seeing in the backtest still matches reality.

To be honest, I’m still a bit lukewarm to this strategy. The ROI is pretty low due to the number of bets placed, but it’s been a good exercise and it's currently performing well so let’s see what happens!

FAQs

Give us the strategy!
1. Firstly, I don’t think we’ve held much back 😂 You could probably re-read our posts and piece it together pretty well, but as far as sending it out as a pre-packaged betting strategy, we’re not going to do that. We get endless stick from ~~tro~~ "concerned parties" who like to strawman us as tipsters, which we’re not. Secondly, the point of this project was to go through the entire process of building a trading bot, warts and all. If you’ve taken in the information presented, you should be well on your way to knowing how to come up with your own trading strategies and dealing with the ups and downs that come up along the way. That’s going to be way more valuable than being spoon-fed.

Hope you enjoyed reading this. Until next time!

Adam

Subscribe