Uncover Hidden Market Relationships Using Fuzzy Logic

In the previous posts (pt1, pt.2,  pt.3) we talked a bit about how to take various indicators and fuzzify them.

Now I will show how we can quickly test for relationships in indicators.
So the question is: Can we only use RSI(3) and RSI(14) (short term and medium term Relative Strength Index) to trade the S&P500?
So in the Fuzzy Model afl script that I wrote, we input rsi3 and rsi14 as inputs and “Low,Neutral,High” as qualifiers.

We then let it optimize using the  CM-AES non-exhaustive optimizer (with all default settings). After a couple of minutes we get the results:

So the rules that the optimizer came up with are:

rule   0   rsi3  Low  AND  rsi14   Low  Then  SELL
rule   1   rsi3  Low  AND  rsi14   Neutral  Then  Do NOTHING
rule   2   rsi3  Low  AND  rsi14   High  Then  BUY
rule   3   rsi3  Neutral  AND  rsi14   Low  Then  Do NOTHING
rule   4   rsi3  Neutral  AND  rsi14   Neutral  Then  BUY
rule   5   rsi3  Neutral  AND  rsi14   High  Then  Do NOTHING
rule   6   rsi3  High  AND  rsi14   Low  Then  Do NOTHING
rule   7   rsi3  High  AND  rsi14   Neutral  Then  SELL
rule   8   rsi3  High  AND  rsi14   High  Then  SELL

Do these make sense? Unlike Neural Networks, these rules can be read and checked by common sense. What the above rules tell us is that:

Rule 2:
Buy when rsi14 is high (meaning we are in a mid-term bullish move) and rsi3 is low (meaning we are at a short-term correction). That does make sense!
rule   4 :
  rsi3  Neutral  AND  rsi14   Neutral  Then  BUY – Hmmm… This is another way of saying that SPY has a positive bias overall.

Many more possibilities

No-ones restricts us to a 2 input model (although complexity goes up extremely fast). No one also restricts us to oscillator-type of indicators. One can try the same with COT (commitment of traders) data or with the price of Gold vs Price of Bonds ratio or whatever. The bottom line is that this tool can help you search for fairly robust and linguistic base strategies that can be cross-checked with common sense.

Here is some interesting links:

http://www.nisis.risk-technologies.com/events/symp2007/papers/BB25_p_kaymak.pdf
http://www.iaeng.org/publication/WCE2007/WCE2007_pp142-147.pdf
http://www.cs.wayne.edu/~mdong/fuzzytech.pdf
http://sedok.narod.ru/s_files/poland/Dourra_Siy.pdf

* I am neither a programmer nor a fuzzy logic expert. The information given is to the best of my ability/knowledge and meant to un-intimidate and motivate self directed investors to use tools that proffesional Quants use. The information is not necessarily written with accuracy in mind but with practical usability for someone who trades.

Fuzzy Logic Optimization -The “WOO-FOO-SMASH-EFIS-MEE-COPT” model

Fuzzy Logic pt.3: Auto-generate model parameters

I often tire of reading these complicated academic papers that are filled with impossible jargon only to make my life harder. I have found that often times the difficult terminology only hides the simple concepts that lie behind.

So here I will use a difficult term myself,  just to sound smarter than I am. In today’s post I will implement a WFCMAESFISMICOPT( pronounced  “woo-foo-smash-efis-mee-copt”) model.
It stands for: A Walk-Forward Covariance-Matrix-Adaptation-Evolution-Strategy on a Fuzzy-Inference System On Multiple Instruments and A Custom Optimization Target. How is that for a title!

You think that’s complicated? It’s not. We ‘ll start with Matlab’s standard ANFIS model to see where the idea comes from.

Matlab’s Fuzzy Logic module includes what is called ANFIS or Adaptive Neuro-Fuzzy Inference Systems.
Now that sounds complicated too. Well it is, if you were to built it. But the concept behind it is simple.

An ANFIS system is a fuzzy model where the parameters are not set by an “expert” but are found using Neural Networks.
In a normal Fuzzy Model an “expert” is invited to give statements like
1. “I consider RSI  to be Low when  it is between 0 and 40. The closer to 0 the “Lower” it is”.
2. “A Low RSI(3) gives a possible Buy signal”.

read part 1 and part 2 of this article.

An ANFIS system needs no expert. Given a optimizing target (i.e.,profit), it comes up with both the membership functions and the rules. So in theory it can expose relationships that humans may not be able to see. One can argue, why not use Neural Networks directly? Anfis has the advantage that once we get the results, the results are translatable to Human understandable concepts like “Buy at Low RSI and Mid-month”.                  

ANFIS models get bogged down at 5+ Inputs and the optimization target is usually tomorrow’s return. Matlab’s ANFIS cannot (without custom code) optimize for Profit% or CAR/MDD or Sharpe Ratio.

Now Amibroker (a Software for under $300) can do that. It may not have Neural Nets but has something possibly better for optimizing the parameters: the CMA-ES optimizer. And it’s fast.

That means we can build the Fuzzy model in an Amibroker script and optimize all the parameters, using whatever optimizing target we want (i.e., the default CARMDD or our own custom). We can also run it on a portfolio of multiple instruments as well as Walk-Forward it so that we know it has no “prior knowledge” built into it.

You may want to read part 1 and part 2 of this article.

Let’s look at the TRIANGULAR memberships we build before. This basically says that
if RSI(3) is between 0 and 40 it is “Low”. The close to 0 the lower it is.
if RSI(3) is between 10 and 90, its “Mid” where it’s most “Mid” at RSI(3)=50
if RSI(3) is between 60 and 100, its “High”. The closer to 100 the higher it is.

Here’s the  TRIMember function:
TRIMember(variable, leftBottom, MidHigh, RightBottom)
So these can be written as

 SetMember(  “rsi3”, “Low”, TRIMember( rsi3, 0, 0, 40 ) );
 SetMember(  “rsi3”, “Neutral”, TRIMember( rsi3, 10, 50, 90 ) );
 SetMember(  “rsi3”, “High”, TRIMember( rsi3, 60, 100, 100 )  );
or for that matter

 SetMember(  “rsi3”, “Low”, TRIMember( rsi3, a, a+b, a+b+c) );
 SetMember(  “rsi3”, “Neutral”, TRIMember( rsi3, d, d+e, d+e+f) );
 SetMember(  “rsi3”, “High”, TRIMember( rsi3, g, g+k, g+k+m)  );

and as Amibroker users know we can do this:
a=optimize(“a”,0,0,50,10);
b=optimize(“b”,0,0,50,10);
c=optimize(“c”,0,0,50,10);
etc…

We do have a practical problem here: Too many overlaping parameters. We can help a bit by using Membership Function with 2 parameters like Gaussian Membership Function and Sigmoid Membership function.  Or we can forget about optimizing the membership functions and just optimize the rules.

That we will do in the next post:
How to uncover hidden relationships in the markets in under 5 minutes.

* I am neither a programmer nor a fuzzy logic expert. The information given is to the best of my ability/knowledge and meant to un-intimidate and motivate self directed investors to use tools that proffesional Quants use. The information is not necessarily written with accuracy in mind but with practical usability for someone who trades.

The case for Fuzzy Logic in Trading – Pt. 2: Amibroker

Backtesting Fuzzy Logic in Amibroker

In pt.1 we backtested a “fuzzified” trading strategy in Matlab (read part 1):

Buy when RSI(3) is Low and it’s before Expiration week.

Although it was easy to develop this fuzzy model in Matlab, due to the intuitive GUI, it is difficult to get meaningful backtest results, visual signals, statistics and, of course, portfolio level backtesting. Everything is possible in Matlab but it takes considerable time and effort. I would rather backtest in Amibroker and get all my usual statistics, equity curves, visual arrows, as well as 1 click portfolio backtesting.

So how do we code this in Amibroker?

Again here’s the basic idea:

You can say buy when RSI(3)<25
Or you can say Buy when  RSI(3)  is fairly Low
You can say Buy on Monday of Expiration Week
Or Buy around the middle of the month.

I will be using some custom functions I developed:

//Let’s keep these for reference
 indicators = “rsi3,Day_M”;
Property_ind1 = “Low,Neutral,High”;
Property_ind2 = “EarlyMonth,AroundExpirationWeek,EndMonth”;

Output = “Action”;
Property_Action = “Buy,Hold,Sell”;

  RSI3 = RSI( 3 ); //0-100
  Day_M = Day(); //1-31

 SetMember(  “rsi3”, “Low”, TRIMember( rsi3, -40.5, 0.475, 40.34 ) );
 SetMember(  “rsi3”, “Neutral”, TRIMember( rsi3, 10, 50, 90 ) );
 SetMember(  “rsi3”, “High”, TRIMember( rsi3, 60, 100, 140 )  );
 SetMember( “Day_M”, “EarlyMonth”,  GaussMember( Day_M, 5.27, -0.164 )  );
 SetMember( “Day_M”, “AroundExpirationWeek”,  GaussMember( Day_M, 3.94, 15.83 ) );
 SetMember( “Day_M”, “EndMonth”, GaussMember( Day_M, 5.266, 31) );
Now, these are custom functions:
SetMember() creates an array named “rsi3Low”, and fills it with values from TRIMember() function.
TRIMember() function “transforms” the RSI(3) values to a membership function [0 1] of ‘Low” Rsi.
If the above make no sense, don’t worry, it’s much easier than you think.
Now, how did these numbers come up: TRIMember( rsi3, -40.5, 0.475, 40.34 ).
Well if you look at the Matlab membership function we created in the last post, these are the numbers under “parameters”.
In other words   
SetMember(  “rsi3”, “Low”, TRIMember( rsi3, -40.5, 0.475, 40.34 ) );
 SetMember(  “rsi3”, “Neutral”, TRIMember( rsi3, 10, 50, 90 ) );
 SetMember(  “rsi3”, “High”, TRIMember( rsi3, 60, 100, 140 )  );
is basically this:

and

 SetMember( “Day_M”, “EarlyMonth”,  GaussMember( Day_M, 5.27, -0.164 )  );
 SetMember( “Day_M”, “AroundExpirationWeek”,  GaussMember( Day_M, 3.94, 15.83 ) );
 SetMember( “Day_M”, “EndMonth”, GaussMember( Day_M, 5.266, 31) );
is this:

So now we’ll use two more custom functions that create the rules
1. FuzzyAndInf() – Stands for Fuzzy “AND” inference
2.FuzzyInf()

   FuzzyAndInf( 0, “rsi3”, “Low”, “Day_M”, “AroundExpirationWeek”, “Action”, 1 );
   FuzzyAndInf( 1, “rsi3”, “High”, “Day_M”, “EarlyMonth”, “Action”, -1 );
   FuzzyInf(    2, “rsi3”, “High” , “Action”, -1 );

that roughly corresponds to these statements:
If RSI(3) is Low and it’s AroundExpirationWeek then Action is Buy 
If RSI(3) is High and it’s  EarlyMonth  then Action is Sell
If RSI(3) is High then Sell

Then we have to weight the rules and defuzzify. Here  I will simplify things since I cannot yet code a proper output membership function and de-fuzzify algorithm (help would be appreciated from math inclined people…).

Anyways, here are the reults. Remember this is a very simple concept of low RSI and expiration week positive bias…

1. SPY from 1995-August 2012
2. $0.006 commission per trade
3. Start with 100,000. Long only

Profit = 1359462.71 (1359.46%),
CAR = 16.37%,
MaxSysDD = -358582.04 (-43.88%),
CAR/MDD = 0.37,
# winners = 363 (68.75%),
 # losers = 165 (31.25%)

So here’s the whole code:

 indicators = “rsi3,Day_M”;
Property_ind1 = “Low,Neutral,High”;
Property_ind2 = “EarlyMonth,AroundExpirationWeek,EndMonth”;
Output = “Action”;
Property_Action = “Buy,Hold,Sell”;

    RSI3 = RSI( 3 ); //0-100
    Day_M = Day(); //1-31

    SetMember(  “rsi3”, “Low”, TRIMember( rsi3, -40.5, 0.475, 40.34 ) );
    SetMember(  “rsi3”, “Neutral”, TRIMember( rsi3, 10, 50, 90 ) );
    SetMember(  “rsi3”, “High”, TRIMember( rsi3, 60, 100, 140 )  );

    SetMember( “Day_M”, “EarlyMonth”,  GaussMember( Day_M, 5.27, -0.164 )  );
    SetMember( “Day_M”, “AroundExpirationWeek”,  GaussMember( Day_M, 3.94, 15.83 ) );
    SetMember( “Day_M”, “EndMonth”, GaussMember( Day_M, 5.266, 31) );

    FuzzyAndInf( 0, “rsi3”, “Low”, “Day_M”, “AroundExpirationWeek”, “Action”, 1 );
  FuzzyAndInf( 1, “rsi3”, “High”, “Day_M”, “EarlyMonth”, “Action”, -1 );
    FuzzyInf(    2, “rsi3”, “High” , “Action”, -1 );
//FuzzyAndInf( 3, “rsi3”, “Low”, “Day_M”, “EndMonth”, “Action”, 0 );

   weight=0;
    weight[0]=Param( “w0”, 1, 0.1, 1, 0.1 );
    weight[1]=Param( “w1”, 1, 0.1, 1, 0.1 );
    weight[2]=Param( “w2”, 1, 0.1, 1, 0.1 );
   // weight[3]=Param( “w3”, 1, 0.1, 1, 0.1 );

 Sellout = Holdout = Buyout = 0;

    for ( k = 0;k <= 2;k++ )
    {
        Buyout = Buyout +( Nz( getRuleRes( 1, k ) ) ) * weight[k];
        Holdout = Holdout +( Nz( getRuleRes( 0, k ) ) ) * weight[k];
        Sellout = Sellout + (Nz( getRuleRes( -1, k ) ) )* weight[k];
    }

    decision = ( 0 * Sellout + 0.5 * Holdout + 1 * Buyout ) / ( Sellout + Holdout + Buyout );
   decision=IIf( ( Sellout + Holdout + Buyout )==0,0 ,decision );

//Plot(decision,”decision”,colorRed);

Buy=decision>0.5;
Sell=decision<0.5;

Trading with Fuzzy Logic

The case for Fuzzy Logic in Trading

The more I backtest strategies the more I feel the need for robustness in a system. There is no point to optimize return. One should optimize certainty of positive return. Most strategies that do really well in the past are over complicated and over-fitted and tend to loose money.

One way to achieve robust results is to use approximate values.

You can say buy when RSI(3)<25
Or you can say Buy when  RSI(3)  is fairly Low

You can say Buy on Monday of Expiration Week
Or Buy around the middle of the month.

So here’s a system:
Buy when RSI(3) is Low and we are before Expiration week. Let’s test it.
How do we code this in
1. Matlab
2. Amibroker

1. Matlab –
First we construct a Fuzzy Model that will take 2 Inputs
a. RSI(3)
b.DayofMonth     (i.e., 1…31)

Type “fuzzy” in the Terminal and voila…
We then add a second input and name both.


Set the ranges, 0-100 for rsi3 and 1-31 for day of month.
We then move around the MF’s (memebership functions) until we come up with this:

We then come up with the rules:
If rsi is low and month is around the middle then Buy
If rsi is high and month is early Sell
if rsi is High then Sell

Here’s what the rules look like visually:

The Lower the RSI the closer to 1 the output.
The More in the middle of the month the closer to 1 the output.
Keep in mind the Output ranges from 0 to 1. Towards 1 is Buy, towards 0 is Sell.

Here’s how the fuzzy model incorporates the “Common sense” non linearity (for a simple 2 input model)

This may look “scientific” but all it does is say “I like Mid-month and Low RSI”.

So now we have our model. We will save it as “RSI3_ExpWeek_Fis.fis”.

Now on to the script. We need daily prices of SPY to calculate RSI(3) and the day number. We ‘ll get it from Yahoo.

%START CODE

 %Download “SPY” from Yahoo
ticker_fints=GetTickerV(‘SPY’,’15y’,’d’);

DatesinCode=ticker_fints.dates;
close=fts2mat(ticker_fints.close);
open=fts2mat(ticker_fints.open);
dayofmonth=day(DatesinCode);
rsi3=rsi(close,3);

inputs=[rsi3 dayofmonth];

%open the fis model
b = readfis(‘RSI3_ExpWeek_Fis.fis’);

%Feed the inputs and get result (0…1)
result=evalfis(inputs,b);

%if result <0.5 Buy, else Sell
signal=iif(result>0.5,1,iif(result<0.5,-1,0));

[pnl,pnlvector, sh]= backtestlongAmount(ticker_fints,signal,’open’,1,100000);

%END CODE

*This code uses Custom Functions. Read the article and download them  here.
So how did we do?

On the next post I will show how to code this in Amibroker.

The Yield Curve as a predictor for the SP500

*This post was drafted on September 2011 and not posted. I am posting it now with an updated chart at the end.

The question is: Does Treasury Yield data contain usable information on trading the SP500?

The answer seems to be yes.

I am using 3 month, 5 year, 10 year and 30 year Treasury yield data from 1977 to September 2011. The data comes from Yahoo.

The red line in the middle pane is a weighted aggregate of the yield spreads.
 –w1* (30y-10y)+w2*(10y-5y)+w3*(5y-3M).
In other words the more far apart the yields are (the steeper the yield curve), the higher the red line is. The closer together the yields are the lower the red line is.

My preferred method to quickly check if there is some kind of usable relationship  is to set up a linear perceptron with the 3 yield spreads as inputs and then optimize the weights of the inputs. The outcome could not have been traded (as it is optimized) but as you see it does say something about the usability of the data. It confirms common sense: It’s good to buy on a steep yield curve and sell on flat yield curves. Out of the three “spreads” the 30Y yield minus 10Y yield is the on that matters most (has the most weight in the perceptron).

Conclusion: Yield curve data seems to give one useful macroeconomic signal. The possible end of a multi-year Bull market. For example, at present time (September 2011) and after a panic driven -17% drop in the SP500, this particular model seems to imply that this is a short to medium term correction rather than the start of a multi-year bear market. The red line is currently above 140 and a long way from the sell signal at 10. This is due to immense Fed intervention that keeps yield extremely low.

Update – September 2012 – 
Well it was a short to medium term correction and the SP500 advanced some 20%+ since then. Still it seems the red middle line is nowhere near the bottom threshold where this simplistic model would issue a sell signal.