All posts by mintegration

Digital Assets

Love it or hate it, crypto is here to stay!

The digital asset world is anxiously waiting for the Securities and Exchange Commission’s decision on the  Winklevoss Bitcoin Trust ETF (COIN). Digital Asset Services, LLC, the sponsor of the Winklevoss Bitcoin Trust, has filed a preliminary registration statement for the Trust with the U.S. Securities and Exchange Commission (SEC) to offer its Winklevoss Bitcoin Shares to investors after the Trust’s registration statement is declared effective by the SEC. The Trust’s purpose is to hold only Bitcoin. The Shares will be listed on the Bats BZX Exchange under the ticker symbol “COIN”

For some firms, regulated investment vehicles are the only practical way to obtain these new digital assets like Bitcoin. However, COIN is not the first product available to investors.

Global Advisors, regulated by the Jersey Financial Services Commission have the Global Advisors Bitcoin Investment Fund plc (GABI) which is listed on the Channel Islands Securities Exchange. In addition to GABI, Global Advisors also manages two bitcoin-tracking Exchange Traded Notes (ETN), both listed on Nordic Nasdaq under tickers COINXBT & COINXBT. These are issued by XBT Provider AB located in Stockholm, a company acquired by Global Advisors.

Grayscale Investments have the Bitcoin Investment Trust. Eligible shares of the Bitcoin Investment Trust are quoted on OTCQX® under the symbol GBTC.

The Zurich-based private bank Vontobel AG offers a tracker certificate, listed on the Swiss SIX Structured Products Exchange.

Revoltura  have an Exchange Traded Instrument (ETI) listed on the Gibraltar Stock Exchange and trades on the Deutsche Börse.

Maltese Prime Broker EXANTE has a Bitcoin Fund which is traded exclusively on the EXANTE fund platform. They are Authorized and regulated by the Malta Financial Services Authority.

Investment firms Fortress Investment Group (FIG), Benchmark Capital and Ribbit Capital have teamed up with Pantera Capital, the U.S. Bitcoin investment firm to launch a Bitcoin investment fund called the Pantera Bitcoin Partners Fund.

SolidX Partners Inc have filed a registration statement with the SEC to offer a Bitcoin Trust on the NYSE Arca exchange.

The second product to be launched by Grayscale Investments, Ethereum (ETC) Investment Trust, allows investors to gain exposure to the price movement of Ethereum Classic through the purchase of a titled security.

The world’s leading and most diverse derivatives marketplace, CME Group, is getting in on the action too. CME Group has launched the CME CF Bitcoin Reference Rate (BRR) and CME CF Bitcoin Real Time Index (BRTI), a standardized reference rate and spot price index with independent oversight. These tools are expected to boost the adoption of Bitcoin trading and hopefully establish digital assets as a new asset class.

If you aren’t investing in or trading digital assets yet, you should pay attention. This space could get really big!

I’m offering a “Cryptocurrency Trading with Python” online workshop – moderated by Ernest Chan.

It’s on March 11th / 18th  (2 Consecutive Saturdays). Registration is here:

Gemini‘s Sandbox environment will be used, which offers full exchange functionality using test funds, for testing API connectivity and the execution of strategies.

The goal of the workshop is to help raise awareness of crypto. With exchanges acting as the on/off ramps between digital and fiat currency, more liquidity at the exchanges will ultimately help with the adoption of digital assets. This is a good thing – I am a believer and I am long crypto. More info can be found on The Cointelegraph.


Intra-day data from Quandl and a new tick database in town – party time !

(with permission from Quandl)

Quandl will soon be offering intra-day data (1 min bars). Rock on !

I was kindly given some data to test out (see below). I can’t say much more than this but keep an eye out for an official announcement soon 🙂

With both QuantGo and Quandl offering reasonably priced intra-day data, smaller trading shops have never had it so good.

I’ve been involved in the integration (i.e. messaging) of trading systems since my days working at IBM nearly 10 years ago. My current research is centred around the development of predictive models for trading systems, looking at different data sources to feed into my models. In today’s world, data is streaming from all directions and so the successful integration of data is vitally important.

Kerf is a new columnar tick database and time-series language designed especially for large volumes of numeric data. It is written in C and natively speaks JSON and SQL. Kerf has support for both real-time and historical databases and it provides data loaders for all the common Quandl formats, so you can run queries against Quandl data right away.

I’ve taken Kerf for a quick spin and via it’s Foreign Function Interface (FFI), which makes it possible to call out to C libraries from Kerf and call into Kerf from C, I can query the data using R (very easily). With ZeroMQ for my messaging, it could stack up to be a seriously good trading or analytics system.

I really like what I’ve seen so far. Is this a true competitor to Kx‘s kdb+ database ? Possibly. You should check it out for sure !


Here is the output from a simple request/response using S&P 500 One Minute Bars from Quandl. A C application acts as the handler for Kerf requests. It calls into Kerf and responds back to the client with the data. A R script requests the closing prices for AAPL, but just as easily C++, Java or Python could be used. The messages are in JSON format and these are then transformed to xts in R. Messaging uses ZeroMQ.

R (rzmq) -> C (libzmq) -> kerf (FFI) -> C (libzmq) -> R (rzmq)

I’ve decided to further develop the R API and to start integrating my real-time Interactive Brokers feed to Kerf (e.g. Java API and ZeroMq). If anyone is interested in this, please get in touch.




Strategy Replication – Evolutionary Optimization based on Financial Sentiment Data

Wow, I enjoyed replicating this neatly written paper by Ronald Hochreiter.
Ronald is an Assistant Professor at the Vienna University of Economics and Business (Institute for Statistics and Mathematics).

In his paper he applies evolutionary optimization techniques to compute optimal rule-based trading strategies based on financial sentiment data.

The evolutionary technique is a general Genetic Algorithm (GA).

The GA is a mathematical optimization algorithm drawing inspiration from the processes of biological evolution to breed solutions to problems. Each member of the population (genotype) encodes a solution (phenotype) to the problem. Evolution in the population of encodings is simulated by means of evolutionary processes; selection, crossover and mutation.
Selection exploits information in the current population, concentrating interest on high-fitness solutions. Crossover and mutation perturb these solutions in an attempt to uncover better solutions. Mutation does this by introducing new gene values into the population, while crossover allows the recombination of fragments of existing solutions to create new ones.

After reading Ronald’s paper I immediately wanted to test the hypothesis that the model is good at predicting the 1-day ahead direction of returns. For example, when the rule determines to go long, are the next day returns positive and when the rule determines to exit the long position or stay flat, are the next day returns negative. The results are not much better than a flip of a coin (see the results in the attachment below). Also, turnover is high (see the plot) which may warrant the strategy useless.

However, many variations on this genetic algorithm exist; different selection and mutation operators could be tested and a crossover operator could be added. Instead of using financial sentiment data a variety of technical indicators could be used for generating an optimal trading rule – e.g. see “Evolving Trading Rule-Based Policies“.

I emailed Ronald to get clarification regarding several questions I had. He kindly and swiftly responded with appropriate answers.

  • No crossover is used as the chromosome is too short.
  • The target return for the Markowitz portfolio is calculated as the mean of the scenario means, i.e. mean of the mean vector.
  • Pyramiding is not considered. The rule just checks whether we are invested (long) in the asset or not.
  • A maximum number of iterations is specified as the stopping rule.

Like my earlier post, End-of-Day (EOD) stock prices are sourced from QuoteMedia through Quandl’s premium subscription and the StockTwits data is sourced from PsychSignal.

The following comparisons and portfolios were constructed:

  1. In-Sample single stock results – Long-only buy-and-hold strategy vs. Optimal rule-based trading strategy
  2. Out-of-Sample Buy-and-hold optimal Markowitz portfolio
  3. Out-of-Sample Buy-and-hold 1-over-N portfolio
  4. Out-of-Sample Equally weighted portfolio of the single investment evolutionary strategies

I used R packages quadprog and PerformanceAnalytics but I wrote my own Genetic Algorithm. I’ll continue using this algorithm to evaluate other indicatorssignals and rules  🙂

Here’s some code with the results. The evolutionary risk metrics (pg. 11) are not as good as those in the original paper (I used 100 generations for my GA) but as you can see, my output is almost identical to Ronald’s. A clone perhaps – hehe.


If you have a specific paper or strategy that you would like replicated, either for viewing publically or for private use, please contact me.

Risk Budgeted Portfolios with an Evolutionary Algorithm

In my previous post I demonstrated how a risk parity portfolio could be calculated numerically using a Newton method. However, the more common problem is to be able to set risk budget constraints as opposed to having equal risk. We can do this with PCTR constrained optimization by setting PCTR inequality constraints on portfolio assets, or groups of assets. This results in an optimization problem with non-linear inequality constraints. As such, Linear or Quadratic programming won’t work and a more powerful optimization technique is needed. Differential Evolution (DE) is one such solution.

DE is a search heuristic introduced by Kenneth Price and Rainer Storn. It’s a very simple yet powerful population based, stochastic function minimizer that has been shown to be effective for global optimization of multidimensional, nonlinear, multimodal, and highly-constrained functions. It does not require that the function be either continuous or differentiable.

DE belongs to the class of evolutionary algorithms which use biology-inspired operations of crossover, mutation, and selection on a population in order to minimize an objective function over the course of successive generations. As with other evolutionary algorithms, DE solves optimization problems by evolving a population of candidate solutions using alteration and selection operators. DE uses floating-point instead of bit-string encoding of population members, and arithmetic operations instead of logical operations in mutation.

The DEoptim function in the R package with the same name performs evolutionary global optimization via the Differential Evolution algorithm.

DEoptim can be used to optimize a risk parity portfolio also. We can apply the Gini coefficient as a measure of portfolio concentration by computing risk contribution inequality (or risk contribution concentration). The Gini coefficient has an advantage over the more popular Herfindahl–Hirschman Index (HHI) measure of portfolio concentration in that it is always unitized between 0 and 1. For reference, see “Efficient Algorithms for Computing Risk Parity Portfolio Weights

First I demonstrate PCTR constrained optimization by setting PCTR inequality constraints on the portfolio assets I used in my previous post. Specifically, I set the upper threshold for each PCTR to be 20%.


# N assets
n <- ncol(returns) 

portfolio_risk_budget_obj <- function(w, r, upper_pctr_threshold) {
    # Instead of adding a penalty to the full investment constraint, ensure the 
    # weights sum to 1. (see Ardia, 2011, Differential Evolution with DEoptim)
    if (sum(w) == 0) {
        w <- w + 0.01
    w <- w/sum(w)
    # Calculate the component contribution to standard deviation of the portfolio
    component_contrib <- StdDev(r, portfolio_method = "component", weights = w)
    # ###########################
    # Now compute the penalties #
    # ###########################
    # Penalty for portfolio volatility
    port_risk_penalty <- 200 * as.numeric(component_contrib[[1]])
    # Assets above the upper PCTR threshold
    assets_above_pctr_threshold <- component_contrib[[3]][component_contrib[[3]] > 
    # Penalty for risk budget (upper)
    risk_budget_penalty <- sum(assets_above_pctr_threshold)
    # Objective function to minimize
    obj <- port_risk_penalty + risk_budget_penalty

# ##############################################################################
# Perform evolutionary global optimization via the 
# Differential Evolution algorithm.
# ##############################################################################

# Set the seed of the random number generator before calling DEoptim because DE 
# is a randomized algorithm, and the results may vary between runs. This allows
# the exact reproduction of results.

# Lower and Upper values for the weights (long-only)
lower <- rep(0, n)
upper <- rep(1, n)

# Number of population members
NP <- 20 * n

# Optimize
out <- DEoptim(fn = portfolio_risk_budget_obj, # the function to be optimized
               lower = lower, # lower bounds on each parameter to be optimized
               upper = upper, # upper bounds on each parameter to be optimized
               control = list(
                   itermax = 200, # maximum iteration (population generation) allowed.
                   NP = NP,       # number of population members
                   strategy = 1,  # DE / rand / 1 / bin (classical strategy)
                   storepopfrom = 1, # store intermediate populations from 1st gen
                   parallelType = 1, # use all available cores
                   trace = FALSE,    # do not print out progress at each iteration
                   packages = c("PerformanceAnalytics")), # load package for fn
               # two arguments to be passed to fn portfolio_risk_obj
               r = returns, 
               upper_pctr_threshold = 0.20)

# The best set of parameters (i.e. weights) found.
(w <- out$optim$bestmem / sum(out$optim$bestmem))
##        par1        par2        par3        par4        par5        par6 
## 0.002915525 0.244992874 0.046066171 0.056544851 0.165894008 0.145604661 
##        par7        par8        par9 
## 0.119851233 0.007811000 0.210319676
component_contrib <- StdDev(returns, portfolio_method = "component", weights = w)

# Display each asset's PCTR
round(component_contrib[[3]], 4)
##   AAPL      T    BAC   CSCO      F     GE   INTC   MSFT    PFE 
## 0.0032 0.1994 0.0517 0.0628 0.1785 0.1599 0.1381 0.0098 0.1968

As is shown above, each asset’s PCTR is constrained to be below 20%.

A risk parity portfolio using the Gini coefficient is now demonstrated.

portfolio_risk_parity_obj <- function(w, r, n) {
    # Instead of adding a penalty to the full investment constraint, ensure the 
    # weights sum to 1. (see Ardia, 2011, Differential Evolution with DEoptim)
    if (sum(w) == 0) {
        w <- w + 0.01
    w <- w/sum(w)
    # Calculate the component contribution to standard deviation of the portfolio
    component_contrib <- StdDev(r, portfolio_method = "component", weights = w)
    # #####################################
    # Now compute the "Gini" coefficient  #
    # #####################################
    # First sort the PCTR in non-decreasing order
    risk_contributions <- sort(component_contrib[[3]])
    # G = 2/N * SUM(i * (PCTR_i - mean(PCTR)))
    gini <- 0
    for(i in 1:length(risk_contributions)) {
        gini <- gini + (i * (risk_contributions[i] - mean(risk_contributions)))
    gini <- 2/n * gini


# Optimize
out <- DEoptim(fn = portfolio_risk_parity_obj, # the function to be optimized
               lower = lower, # lower bounds on each parameter to be optimized
               upper = upper, # upper bounds on each parameter to be optimized
               control = list(
                   itermax = 200, # maximum iteration (population generation) allowed.
                   NP = NP,       # number of population members
                   strategy = 1,  # DE / rand / 1 / bin (classical strategy)
                   storepopfrom = 1, # store intermediate populations from 1st gen
                   parallelType = 1, # use all available cores
                   trace = FALSE,    # do not print out progress at each iteration
                   packages = c("PerformanceAnalytics")), # load package for fn
               # two arguments to be passed to fn portfolio_risk_obj
               r = returns, 
               n = n)

# The best set of parameters (i.e. weights) found.
(w <- out$optim$bestmem/sum(out$optim$bestmem))
##       par1       par2       par3       par4       par5       par6 
## 0.09361924 0.15820304 0.09990865 0.09989595 0.11490732 0.11324291 
##       par7       par8       par9 
## 0.10180668 0.08351423 0.13490198
component_contrib <- StdDev(returns, portfolio_method = "component", weights = w)

# Display each asset's PCTR
round(component_contrib[[3]], 4)
##   AAPL      T    BAC   CSCO      F     GE   INTC   MSFT    PFE 
## 0.1118 0.1112 0.1115 0.1102 0.1105 0.1113 0.1113 0.1108 0.1114

As is shown above, each asset’s PCTR is 11.11% (i.e. 1/n)

A Trendy and Innovative Investment Strategy

“It has been a cruel summer for one of the trendiest, most innovative investment strategies of the asset management industry”.

This quote was taken from an article last week in the FT.

• Title:  Risk parity funds suffer a cruel summer
• Author: Robin Wigglesworth
• Source: Financial Times (FT)

Here I will demonstrate how a risk parity portfolio can be calculated quite easily using R.

By definition, a risk parity portfolio is one for which all percentage contributions to risk (PCTR) are equal. By the same definition, it means that the total contributions to risk (TCTR) are all equal also.

Using some basic calculus, it can be shown that the volatility of a portfolio can be decomposed as the weighted sum of each asset’s marginal contribution to risk (MCTR). MCTR tells us the impact of an infinitesimal increase in an asset’s weight on the total portfolio risk. With each MCTR known, each assets PCTR can be derived.

In general, a risk-parity portfolio needs to be solved by some numerical method, for example a Newton algorithm.
The setup for the Newton method requires writing the risk parity problem as a system of nonlinear equations (imposing the restriction that the weights add up to one), finding the Jacobian (using multivariable calculus) and taking a one-term Taylor expansion approximation. Then the Newton iteration can be formulated by defining a stopping rule and starting point.

For reference, see “Efficient Algorithms for Computing Risk Parity Portfolio Weights

So using simple arithmetic and matrix operations, I demonstrate how this can be implemented. My example portfolio consists of 9 liquid stocks using end-of-day (EOD) prices sourced from Quandl.


As we can see in the attachment, each asset’s PCTR is 11.11%
Now, is that trendy and innovative or what?

Thanks to Doug Martin for his lectures on portfolio optimization.

Strategy Replication – Nonlinear SVMs can systematically identify stocks with high and low future returns

I’ve replicated the following academic paper from my favourite journal;

• Title: Nonlinear support vector machines can systematically identify stocks with high and low future returns
• Authors: Ramon Huerta, Fernando Corbacho, and Charles Elkan
• Journal: Algorithmic Finance (2013) 45-58 45, DOI 10.3233/AF-13016, IOS Press,


The authors explore if there are features in accounting data and in historical price information that can help predict the stock price changes of companies.

The original source data was from the CRSP(Center for Research in Security Prices)/Compustat merged database (CCM); 7 technical features are calculated from CRSP and 44 fundamental features are obtained from Compustat.
All U.S. stocks between 1981 and 2011 are used.

Support Vector Machines (SVM) is used as the classifier technique to help predict the future direction of the stock price returns. A distinct contribution by the authors is the selection of hyper-parameters of the SVM model by a type of reinforcement learning.

Stocks that do not seem to have strong correlations with the technical and fundamental features are removed from the training data set. This leads to a significant reduction in computational time without hindering the predictive power of the model.
When forming the tail sets that constitute the positive and negative classes of the training data, an ordered list of stocks with volatility-adjusted returns is created. The estimate of the volatility is an exponential moving average, using a type of absolute deviation calculation.

The fundamental features come from accounting data; Income Statements, Balance Sheets, Statement of Cash Flows (e.g. BV, EPS). Technical features were selected by the authors based on the following claims:
• Stocks with high (low) returns over periods of three to 12 months continue to have high (low) returns over subsequent three to 12 month periods.
• Volume is a way to characterize under-reactions and over-reactions in stock price changes.
• The number of n-day highs and n-day lows as suggested in prominent literature.
• The maximum of daily returns is considered an indicator of the interest for traders with few open positions.
• A proxy for the resistance level is used (i.e. the percentage difference from the closest peak in the past) because, at least psychologically, it can be an important factor for traders.

A model for each sector is built, as defined by the Global Industry Classification Standard (GICS); Energy (10), Materials (15), Industrials (20), Consumer Discretionary (25), Consumer Staples (30), Health Care (35), Financials (40), Information Technology (45), Telecommunication Services (50), and Utilities (55). Stocks without a sector are omitted and because the Telecommunication Services and Utilities sectors have so few stock, they too are omitted.

My implementation

I was able to download the CCM data through the Wharton Research Data Service (WRDS) interface. Substantial effort was required to load, subset, merge and clean the data.

It is worth noting that a Google Summer of Code project to develop a WRDS CRSP R package is in progress and it’s use is something to consider in the future.

Typically, the number of features characterizing each stock varies from 7 to 51 depending on whether technical data, fundamental data, or both are used.
However, using all these features with 30 years of data for ALL stocks in the U.S. resulted in the replication taking far too long to simulate. The SVM model was also re-trained every day. I contacted the original authors of the paper and they told me I may reproduce the results in 3-4 months of computation if I have access to several multicore computers.

Therefore, to ensure I could at least work through the strategy replication process and test the main hypothesis, I had to reduce the dataset as follows:
• Removal of the fundamental features. The paper demonstrated that these were not as good as the technical features in characterizing the stocks (i.e. the predictive power was not as good).
• The stocks are divided into 8 sectors. I have concentrated on one sector only (Energy).
• Reduce the time series to 2 years only.
• I did not apply the stock filters (liquidity (LIQ) and dollar trading volume (DTV)). These filters eliminate stocks that do not have sufficient capacity to be traded by large mutual funds only.

I developed the SVM model using the e1071 R package. Each day, the model was trained on tail sets using a quantile of 25% (I.e. the 25% highest and 25% lowest ranked vol-adj-returns) using a history of 10 days. The model was then used to predict the future (1-day-ahead) direction of stock prices. After each prediction was made, the model was tested for accuracy.

I was unable to use the R package PortfolioAnalytics because the portfolio rebalancing was too complicated – i.e. form portfolios of 10 equally weighted long and 10 equally weighted short positions. Each position is closed at the end of the last trading day in the following 91 days. Every 28 days we open an additional 20 positions.
Hence, I wrote my own code to manage and track the sub-portfolios (i.e. I used a “queue”. The first in the queue is always the sub-portfolio to be sold first).

I implemented all 7 technical features from the original source paper:
• Momentum 3 months
• Momentum 1 year
• Volume change 3 months
• Volume change 1 month
• 12-month Highs/Lows
• maxR
• Resistance levels

A full set of summary statistics was written, utilizing R packages PerformanceAnalytics and quantmod when needed:
• Annualized Returns
• Cumulative Returns
• Annualized Sharpe ratio (SR)
• Non-normality adjusted Standard Error (se) for the Annualized SR
• Annualized STARR using a Non-Parametric Expected Tail Loss (ETL) estimate
• Maximum Drawdown
• Average Turnover
• Average Diversification
• Accuracy of the SVM model

Please note, when constructing and rebalancing the sub-portfolios, transaction costs were ignored and it was assumed that execution was instant (i.e. no price slippage).
An especially common error when backtesting and making a decision at time t is to use/include the data at time t+1. Thus, particular attention was given to any possibility of look-ahead bias. For example, when using the model to make predictions for time t+1, only data at time t was used; i.e. the 7 technical features from EOD yesterday were used to predict the price direction for today.



## [1] “====================================================================”
## [1] “====================================================================”
## [1] “”
## [1] “ENERGY SECTOR (U.S. STOCKS) 2009-01-01::2010-12-31 SVM MODEL”
## [1] “”
## [1] “”
## [1] “Annualized return (arith): 9.9 %”
## [1] “Annualized Geo mean return: 8.7 %”
## [1] “Cumulative returns: 131.9 %”
## [1] “Annualized volatility: 15.2 %”
## [1] “Annualized sharpe ratio: 0.65 (0.326)”
## [1] “Ratio (SR/se(SR)): 0.502”
## [1] “STARR (5% ETL): -24.321”
## [1] “NSTARR (5% NETL): 2.151”
## [1] “Annualized STARR (5% ETL): -175.384”
## [1] “Annualized NSTARR (5% NETL): 15.511”
## [1] “Max Drawdown: 0.366”
## [1] “SVM Accuracy (Mean): 48.8 %”
## [1] “SVM Accuracy (Max): 58.1 %”
## [1] “SVM Accuracy (Min): 41.1 %”
## [1] “====================================================================”


The SVM model accuracy is not good at all. It’s possible that the technical features do not characterize the stocks well enough or the hyper-parameters are not tuned.

The returns and SR are similar to those demonstrated in the original source paper but the backtest needs to be run for longer to have statistical significance in its results.

If the model was not re-trained every day, a significant reduction in the time needed to run the backtest would be achieved. How often the classifier should be trained is an active area of research.

It was fun to replicate the paper and much was learned. The paper is definitely worth reading !