Speed-the only HFT advantage ? Not so fast

Michael Lewis - Flash Boys

Is speed really the only advantage of HFTs? This NYU professor and former HFT trader says not so fast - there's more.

In his new book, " Flash Boys ," Michael Lewis writes that Regulation National Market System (known as Reg NMS) favors "a small class of insiders with the resources to create speed." But is speed really the only advantage, or is it the intelligence in their systems that give high-frequency traders their actual edge? I exploited the slow stock market for several years through clever algorithms that used high-frequency data to identify short-term patterns in prices. The program had, on average, one losing day a month. Why?

Asking the right questions of the data was key to finding the exploitable patterns buried in massive amounts of data. Interestingly, the program stopped working when Reg NMS came into effect. The regulation had, in effect, eliminated my advantage and created new opportunities for a new breed of players who could exploit NMS to their advantage.

An important overlooked fact in the current debate is that the same players who invested in speed have also invested in big data and sophisticated predictive analytics. If we consider the sheer volume of data generated by the stock market - quotes, orders, trades, cancellations, prices, messages - it is enormous. When combined with news releases and economic data, the fire hose of information is overwhelming to all but the most analytically astute. A player with the ability to discover buried, but exploitable, patterns is at a huge advantage over the significant majority who are not well positioned to find them, including regulators.

Read More How to unrig the market: 'Fast Money' trader Jon Najarian

Patterns emerge before reasons for them become apparent. But for a pattern to emerge, a human or a computer must first ask the right question. Brad Katsuyama, the whistleblower at the center of the HFT firestorm, noticed that the larger the number of exchanges across which he spread a large order, the smaller was the percentage of the order that got filled. This spurred him into asking questions and conducting experiments whose results suggested that the arrival of part of his order at one exchange alerted someone that there was more to come at the other venues. That someone, with faster connections, could get to all of the other market venues ahead of Brad's incoming order using advanced predictive analytics and force him to trade at inferior prices. The prediction didn't have to be perfect, just significantly better than random.

In any market, people closest to the trading develop a feel for the nuance of that market, which can give them an edge. That edge now resides with people who not only have the fastest computers, but also know what questions to ask of their data resources and analytic tools as markets change. Their computers spot patterns with algorithms and pass the insights along to their human masters. Those humans, by virtue of their deep knowledge of exchanges, latencies, and the behavior of other players, know what types of questions to ask. Their computers find the answers, complete with probabilities and risks, and trade them algorithmically.

Is this illegal? Not under current regulations. Is someone front-running by breaching their fiduciary responsibility as a broker and using knowledge of customer orders to profit at their customer's expense? Not always. If a high-frequency-trading operation invests heavily in technology to sniff out predictive patterns across the exchanges and take risks with its own capital, it isn't front running. It is seeking a positive return on its own investment in big data and analytics, which firms such as Amazon have done in other industries and newer entrants have begun in online advertising auctions. On the other hand, if a firm benefits by trading its own "proprietary" capital on the basis of order flow from its customers and to their detriment, it is, by definition, front-running.

Read More Dom Chu goes to high-frequency trading school

How do we improve market conditions for all investors? Quite incredibly, with additional transparency.

What if the entire data trail of the stock market were available to everyone after the end of the trading period, such as a day or a week? Information technology disrupted the old way of trading in the last century by making markets more transparent and reducing the edge held by floor traders or market makers. However, somewhere along the way, technology coupled with regulation and market forces that fragmented the stock market started to increase the complexity of trading to a point where few people understood it, let alone knew how to regulate it. We have found ourselves with complicated, fast-moving markets whose regulations were developed for a different era of trading and that have had consequences that were not anticipated. The current HFT powerhouses seized on this opportunity and kept it under wraps while advancing their analytical capability and creating a proprietary advantage.

There's nothing wrong with this in competitive markets as long as they don't engage in front-running. It is useful to note that HFTs provide a service to the market, namely, some additional liquidity, which has arguably reduced costs of trading for the small investor.

The drum beat to re-regulate trading and turn the clock back is growing. Let's not risk our markets to a populist-based reaction that asks whether HFTs do any "social good," but rely instead on an objective analysis of the "big data" that emanates from the markets. This would go a long way toward creating a more level playing field that benefits everyone instead of select players, and avoids more regulation and additional unintended consequences.

Read More Co-founder of HFT firm: HFTs can't front-run anyone

Vasant Dhar is professor and co-director for the Center for Business Analytics at the Stern School of Business at NYU. He has been trading in equity and futures markets for 20 years. He created the first systematic machine-learning based Commodities Trading Program at Morgan Stanley in the 90s that currently trades in futures markets. He is also the editor-in-chief of Big Data, a peer-reviewed journal devoted to issues around big data. Follow him on Twitter @vasantdhar.

JefferiesAnd the Best Place to Work in the global financial markets 2017 is...

Register for Financial Markets News Alerts