By Stuart Tarmy, Global Director of Financial Services Industry Solutions, Aerospike
In today’s capital market, investment firms need to manage and analyze massive volumes of historical and streaming data (‘tick data’) to help forecast market trends, back-test trading strategies, manage risk, identify high alpha trading opportunities, and comply with industry regulations. This can be particularly time-consuming and data-intensive because of the time series nature of market data.
In the investment industry, time series data is a sequence of data points, such as a security’s price, tracked over a specific period of time. Investors analyze time series of tick data, which is data for every trade of any given security. Tick data includes the security symbol, execution price, lot size, and time stamp. Other information often included in the tick data is the security exchange and the security industry sector. Tick data is available for many different securities types, including stocks, bonds, ETFs, mutual funds, options, derivatives, and futures.
It’s a lot of data. For example, a capital market firm might want to store every trade of a particular company every day for seven years. And then do this for all other stocks. FirstRate Data, a provider of high-resolution intraday stock market, crypto, futures, and FX data, says it has an archive that includes trade and quote data for over 4,500 tickers aggregated across 12 exchanges. The total archive is about 200 terabytes of uncompressed data.
Understanding Time Series
Forecasting is one of the main uses of time series. Capital market firms build models with historical tick data to identify future trends and directions — with the idea that understanding the past will predict the future.
Time series analysis can be used to detect patterns and trading opportunities within a given asset, across a given asset class, or across different asset classes. As an example of observing patterns across various asset classes, investors may want to compare a company’s stock movement to the movement of its bonds with different maturities. They might find that the bond prices are deteriorating or indicating a bond downgrade from the bond rating companies, which gives them insight into what might happen with the stock’s future performance.
The main behaviors of time series are trends, seasonal variation, and cyclic. A trend is a pattern in the data that reveals an increase or decrease of the series over a long period of time, and it doesn’t repeat. For example, a capital market firm may want to analyze a stock’s daily closing price for one year to help evaluate its performance. Seasonality is when patterns or cycles repeat regularly over time. A capital market firm may want to analyze a stock’s time series to see if drastic highs or lows correlate to a particular season, either traditional calendar or retail holiday seasons. Cyclic behavior is when a stock’s price increases and decreases due to economic conditions like unemployment rates or interest rates. The fluctuations are not a fixed frequency like seasonality.
Measuring how a given asset, security, or economic variable changes over time can be valuable. The leading quantitative trading firms use time series to look for exploitable patterns in the tick data to drive their investment decisions. Financial theory states that a stock’s performance is related to the underlying market (beta) and the specifics only to that stock (alpha). They are looking for alpha and beta measurements to evaluate a stock’s performance. Analyzing time series data can help identify the alpha to see how a stock performs independently of the market.
A number of progressive, research-driven asset management firms are also building quantitative capabilities to overlay their investment research to see if it’s the right time to make a purchase. These firms have learned that even though they might uncover a great company to invest in and the price looks attractive relative to expected cash flows, the stock price keeps going down because of the influence of quantitative trading. It ends up being a losing trade. While they may not understand or endorse it, they’ve become aware that quantitative investing influences their trading decisions and results.
Managing Time Series Data
Capital market firms using traditional data architectures struggle to store and quickly analyze such enormous amounts of time series data. This type of data amasses fast because each record is a new measurement at a specific time interval that is added to the existing data set. It is not simply an update to an existing record. Not only is there a huge volume of data, but the list of data sources is extensive when accounting for investments across the different asset classes.
Traditional data architectures can’t keep pace as they weren’t designed to handle real-time scale and data complexity at a large scale (terabytes, petabytes). They can suffer from high operational costs, unpredictable performance, inconsistent data, latency, and availability issues. Delays in accessing and processing time series data can lead to incorrect or incomplete investment models, degrading investment performance, and introducing additional risk.
On the other hand, modern data platforms have been designed to address these issues to efficiently store, retrieve, and process time series data in real time. These platforms contain the latest processor, storage, and networking technologies, and use geographically distributed scale-out architectures to better manage growing data sets.
These newer platforms include three key capabilities that enable firms to adjust to market conditions in real time and build and deploy their forecasting models. They can 1) ingest data fast from multiple data sources, 2) develop and train sophisticated analytic models, including artificial intelligence (AI) and neural nets, and 3) deploy these in real-time production environments.
Modern data platforms are now available to enable capital market firms to unlock the value of time series data — uncovering insights to inform data-driven investment decisions and better manage risk.
Stuart Tarmy is the Global Director of Financial Services Industry Solutions for Aerospike. He has over 25 years of experience as a General Manager and head of sales, partnerships, and product management for leading global financial services technology, electronic payments, eCommerce, artificial intelligence/machine learning (AI/ML), data privacy, and regulatory compliance companies. He has held executive roles with Fiserv, MasterCard, Bankers Trust, and McKinsey & Company. Stuart began his career as a design engineer at Texas Instruments, developing AI/ML-based computing systems. Stuart holds an MBA from the Yale School of Management, an MS in Electrical and Computer Engineering from Duke University, and an Sc.B. with Honors in Electrical and Computer Engineering from Brown University.
