According to a survey of buy side attendees at Asia Trader Forum 2016, 67% of respondents evaluate their execution performance by using benchmarks which are provided by either the portfolio managers (37%) or trading heads (30%). While VWAP is still the most commonly used benchmark in Asia, it has several well-studied shortcomings. First, interval VWAP does not account for order timing, in a sense that it cannot show the opportunity cost if the execution is started too early or too late.
The benchmark calculation usually starts with the first execution and continues until the last fill. Second, VWAP is not able to measure market impact from large orders. In particular, if an order is so big that it becomes 100% of the market volume, the performance vs. VWAP will become zero as the order becomes the benchmark itself. Finally, benchmarking vs. VWAP does not give strong incentive to search for block trades. While blocks are best used to prevent information leakage and market impact, VWAP fails to measure them appropriately.
Figure 1. Shortcomings of benchmarking vs VWAP. Source: Bloomberg Tradebook.
When we switched the benchmark from VWAP to arrival price, we found that there is a 41% chance that an outperforming order will become underperforming and vice versa (Table 2). The average absolute difference of the performance between the two benchmarks is at a sizable 112 bps. Correlation of the performance measures is also very low at only 11%.
One reason why there is such a big difference is because VWAP is much more forgiving than arrival price. In Figure 2, we can see that the standard deviation of performance vs. arrival price can be as much as four times larger than VWAP. On the other hand, the average performance in Figure 3 is insufficient to identify such difference in the risk measures.
The more relaxed measure by VWAP is also insufficient to identity outliers. As we can see from Figure 4, it is not uncommon that an order makes a significant market impact but its performance vs. VWAP is right on par. Failure to identify outliers is not only important for performance evaluation, but also critical for compliance and regulators to investigate suspicious trading activities.
Figure 4. Outliers hidden by benchmarking vs. VWAP. Market: Asia. Period: Year 2015. Source: Bloomberg Tradebook.
An appropriate benchmark is necessary to measure a buy side traders true value-add to investment returns. Although VWAP is still the most popular benchmark in Asia, we have illustrated its insufficiency to provide strong incentive for best execution or to identify the true underlying risk. Arrival price is arguably the most natural benchmark to measure execution value-add to the investment return, as it calculates a hypothetic return of the order as if traded at the arrival price. PWP is also gaining popularity with its VWAP-like stability, but is also less game-able like Arrival Price. A TCA with multiple benchmarks would also be a good approach to evaluate executions from different angles in order to obtain a more comprehensive view on a traders performance.
