Research: How Are Paeg/ls Formed

Betler

Building COMMENTARY

9rorrlt Perf

ormernce

77

JULY

2OO3

RESEA RCH: HOW ARE PAEG/L@S FORMED? Market discipline and regulatory pressure are forcing traders to evaluate the quality of their trading. This can be done by peer group comparisons or against a cost benchmark, preferably both. As the benchmark becomes more relevant to trade evaluations, traders want to know what's behind the PAEG/L concept applied by Plexus since 1994. This Commentary discuss the benchmarking process in detail.

accu rately attributed

Why Did Plexus Create The PAEG/L Concept?

;

2. Managers and traders develop a better sense of when and why trading costs rise; and 3. Valuable insights lead to action items that directly enhance future pedormance.

- Plexus Average Execution Gain/Loss benchmark for evaluating the quality of execution of a trade. The Alpha Capture@ Without a good benchmark, organizations often implementation shortfall approach answers the cannot make the critical first leap from ouestion: What did it cost to execute this trade? measurement to evaluation that leads to The PAEG/I- puts this cost measuring in context improvement. PAEG/L

is a

by answering the question "What should it cost to execute this trade?"

"What should

it

cost?" refers

to the

typical experience of professional investment managers, traders, and brokers executing simtlar trades in si milar circumstances.

The PA.trGi! r'neasrrre of eynacted coqt i-e d45ir.rarl from a statistical regression applied to recent trade data. Plexus' unique database contains a very large sample of manager-to-trader-to-broker linked trade data and provides the ability to create intelligent estimates of costs through time.

Why fs lhr's Important?

A good benchmark

or

poor performance by managers, traders, and brokers. The benefits of better benchmarks include:

1.

The objective of PAEG/L research is to explain the variation in trade cost. This requires: 1.

A rich database of observations, consisting of

the experienced costs (the dependent variable) and a variety of potential causative factors; 2. A high-power regression package, capable of running very large problems; 3. A test-bed facility to determine not only how well we can explain the variation within the

sample (explanatory power) and

signals whether actual

transaction costs reflect good

Sfafisfrcally Speaking, What Is the Goal of PAEG/L Research?

Trading and brokerage skills are

more

more importantly, the ability to explain costs in out-ofsample, ntothe-futu re appl ications (forecasti n g power); 4. A deep understanding of the economics of trading and the vagaries of trading data. i

quarterly to reflect evolving market structure and conditions;

How Do You Select Explanatory Factors?

3. Art as well as science is reouired. The

An appropriate benchmark should take

into account trade- and stock-specific factors. For example, it should reflect the fact that more

difficult trades such as large trades usually cost more than small trades. A regression-based approach, such as PAEG/L, permits us to try various combinations of variables to see what works best as a forecaster. Every trade is different and traders assess many factors as they work trades. Defining an exhaustive list of variables and conditions that capture ihe fuii essence of trading is impossible. The trick is to identify a set of factors that account for as much of the variability in trade cost as possible. Finding these factors is a trial-

and-error process. The modeling process is quite similar in scope and in accuracy to what a

research director would

go

through while

developing a stock valuation model.

Over the past fifteen years, we have identified and tested dozens of factors in combination, including various measures of liquidity, volatility, trend, market, size, and nature of the trading desk (size, style, etc.) We often find, sometimes much to our surprise, that important-sounding factors do not add forecasting power. This happens when one of the existing factors is a strong surrogate for another and, so to speak, steals its thunder. For example, tagging trades according to manager style (e.9. growth, value) doesn't help because the trade size, company capitalization and short-term price momentum already capture the differences between manager styles.

How Are The Equations Produced? PAEG/Ls are derived quarterly through a multi. step process: 1. Measure total transaction cost from timestamped manager order through completion; 2. Form a rolling six month client data universe for trades world-wide. Equations are updated

4.

regression equations are carefully screened for outlier observation effects that can easily distort the equations. An example: a very large trade that fortuitously found the other side for a cross al near-zero cost. lt would be unfair to expect a trader to duplicate those circumstances, so these irreproducible observations are d iscarded ; A parallel procedure is used to calculate Broker PAEG/Ls, which benchmark costs from broker release through execution based on the most recent quarter's data.

What Are The Strengths Of The Database? Probably the greatest strength is the diversity of sample trades. The sample runs the gamut of institutional trading from the simplest to the most comolex. lt includes trades from hundreds of managers, traders, and brokers, all striving to produce "best execution" and maximal performance. lt is an unbiased, peer-based totalcost comparative standard.

The second strength is the number

of observations: over a million orders go into the computation of U.S. PAEG/Ls each quarter, and over 300,000 orders are used in the computation of non-U.S. equations. Each observation contains

+lr^ tl l('

.l^+^ (lcll,cl

+^ ((J Im^^^!rra llEdDUlY

aaa*a f.^* ^^:^+ r! \Jr -r .- -rr-titJtJ>l,D ll t/r | | PUil Pvr !rvllv

manager inception through all partial completions. Finally, we go to extraordinary efforts to scrub the data to avoid the GIGO principle. We estimate that we spend 2-4 man-years per quarter validating the data submitted to us.

What Does a Typical PAEG/L Equation Look Like? Factor

Interpretation

Coefficient

Standard Error

T-test

Intercept

Absent any other factor effects, transacting is expected to lead to costs.

-62.39

2.54

-24.5

MomSize

Market buy/sell balance effect, measured as the product of the size of the trade and the two dav price momentum.

-4.23

0.023

182.0

-0.53

0.0'18

-29.4

PercVol

Liquidity demand, measured by the percentage of daily volume this trade reDresents.

Size

Size of the trade. in shares.

0040622

0.0000030

-20.6

LogCap

Log of the Capitalization of the company

8.29

0.61

13.6

17.i3

0.99

17.4

SizeDummy

A small-trade dummy factor equal to 1 if the trade is under 10.000 shares

The PAEG/L equation is estimated separately for !.1' qrr

rrc call nrr.{nr^ ih li.:+4/.1 qtanlzc onr{ hr rrr trc eall

orders in NASDAQ stocks. The equation above is the Buy/listed equation applied to the fourth quarter of 2004. A parallel set of equations evaluates broker executions. In addition, we

calculate separate PAEG/L equations each quarter for Canada. Latin America, Europe (exUK), UK, Emerging Europe, Japan, Asia (exJapan), and Emerging Asia,

P/ease lnterpret The Factors For Me . The Intercept is negative: in the absence of any

.

other factors. the expectation is that transacting will lead to costs. MomSize captures the movement of the stock and provides a measure of the degree to which the trade is liquidity demanding or supplying. Rising orices during a buy execution is an adverse situation, so the coefficient should be negative. Percent Volume is a measure of relative trade size. We exoect that it will be more difficult to find the other side of the trade for high levels of PercVol, so its coefficient should be negative. Large orders are more difficult to transact and should imply higher costs, so the Size coefficient is also expected to be negative. Log Cap is a proxy for stock liquidity, so it is expected that it should have a positive coefficient. Trades less than 10,000 shares are easier to execute, so we would also expect a positive coefficient for the Small Trade Dummv factor.

The T-Test column shows that all factors are significent at the 99%+ !eve!. l-lcv;ever, the explanatory power is dominated by the MomSize factor, which reflects the compounded effect of trading Iarge orders into strongly favorable or adverse market conditions. This is the situation in which trades are most likelv to be costlv.

Is the Equation Statistically Significant?

The table below compares the

in-sample power of the equations developed on explanatory second and third ouarter 2002 data to the predictive power as applied in the fourth quarter, 2002. In-Sample Explanatory Power

Next Quarter Forecasting Power

ru/v

"09'i4

130

.0848

NASDAQ Stocks Buys

.1202

091 3

NASDAQ Stocks Sells

.1236

0778

PAEG/L equation

Listed Stocks - Sells

1

Remember that the equation is tuned for maximum forecasting power (the second column.) We can easily come up with better Rsquareds for the in-sample model, but the power derives from over-fitting and is spurious. For the four equations above, the deterioration as we move out of sample is about 25%.

Can We Trust PAEG/L? What's the Quality of the Estimate?

What's the future of PAEGIL? Research

Absolutely YES, but it is important to understand what the PAEG/L is telling you. lVarkets, in the classic understatement. fluctuate. which means the costs we are trying to explain vary strongly from day to day depending on market conditions. The PAEG/L is subject to the same rules as any statistical analysis. Therefore, the statistic carries

more weight when there are more trades to reference. We always highlight and recommend to clients that they should be skeptical about any statistic.

Can I Have Access to the PAEG/L equations? Yes. Ask your consultant to discuss them with you. Our research is open to clients and we encourage independent testing and review of our equations.

How Do I Use PAEG/Ls in a Predictive Environment Such as TransPort@? The conundrum of prediction is that today we don't know tomorrow's market environment" lf the market rises, sells will be cheap and buys will be more expensive; vice versa if the market falls. Our method for dealing with that uncertainty is to build a Monte Carlo distribution of possible costs by computing the benchmark cost for each of the last 100 days. Buys will be more expensive on rising days and cheaper on falling days. The distribution shows managers and traders the range of possible outcomes in tomorrow's markets. Of course, when we know the market conditionq after the fact, we can make the appropriate adjustment and hone in the estimate.

is a

continuing challenge and we

always welcome new ideas and

new technologies. We've hired the best academicians we can find to come up with better forecasting power. What we use now is the best result to date. lt is not, and never will be, the final answer.

Where Can I Go for More Information?

Check out the Co m me nta ries on www"plexusgroup,com particularly the one entitled "A Look Under the Hood of U.S. PAEG/Ls." Plexus Neurs Plexus Group's Ninth conference will be held September 2124, 2003 at Silverado Country Club & Resod located in Napa Valley, California. Please look for the program and all reseruation forms on our website at: www.plexusgroup.com Plexus canferences gather together managers, traders, brokers, exchanges and regulators in a format of open interchange af ideas on markets, trading and investment performance. This year's conference will feature an updated format, enhancing the take-away value for the pafticipants.

Re/ease 2.0 of our lceBreaker'drill-down'tool, and a new on-line application for reviewing daily trading activity will be available mid-August. More detailed descriptions and instructions will be communicated very soon, or contact your consultant to qet the latest infarmation.

Reprint any portion with credit given to:

lexrr Dlexrrsgrorrp 11150 W. Olympic Blvd., #9AA Los Angeles, CA90064 PH: 31 0.312.5505 FAX: 31 0.31 2.5506 www.plexusgroup.com

Plexus Group is a wholly owned subsidiary of JPMorgan lnvestor Services Company, a division of JPMorgan Chase. @

2Affi Pbxus Group, lnc.