Investment Manager Selection Framework

Evaluating an investment manager is a complex and detailed process that encompasses a great deal more than analyzing investment returns. The investigation and analysis in support of an investment action, decision, or recommendation is called due diligence.  

Manager Selection Process

Typically, a search starts with a benchmark that represents the manager’s role within the portfolio. The benchmark also provides a reference for performance attribution and appraisal. There are several approaches to assigning a manager to a benchmark:

  • Third-party categorization: Database or software providers and consultants typically assign managers to a strategy sector. This categorization provides an easy and efficient way to define the universe.
  • Returns-based style analysis: The risk exposures derived from the manager’s actual return series has the advantage of being objective. The disadvantage is additional computational effort and the limitations of returns-based analysis.
  • Holdings-based style analysis: This approach allows for the estimation of current factor exposures but adds to computational effort and depends on timing and amount of transparency.
  • Manager experience: The assignment can be based on an evaluation of the manager and observations of portfolios and returns over time.
Key aspectsKey Question
Defining the universeWhat is the feasible set of managers that fit the portfolio need?
SuitabilityWhich managers are suitable for the IPS?
StyleWhich have the appropriate style?
Active vs. passiveWhich fit the active versus passive decision?
Quantitative Analysis
Investment due diligenceWhich manager “best” fits the portfolio need?
QuantitativeWhat has been the manager’s return distribution?
Attribution and AppraisalHas the manager displayed skill?
Capture ratioHow does the manager perform in “up” markets versus “down” markets?
DrawdownDoes the return distribution exhibit large drawdowns?
Qualitative Analysis
Investment due diligenceWhich manager “best” fits the portfolio need?
QualitativeIs the manager expected to continue to generate this return distribution?
PhilosophyWhat market inefficiency does the manager seek to exploit?
ProcessIs the investment process capable of exploiting this inefficiency?
PeopleDo the investment personnel possess the expertise and experience necessary to effectively implement the investment process?
PortfolioIs portfolio construction consistent with the stated investment philosophy and process?
Operational due diligenceIs the manager’s track record accurate, and does it fully reflect risks?
Process and procedureIs the back office strong, safeguarding assets and able to issue accurate reports in a timely manner?
FirmIs the firm profitable, with a healthy culture, and likely to remain in business? Is the firm committed to delivering performance over gathering assets?
Investment vehicleIs the vehicle suitable for the portfolio need?
TermsAre the terms acceptable and appropriate for the strategy and vehicle?
MonitoringDoes the manager continue to be the “best” fit for the portfolio need?

Type I and Type II Errors in Manager Selection

In making decisions on whether to hire a new manager or to keep or fire an existing manager, hypothesis testing can be used. The null hypothesis (H0) is that there is no value added.

There could be a Type I error whereby the null hypothesis is rejected, when in fact, there was no value added.

Alternatively, there could be a Type II error whereby the null hypothesis is not rejected, when in fact, there was value added.

Decision makers appear predisposed to worry more about Type I errors than Type II errors. Potential reasons for this focus on Type I errors are as follows:

  • Psychologically, people seek to avoid feelings of regret. Type I errors are errors of commission, an active decision that turned out to be incorrect, whereas Type II errors are errors of omission, or inaction.
  • Type I errors are relatively straightforward to measure and are often directly linked to the decision maker’s compensation.
  • Type I errors are more transparent to investors, so they entail not only the regret of an incorrect decision but the pain of having to explain this decision to the investor. Type II errors,

Type II errors are difficult to determine; for example, how can one objectively determine how a manager (who was not hired) would have performed?

The goal of monitoring is to determine the following:

  • Are there identifiable factors that differentiate managers hired and managers not hired?
  • Are these factors consistent with the investment philosophy and process of the decision maker?
  • Are there identifiable factors driving the decision to retain or fire managers?
  • Are these factors consistent with the investment philosophy and process of the decision maker?
  • What is the added value of the decision to retain or fire managers?

An excessive number of Type II errors would be indicative of a problem with the hiring and firing of managers. The obvious solution to minimizing Type II errors would be to track the subsequent performance of managers who were not hired as well as those who were fired. Ultimately, it is important not to hire or fire managers because of short-term performance or because of behavioral biases.

Type I errors result in costs associated with retaining managers who are weak, while Type II errors result in costs associated with not retaining managers who are strong. Therefore, assuming two separate groups of managers, the greater the differences in sample size and mean, the greater the costs of Type I and II errors.

The wider the dispersion of returns between strong and weak managers, the easier it is to distinguish between their relative skills. Therefore, it makes it less likely to have a Type I or II error which results in a lower expected cost of a Type I or II error.

In an efficient market, the dispersion of return distributions between the two groups is probably smaller due to greater difficulty in achieving alpha through active management, which would lessen the costs of hiring or retaining weak managers.

If markets are mean-reverting, then Type I errors may occur when firing a poor performer, only to have performance improve subsequently or hiring a strong performer only to have performance deteriorate subsequently.

Type II errors occur in mean-reverting markets when strong managers are retained for too long ( or managers who have weaker short-term performance are not hired and they subsequently outperform when the market goes up.

Table of Contents

Leave a Comment