Evaluating an investment manager is a complex and detailed process that encompasses a great deal more than analyzing investment returns. The investigation and analysis in support of an investment action, decision, or recommendation is called due diligence.
Manager Selection Process
Typically, a search starts with a benchmark that represents the manager’s role within the portfolio. The benchmark also provides a reference for performance attribution and appraisal. There are several approaches to assigning a manager to a benchmark:
- Third-party categorization: Database or software providers and consultants typically assign managers to a strategy sector. This categorization provides an easy and efficient way to define the universe.
- Returns-based style analysis: The risk exposures derived from the manager’s actual return series has the advantage of being objective. The disadvantage is additional computational effort and the limitations of returns-based analysis.
- Holdings-based style analysis: This approach allows for the estimation of current factor exposures but adds to computational effort and depends on timing and amount of transparency.
- Manager experience: The assignment can be based on an evaluation of the manager and observations of portfolios and returns over time.
| Key aspects | Key Question |
|---|---|
| Universe | |
| Defining the universe | What is the feasible set of managers that fit the portfolio need? |
| Suitability | Which managers are suitable for the IPS? |
| Style | Which have the appropriate style? |
| Active vs. passive | Which fit the active versus passive decision? |
| Quantitative Analysis | |
| Investment due diligence | Which manager “best” fits the portfolio need? |
| Quantitative | What has been the manager’s return distribution? |
| Attribution and Appraisal | Has the manager displayed skill? |
| Capture ratio | How does the manager perform in “up” markets versus “down” markets? |
| Drawdown | Does the return distribution exhibit large drawdowns? |
| Qualitative Analysis | |
| Investment due diligence | Which manager “best” fits the portfolio need? |
| Qualitative | Is the manager expected to continue to generate this return distribution? |
| Philosophy | What market inefficiency does the manager seek to exploit? |
| Process | Is the investment process capable of exploiting this inefficiency? |
| People | Do the investment personnel possess the expertise and experience necessary to effectively implement the investment process? |
| Portfolio | Is portfolio construction consistent with the stated investment philosophy and process? |
| Operational due diligence | Is the manager’s track record accurate, and does it fully reflect risks? |
| Process and procedure | Is the back office strong, safeguarding assets and able to issue accurate reports in a timely manner? |
| Firm | Is the firm profitable, with a healthy culture, and likely to remain in business? Is the firm committed to delivering performance over gathering assets? |
| Investment vehicle | Is the vehicle suitable for the portfolio need? |
| Terms | Are the terms acceptable and appropriate for the strategy and vehicle? |
| Monitoring | Does the manager continue to be the “best” fit for the portfolio need? |
Type I and Type II Errors in Manager Selection
In making decisions on whether to hire a new manager or to keep or fire an existing manager, hypothesis testing can be used. The null hypothesis (H0) is that there is no value added.
There could be a Type I error whereby the null hypothesis is rejected, when in fact, there was no value added.
Alternatively, there could be a Type II error whereby the null hypothesis is not rejected, when in fact, there was value added.
Decision makers appear predisposed to worry more about Type I errors than Type II errors. Potential reasons for this focus on Type I errors are as follows:
- Psychologically, people seek to avoid feelings of regret. Type I errors are errors of commission, an active decision that turned out to be incorrect, whereas Type II errors are errors of omission, or inaction.
- Type I errors are relatively straightforward to measure and are often directly linked to the decision maker’s compensation.
- Type I errors are more transparent to investors, so they entail not only the regret of an incorrect decision but the pain of having to explain this decision to the investor. Type II errors,
Type II errors are difficult to determine; for example, how can one objectively determine how a manager (who was not hired) would have performed?
The goal of monitoring is to determine the following:
- Are there identifiable factors that differentiate managers hired and managers not hired?
- Are these factors consistent with the investment philosophy and process of the decision maker?
- Are there identifiable factors driving the decision to retain or fire managers?
- Are these factors consistent with the investment philosophy and process of the decision maker?
- What is the added value of the decision to retain or fire managers?
An excessive number of Type II errors would be indicative of a problem with the hiring and firing of managers. The obvious solution to minimizing Type II errors would be to track the subsequent performance of managers who were not hired as well as those who were fired. Ultimately, it is important not to hire or fire managers because of short-term performance or because of behavioral biases.
Type I errors result in costs associated with retaining managers who are weak, while Type II errors result in costs associated with not retaining managers who are strong. Therefore, assuming two separate groups of managers, the greater the differences in sample size and mean, the greater the costs of Type I and II errors.
The wider the dispersion of returns between strong and weak managers, the easier it is to distinguish between their relative skills. Therefore, it makes it less likely to have a Type I or II error which results in a lower expected cost of a Type I or II error.
In an efficient market, the dispersion of return distributions between the two groups is probably smaller due to greater difficulty in achieving alpha through active management, which would lessen the costs of hiring or retaining weak managers.
If markets are mean-reverting, then Type I errors may occur when firing a poor performer, only to have performance improve subsequently or hiring a strong performer only to have performance deteriorate subsequently.
Type II errors occur in mean-reverting markets when strong managers are retained for too long ( or managers who have weaker short-term performance are not hired and they subsequently outperform when the market goes up.