There are a number of factors which make analyzing the performance of hedge funds using traditional portfolio analysis techniques difficult. The first is that expected returns are difficult to forecast due to aspects of finance known as biases. The main one, which affects hedge fund data is survivorship bias.
Since hedge fund managers are typically quick to close poorly performing hedge funds, when looking back to analyze the performance of hedge funds over say the last five years, this will typically only include hedge funds which are still in existence today, and which therefore have a full five years worth of historic performance data. As such, this historic return of hedge funds would not capture funds which were unable to survive, and therefore, such metrics overestimate average performance and underestimate investor risk. Since historic data only includes the performance of funds which have survived and are therefore the relatively good hedge funds. In addition, hedge funds can be influenced by stale prices which occur due to the illiquid nature of hedge funds and their underlying investments. This means that mark to market price verification is not possible in real time. This also reduces the perceived volatility of hedge fund performance since the movements in the value of the fund between each valuation point aren't captured.
Hedge funds also experience more dynamic performance than traditional funds since hedge fund strategies are not static and change over time.
For example, if systematic risk often measured by the beta of the portfolio is considered, if a equity long short portfolio manager believes stock prices will rise, they'll have more long than short positions and have a positive exposure to broad market movements or beta, which will add to return if they're correct. However, if they subsequently have a view that markets will fall and have more short positions than long, they'll have a negative beta. But if they're correct, this will still generate positive returns it then becomes more difficult to link returns to the level of systematic risk taken since positive returns have been generated both with positive and negative beta positions in the portfolio, there is a similar effect in relation to calculating volatility and correlation between long and short positions in a portfolio, both of which are metrics typically used in traditional portfolio analysis.
Finally, standard deviation is an incomplete measure of risk, meaning that interpreting standard deviation as our sole measure of risk requires an assumption that returns follow a normal distribution. For hedge funds this is unlikely to be true. Hedge fund returns often have significant negative skew, which means they suffer unusually large losses more frequently than they make unusually large gains, and they have fatter tails than predicted by the normal distribution, meaning the actual likelihood of an unusual outcome. Both gains and losses is greater than the normal distribution would predict. Both of these factors mean that standard deviation underestimates the full risk faced by hedge fund investors.
