It is the inevitable backlash to a boom that went sour. While stocks were rising, few felt the need to question the system. Now, amid investor recriminations and growing regulatory scrutiny, the analyst community is engaging in an unprecedented bout of breast-beating and navel-gazing. The outcome of the process may fundamentally alter the way equity research is prepared and communicated.

Analysts stand accused of being too optimistic for too long, compromised by their personal interests and by the need to keep their firms' investment banking arms happy, of neglecting their responsibilities towards investors and - not least - of being overpaid.

Thomas Bowman, president and chief executive of US-based Association for Investment Management and Research (AIMR), agreed there was a crisis of confidence in analyst research. While the conflicts that had undermined public trust applied only to 'sell-side' analysts - those that work for brokerages and investment banks - the ripples threatened to embrace the entire profession, he said.

'I think it is important to point out that the analysts that have this conflict are but a very small part of what we consider to be the financial analyst profession,' Mr Bowman said during a visit to Hong Kong. 'Most of our members and most of the analysts do not have these conflicts. A lot of them work for buy-side firms that do not have investment banking arms: they work for mutual funds or small retail firms.

'Really, when we talk about the 'analysts', we are talking about the analysts of the big, sell-side retail firms like Merrill Lynch, Goldman Sachs, Morgan Stanley and all of the rest. And unfortunately, even though that represents a relatively small number of research analysts, we are all tainted by that brush.'

AIMR sees itself as the conscience of the analyst community. It has more than 50,000 members around the world and administers the Chartered Financial Analyst (CFA) programme, a respected industry qualification. Candidates for the CFA charter must pass three rigorous examinations and adhere to an extensive code of ethics.

AIMR is taking the analyst backlash seriously.

In July it released a consultation paper titled 'Preserving the Integrity of Research', commenting: '[AIMR] is concerned that the deterioration of investor confidence in the objectivity of sell-side research, should it continue, could greatly undermine the integrity of this research and harm the reputation of the entire investment profession.'

With US politicians and regulators taking a closer interest in the work analysts do, the paper represents an attempt to head off government intervention and preserve self-regulation. It prescribes a set of standards for firms designed to safeguard the independence of analysts.

'Very few people that make decisions on the investment banking side of these firms are AIMR members. If they were all CFA holders, this problem should not exist,' he said. 'We are an organisation of individuals. We can control individual members but [not firms].'

The standards are voluntary, but AIMR is hoping market and peer-group pressure will eventually force firms to adopt them.

The controversies that have embroiled equity research reflect the evolution of financial markets, particularly in the US, over the past 25 years. Brokerages used to make most of their money from trading commissions; they produced research to help clients make investment decisions and to induce them to trade. But deregulation of commission fees drove them down to an insignificant fraction of firm revenues. At the same time, corporate finance work - arranging initial public offerings and other fund-raising exercises for companies - began to rise in importance.

The dynamics of the business changed: research became more of an adjunct to the investment-banking business than a service for the investor. Supposedly, research and investment banking are separated by 'Chinese walls' but the conflicts are not hard to see: if a firm stands to make huge fees from arranging an equity issue for a company, then it is not helpful, to say the least, if the firm's research analyst has a 'sell' on the stock.

As the technology boom got under way in the late 1990s, what had always been a latent conflict turned into a feeding frenzy. The compensation of leading technology analysts in many cases was linked to the success of the firm's investment-banking business, inevitably compromising the independence of research. The number of sell recommendations shrank to absurd lows.

'It is because the investment-banking side has become such a huge profit centre, such a huge source of revenues, that this problem has arisen,' Mr Bowman said.

As corporate finance work was becoming a more important part of the brokerage business, a significant parallel development was taking place: individual investors were pouring into equities.

This created its own problems. As investing in the 90s bull market became a daily activity for millions of Americans, the demand for information rose. In response, business television stations such as CNBC sprang up and flourished. Equity research is complex and multi-faceted - and aimed largely at professionals - but was dumbed down into quick soundbites for a mass audience, and many in that audience were unaware of the conflict of interest at the heart of sell-side research.

'Investment professionals on the buy-side who manage money and do serious research work have always known that there are inherent conflicts in sell-side research,' Mr Bowman said. 'They might use sell-side research as background, but as professional investors they would never run right out and buy stock [because a sell-side analyst says so]. They do independent research of their own.'

Growing media coverage also fostered the celebrity culture that helped to create market-moving super-analysts.

'Things get very simplified. You have got these basically unsophisticated investors out there looking at the TV. So-and-so says a stock is a buy and suddenly they go out and buy, and everybody's happy as long as the market keeps going up,' said Mr Bowman. 'The whole way recommendations are communicated is an issue. They need to be communicated in context.'

Whether a stock is a buy depends on the risk profile of the investor: a single 25-year-old has a far higher risk tolerance than a 75-year-old pensioner. Under the new AIMR standards, firms will have to build in risk and time horizons into stock recommendations. The standards will also require firms to delink analyst compensation from investment-banking deals, among other measures.

Whether the AIMR standards are sufficient to push away the heavy hand of government remains to be seen. Mr Bowman conceded that AIMR might not have helped its case by opposing the US Securities and Exchange Commission's (SEC) fair-disclosure regulation, introduced last year.

The credibility of analysts' research has been undermined as much as anything by the practice of earnings guidance. Companies commonly steer analysts towards an earnings forecast they know they can just surpass. Under this symbiotic relationship, the analyst looks good for having an accurate forecast, while the company looks good for beating it. With earnings guidance, there is little if any incentive for independent research. The fair-disclosure regulation requires companies to release material information to the market simultaneously.

'The SEC has the better side of the argument in terms of the public - there is no question about that,' Mr Bowman said. 'The public perceives this cosy relationship, and that the [Wall] Street has all this information and we don't have any . . . It puts us in a difficult position of justifying why we would want to reinforce this whole problem.'

AIMR was not against what the SEC was trying to do in terms of increasing transparency and openness, but doubted whether fair-disclosure regulation was the right way of achieving it. The issue was controversial even among AIMR members, he said.

'Unfortunately, nothing in this world is simple. All we are saying is that maybe there are other ways that we could work to ensure there is a free flow of information, without somehow or other handcuffing the analysts.'

Graphic: anal23gbz