Comprehensive Content Analysis on Disclosure Quality Assessment

HSBC Holdings plc in Canary Wharf, London. Photo taken in 2005.

In recent years, the nature of business has changed fundamentally. Competitive advantage increasingly involves value creation processes that rely on intangible assets not recognised in the financial statements. To serve the information needs of the market and provide the information required for corporate transparency and accountability, there is now a consensus that the business reporting model needs to expand beyond the traditional financial reporting model that emphasises backward-looking, quantified, financial information. The general thrust of these articles and reports is that there is a need for more information that is forward-looking and non-financial in nature. It is recognised that much of this new information will be ‘soft’, i.e., either unquantified or unquantifiable.

Of particular note is the report published by the AICPA (1994), which has become extremely influential (the Jenkins Report). This set out to improve business reporting by adopting a customer focus, i.e., by meeting the information needs of investors and creditors. The report proposed a comprehensive model of business reporting that embraced a ‘broader, integrated range of information’ (p. 131). This model comprised eight main topics (financial data, operating data, management analysis, forward-looking information, information about management and shareholders, objectives and strategy, description of business and industry structure) and many sub-topics. In response to the report, FASB set up a business reporting research project to consider the types of information that companies are voluntarily providing and the means for delivering it. Reporting on the first of these two issues, FASB stated that ‘the importance of voluntary disclosures is expected to increase in the future because of the fast pace of change in the business environment’ (2001b, p.v).

Worldwide, narrative communication in annual reports is viewed as the crucial element in achieving the desired step-change in the quality of corporate reporting and regulators are focussing attention on the management discussion and analysis statement in the annual report (referred to as the MD&A in most countries and the operating and financial review (OFR) in the UK). In some jurisdictions, guidelines are being extended and revised; while in others, disclosures are becoming mandatory. In the US, post-Enron, MD&A regulations are being strengthened. In Canada, the Canadian Institute of Chartered Accountants (CICA) issued more detailed MD&A guidelines that set out six disclosure principles and develop a five-part integrated disclosure framework that covers strategy, key performance drivers, capabilities, results and risks. In the UK, the Accounting Standards Board issued revised OFR guidance, which draws upon the Jenkins framework. Company law reviews in both Australia and the UK are proposing mandatory OFRs for listed companies. Draft regulations were recently issued in the UK. Finally, consideration of MD&A statements was put on the IASB agenda in 2002.

Meanwhile, accounting researchers have increasingly focused their efforts on investigating disclosure, in particular the determinants of disclosure and the capital market consequences. Healy and Palepu observe that ‘one of the limitations of the [studies on voluntary disclosure] is the difficulty in measuring the extent of voluntary disclosure’ (2001, p. 32), while Core notes that ‘improved measures of disclosure quality also need to be developed’ (2001, p. 16). To date, two principal ways of measuring disclosure have been employed. The first approach has been to use subjective analyst disclosure quality rankings. Although this approach is not without conceptual problems, a real practical problem for US researchers is that the AIMR discontinued its rankings in 1997 (after ranking fiscal year 1995). Many other countries have never had similar rankings available. The second approach, which has a long history, has been to use researcher-constructed disclosure indices where the amount of disclosure is used as a proxy for disclosure quality.

Given the limitations and weaknesses of these two approaches, there is clearly a pressing need for research effort to be devoted to developing new ways of documenting disclosure practices, identifying dimensions of disclosure quality and exploring possible measurement proxies. It is suggested that developments of this nature have two main advantages. First, the development of a comprehensive disclosure profile serves as a practical tool, permitting the benchmarking of current practices. This allows inter-company, inter-industry and inter-country comparisons to be made and also allows changes over time to be monitored. Second, a richer set of objective measures relating to disclosures can permit much more powerful tests of many research questions that relate to narrative disclosures.

To this end, the present paper has two main objectives. First, the paper introduces to the academic literature a methodology for generating a rich descriptive profile of a company’s narrative disclosures. The basis of this profile is a comprehensive four-dimensional framework for the holistic content analysis of narratives, based on the coding of topic and three type attributes. The topic analysis is based on the Jenkins report, which proposes a ‘comprehensive model of business reporting’. The type analysis captures the time orientation, financial/non-financial and quantitative/qualitative attributes of each text unit. A text unit is defined as a phrase containing a single piece of information. Associated with this framework is a detailed set of coding procedures and a computer-assisted methodology for implemention. Second, the paper discusses the complex concept of quality and the problematic nature of quality measurement. It makes a preliminary attempt to identify some of the attributes of quality, suggests observable proxies for these and offers a tentative summary measure of disclosure quality.

The remainder of this paper is structured as follows. Section 2 describes and critiques the different extant approaches to the study of narratives in annual reports. Section 3 outlines the general principles of content analysis. Section 4 explains the new computer-assisted methodology that generates a comprehensive descriptive disclosure profile of annual report narratives. Section 5 presents an illustrative application of the procedure to the narratives in Cadbury Schweppes’ 1999 annual report. The concept of quality is explored in Section 6 and proposals are made regarding possible attributes of quality, observable proxies for some of these and aggregation into a summary measure of disclosure quality. The final section summarises and concludes.

The extant literature adopts a variety of approaches to the analysis of narratives in annual reports. These are summarised in Fig. 1. Although what is being measured varies across approaches, the implicit underlying construct of interest is generally the ‘quality’ of disclosure. The major distinction to be made is that between subjective analysts’ ratings and semi-objective approaches. Of the semi-objective approaches, some specify ex ante a list of items and scrutinise the text for their presence, ignoring sections of the text that do not relate to this list. This is the approach taken by the large body of disclosure index studies and it is characterised in this paper as a partial type of content analysis. It is a fairly objective, form-oriented content-analytic method. Other approaches encompass all of the text (textual analyses). These include thematic, meaning-oriented content analysis (where the whole text is analysed), readability studies and linguistic analysis. Each of these five approaches is discussed below in turn.