Accounting Bits

Adequacy of Student Surrogates for Non Students

The adequacy of students as surrogates for non-students is a controversial issue in behavioural research. Some research studies question the external validity of results based on students on the basis that students fail to represent non-students. Others, however, argue that it is the subject’s interpretation of the task and meanings they attribute (i.e., experimental realism) that affect the external validity more than the type of subjects.

Despite receiving considerable emphasis from researchers in a wide range of disciplines (e.g., psychology, organisational behaviour, management, accounting, etc.,), the accounting literature suggests that there is a widespread belief that students are inadequate surrogates for accounting practitioners. A review of the literature on student surrogates, however, suggests that decision-making studies are an area where the adequacy of student surrogates for accounting practitioners should be further examined. Furthermore, and from a somewhat more pragmatic perspective, Libby, Bloomfield, and Nelson (2002) have added impetus for the need to assess the adequacy of student surrogates for accounting practitioners. Their particular position concerns the cost and benefits associated with the overuse of professional subjects, and that negative externalities could be avoided in some cases by using student subjects. Should students be shown to be adequate surrogates then considerable research findings could be built up before seeking to confirm those results on less frequently used, and more expensive samples of professional subjects. Understanding more about the types of decision tasks and conditions under which accounting students may prove adequate surrogates for their practitioner counterparts is warranted.

This study investigates the adequacy of student surrogates when making investment decisions in the context of disclosures of significant environmental liabilities information. The present study replicates the investment decision-making task used in Milne and Patten (2002). To date, a series of experimental studies have examined how accountants react to social and/or environmental information disclosures in corporate annual reports when making investment decisions, and in this study we investigate such decisions using accounting undergraduates as research subjects, and compare the results with those gained from their practising counterparts.

In the next section we review the literature on both the issue of student surrogates and the research findings on the impact of social and/or environmental disclosures on investment decisions. The following section then describes the experimental research method used in Milne and Patten (2002) and followed in this study. Finally, we report the results and conclusions of the study.

A major criticism of using students as surrogates is the belief that there are significant differences between students and non-students. “Surrogate”, according to Watson (1974, p. 533), “denotes an object which substitutes for or takes the place of a second object”. Skills, experience, age and wealth are some of the factors, which may distinguish students from non-students, but whether these differences interfere with human information processing capacities to significantly effect decision/response outcomes is less clear cut. Aronson, Brewer and Carlsmith (1985) distinguish between “experimental” and “mundane” realism in experiments. Experimental realism, they suggest, occurs where the situation is involving to the subjects, where they are forced to take it seriously, and where it has an impact on them. Mundane realism, on the other hand, occurs to the extent that events occurring in the research setting are likely to occur in the normal course of the subjects’ lives, that is, in the ‘real world’. Simply matching features (actors, contexts, behaviours) that are present in the real world to one’s experimental situation may not necessarily contribute to the external validity of the findings, for such features may be largely surface orientated and unimportant in producing the effects.

The empirical results available on the student surrogate issue are mixed. Gordon et al. (1986) review thirty-two studies in the areas of organisational/industrial psychology and organisational behaviour in order to test whether student subjects are significantly different from their non-student counterparts. Gordon et al. (1986) found ‘significant between group differences in 55% of studies’ using a statistical method of analysis. They argue that: “… it is clear that problems exist in replicating with non-student subjects behavioral phenomena observed in student samples”. The review undertaken by Gordon et al. (1986), however, has attracted criticism. For example, Greenberg (1987) and Dobbins, Lane and Steiner (1988) have questioned the research approach of Gordon et al., and note their review focuses on examining comparative effect size differences in results. Dobbins et al. (1988) argue that:

A more sensible approach … would be to determine whether the principles based upon research with students converged with the principles based upon research with non-students when all studies are included in the comparison.

This was the approach taken by Locke (1986). He asked experts in the areas of organisational behaviour and human resource management to review studies to determine to what extent the findings of laboratory studies have been replicated in the field. Unlike Gordon et al. (1986), Locke asked the authors to “focus primarily on similarity in the direction of effects obtained and only secondarily on effect size”.

The results of Locke’s work indicate that findings of laboratory experiments can be generalised to field settings, despite the differences in subjects, tasks, and settings. According to Locke (1986, p. 6):

The results reported in this book indicate that we must begin to rethink the whole issue of external validity. The evidence indicates that a detailed, point-by-point similarity with respect to subjects, tasks, settings, and so forth is not necessarily required in order to achieve generalizability. Both college students and employees appear to respond similarly to goals, feedback, incentives, participation, and so forth, perhaps because the similarities among these subjects (such as in values) are more crucial than their differences. Task differences do not seem overwhelmingly important. Perhaps all that is needed is that the participants in either setting become involved in what they are doing.

Locke suggests that the available evidence points out that experimental realism has a better chance of generalising the results to other situations, settings, or people than the superficial similarity of representativeness.

Although the studies reviewed in this book did not explicitly measure the meaning subjects assigned to the experiments they were in, the results suggest that generalizability is determined by something other than literal similarity.

Tags: student surrogates