Formal motorcycle rider education/training has been viewed as critical to the novice's mastery of the demanding skills necessary to operate and control a motorcycle. Many believe that the unique handling characteristics of the motorcycle and the rider's vulnerability to perceptual, aerodynamic and roadway disturbances require the acquisition of a high level of skill -most effectively obtained in a formal training situation. Such initiatives are founded on the fundamental and compelling assumption that students who are exposed to the education/training will be at lower risk of traffic mishap than those who are not.

Similar to early studies on the effectiveness of driver education, almost all of the early evaluations of the effectiveness of motorcycle education courses produced encouraging results-they found that formally trained riders had a lower risk of collision than persons not so trained. However, virtually all of these investigations suffered from some methodological flaw (e.g., no control or comparison group of untrained riders whatsoever, small sample size, no random assignment of subjects to trained and untrained groups, no control for exposure), which made the reliability of the positive findings questionable (Collins 1979; Satten 1980).

By contrast, the better designed studies generally produced disappointing results-they found that formally trained riders were not at lower risk of a collision than riders who did not receive the instruction. In addition, several evaluations actually found that formally trained riders had higher crash rates (per miles ridden) than those who were informally trained.

This section examines findings from the more rigorously designed studies. Similar to the previous section, the discussion is organized by jurisdiction. This approach is necessary given that the evaluations have examined courses that differ in many respects-i.e., in length of instruction, time spent on-bike versus in-class instruction, content, and quality of instruction. Accordingly, findings in one jurisdiction do not necessarily apply to courses offered elsewhere. Nevertheless, the overall direction of the results or the "convergence of evidence" has relevance to the current investigation.

As will become evident from examining this section, developments in motorcycle rider education/training largely parallel those in formal driver instruction. Surprisingly few evaluations have been undertaken to determine the effectiveness of motorcycle rider education/training despite the importance ascribed to these programs in most jurisdictions. As suggested above, the few evaluations that have been undertaken have generally produced discouraging findings and led many to question the value of formal motorcycle rider education and training as an effective loss reduction measure. In spite of these findings, formal training has continued to be popular.

An evaluation of the MRC in South Dakota, using self-reported information on formally trained and non -trained riders, did not find that those who took the course had a lower crash rate (per miles ridden) (Osga 1980). And Mortimer (1984) compared the self-reported collision experience of 213 MRC graduates with that of 303 riders who had not taken the MRC course in Illinois. He found that "when controlling for age and years licensed, those who took the course did not have a lower accident rate (per million miles ridden) than the control group."

More recently, using a much larger sample, Mortimer (1988) replicated the evaluation of the MRC in Illinois. In this study, he compared 914 MRC graduates with 500 riders who had not taken the MRC on a wide range of safety measures. He reports that "compared to the control group, those who had taken the course did not have a lower violation rate, a lower accident rate, a lower total cost of damage to accident involved motorcycles, a significantly lower mean cost of injury treatment per accident, or a lower total cost of injury treatment."

On the positive side, although there was no difference between the groups in the total cost of damage to collision-involved motorcycles, the average cost per collision was somewhat lower among trained riders and the trained riders made more use of protective clothing (this did not, however, produce significant differences between the groups in the mean cost of injury treatment per collision or the total cost of injury treatment).

One of the most ambitious and sophisticated evaluations of rider education and training was initiated in New York State in the early 1980s. In an effort to overcome the methodological difficulties inherent in most previous studies, the National Highway Traffic Safety Administration (NHTSA) sponsored a large-scale project to evaluate the effectiveness of motorcycle rider education, in combination with improved tests of motorcycle knowledge and skills. The study used an experimental design in which nearly 26,000 new motorcycle license applicants were randomly assigned, over a three-year period (1981-1983), to one of four groups who were to take either (1) the existing New York State motorcycle license test consisting of a brief knowledge test and an in-traffic skill test; (2) the Motorcycle Operator Skill Test (MOST 11) developed by NHTSA; (3) three hours of training plus the MOST 11; or (4) 20 hours of training (MSF's Motorcycle Rider Course that involves 8 hours of classroom instruction and 12 hours of off-street on-cycle training) in addition to the MOST II test.

The motorcycle safety community in the United States viewed the project with considerable interest and optimism since it would "yield valuable data on the effectiveness of different kinds of novice training courses" (Hartman 1982). The study was not expected to produce results that confirmed the negative findings of previous, and what many perceived as less rigorous, research. However, the study found no statistically significant differences in crash rates as a function of the different training and licensing systems. Of particular interest, riders exposed to the 3-hour or 20-hour Motorcycle Rider Course had crash rates that did not differ from the rates of those who had not received formal training.

The study has, however, been criticized on operational and administrative grounds that may have seriously biased the findings. For example, (1989) indicated that "the applicant flow was considerably less than had been expected and administrative problems led to long delays between the dates of application, training and license testing." However, these problems would only be important to the validity of the results if they differentially affected the subjects in the various groups. It is not known to what extent this may have been the case. McPherson also noted that "the applicants utilized in this study were found not to be novices, having an average three years riding experience." This criticism is perhaps more substantive since it could be argued that the training or testing programs cannot be expected to improve the performance of, or differentiate among, more experienced riders. Accordingly, the study may not have been an adequate or appropriate test of either MOST II or the MSF Motorcycle Rider Course.

Similar results were reported by McKnight (1987) from an evaluation of the Pennsylvania Motorcycle safety program. That study examined the collision experience of over 3,000 pairs of trained and untrained motorcycle license applicants, who were matched on the basis of age, sex, and prior driving records. He found no significant differences between the mileage adjusted collision rates of trained and untrained riders.

More recently, Billheimer (1996) published the results of an evaluation of the California Motorcyclist Safety Program (CMSP). Initially implemented in July 1987, the CMSP training program became mandatory in January 1988 for all riders under 18 seeking a California motorcycle license. The mandatory age was subsequently raised in January 1991 to include all riders under 21 years of age. In 1994, a further amendment to the legislation permitted students who successfully completed the course to forego the riding skills test required for licensing by the Department of Motor Vehicles.

The evaluation included several quasi-experimental designs: (1) a pre-post design compared motorcycle collisions before implementation of the CMSP and the introduction of mandatory training to collision experience after its implementation; (2) a pre-post design, with a "control", compared trends in motorcycle collisions in California to overall trends in the United States and (3) a matched group design compared the collision involvement of samples of trained and untrained riders from the Los Angeles area over three time periods following training-six months, one year and two years.

Pre-post comparisons of collision trends revealed that the total number of motorcycle collisions and fatal motorcycle collisions in California had decreased since the implementation of CMSP, especially among younger riders, the primary target for the program. For example, crashes among riders under 25 years of age dropped by 84 percent between 1987 (when the program was introduced) and 1995, compared to only a 50 percent decline among riders over the age of 25 during this same period.

Moreover, riders under 18 years of age experienced an 88 percent drop in crashes from 1988 (when training became mandatory for them) to 1995, compared to a 61 percent decline among riders over the age of 18. Similarly, those age 18 to 20 experienced a 78 percent drop in crashes from 1991 (when training became mandatory for them) to 1995, compared to a drop of only 45 percent among riders age 21 and over.

Although these findings suggest a positive effect of the CMSP, many factors other than training could account for the greater reductions in the number of collisions among young riders-e.g., changing demographics, fewer young licensed riders, relatively less riding by young people. Foremost among these would be changes in the relative numbers of young and older licensed riders. And indeed, the number of licensed riders under the age of 25 declined 68 percent between 1987 and 1995 even though the total number of licensed riders was 5.6 percent higher than in the pre-CMSP period. As a consequence, the crash rate (number of collisions per 1,000 licensed riders) for those under 25 declined only 50 percent after the implementation of the CMSP, compared to a drop of 70 percent for older riders (the crash rate for those under 25 declined from 146 to 72; the crash rate for older riders declined from 39 to 12). Thus, the reduction in collisions among riders under 25 the primary target for the CMSP-was actually less than the reduction among all riders.

Billheimer observes, however, that the number of licensed riders is not an accurate measure of the number of active riders. This is so because older riders renew their license without riding and several years pass before the records of inactive riders are purged from DMV files. This means that the estimate of the number of older licensed riders is spuriously high, producing an artificially low crash rate for that group. Importantly, however, an artificially low crash rate for older riders should apply throughout the study period-i.e., there is no reason to believe that difficulties in accurately estimating the number of older licensed riders became worse in the period after the implementation of the CMSP. Accordingly, the above finding that the decline in collision rate was more pronounced among older riders still holds.

Given his concerns with the licensing data, Billheimer used the number of registered motorcycles, which represents a more accurate estimate of the number of active riders than the number of licensed riders, as the denominator of exposure to calculate fatality rates in the periods before and after the 1987 implementation of the CMSP. This analysis showed that the number of fatalities per registered motorcycle dropped 59 percent from 1987 to 1995-1.2 fatalities per thousand registrations in 1986, to 0.50 fatalities per thousand registrations in 1995. Again however, the extent to which the CMSP contributed to this decline is unknown because the reductions could be attributed to factors other than mandatory training-e.g., an overall decline in riding, the introduction of a mandatory helmet law in 1991. Of special note, CMSP trainees accounted for only 18 percent of active riders by 1995-a total of 95,175 trainees and 525,000 registered motorcycles. This means that the vast majority of active riders had not taken training during the period in which the declines in collisions and fatalities occurred. Unfortunately, the author was unable to disaggregate the motorcycle registration data for riders of different age to determine whether declines in fatality rates were more pronounced among young riders, the primary target for the CMSP, than among older riders.

The author also compared trends in motorcycle collisions in California to those in the rest of the United States. These analyses showed that the number and rate of overall motorcycle collisions and fatal motorcycle collisions have also been dropping throughout the United States but that the declines have been greater in California. For example, the author reports that "on the average, motorcycle fatalities per thousand registrations in California have dropped by 33.1 percent since the formation of the CMSP in 1987. Over the same period, the level in the remainder of the United States only dropped 17.3 percent." (p. 39).

Although these comparative analyses as well as the trend analyses described earlier are suggestive, they do not provide convincing evidence that CMSP contributed to the declines. As Billheimer observes:

While the overall decline in accident trends since the start of the CMSP is gratifying, several factors in addition to the CMSP are likely to have contributed to these trends. These factors include a drop in motorcycle riding, the introduction of a mandatory helmet law in January 1991, and the aging population. (p. 4-1)

Given that factors other than the CMSP are uncontrolled in the analyses, the conclusion that the CMSP did contribute to these crash trends is not supported by the analyses. In fact, the finding that crashes per licensed driver decreased more among older than younger riders-the primary target of the program -could be interpreted as demonstrating that the CMSP actually eroded the decline among young riders. That is, if these riders had not taken training, their crash rates would have dropped even further, to a level similar to that of older riders. Of course, this interpretation is also highly speculative and unsubstantiated. Accordingly, the most that can be concluded is that California has experienced a dramatic decline in motorcycle crashes that has outstripped declines in the rest of the United States and that the CMSP may have contributed to this decline.

Billheimer also used a more rigorous evaluation design to examine the impact of the CMSP. He compared the collision involvement of matched pairs of trained and untrained riders in the Los Angeles/Orange County area over periods of six months, one year and two years. The sample of untrained riders was obtained by means of interviews each year from 1989 to 1994 at places frequented by motorcyclists (dealerships, accessory shops, schools, malls etc.) and at high schools and colleges. This sample was obtained at the same time riders were being trained to minimize potential sources of bias arising from different time frames.

The trained and untrained riders were matched in terms of gender, age, years ridden ("within at least six months under two years"), miles ridden last year (defined only as "within 1,000 miles under 5,000 miles" by the author, p. 4-6) and commute history ("is the motorcycle used to commute?"). Over the five-year study period, 1,158 untrained riders were successfully matched with riders who had completed the basic training program for novice motorcyclists, which lasts a minimum of 16 hours. The matching occurred from a pool of 14,832 trained riders and 16,600 untrained riders who had been interviewed.

In the analyses, Billheimer split the matched pairs of basic course graduates and untrained riders into two groups based on their pre-training (or interview) riding experience: 1) riders with 500 miles or less of prior driving experience and 2) those with more than 500 miles of prior riding experience. This yielded a 2x2 design with training (trained/untrained) and prior riding experience (500 miles or less vs. more than 500 miles) as the variables. These groups were compared in terms of crash rates during four time periods-one year prior to training (or interviewing), six months after training, one year after, and two years after. Based on this analysis, the author concluded that:

... it appears that training has an impact on riders, particularly inexperienced riders, for at least six months following training. Accident rates for untrained riders with little or no prior experience were more than double the rates of trained riders during the first six months after training. Beyond this time, riding experience levels the playing field and accident rates of the trained and untrained groups become indistinguishable. (p. 5-8, 5-9).

Thus, the only statistically significant (i.e., p<.10) difference in crash rates between trained and untrained novice riders occurred among those with little or no prior riding experience in the six month period following training (or interviewing)-the six month crash rate for untrained riders was 0.85 collisions per 100,000 miles, compared to only 0.39 collisions per 100,000 miles for trained riders (statistically significant difference at the 0.10 level). In absolute terms, in the six months following training, 615 trained riders had ridden 1,288,000 miles and had five collisions; by comparison, over the six-month period, the 615 matched untrained riders had ridden 820,000 miles and had seven collisions.

These results suggest a short-term positive effect of training but several methodological weaknesses in the research design cast doubt on their reliability. One issue concerns the extent to which the matching of trained and untrained riders successfully controlled for potentially confounding variables. As noted earlier, Billheimer did match trained and untrained riders on key variables such as gender, age, and riding experience to ensure group comparability and to minimize biases associated with self-selection-i.e., riders who take training may be more safety conscious than those who do not and therefore might have lower crash rates than untrained riders even without the course. However, the self-selection bias has not been completely eliminated because some in the study took basic training voluntarily-e.g., this would apply to 18, 19 and 20-year-olds prior to the mandatory training requirement in 1991, as well as those older than 20 for the duration of the project.

Moreover, Billheimer notes that because of the mandatory training requirement he experienced difficulty identifying untrained riders under the age of 18 for matching with trained riders. To overcome this problem, he included in the untrained group riders who ignored the mandatory training and licensing requirement and rode untrained and, therefore, unlicensed. Thus, some of the matched untrained riders differed from the trained riders on a critical factor -i.e., willingness to obey the training and licensing requirements. This difference is critical because unlicensed riders are overrepresented in fatal crashes. In fact, in another section of the report, Billheimer shows that unlicensed riders comprise about 26 percent of the riding population but account for 65 percent of the fatalities in California. Thus, the untrained group may include riders who have a higher likelihood of collision involvement.

Another design concern arises from the choice of dependent measures -collisions per mile ridden. Billheimer selected this to control for exposure, a control that was necessary because riding increased dramatically among those who took the course. Although an increase in riding was also characteristic of untrained riders in the period after their interview, it was much less pronounced. According to Billheimer, "there is no way of knowing whether confidence instilled by training leads to added riding, or whether the decision to ride (or ride more often) leads riders to take a training course" (p. 4-13). Importantly, if this latter possibility applies and the decision to ride leads to riders taking the course, trained riders once again differ from untrained riders on an important factor-i.e., intentions regarding riding.

A somewhat puzzling finding from the matched group analysis is that the trained novice riders had a higher collision rate than the group of untrained novice riders in the year prior to training-7.1 crashes per 100,000 miles compared to 2.2 crashes per 100,000 miles. Although this difference is not statistically significant, it is substantial and suggests that the trained and untrained groups may have differed in terms of their crash involvement at the outset. The fact that the trained group went from a high crash rate to a lower crash rate in the six months following training (interviewing) could be explained in part by regression to the mean.

Finally, Billheimer suggests that beyond the six month period from training (or interviewing), "riding experience levels the playing field and accident rates of the trained and untrained groups become indistinguishable." Although this may be the case, this same interpretation can be used to explain why trained riders have a collision rate that is lower than the untrained group in the six months following training. Indeed, riding experience increased by 2,200 percent among trained riders (from 56,000 miles in the year before training to 1,288,000 miles in the six months after training), compared to an increase of only 259 percent among untrained novice riders (from 228,000 miles in the year before the interview to 820,000 miles in the six months following the interview). Thus, it could be argued equally that the lower crash rate for trained riders was attributable to greater experience and not the training per se.

Given such methodological concerns, it must be concluded that the study provides little in the way of conclusive evidence that the CMSP contributed to the overall crash reductions witnessed in California from 1987 to 1995, or that it reduced the crash involvement of novice riders during the six months following training.

In summary, several studies have failed to provide definitive conclusions about the effectiveness of rider education and training in reducing crashes.

A simple comparison of the number of collisions showed that MTP graduates were less likely than informally trained riders to be involved in collisions. However, when the analyses controlled for differences between the groups in exposure (miles reported ridden) and rider characteristics, such as age and sex, the differences disappeared-the crash picture was the same for those who were formally trained as it was for those who were not.

More recently, Rothe and Cooper (1987), using both telephone interviews and driver records of those interviewed, gathered information on three samples of motorcyclists: a group of formally trained motorcyclists (those who completed the British Columbia Safety Council's Motorcycle Training Program) who had also passed the Motor Vehicle Department's licensing exam; a group of formally trained motorcyclists (also took the B.C. MTP) who initially failed the licensing exam; and a group of informally trained motorcyclists. The results showed that if significant characteristics of the rider were controlled for in the analysis, motorcycle training was not significantly associated with fewer motorcycle crashes. Rothe and Cooper also repeated the analysis separately for 16-19 year olds and for those aged 16-24 to determine whether motorcycle training was more effective for very young novice riders. They indicate that "in neither case was motorcycle training identified as a potentially significant variable relating to motorcycle accidents" (p. 148). In addition, an examination of the frequency of collisions in the first year or first few years of riding was not significantly related to motorcycle training. Thus according to the authors, "this study could not find any evidence to suggest that motorcycle training (as represented by the single course in question) might have had a salutary safety effect as defined by police-reported accidents even during the initial stages of riding experience" (p. 149).

In contrast to these findings, McDavid et al. (1989) reported that the British Columbia Safety Council's 37-hour motorcycle safety program was associated with fewer motorcycle crashes as well as fewer collisions of all kinds. In this study, the authors compared the crash records of two matched groups of formally trained and informally trained motorcyclists. The matching procedure was adopted to overcome methodological weaknesses of previous evaluation studies related to the lack of similarity between persons who seek motorcycle training and those who do not (self-selection). Basically, the groups were matched in terms of sex (all males), age, location of licensing, the month in 1979 in which they had obtained their motorcycle license, possession of a motor vehicle license prior to acquiring their motorcycle license and previous driving record (crash and moving violation records). This matching procedure resulted in a comparison of 139 trained and 139 untrained riders.

Based on their analyses, McDavid et al. (1989) conclude:

... the findings of this study indicate that trained riders are involved in fewer accidents over time and tend to be involved in less severe accidents. The putative effects of training are not large, but they are persistent, and they can be stated with more confidence than findings from previous research (p. 71).

No doubt the confidence attached to their findings is in part due to the rigorous methods they used for controlling potentially confounding variables. However, the one variable that was not controlled was the amount of exposure. It is possible that formally trained riders had fewer collisions than informally trained riders because they traveled less often and less far on motorcycles. Of course, it could then be argued that formal training does have a positive effect-it results in people riding less often, perhaps even encouraging them not to ride during inclement weather or other potentially hazardous conditions. This is an interesting hypothesis that might be worthy of testing. Formal training may sensitize novice riders to the complexity of the task and to their increased vulnerability under certain conditions, such as low visibility.

Unfortunately, the study by McDavid et al. provides no data that bear on this hypothesis. All that can be concluded from their study is that the group of formally trained riders did have fewer overall collisions than those who were not formally trained, but the importance of this finding is diminished in the absence of exposure data. Moreover, if the analysis is restricted to only motorcycle crashes, this difference is not statistically significant.

An evaluation of rider training has also been conducted in Quebec, the only Canadian province that requires training as a prerequisite for obtaining a motorcycle license. Simard (1987) evaluated the effect of a 1985 law that required all prospective motorcycle riders to take a training course, even if they had previously taken a mandatory automobile driver training course. The crash records of two groups of first year novice riders were compared. One group had taken only the driver education course in 1983 or 1984; the other group had taken both the automobile driver course and the motorcycle rider training courses in 1985. An examination of the total number of motorcyclists injured in collisions revealed that the group subject to the new law (both courses) had considerably fewer collisions in their first year riding (125) than did the group that took only automobile driver training (243).

However, the greater licensing demands associated with the new law also resulted in a pronounced decrease in the number of motorcyclists who took the rider course and became licensed in 1985. This decrease could account for the lower number of collisions (i.e., fewer new riders to have collisions in 1985 compared to 1984). This possibility is supported by the finding that standardized comparisons using crash rates (numbers of collisions per motorcyclist) were actually higher for those who took a rider training course in addition to the driver education course (rate of 85 per 1,000 motorcyclist) than for those who took only the automobile driver training course (rate of 73).

Since these comparisons involved collisions that occurred in different years, factors unrelated to the introduction of the new law in 1985 could have influenced crash frequencies as well as the number of new motorcyclists in 1985. To control for this possibility, Simard also examined the pre- and post-law crash record of two other comparison groups shown below:

The first comparison "Control 1"-involved persons who had taken only the motorcycle course either in the pre- or post-law period. Since these motorcyclists had not previously taken automobile driver education, the legal change that occurred in March 1985 did not apply to them. Accordingly, any changes in collision frequencies or in the number of motorcyclists taking the motorcycle rider course from 1984 to 1985 would be accounted for by other factors. The number of these motorcyclists involved in collisions was very similar in 1984 (147) and 1985 (148), and their total numbers declined only slightly in this period (1,486 in 1984 to 1,430 in 1985). Thus the crash rate for persons who were trained and licensed to ride only a motorcycle (unaffected by the new law) increased slightly from 1984 (99) to 1985 (103).

This pattern was considerably different from the dramatic increase in the crash rate experienced by riders most affected by the law, as discussed previously. In contrast the second comparison ("Control 11"), which focused on motorcyclists licensed to drive both types of vehicles but who had taken only a driver education course for an automobile prior to the introduction of the new law (and were presumably unaffected by it), revealed a pattern similar to that of the initial (test) comparison. Fewer motorcyclists were involved in collisions in 1985 than in 1984 (118 compared to 133) but there was an even fewer number of motorcyclists in this group after the law (2,879 in 1985 compared to 3,776 in 1984). This yielded a much higher injury crash rate after the law (rate of 41 in 1985 compared to 35 in 1984).

The author speculates that the dramatic drop in the number of riders in this group may have resulted from the introduction of the new law. Although the law did not come into effect until March 1985, it was actually approved by the National Assembly in June 1984 and publicized extensively throughout that year. Accordingly, some of those licensed to drive an automobile in 1984 who had already taken driver education may have been dissuaded from becoming licensed to ride a motorcycle because they thought the law was already in effect and would require rider training as well.

In summary the Simard study suggests that a law requiring new riders to take a motorcycle training course, even if they had previously taken automobile driver training, is associated with an increase in the crash rate. This occurs even though collision frequencies have dropped because the number of riders has declined more precipitously. Simard (1987) speculates that "a direct effect of the mandatory rider course is to keep off the road those riders who would have, otherwise, shown a lower accident rate ... these riders do not ride very often in the first place. Thus, the low riding frequency rather than the rider's abilities on the road could explain the low accident rate." Perhaps the inconvenience of taking an additional training course dissuaded those who would have ridden infrequently from becoming motorcyclists. If the individuals who took both courses rode frequently, greater exposure to risk explains their higher crash rate

More recently in December 1990, compulsory basic training was introduced as part of an effort to improve the road safety of motorcyclists. The safety impact of this change is currently being evaluated by the Transport Research Laboratory but the results are not yet available.