determination. Although O*NET was envisioned as a replacement for the Dictionary of
Occupational Titles (DOT), the Social Security Administration (SSA) continues to rely on the
DOT when making disability determinations. The chapter first reviews the history of the use
of occupational information in the process of disability determination. It then discusses prior
interagency efforts between the Department of Labor (DOL) and SSA to develop an
occupational information database suited to the process of disability determination and prior
research on the use of O*NET for disability determination. The third section evaluates the
potential use of O*NET data vis à vis the specific types of occupational information required
in the disability determination process, and the final section presents the panel’s conclusions
and recommendations in this area.
OCCUPATIONAL INFORMATION NEEDS OF THE SOCIAL SECURITY
ADMINISTRATION
The Social Security Act (Section 223(d)(2)) establishes that disability determination
requires that an individual’s physical or mental impairment is of such severity that she or he is
not only unable to do his or her previous work but cannot, considering his or her age,
education, and work experience, engage in any other kind of substantial gainful work that
exists in the national economy. “Work which exists in the national economy” is defined as
work found in jobs that exist in significant numbers in either the region where such individual
lives or in several regions in the country. To answer the question of whether or not “work
exists in significant numbers” in the national economy, the Social Security Administration
(SSA) took administrative notice of the Dictionary of Occupational Titles (DOT). That is,
under the assumption that only occupations that existed in significant numbers were reflected
in the DOT, this O*NET predecessor served as a primary tool for determining whether a
Social Security claimant had the capacity to work.
In 1996, SSA requested that the Institute of Medicine, in collaboration with the
National Research Council’s Committee on National Statistics, conduct an independent
review of the statistical design and content of its research plan for redesigning the disability
decision process. The study committee concluded that the DOT replacement (i.e., O*NET),
“will not meet SSA’s needs to define the functional capacity to work without major
reconstruction” (Institute of Medicine, 2002, p. 9). The report continues:
Barring some resolution, SSA will be left with no objective basis upon which to justify
decisions concerning an individual’s capacity to do jobs in the national economy. SSA
might be cast back into the era when it relied extensively on the testimony of
“vocational experts” or their written evaluations [emphasis in the original].
Given that occupational information is critical for use in disability determination, our
panel invited Sylvia E. Karman, a representative of SSA, to make a presentation on this issue.
SSA appears to think that O*NET is not able to fulfill the needs of vocational rehabilitation
experts and others involved in the process of disability determination. In a letter to
administrators of disability determination services, SSA advises disability adjudicators and
reviewers not to use O*NET when making disability decisions (Social Security
Administration, Office of Disability, 1999). Sylvia Karman (2009) indicated that this view is
widely shared, presenting a list of four reports (General Accountability Office, 2002a, 2002b;
Social Security Advisory Board, 2001; Institute of Medicine, 1998) which, she said, either
state that “both SSA and DOL acknowledge that O*NET cannot be used in SSA’s process”
and/or “recommend that SSA investigate other alternatives.”
THE FUTURE OF OCCUPATIONAL INFORMATION FOR DISABILITY DETERMINATION
Having ruled out the use of O*NET for disability determination purposes, SSA has
begun taking steps to develop its own occupational information system. In December 2008,
the commissioner of social security established the Occupational Information Development
Advisory Panel. The advisory panel was charged with providing independent advice and
recommendations on plans and activities to replace the DOT currently used in the SSA
disability determination process (Astrue, 2008). The panel’s report, issued in September,
2009, recommends the creation of a new “Social Security Administration Occupational
Information System” for use in disability determination (Social Security Administration
Occupational Information Advisory Panel, 2009). The panel recommends developing an
initial, empirically derived work taxonomy; research to examine various job classification
methods; development of internal and external research capacity at SSA; basic and applied
research on work-side and person-side job attributes and demands; development of scales and
measures; and communication with users, the public, and the scientific community.
SSA has concluded that, in its current form, O*NET does not fulfill the needs of the
disability determination process. At the same time, the Department of Labor (DOL) has
concluded that O*NET fulfills its needs for occupational information; other chapters of this
report show that O*NET meets many other occupational information needs. However,
disability determination was an important use of the DOT and because O*NET was created to
replace the DOT, it seems fair to conclude that O*NET has failed to replace the DOT in this
particular usage.
The panel understands that, pending SSA’s response to the advisory panel
recommendations, the jury is still out on the topic of whether and to what extent O*NET
should be changed or expanded to meet SSA’s needs. However, given public demand for
budgetary restraint and efficient government, which acquire additional importance in times of
economic recession and slow economic growth, duplication in government functions should
be prevented. Therefore, the development of parallel, possibly redundant, occupational
information systems, one for general purposes termed O*NET and the other tailored to the
needs of SSA, is of concern to taxpayers. In addition, dual data collection processes would
seem unnecessarily expensive.
The panel is not advocating the adoption of O*NET by SSA or the development of a
hybrid O*NET-Disability system in the disability determination process. However, we
conclude that a considerably modified and expanded O*NET may be capable of informing the
disability determination process. There are also some potential economies of scale to be
derived from the development of a single occupational information system to be used by both
agencies, which may allow cost-sharing of resources in such functions as data collection and
system maintenance.
An occupational information system that facilitates the process through which
individuals with disabilities obtain gainful employment would help relieve the financial
pressure on the SSA system and also contribute to the mental health of those who become
productive members of society.
Not all stakeholders share the opinion that O*NET cannot be amended to meet the
needs of those involved in disability determination. In fact, the Committee to Review the
Social Security Administration’s Disability Decision Process called for interagency
collaboration (Institute of Medicine, 1998). Its 1998 report encouraged SSA to explore some
interagency agreement “to initiate a version of O*NET that would collect information on
minimum as well as average job requirements to better serve SSA’s needs to assess ability to
engage in substantial gainful activity” (p. 24).
We found evidence suggesting that these calls for collaboration between DOL and
SSA were heeded. In 2000, vocational rehabilitation professionals initiated discussions with
DOL and SSA which led to the creation of the Inter-Organizational O*NET Taskforce with
representatives of 16 associations of physicians, psychologists, therapists, counselors, insurers
and educators (Cannelongo, 2009). The group met for 4 years and proposed development of a
modified version of O*NET called O*NET –D (for Disability) that would incorporate
occupational information gathered in the field by disability professionals trained in job
analysis, using standardized questionnaires. A pilot study of the approach funded by DOL
yielded promising results. Although SSA staff initially agreed with the plan and submitted it
to the SSA administrator, the agency later withdrew its support.
At around the same time, SSA commissioned the American Institutes for Research
(AIR) to examine the suitability of O*NET for the disability determination process
(Gustafson and Rose, 2003). Based on an analysis of the initial O*NET database (the
“occupational analyst” database), the AIR research team found that reliability, definitional,
and anchoring issues could lead to problems if O*NET data were used for disability
determination. At the same time, however, the authors identified specific steps for addressing
these problems. For example, they suggested that a disability decision maker could use
Copyright © National Academy of Sciences. All rights reserved.
O*NET task lists and other descriptive information to help determine the activities of
claimants’ current jobs and described an approach to using selected O*NET descriptors that
would adjust for the positively skewed distributions of ratings of these descriptors. Gustafson
and Rose (2003, p. 15) concluded that “SSA could implement into the [disability
determination process] a version of O*NET that is legally defensible and acceptable to
decision-makers and claimants alike.”
Another piece of evidence, suggesting the continued possibility of collaboration
between DOL and SSA, is the testimony provided by former O*NET director, James Woods,
to the SSA advisory panel on January 13, 2009 (Woods, 2009). In his address, he regretted
that earlier efforts to accommodate the SSA needs into O*NET did not bear fruit; however, he
remained hopeful that O*NET
may provide a basis to help SSA focus on a specific set of data needs and to organize
data within the O*NET framework—for SSA’s specific needs. O*NET, or at least the
lessons learned in developing the O*NET system, may provide a starting point rather
than SSA starting from scratch.
In spite of such past interagency efforts, communication and collaboration between
DOL and SSA regarding a common occupational database now appears quite limited. An
inspection of their most recent communications suggests that both agencies have reached the
implicit conclusion that DOL will not modify O*NET to accommodate disability
determination users, and that SSA will build an entirely different occupational information
system for its purposes. The fact that SSA’s newly formed advisory panel does not include a
DOL liaison suggests that the development of an SSA-sponsored system may proceed
relatively independent of O*NET.
It is important to consider why SSA and other stakeholders deem O*NET inadequate
for disability determination purposes. In the next sections, we present the primary issues that
preclude the use of O*NET in the disability determination process in the eyes of stakeholders,
the available evidence, and our conclusions regarding the type and the extent of the O*NET
modifications called for by each of the issues.
conditions, and impairments to the functional limitations caused by these factors (Institute of
Medicine, 2002, p. 4). A key element implicit in the contemporary view of the disability
determination process is the need for an increased understanding of the physical and social
factors in the work environment that may impact the Social Security claimant’s functional
capacity.
There are a number of mental, physical, and psychosensory disabilities that affect the
capacity to work. In the context of disability determination, the role of occupational analysis
is to determine the important job behaviors and the type and level of ability that is required to
perform them. The Americans with Disabilities Act (ADA) states that job requirements
should be linked to essential (not marginal or peripheral) job functions, and that alternate
ways of performing job functions that may require reasonable accommodations should be
considered.
In spite of their importance for job performance, physical abilities have not received as
much attention as cognitive abilities have in the occupational analysis literature (Guion and
Highhouse, 2006). Much of what is known about physical abilities appears to have its origins
in the work of Fleishman and his associates (Fleishman and Reilly, 1992). An overall
summary of this work (Hogan, 1991) suggests that physical abilities can be subsumed into
three general fitness factors: (1) muscular strength, or the ability to apply or resist force by
contracting muscles; (2) cardiovascular endurance, or aerobic capacity; and (3) coordination,
or quality of movement.
Physical abilities are not the only type of ability impacted by disability. Cognitive,
psychomotor, and sensory/perceptual abilities can also be impaired. Fleishman’s approach to
abilities is particularly important because his measures of ability requirements—including
cognitive, physical, psychomotor, and sensory abilities—were adopted in O*NET (Fleishman
and Quaintance, 1984). For example, the 52 ability scales used in O*NET were drawn almost
verbatim from Fleishman and Reilly (1992). Even though all of these 52 abilities conceivably
have implications for disability determination, the panel heard from stakeholders who
questioned their utility in the disability determination process.
in the O*NET approach to the measurement of physical requirements (Karman, 2009). These
shortcomings, along with a critical examination of their rationale, are reviewed below.
occupational requirements along each physical ability dimension. As incorporated into the
O*NET content model, these scales provide a definition of the ability as well as examples of
tasks or job behaviors situated at various points on the scale.
This approach is most useful for a construct or a criterion-related approach to the
validation of measures of physical requirements that are used for selection purposes (Hogan,
1991). For example, a number of tests are available to assess each of these abilities that can
be used to identify capable individuals and can be validated in criterion-related studies
(Fleishman and Reilly, 1992). However, from the point of view of disability determination,
these abilities represent nonspecific, psychologically worded, or unobservable constructs that
cannot be easily tied to specific disabilities or specific groups of muscles, such as those
involved in lifting, kneeling, etc.
Consider, for example, the O*NET ability, Static Strength, which is defined as “the
ability to use muscle force in order to lift, push, pull, or carry objects. It is the maximum force
that one can exert for a brief period of time using the hand, arm, back, shoulder, or leg”
(National Center for O*NET Development, no date; see Figures 4-1 and 4-2). Clearly, this
physical ability construct cuts across very different muscle groups and different body limbs.
In contrast to this type of definition of a physical construct, the SSA disability determination
process relies on the notion of Residual Functional Capacity (RFC), which measures the
ability to perform specific physical tasks, such as lifting 20 pounds with hands and arms. In
the RFC assessment of the claimant, the focus is on specific and observable functions or
behaviors related to lifting, standing, sitting, and pushing, as well as similarly verifiable
(medically and otherwise) postural limitations regarding balancing, crouching, and crawling
(Form SSA-4734-BK, 08-2008).
There is a series of O*NET work context descriptors related to how much time the
occupation requires sitting, standing, climbing, walking or running, and keeping one’s balance
(items 34 through 39 in the work context O*NET questionnaire—National Center for O*NET
Development, no date) that have conceptually parallel items in the RFC assessment, even
though the anchors placed at the various points of these scales are quite different in the two
approaches. For example, SSA uses specific time ranges (e.g., “about 6 hours in an 8-hour
workday”), whereas O*NET uses relative scales (e.g., “about half the time”). The panel also
observes that some O*NET descriptors, such as item 38 in the work context questionnaire,
collapses occupational requirements across posturing, such as kneeling-crouching-stoopingcrawling,
whereas the RFC assessment breaks down each one of these postural limitations.
panel identified smaller differences between the O*NET and SSA approaches with regard to
sensory and perceptual abilities. Note for example the almost perfect equivalence between the
O*NET descriptors of near vision, far vision, visual color discrimination, and depth
perception and the RFC assessment (descriptors of near acuity, far acuity, color vision, and
depth perception). The scales and definition of scale points, however, are still quite different
between the two scales.
environmental factors, such as extreme heat, extreme cold, wetness, humidity, noise,
vibration, and others, using scales ranging from “unlimited” to “avoid all exposure.” Although
the panel did not find perfectly equivalent descriptors in O*NET, the O*NET Work Context
domain involves a number of conceptually similar items related to exposure to either very hot
or very cold temperatures (item 23), such contaminants as gases and dust (item 25), and
whole-body vibration (item 27). The anchors in these O*NET scales range from “never” to
“every day,” and the anchors in the RFC range from “unlimited” to “avoid all exposure.”
in which behavioral anchors representing differing degrees of a construct are placed along the
scale continuum (see Chapter 4). The various degrees of the continuum represented by these
scales are illustrated through “anchors” situated at the corresponding scale points. These
anchors are short statements describing tasks purportedly representing the level of the
construct: “light a candle” is placed next to the scale point 2 in the Ability scale termed Arm-
Hand Steadiness,” “thread a needle” is placed next to the scale point 4 in the same scale, and
“cut facets on a diamond” is placed next to the point 6 in the scale.
Clearly, there are variations in the degree of arm-hand steadiness lying between any of
these pairs of proximal anchors. If the O*NET occupational unit score on arm-hand
steadiness is 3 (requiring a level of arm-hand steadiness between 2–light a candle and 4–
thread a needle), it seems nearly impossible to determine the type of task that a potential
claimant should be able to complete to be deemed capable of performing work in this
occupation unit. For example, two Social Security disability claimants, both of them capable
of lighting a candle and unable to thread a needle, may or may not be able to perform an
occupation with a score of 3 on this ability. This could be true because the two claimants
have different limitations in their degree of arm-hand steadiness, despite the fact that both of
them are unable to thread a needle.
Still another issue related to the behavioral anchors employed in the O*NET physical
ability scales is whether ability requirements are scaled at the level of the ability required by
the most demanding task or the typical (average) task. For example, a Social Security
claimant may be capable of performing the occupation because she or he has enough armhand
steadiness to thread a needle, so long as threading a needle represents the maximal level
of ability that would be required on the job; however, that same claimant may be unable to
perform all the work in the occupation if threading a needle represents the level required by
the typical, average, or everyday task.
viewed the National Center for O*NET Development’s use of trained occupational analysts to
judge ability requirements, using solely a written description of the occupation, as an obstacle
to relying on this information for disability determination purposes (Karman, 2009). This
concern may be accentuated when physical requirements are being determined, because many
of them lend themselves to job observation. The process of disability determination can be
quite litigious, and those in charge of making the determination prefer to minimize the risk of
legal challenges by relying on occupational information gathered directly by trained
vocational or job analysts. As noted in chapter 1, trained occupational analysts gathered
information directly from job incumbents for inclusion in the DOT.
small for disability determination purposes, because each occupation involves multiple,
heterogeneous jobs that may have different physical and education requirements. If her
assertion is correct that there is a wide range of physical and education requirements of jobs
within the same O*NET occupation, then SSA would find it nearly impossible to determine
whether or not a given disability precludes a claimant from performing a specific job in the
occupation.
The process used to create and write descriptions for the 1,122 original O*NET
occupations, referred to as “occupational units” was complex, according to a report of the
National Center for O*NET Development (1998). It entailed the use of the occupational
classification system adopted by the Bureau of Labor Statistics to administer the Occupational
Employment Survey, the development of crosswalks to DOT title codes, cluster analyses of
DOT data, analysis and aggregation of DOT task statements, and multiple reviews by subject
matter experts. As described in this report, even though DOT titles and task data contributed
to the original formation of these occupational units, the occupational units were not the
outcome of a simple clustering of DOT titles, nor were they meant to represent simple
aggregations of DOT titles. It is not unreasonable to conclude that, at the end of this process,
each occupational unit had its own identity independent of—though partially informed by—
DOT titles and task content.
Since that time, the O*NET occupational classification system has been revised
several times, so that the current O*NET-SOC 2009 occupations may exhibit less within occupation
variability than did the occupational units created by the research team in the late
1990s. Nevertheless, the reduction from over 12,000 occupational titles in the DOT to the
current 1,102 occupations in O*NET-SOC 2009 will inevitably be accompanied by some
increase in within-occupation variability.
Karman presented a chart indicating wide variability in education and physical
requirements across 553 DOT titles that she said were clustered into a single O*NET
occupational unit (51-9198, Helpers-Production Workers) (Karman, 2009). Harvey (2009)
presented similar data and drew similar implications regarding what in his view constituted
excessive aggregation in the original O*NET occupational units for the purpose of disability
determination. However, as noted above, the occupational units were not intended to be
merely aggregated DOT titles. The question of the extent of variability in current O*NET
occupations deserves further study.
on assessment of the residual functional capacity of a claimant, focusing on physical functions
or behaviors and postural limitations. Matching the results of the RFC to the descriptors of
physical ability and occupational context employed in O*NET is inherently difficult.
Nevertheless, there are commonalities in the descriptors used in these two systems, even
though substantial differences remain in the level of detail, specificity, and types of scales
employed to measure them. The evidence indicates that occupational descriptors involving
exposure to unusual environmental demands, such as heat or cold, exist in both O*NET and
the RFC assessment used by SSA. However, there is no clear, one-to-one correspondence
between the two types of environmental descriptors, because some environmental factors are
defined and grouped quite differently in the two models. Taken together, the differences and
similarities suggest that continued collaboration between DOL and SSA is in the interest of
efficient use of government resources.
Recommendation: SSA and DOL should create an interagency task force to
study the viability of potential modifications of O*NET to accommodate the
needs of SSA with regard to disability determination. Before implementing these
or similar modifications, however, we recommend that the task force conduct (1)
an in-depth needs analysis of the occupational information required by the
current disability determination process and (2) an interagency cost-benefit and
cost-sharing analysis of the additional resources that would be needed to make
O*NET suitable to the disability determination process.
The reduction from over 12,000 occupational titles in the DOT to the current 1,102
occupations in O*NET-SOC 2009 has been accompanied by some increase in withinoccupation
variability in the physical and mental requirements of the work included in these
two different types of occupational categories. Because the extent of this variability has
important implications for the usefulness of O*NET in disability determination, it should be
studied.
Recommendation: As part of the research on the occupational classification
system recommended in Chapter 3, the Department of Labor should commission
research to determine whether and to what extent O*NET occupations represent
excessively heterogeneous clusters of jobs (in terms of their physical and
education requirements) for the purpose of disability determination. This
research should include gathering evidence from firsthand observations
regarding physical requirements and verifiable survey responses from wellinformed
sources capable of assessing cognitive requirements.
Development Advisory Panel. Federal Register (73), 247, p. 78864
Available:
http://www.socialsecurity.gov/oidap/Documents/federal_register/federal_register.htm
[accessed November, 2009].
Cannelongo, J.M. (2009). Statement to the Social Security Advisory Board. Available:
http://www7.nationalacademies.org/cfe/Social%20Security%20Advisory%20Board%
20Presentation.pdf. [accessed September, 2009].
Fleishman, E.A., and Quaintance, M.K. (1984). Taxonomies of human performance. Orlando,
FL: Academic Press.
Fleishman, E.A., and Reilly, M.E. (1992). Handbook of human abilities. Palo Alto, CA:
Consulting Psychologists Press.
General Accountability Office. (2002a, July 11). SSA Disability Programs: Fully Updating
Disability Criteria Has Implications for Program Design. Available:
http://www.gao.gov/new.items/d02919t.pdf [accessed November, 2009].
General Accountability Office. (2002b, August). SSA and VA Disability Programs
Reexamination of Disability Criteria Needed to Help Ensure Program Integrity.
Available: http://www.gao.gov/new.items/d02597.pdf [accessed November, 2009].
Guion, R.M., and Highhouse, S. (2006). Essentials of personnel assessment and selection.
Mahwah, NJ: Lawrence Erlbaum.
Gustafson, S.B., and Rose, A. (2003). Investigating O*NET’s suitability for the Social
Security Administration’s disability determination process. Journal of Forensic
Vocational Analysis, 6, 3-15.
Harvey, R.J. (2009). The O*NET: Do too abstract titles + unverifiable holistic ratings +
questionable raters + low agreement + inadequate sampling + aggregation bias = (a)
validity, (b) reliability, (c) utility, or (d) none of the above? Paper prepared for the
Panel to Review the Occupational Information Network (O*NET). Available:
http://www7.nationalacademies.org/cfe/O_NET_RJHArvey_Paper1.pdf [accessed
July, 2009].
Hogan, J. (1991). Structure of physical performance in occupational tasks. Journal of Applied
Psychology, 76, 495-507.
Copyright © National Academy of Sciences. All rights reserved.
A Database for a Changing Economy: Review of the Occupational Information Network (O*NET)
http://www.nap.edu/catalog/12814.html
Prepublication Copy
Uncorrected Proofs
8-11
Institute of Medicine. (2002). The dynamics of disability: Measuring and monitoring
disability. Washington, DC: National Academies Press.
Institute of Medicine. (1998). The Social Security Administration’s decision process: A
framework for research, 2nd Interim Report. Committee to Review the Social Security
Administration's Disability Decision Process Research, G.S. Wunderlich and D.P.
Rice (Eds.). Washington, DC: National Academy Press.
Karman, S. (2009). No title. Presentation to the Panel to Review the Occupational
Information Network (O*NET). Available:
http://www7.nationalacademies.org/cfe/Karman%20Power%20point.pdf.
[accessed June, 2009].
National Center for O*NET Development. (no date). Questionnaires. Available:
http://www.onetcenter.org/questionnaires.html. [accessed July, 2009].
National Center for O*NET Development. (1998). Appendix D: The Development of the
Occupational Information Network (O*NET) Analyst Database. Raleigh, NC:
Author. Available: http://www.onetcenter.org/reports/appendix_d.html [accessed
November, 2009].
Occupational Information Advisory Panel. (2009). Content model and classification
recommendations for the Social Security Administration occupational information
system. Report to the Commissioner of Social Security. Available:
http://www.socialsecurity.gov/oidap/Documents/Occupational%20Information%20De
velopment%20Advisory%20Panel.pdf [accessed November, 2009].
Social Security Administration, Office of Disability. (1999, June 30). Policy instruction
DDSAL-507. Washington, DC: Author.
Social Security Administration. (2008). Physical residual functional capacity assessment.
Form SSA-4734-BK (08-2008). Washington, DC: Author.
Social Security Advisory Board. (2001, January). Charting the Future of Social Security’s
Disability Programs: The Need for Fundamental Change. Available:
http://www.ssab.gov/Publications/Disability/disabilitywhitepap.pdf [accessed
November, 2009].
Social Security Administration Occupational Information Advisory Panel. (2009). Content
Model and Classification Recommendations for the Social Security Administration
Occupational Information System. Available:
http://www.socialsecurity.gov/oidap/Documents/Occupational%20Information%20De
velopment%20Advisory%20Panel.pdf [accessed November, 2009].
Woods, J. (2009, January 13). Testimony to the Social Security Administration Occupational
Information Development Advisory Panel. Available: Unpublished document
provided to the Panel to Review the Occupational Information Network.
Copyright © National Academy of Sciences. Permission is granted for this material to be
shared for noncommercial, educational purposes, provided that this notice appears on the
reproduced materials, the Web address of the online, full authoritative version is retained,
and copies are not altered. To disseminate otherwise or to republish requires written
permission from the National Academies Press.