The profession of career and technical education (CTE) at the secondary level has become increasingly complicated. The need or requirement of CTE agencies (local, state, and university) to measure progress, synthesize responsibilities, and integrate disciplines has never been more prominent, nor emphasized. State and local CTE agencies are required in the form of new legislation, regulations, and procedures to take on this administrative burden with, in many instances, dwindling budgets. This has led to the uncomfortable, but all too common, practice of decreasing or eliminating support for basic operational criteria such as the ones defined and validated in this piece of research. For example, the CTE state-level administration in one of the researcher's state has eliminated certain teacher education grants in order to help fund accountability measures put forth by Perkins IV. These teacher education grants were generally used to help improve and maintain CTE teacher education programs and/or recruit quality CTE teacher candidates. As Dr. Richard Lynch, CTE teacher educator, once stated, "...without good teachers, you cannot have good instruction nor good programs. The studies on the education and re-education of good teachers...indicate that effective teacher education takes time and it takes money" ( NCRVE, 1991 , p. 23). Yet, as state and local budgets decrease, some programs and/or initiatives must take the brunt of fiscal belt tightening. Of course, these programs and/or initiatives are generally not the ones that new federal and state guidelines or regulations are requiring state and/or local CTE agencies to increase or emphasize. What effect does this have on the general operation of secondary-level CTE? While no new legislation or accountability measure can account for all of the factors, what does such a change do to the objectives, tasks, programs, and/or initiatives not included in the new legislation or measure?
First and foremost, CTE must identify and agree upon the criteria that make up the general operation of secondary-level CTE. The researchers attempted to accomplish this task with the development and validation of the Operational Infrastructure of Secondary-level CTE (OISCTE). The OISCTE is defined as the basic, underlying framework of criteria that state leaders in CTE (state and local administrators and university-level teacher educators) must develop, grow, and/or maintain in order to effectively and efficiently operate a statewide CTE program. Therefore, the purpose of this study was to define and validate the set of criteria that would embody the OISCTE and thus answer the question: What is the OISCTE?
In order to answer the research question, the researchers conducted a literature review to aid in the development of the OISCTE's criteria statements. The literature review involved synthesizing national and state documents related to the regulation, administration, and evaluation of CTE programs at all levels. Additionally, it required contacting the current Virginia CTE state director for insight into their role within the operation of secondary-level CTE.
Once the criteria statements were established, the researchers validated them using a survey that was completed by local and state CTE administrators as well as university-level CTE teacher educators from Virginia. The survey utilized a Likert scale ( r =5) to identify their level of agreement as to the inclusion of the criteria statements derived from the literature review. Additionally, the participants were given an opportunity to add comments concerning their ratings or the criteria statements. It is important to note that this survey was included in Round 1 of a three-round Policy Delphi. The purpose of the Policy Delphi was to identify the impact of a federal formula change that occurred with the passage of the 1990 Perkins Act on the OISCTE within Virginia ( Manley, 2011 ). Before the subsequent rounds could proceed, the participants needed to validate the set of criteria that embodied the OISCTE so they could begin the process of identifying how the federal funding change affected each OISCTE criterion statement.
Upon participant completion of the survey, the researchers calculated the frequency, median, mean, standard deviation, and interquartile range (IQR) for each criteria statement. Additionally, the researchers analyzed participants' comments for relevancy. For a criterion statement to be considered validated, the statement's mean was required to be at least 4.0 on a five-point scale with 5.0 denoting strong agreement as to the inclusion of the criterion statement.
Since the study that included this survey sought to identify the impact of a policy change that occurred in the early 1990s, the participants must have been working as either a local, state, or university-level CTE administrator or teacher educator within Virginia from 1989 to 1993. The researchers used multiple resources (state library documents, online phone and email databases, university course catalogs, AVA and ACTE rosters, and interviews) to identify and locate potential participants.
The OISCTE is grounded in criteria related to the development of competency-based education (CBE). More specifically, the development of OISCTE followed pertinent specification criteria for describing and assessing competency-based programs as noted by Burke, Hansen, Houston, and Johnson (as cited in Tuxworth, 1989 ). First, Burke et al. noted that competency statements must be "...based on an analysis of the professional role(s) and/or a theoretical formulation of professional responsibilities" (p.12). This criterion was the basis for reviewing the literature related to the statewide operation of secondary-level CTE. In addition, the Burke et al. note that competency statements must "...facilitate criterion referenced assessment" (p.12). This criterion helped researchers determine the construction the OISCTE criteria statements. Lastly, the authors noted that the competency statements must be "...treated as tentative predictors to professional effectiveness, and are subjected to continual validation procedures" (p.12). This criterion helped the researchers understand that while current policy and practice may limit a certain OISCTE criterion from achieving maximum effectiveness, it should not limit the criterions’ inclusion if inclusion equated to overall professional effectiveness as it relates to the operation of secondary-level CTE. Additionally, this criterion was the impetus for validating the OISCTE using a group of CTE leaders.
Review of Related Literature
The regulation, administration, and evaluation of state CTE programs vary widely between states. For example, most secondary-level CTE programs are administered within offices of their Department of Education, but seven states operate their CTE programs through "...an agency other than the State Department of Education..."( NASDCTE, 2010 , p. 2). Additionally, six states politically appoint their state director of CTE rather than engage in the hiring of career employees (2010). While the researchers do believe these variances in administration have an effect on the operation of secondary-level CTE, the criteria they sought to define transcended these ever-changing administrative structures. In addition, much of the literature concerning this topic focused on one level within the administrative structure. For example, research has been done related to the objectives and tasks of state administrators ( Ruff, 1981 ; Stevenson, 1982 ) as well as local administrators ( Magisos & Schroeder, 1974 ; Tennessee State Board of Education, 1975 ; Clark, Farmer, & Welch, 2010 ). Additionally, research has been done related to the objectives and tasks of teacher educators ( Stevenson, 1963 ; Norton, 1977 ; Norton & Harrington, 1987 ). Lastly, another portion of the literature is made up of strategic plans for the operation of CTE. These time-sensitive strategic plans, more often known as state plans, have vision areas that focus on statewide objectives related to the operation of CTE. While promising, these vision areas usually focus on ancillary objectives related to the Perkins Act such as the integration of academic curriculum or assessing CTE completers' content knowledge with standardized assessments.
While all of the aforementioned pieces of literature are significant, the researchers could find no published research that defined and validated a broad set of objectives, tasks, and/or criteria statements that encompassed all levels of CTE within a state. As a result, the researchers began the process of defining the OISCTE.
Defining the OISCTE
The criteria that make up the OISCTE were derived from a synthesis of four national and four state documents (Virginia) related to the regulation, administration, and evaluation of CTE. The result was the development of 13 operational criteria statements that were identified as embodying the OISCTE. Described in the following sections are the national and state resources used to define the OISCTE.
At the national level, four documents assisted the researchers in conceptualizing the "big picture" of operating a statewide CTE program: (a) Carl D. Perkins Career and Technical Education Act of 2006, (b) National Center for CTE’s vocational administrators' competency studies (1977 and 1987), and (c) the text, Administration of Vocational Education ( Wenrich, Wenrich, & Galloway, 1988 ).
The current federal legislation governing CTE programs, the 2006 Perkins Act, provided documentation of the intended purpose of the Act and outlined the required and permissible uses of the Perkins funds. This document helped to define what state-level processes were considered of federal importance, thus important to include in the OISCTE.
Further, at the national level, in response to the need to identify competencies required of vocational administrators, the Occupational and Adult Education Branch of the U.S. Office of Education, under provisions of part C—Research on the Vocational Education Amendments of 1968, funded a project entitled "Development of Competency Based Instructional Materials for Local Administrators of Vocational Education" during the period of 1975-77. This project had two major objectives: (a) to conduct research to identify and nationally verify the competencies considered important to local administrators of vocational education and (b) to develop and field test a series of prototypic competency-based instructional packages and a user’s guide ( Norton, 1977 ).
A study to update the 1977 study was completed in 1987. The purpose of the study was to identify and nationally verify the competencies needed by administrators of secondary and postsecondary vocational and technical education programs and compared the results to the 1977 study. The survey instrument used in the 1987 study was a three-day Developing a Curriculum (DACUM) workshop. The DACUM committee identified 210 tasks that were clustered into 12 duty areas. A validation questionnaire was sent to 188 administrators, 128 of whom returned usable questionnaires. Of the 210 task statements included on the questionnaire, 201 were verified as being important ( Norton & Harrington, 1987 ). The ultimate outcome of these two studies was to rigorously identify the important competencies needed by CTE administrators and, in the context of this study, serves as a rationale for the criteria identified within the OISCTE.
Lastly, the text, Administration of Vocational Education , by Wenrich, Wenrich, and Galloway (1988) served as another source for the development of the 13 criteria statements that embody the OISCTE. This text addressed, among other things, "...administrative structures for delivering vocational education and key aspects of the major functions that any educational administrator must perform" (1988, p. 1).
At the state level, four documents provided the researchers with a local-level perspective. The four documents were the: (a) Regulations Governing CTE in Virginia ( Virginia Board of Education, 2001 ), (b) 1988 Vocational Education Evaluation of Virginia (VEEVA), (c) Arizona's local program evaluation model, and (d) correspondence from the state director of Virginia, Elizabeth Russell.
The Virginia Regulations Governing CTE developed by the Virginia Board of Education in 2001 ( Virginia Board of Education, 2001 ), as well as e-mail communication with the director of the Office of Career and Technical Education Services from the Department of Education assisted the researchers in identifying the CTE administrative structure within their home state (personal communication, December 19, 2008).
Additionally, a review of the Vocational Education Evaluation of Virginia, which was the local program evaluation used prior to the enactment of the 1990 Perkins Act, provided an overall perception of the what the state, at that time, thought were important areas to evaluate when examining the local secondary CTE agencies.
Lastly, a draft copy of Arizona's Career and Technical Education Program Review Guide (Assessment and Action Plan) which was prepared by Elliot and Molina (2007) in collaboration with the University of Arizona's Department of Career and Technical Education, the Division of CTE at Arizona's Department of Education, and Arizona's Agriculture Teacher's Association Curriculum Committee, was examined for a more current perspective of secondary CTE local program evaluation. Although specifically designed to "provide information to local personnel for redirection of the program to meet the present and future needs of agriculture students" and to "serve as a model for reviewing all existing [secondary agriculture] programs and guide a new or expanding programs" ( Elliott & Molina, 2007 , p. 2), the program components assessed in this program review guide embody more than secondary agriculture programs.
The 13 Criteria of the Operational Infrastructure of CTE in Virginia
Table 1 lists each of the criteria statements that embody the OISCTE as well as the identification of previously mentioned national and state resources used in the synthesis of each criterion statement.
The Operational Infrastructure of Secondary-level CTE — Criteria and Related Resources
|1||Provide state coordination, leadership, and technical assistance for local systems to identify, interpret, and comply with local program standards developed and accepted by the CTE profession as well as federal and state regulations.||X||X||X||X||X|
|2||Provide relevant professional and leadership development to teachers, administrators, and teacher educators.||X||X||X||X||X||X|
|3||Develop new programs and curricula to respond to current and projected occupational needs.||X||X||X||X||X||X|
|4||Formulate new and improve existing operational policies (standards) at the local level.||X||X||X||X||X||X||X|
|5||Develop, improve, and maintain quality CTE teacher education programs at the bachelors, master’s, and doctoral level.||X|
|6||Conduct innovative research projects to assist in meeting unmet CTE needs||X||X||X|
|7||Educate and recruit quality CTE teachers and leaders||X||X||X||X||X|
|8||Operate and maintain local CTE facilities and equipment||X||X||X||X||X||X|
|9||Develop and execute external and internal procedures for program planning, development and evaluation procedures for local CTE programs.||X||X||X||X||X||X|
|10||Develop, improve, and maintain state and local community relations (e.g., advisory committee, CTSO, other workforce agencies).||X||X||X||X||X||X|
|11||Develop, improve, and maintain statewide professional associations and CTE teachers and administrators at all levels (local, state, and universities).||X||X||X||X||X|
|12||Develop, grow and maintain student services (e.g.. Student recruitment and admissions, placement, and guidance services)||X||X||X||X||X||X|
|13||Develop, improve, and maintain fiscal management of local CTE programs.||X||X||X||X||X||X||X|
a The 2006 Perkins Act ( U.S. Congress, 2006 )
b Administrator Competency Studies ( Norton & Harrington, 1987 ; Norton, 1977 )
c Administration of Vocational Education ( Wenrich, Wenrich, & Galloway, 1988 )
d Regulations Governing CTE in Virginia ( Virginia Board of Education, 2001 )
e Vocational Education Evaluation of Virginia ( Virginia DOE, 1989 )
f Arizona Local Program Evaluation Model ( Elliot & Molina, 2007 )
g Email correspondence with Elizabeth Russell Director of CTE in Virginia (E. Russell, personal communication, November 11, 2008)
Out of 81 potential participants contacted, 54 initially agreed to participate in the Policy Delphi study. Forty-two of the participants completed the Round 1 survey that housed, among other information, the validation of the OISCTE. This resulted in a 78% Round 1 participation rate. Of the 42 that validated the OISCTE, 21 were local CTE administrators, 12 were state-level administrators, and 9 were university-level teacher educators and researchers. The validation of the OISCTE was strengthened by the fact that the Delphi participants (n=42) that validated the OISCTE had an average of 30.4 years ( S =8.4) of cumulative work experience in CTE within Virginia.
The 42 participants validated each of the 13 criterion statements using a five-point Likert scale of agreement. As evident in Table 2, all 13 criteria statements had a mean above 4.0. In addition, a post-analysis of the data showed an interquartile range (IQR) of 1 or less. The IQR is one of many ways in which consensus concerning a rating can be determined ( Rayens & Hahn, 2000 ). An IQR of 1 on a five-point scale denotes no more than a 20% difference between the 25 th and 75 th percentiles. While there are not set values for determining consensus, an "IQR that is 20% of the rating scale appears to be a conservative but acceptable criterion for determining consensus" ( Plinske & Packard, 2010 , p. 296).
OISCTE Validation Results
|1||Provide state coordination, leadership, and technical assistance for local systems to identify, interpret, and comply with local program standards developed and accepted by the CTE profession as well as federal and state regulations.||5||4.79||.606||0|
|2||Provide relevant professional and leadership development to teachers, administrators, and teacher educators.||5||4.79||.520||0|
|3||Develop new programs and curricula to respond to current and projected occupational needs.||5||4.81||.397||0|
|4||Formulate new and improve existing operational policies (standards) at the local level.||5||4.36||.759||1|
|5||Develop, improve, and maintain quality CTE teacher education programs at the bachelors, master’s, and doctoral level.||5||4.55||.889||1|
|6||Conduct innovative research projects to assist in meeting unmet CTE needs||5||4.48||.804||1|
|7||Educate and recruit quality CTE teachers and leaders||5||4.48||.397||0|
|8||Operate and maintain local CTE facilities and equipment||5||4.62||.661||1|
|9||Develop and execute external and internal procedures for program planning, development and evaluation procedures for local CTE programs.||5||4.55||.670||1|
|10||Develop, improve, and maintain state and local community relations (e.g., advisory committee, CTSO, other workforce agencies).||5||4.64||.618||1|
|11||Develop, improve, and maintain statewide professional associations and CTE teachers and administrators at all levels (local, state, and universities).||5||4.50||.634||1|
|12||Develop, grow and maintain student services (e.g.. Student recruitment and admissions, placement, and guidance services)||5||4.55||.739||1|
|13||Develop, improve, and maintain fiscal management of local CTE programs.||5||4.33||1.004||1|
Conclusion and Recommendations
The findings of this study provide CTE leaders, administrators, and researchers with a set of operational criteria that go beyond the ancillary objectives that are the rationale for many changes in federal and state-level CTE policy. While the sample size was not large (n=42) and only represented CTE administrators and teacher educators in one state (Virginia), the researchers believe that the strong consensus as to the inclusion of the criteria statements, as well as the average participant tenure within the field of secondary-level CTE leadership (30.4 years), increases the generalizability of the OISCTE criteria statements. However, validating the OISCTE on a national level could confirm the generalizability of the OISCTE.
Potential Applications of the OISCTE
There are at least three potential applications of the OISCTE. First, this framework could be used to assess the impact changes within a state's CTE program has on its ability to operate its CTE program. For example, the OISCTE was developed by the researchers to measure the impact the 1990 Carl D. Perkins Act within-state funding formula change had on Virginia's ability to operate CTE. Coupled with the Policy Delphi technique, which is "an organized method for correlating views and information pertaining to specific policy area and for allowing the participants representing such views and information the opportunity to react to and assess differing viewpoints" (Turoff, 2002, p. 83), the criteria that embody the OISCTE were ideal variables for measuring the effect of the change in the within-state funding formula. The findings from that study show that it negatively impacted all but one of the criteria statements. Most notably, the change in the within-state funding formula adversely effected "CTE teacher education, state-level CTE research initiatives, and state-level CTE's ability to provide localities with hands-on technical assistance, professional and leadership development, and coordination. The findings also suggested the legislative changes negatively altered the manner in which program evaluation occurred with the state" ( Manley, 2010 , p. 186).
Another potential application of the OISCTE would be the development of state-level assessments that measure a state's efforts and ability to develop, maintain, and grow their OISCTE. These 13 assessments, one for each criteria statement, could be used to assess the operational performance of a state's CTE program. Additionally, OISCTE assessments would enable higher-level associations and institutions, such as the federal government, determine with more validity, which states have strong CTE programs. States that score particularly high on the OISCTE assessments could be examined for potential modeling.
Lastly, the OISCTE could be used by state-level CTE administration when planning how to develop, grow, and maintain ancillary priorities often set forth by new federal and state regulations. Keeping the OISCTE criteria in mind while allocating resources to these new priorities may ensure that the underlying framework of their CTE program continues to operate efficiently and effectively.
In closing, examining administrative and policy-related changes through the OISCTE lens compels one to ask important questions. For example, how can best practices be researched and disseminated without additional support for research at all levels? How can we recruit and train quality CTE teachers without adequate support for teacher education institutions? How can local-level administrators and teachers effectively integrate academic instruction into CTE on a wide scale without increased support for statewide professional development and technical assistance?
The ability to understand the big picture is one of the workplace readiness skills all CTE teachers within Virginia, and other states, are asked to emphasize and assess in their instruction. The researchers believe it's important that leaders in CTE not lose sight of the bigger picture. State and local leaders must continue to fight for the preservation of funds (federal, state, and local) to accomplish the operational criteria identified within the OISCTE. Failure to do so will make the completion of other ancillary objectives and tasks put forth by new legislation and regulations difficult, if not impossible, to effectively accomplish in a timely manner.
Clark, R., Farmer, E., & Welch, S. (2010). An examination of leadership issues facing Pennsylvania career and technical administrators. Journal of Career and Technical Education Research, 35 (1), 47-62.
Elliott, J. & Molina, Q. (2007). Arizona career and technical education program review guide (assessment and action plan) . Unpublished manuscript.
Magisos, J. & Schroeder, P. (1974). Local administration of vocational education . Columbus, OH: Center for Vocational and Technical Education.
Manley, R. A. (In Press). The decentralization of Perkins: history, impact, and recommendations for future CTE legislation. Journal of Career and Technical Education Research .
Manley, R. A. (2010). The intended and unintended consequences of the 1990 Carl D. Perkins vocational and applied technology Act within-state funding formula change: A modified Policy Delphi study . Unpublished dissertation, Virginia Polytechnic Institute and State University.
National Association of State Directors of Career and Technical Education (NASDCTE) (2010). A Look Inside: A Synopsis of CTE Trends. Focus: Governance. Retrieved from: http://www.careertech.org/uploaded_files/Synthesis_-_CTE_Governance_FINAL.pdf.
National Center for Research in Vocational Education (NCRVE) (1991). The National Assessment of Vocational Education "what it says and what it should say" A Symposium . Paper presented at the AVA Conference, Orlando, FL.
Norton, R. E. (1977). The development of competency-based instructional materials for the preparation of local administrators of secondary and post-secondary vocational education . Columbus, OH: National Center for Research in Vocational Education.
Norton, R. E., & Harrington, L.G. (1987). Administrator competency study: A national identification and verification of the competencies important to secondary and postsecondary administrators of vocational and technical education . Columbus, OH: National Center for Research in Vocational Education.
Plinske, K. & Packard, W. (2010). Trustees perceptions of the desired qualifications for the next generation of community college presidents. Community College Review, 37 (4) 291-312
Rayens, M., & Hahn, E. (2000). Building consensus using the Policy Delphi method. Policy, Politics, & Nursing Practice, 1 , 308-315.
Ruff, R. (1981). A study of state level administration of vocational education . Columbus, OH: National Center for Research in Vocational Education.
Stevenson, W. (1963). The role of vocational-technical education in the university . Paper presented at the Oklahoma Vocational-Technical Education Conference at Oklahoma State University.
Stevenson, W. (1982). An assessment of the governance and administration of vocational-technical education in Utah . Columbus, OH: National Center for Research in Vocational Education.
Tennessee State Board of Education. (1975). Guidelines for comprehensive vocational-technical education programs. A handbook for superintendents, principals, local directors, and supervisors of vocational education . Knoxville, TN: Occupational Research and Development Coordinating Unit.
Tuxworth, E. (1989). Competence based education and training: Background and origins. In J. Burke (Ed.), Competency based education and training (pp. 9-22). Bristol, PA: The Falmer Press.
U.S. Congress. (2006). Carl D. Perkins career and technical education Act of 2006 . Washington, DC: U.S. Government Printing Office.
Virginia Board of Education. (2001). Virginia regulations governing CTE . Retrieved March 23, 2009, from http://www.doe.virginia.gov/VDOE/Instruction/CTE/regulations/regulations.pdf
Virginia Department of Education (Virginia DOE). (1989). Vocational education evaluation in Virginia: VEEVA summary report 1988-1989 . Richmond: Author.
Wenrich, R. C., Wenrich, J. W. & Galloway, J.D. (1988). Administration of vocational education . Homewood, IL: American Technical Publishers.