3.3 Institutional Effectiveness
3.3.1 The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.
R Compliance
€ Partial Compliance
€ Non-Compliance
NARRATIVE:
Trinity Valley Community College identifies intended outcomes, assesses and reports the achievement of these outcomes, and provides evidence of improvements in a number of ways.
Institutional effectiveness processes emerge from a desire of TVCC constituents to understand and construct meaning from the activities of the College. This is also a means through which TVCC can be proactive in meeting the accountability mandates from the Texas State Legislature and the Texas Higher Education Coordinating Board. State-mandated processes include the THECB five-year Institutional Effectiveness Desk Review, the THECB Core Curriculum Review, and the annual Legislative Budget Board Performance Measures. Internal efforts include the development of the College's strategic plan, the establishment of linkages between the strategic plan and administrative outcomes that are evaluated annually, and the establishment of the relationship between learning outcomes and general education outcomes. For a graphic depiction of how TVCC views these linkages, please view the attached graphic (Planning and Evaluation Graphic). The relationships between administrative outcomes, learning outcomes, and the overall strategic plan become more clear when reviewing the reporting format for each [Administrative Outcomes, Learning Enhancement Annual Plans (LEAPs)]. TVCC's annual strategic planning process, as discussed extensively in Core Requirement 2.5, specifies goals for the College, which are linked to the outcomes of the planning units.
Identification of Outcomes
Beginning Fall 2005, TVCC implemented a new institutional effectiveness process in order to more systematically differentiate between learning outcomes and administrative outcomes. Each academic program provides student learning outcomes that support the overall general education goals of TVCC, while each educational and administrative support unit provides outcomes that support TVCC goals. Educational planning units now submit Learning Enhancement Annual Plans (LEAPs), which identify intended learning outcomes specifically linked to general education outcomes (LEAPs form, items 5 & 6). This allows for analysis of how educational units contribute to the accomplishment of the general education goals of the College (see Comprehensive Standard 3.5.1 for details of general education outcomes). The LEAPs process was added to TVCC's institutional effectiveness processes in the 2005-2006 academic year, with results of assessment from the first round of LEAPs being available in July 2006. Prior to 2006, planning for learning outcomes were included as Administrative Outcomes (TVCC Administrative and Learning Outcomes Website). The office of planning and institutional effectiveness requests plans to be submitted annually by each planning unit. From 2002-03 through 2004-05, all administrative, educational support services, and educational programs submitted annual goals and objectives, which incorporated the appropriate outcomes for that planning unit.
Each administrative outcome is linked directly to at least one strategic planning goal, which allows for analysis of the planning unit's contribution to the accomplishment of the strategic plan. This is formalized in the administrative outcomes form (Items 5 and 6). Between 2002-2003 and 2004-2005, TVCC handled learning outcomes as a part of the administrative outcomes processes, and the results of these may be viewed online under the headings of "Goals and Objectives" for academic planning units (Example: Biology 2003-2004 Goals and Objectives). Beginning with the 2005-2006 academic year, learning outcomes are reported as [Learning Enhancement Annual Plans (LEAPs); Biology 2005-2006 example]. Administrative and educational support units submit administrative outcomes, in which all planning units define their expected outcomes for the academic year and outline implementation strategies.
TVCC Board Policy BI (Local) describes the annual institutional effectiveness, planning, and budgeting cycles, which include the following:
-
The Strategic Planning Committee reviews and/or updates the strategic planning goals annually.
-
New strategic planning goals are approved by appropriate committees, the College president, and the Board and disseminated to all units for use in updating administrative outcomes and Learning Enhancement Annual Plans (LEAPs) for the coming academic year.
-
Budget requests from departments and units are sent to division chairs, deans, and vice-presidents, with requests to prepare budgets, administrative outcomes, and learning outcomes that are in alignment with strategic planning goals for the upcoming year.
-
The administrative outcomes for the upcoming fiscal year are submitted to appropriate deans and vice presidents for approval. Administrative outcomes include a statement of desired outcomes, outcome descriptions, implementation strategies, assessment methods, assessment criteria, relationship to strategic planning goals, and budgetary implications.
-
Faculty submit LEAPs for the upcoming year to the appropriate dean for approval. LEAPs include a statement of the desired outcomes, outcome description, implementation strategies, assessment methods, assessment criteria, and relationship to general education outcomes.
-
Prior to beginning the academic year, administrative outcomes and LEAPs are modified if budget revisions impact the plans. All administrative outcomes and LEAPs are approved by the appropriate dean or vice president.
-
Administrative outcomes and LEAPs are implemented throughout the fiscal year.
-
Administrative outcomes and LEAPs are evaluated to determine progress toward goals.
-
Using feedback from the results on administrative outcomes and LEAPs, the strategic plan is evaluated and new administrative outcomes and LEAPs are developed for future years by repeating the steps above.
Outcomes specific to administrative, support, and educational programs are identified through the internal institutional effectiveness processes described above. In addition, the institutional effectiveness process integrates standards and measures established by external reviews:
-
The Texas Higher Education Coordinating Board (THECB) institutional effectiveness process that identifies outcomes for access, retention, graduation, and placement for each Texas community college. The THECB Institutional Effectiveness Desk Review was conducted in 2002, which was used to enhance the College's assessment of its educational programs and support services outcomes (Institutional Effectiveness Desk Review, 2002).
-
The Legislative Budget Board requires Annual Performance Measures which track course completion, contact hours taught by full-time faculty, percent of students who transfer to a university, enrollment of disadvantaged populations, developmental education success, licensure pass rates, administrative cost as a percentage of the budget, and the number of degrees or certificates awarded (LBB Performance Measure Data Report).
This system allows TVCC to compare its performance to other institutions. For example, TVCC is serving students with similar academic challenges (34.2% categorized as academically disadvantaged, state=36.4%) as well as economic status (29.2% economically disadvantaged compared to the states 29.5%) (LBB Performance Measure Data Report). TVCC also performs well with these students, as TVCC has received additional state funding for its success in developmental education. Other comparisons reveal that TVCC consistently produces more graduates than most institutions and is above the state average (83.96%) on course completions (TVCC=86.03% in 2005 Legislative Budget Board Report).
-
The Texas Higher Education Coordinating Board Curriculum review process requires an evaluation of student learning outcomes in each of the core areas. TVCC's last review under this process was conducted in October, 2004 (Core Curriculum Report). See Comprehensive Standard 3.4.1 for a complete discussion of this process.
-
The Associate Degree in Nursing and Surgical Technology programs hold a national accreditation. The accreditation process for each of these programs requires faculty to gather and analyze program data to demonstrate achievement of educational program outcomes and outcomes that relate to student support services.
Assessment of Outcomes
The office of planning and institutional effectiveness is primarily responsible for carrying out the institutional research timelines of the College. The dean of planning and institutional effectiveness provides the leadership for these activities (Dean of Planning and Institutional Effectiveness Job Description). The institutional research associate assists in the accomplishment of the objectives of the department (Institutional Research Associate Job Description). This office is responsible for the design, deployment, and administration of a variety of surveys, including TVCC Graduating Student Survey-Learning Environment, the TVCC Graduating Student Survey-Services, the TVCC Student Evaluation of Course and Instructor, the evaluation of various nursing courses and clinicals, the Community College Survey of Student Engagement (2005), and the Community College Faculty Survey of Student Engagement. Students are informed of their role in assessment in the TVCC Catalog (p. 69; Assessment of Student Outcomes). A complete listing of surveys is available on the Office of Planning and Institutional Effectiveness Website, Surveys.
Data are provided in several formats to facilitate the review and analysis of the College's outcomes. The TVCC Fact Book and TVCC Fact Sheet are two documents that contain institution-wide data which is distributed for use in the assessment of outcome achievement. The College utilizes the Annual Data Profile to assess achievement of institutional outcomes and student access to the institution. This report contains a profile of each educational program by CIP code, including retention, course completion, and graduation rates (Annual Data Profile 2003, 2004, 2005). The Texas Higher Education Coordinating Board provides additional data for use by the intuition (THECB Data Website). Additionally, the office of planning and institutional effectiveness provides supports the research needs of various offices on an ad hoc basis. An example, the dean’s involvement in the College’s quality enhancement plan, the Learning Initiative, has generated research looking at student reading comprehension abilities, levels of student engagement, and the connection between engagement levels and gains in reading comprehension (Learning Initiative Website).
Another example is the evaluation of distance learning courses. Students are provided an opportunity to comment on their distance education learning experiences through an online survey (Distance Learning Student Survey Spring, 2003). TVCC provides each student a form that solicits feedback on the distance medium, the instructor, support services, and the overall college performance and provides the opportunity for students to comment on their learning experience. WebCT, TVCC's online learning provider, facilitates an online evaluation form that includes feedback on the learning management system. TVCC forwards all forms to the instructor and the instructional department chair or designated administrator. Necessary action by the faculty is based upon the contents of student evaluations.
Additional instruments are used to evaluate educational outcomes, including the THECB Core Curriculum Review (Core Curriculum Report), and the activities of the Curriculum and Instruction Committee. The demonstration of evaluating program and learning outcomes is discussed further in Comprehensive Standard 3.4.1. Discussions of how graduates attain college-level competencies in general education is further discussed in Comprehensive Standard 3.5.1.
The office of planning and institutional effectiveness requests assessment reports each year from each primary college department/unit and assesses the the unit's success in meeting its stated outcomes. The assessment of those outcomes is clearly identified for FY 2003-2005 in the "Results of Assessment" area of the administrative outcomes form (Sample Administrative Outcomes Form, Vice President of Instruction). The assessment of those outcomes for FY 2005-2006 is clearly delineated on the Administrative Outcomes form (item 13) and the LEAPs form (item 8) (Sample Administrative Outcomes Form, Sample LEAPs Form).
Evidence of Improvement Based on Analysis of Results
The annual Administrative Outcomes and LEAPs process requires each planning unit to identify changes that were made as a result of what was learned through the institutional effectiveness process. Changes made as a result of the assessment of outcomes is clearly identified for FY 2003-2005 in the "Impact of Assessment Findings" area of the administrative outcomes form (Sample Administrative Outcomes Form, Vice President of Instruction). Changes made for FY 2005-2006 are clearly delineated in the "Changes as a Result of Evaluation" area on the Administrative Outcomes form (item 14) and the LEAPs form (item 8) (Sample Administrative Outcomes Form, Sample LEAPs Form).
The THECB requests that educational programs evaluated in the Annual Data Profile identify any deficiencies and develop an improvement plan for the educational programs and support services as outlined in the state required Program Improvement Plan and the individual program evaluations. Improvement efforts are documented by reviewing data to determine if the state standard measures are met or exceeded in the areas of graduates, placement, and licensure pass rates (Annual Data Profile 2003, 2004, 2005).
As a result of TVCC's strategic planning, evaluation, and analysis of the achievement of outcomes, plans are developed to improve the College's educational programs and the administrative and educational support services. A complete detailing of all planning units' Administrative Outcomes and Learning Enhancement Annual Plans (LEAPs) are available on the TVCC Administrative and Learning Outcomes Website. Comprehensive Standard 3.4.1 includes a detailed narrative related to improvement in educational programs, and Comprehensive Standard 3.9.3 provides examples of improvement in student services. These examples are in varying phases of implementation, and provide evidence of continuous improvement efforts in TVCC's educational programs and administrative and educational support services (Annual Data Profile 2003, 2004, 2005).
The activities described above formulate the framework from which TVCC approaches institutional effectiveness and assists the College in assuring compliance with TVCC Board Policy BI (Local) and BIA (Legal).
DOCUMENTATION:
|