2a.1. How does the unit ensure that the assessment system collects information on candidate proficiencies outlined in the unit's conceptual framework, state standards, and professional standards?
COE requires each department to collect and monitor assessments for its own candidates. Collection and assessment of candidate proficiency commences at program entry through the application process, proceeds to program exit through the Institutional Recommendation, and continues beyond graduation through alumni surveys. Table 6 Unit Assessment System: Transition Point Assessments provides links to samples of formative and summative assessments as well as summaries of key assessment data at selected transition points. Target knowledge, skills, and dispositions are communicated during advising and program orientations, in syllabi, and in standards-based assignment rubrics and internship requirements. The College requires faculty to submit syllabi that communicate outcomes aligned with our conceptual framework, state standards, and professional standards. Key assessments, and the scoring guides used in the evaluation of these assessments, are aligned with the appropriate professional organizations’ standards and/or required state performance standards. Exit, alumni, and employer surveys are also aligned with professional and state standards as well as our conceptual framework.
The initial teacher preparation programs’ use of the recently adopted Alaska Beginning Teacher Standards as program exit criteria exemplifies how COE’s assessment system ensures collection of candidate proficiency data. Program faculty have aligned specific professional organizations’ standards with these state standards. These standards inform key assignments and assessments, are used in candidate classroom performance evaluation, and provide the guiding standards for candidate creation and faculty evaluation of their portfolios. Another example is the alignment of the ISSLC/ELCC and Alaska Administrative Standards and their use to frame the principal preparation program as a whole, to structure the portfolio, and to inform candidates’ reflection process.
Candidates admitted to various COE programs are introduced to the appropriate standards early in their experiences. Their familiarity increases during formative performance assessment and final evaluation processes. For example, secondary education candidates go through an orientation to program and internship expectations during a beginning foundations course. As part of this orientation, they learn about the Alaska State Beginning Teacher Standards. Candidates are shown how the Alaska standards form the basis for the program’s standards-based assessments. Candidates clearly understand the expectation that they will produce evidence showing how they meet the required standards in order to successfully complete the program. In subsequent content-specific methodology courses, candidates learn how the appropriate professional organization’s standards relate to and extend the Alaska standards.
In addition, the College’s conceptual framework is connected to standards, course goals, and assessments, particularly through the core values. Each course syllabus emphasizes connections between the framework and course goals. COE awards select faculty and candidates “core value” plaques at the end of the year. This celebration provides concrete evidence of how the conceptual framework imbues teaching, service, and scholarly activity.
TaskStream is used to collect key assessments, and information can be aggregated from that database. Each candidate establishes a personal account, and standards-based assessments are stored there. The College monitors the aggregated data, and thereby ensures that 1) candidates are making progress toward proficiency as they approach graduation, and 2) departments are adequately assessing candidates on the conceptual framework, state standards, and professional standards.
2a.2. What are the key assessments used by the unit and its programs to monitor and make decisions about candidate performance at transition points such as those listed in Table 6?
Unit Assessment System: Transition Point Assessments
2a.3. How is the unit assessment system evaluated? Who is involved and how?
COE has structures in place that facilitate participation in the evaluation and refinement of the unit’s assessment system by faculty, staff, school- or community-based personnel and other community members, and administrators. While participation in the evaluation and refinement process is broad, there are two structures that ensure that the process is focused and systematic: the COE Assessment and Accreditation (A&A) Committee and the COE Leadership Team.
The A&A Committee includes faculty representation from each department, the Associate Dean, the Data Manager, and department chairs. This committee is responsible for on-going monitoring and review of the assessment process within the College, including the College-wide assessment system (Std1a.Exh1), particularly as it relates to candidate data. In monthly meetings, the committee reviews, discusses, and takes action on matters related to assessment and accreditation (Std2a.Exh4). The A&A Committee is specifically involved in the University’s Assessment Plans and Reports Process, which is focused on assessment of candidate learning outcomes. Each spring semester departments participate in an evaluation process including a faculty review of published program outcomes that results in an assessment report. This report is reviewed by university-wide faculty peers during the summer. Each fall semester the A&A Committee reviews the peer evaluations from the UAA Assessment Peer Review Committee, which then pass to departments for discussion and action. Peer evaluations of the assessment reports often result in recommendations about assessment procedures to departments and the college from the A&A Committee. The A&A Committee also oversees the College-wide exit, alumni, and employer survey process to assist the College in a systematic collection and analysis of data from both alumni and employers. The Committee reviews survey documents once per year and update or modify them as appropriate to ensure that the instruments collect the data needed to inform practice at the program, department, and unit levels.
The COE Leadership Team (LT) includes the Dean, Associate Dean, department chairs and directors, Data Manager, Assistant to the Dean, and the Fiscal Manager. The LT is responsible for conducting on-going monitoring and review of scheduling, budgeting, enrollments, personnel, and all other matters related to unit operations. The LT has weekly meetings and utilizes input from departments, central administration, and advisory boards and councils to shape the College’s direction and evaluate the unit operations.
The LT also participates in the annual performance-based budget activities where unit operations are evaluated by central administration and important decisions about space, technology, and staffing are made. Advisory boards and councils meet on either a quarterly or monthly basis to review operations and provide feedback. These boards and councils include representatives from professional education faculty, arts and sciences faculty, staff, P-12 educators, and leaders from business and community organizations.
2a.4. How does the unit ensure that its assessment procedures are fair, accurate, consistent, and free of bias?
Ensuring that assessment procedures are fair, accurate, consistent, and free of bias is ongoing (Std2a.Exh5). During AY09, all departments were required to review the Assessing the Assessments document available from NCATE. LT and College-level committees, such as the Graduate and A&A committees, are especially attuned to these matters. UAA also has a dispute/complaint process for grades and other academic actions.
Departments take several actions to ensure fairness. Program outcomes are widely published in the university catalog and other program documents. Standards-based scoring guides for key assessments are available to candidates through course handouts, TaskStream, program handbooks, or Blackboard prior to assignment due dates. TaskStream allows faculty to return assignments to candidates for revision and subsequent resubmission. Many program areas have completed curriculum maps or similar documents to ensure that candidates have ample opportunity in their coursework and field experiences to learn, practice, and demonstrate the knowledge, skills, and dispositions required by the various standards and the conceptual framework. Key assessments are evaluated using professional and state standards so candidates, as they progress through the program transition points, are exposed to multiple instructors using multiple methods.
While individual faculty examine their course assessments each semester for accuracy (measure what they purport to measure), departments examine key assessments on an annual basis to determine if the assessment procedures are accurate. This is achieved through the UAA Outcomes Assessment Report Process described in 2a.5. In addition, departments apply appropriate methods for assessing knowledge, skills, and dispositions. For example, observational instruments, not tests, are used in the evaluation of field experiences.
The UAA Outcomes Assessment Report process also helps the College ensure consistency in assessments. These reports allow departments to examine key assessment results over time. In addition, department faculty engage in meetings to discuss candidate work and consistency of expectations. Inter-rater reliability is addressed in meetings of clinical faculty and program faculty where scoring guides are discussed. Samples of candidate work are reviewed and common ground and agreement are reached.
Key assessments are designed to avoid bias. Attention to culturally responsive teaching is a significant endeavor in the unit, partly due to Alaska’s rich cultural context. The conceptual framework illustrates COE’s commitment to culturally responsive education. All faculty participate in some form of culturally responsive education through College and department meetings, study groups, or other professional development. COE uses the Alaska Standards for Culturally Responsive Schools and Guidelines for Preparing Culturally Responsive Teachers for Alaska’s Schools to inform instruction and practice.
2a.5. What assessments and evaluation measures are used to manage and improve the operations and programs of the unit?
A variety of measures inform unit operations and programs. Candidate evaluations of instructors and courses; faculty and staff evaluations of the dean; evaluations of faculty performance; surveys of candidates, alumni, and employers; annual program assessment reports; and committee and board review of policies, procedures, and curriculum all contribute to the ongoing management and improvement of the College.
UAA uses IDEA (Individual Development and Educational Assessment) for purposes of evaluating instructors and college deans. IDEA’s Student (candidate) Ratings of Instruction system solicits candidates’ feedback on their own learning progress, effort, and motivation, as well as their perceptions of the instructor's use of instructional strategies and teaching methods. IDEA’s Feedback for Deans system solicits feedback from faculty and staff on the Dean’s administrative performance.
Faculty performance in all workload areas is conducted in accordance with UA Board of Regents Policy 04.04; UAA Policies and Procedures for Faculty Appointment, Review, Promotion, and Tenure (UAA Handbook, Chapter 3); and COE Guidelines for Evaluating Teaching, Service, and Research and Creative Activity (COE Handbook, Appendix A).
In accordance with University of Alaska Board of Regents Policy 10.06, all academic programs and units at UAA are required to engage in continuous improvement processes that result in recommendations for program enhancements based on critical self-review and performance evaluation. These processes include, but are not limited to, annual candidate outcomes assessment review and cyclic program review. The review processes determine if the program aligns with the mission of the University and the academic unit, effectively achieves its intended outcomes, and operates efficiently and is a good steward of its resources. The College actively participates in these continuous improvement processes.
The A&A Committee oversees the administration of exit, alumni, and employer surveys. These surveys collect information about College, department, and program services; technology; general program outcomes; state and national standards; the conceptual framework; and graduate performance. These surveys also solicit feedback on improving programs and services.
The UA System uses the SunGard Higher Education Banner database for student, financial aid, finance, and human resources applications. UAA also uses OnBase content management software to facilitate applicant and candidate document imaging and workflow. In addition, COE utilizes a user-friendly website known as PETaL (Preparing Educators to Transform Lives, the College motto), which is a repository of regularly updated and automated dynamic database-driven reports on programs and courses that uses institutional enterprise data from Banner and document images of candidate records to facilitate program management and communications.
COE’s governance structure (see COE Handbook) cultivates an intellectual community that engages in collaborative and cooperative decision making and communication among faculty, candidates, administration, staff, and community stakeholders. This structure includes College-wide committees and assemblies as well as advisory boards and councils that review and evaluate data on an ongoing basis for purposes of managing and improving operations and programs.
2a.6. Optional exhibit found in Standard 2 Exhibits.