Preceptor Evaluations

1^st year students:  At least 60% of students in ATEP 292/293 will receive a score of two (2) or higher on questions 11-13 on the End of Semester Evaluation by Preceptors examining advanced knowledge and use of Evidence-based medicine.

2^nd year students: At least 70% of students in ATEP 392/393 will receive a score of two (2) or higher on questions 11-13 on the End of Semester Evaluation by Preceptors examining advanced knowledge and use of Evidence-based medicine.

3^rd year students: At least 80% of students in ATEP 492/494 will receive a score of two (2) or higher on questions 11-13 on the End of Semester Evaluation by Preceptors examining advanced knowledge and use of Evidence-based medicine.

Research Article Abstracts

At least 80% of students will score higher than 80% on the research article abstract assignments in all clinical classes.

Research Project (ATEP 494)

At least 80% of students will score higher than 80% on the research project assignment in ATEP 494.

Preceptor Evaluations

Students are evaluated each semester by their assigned clinical preceptor as part of their required series of clinical classes.  Preceptor evaluations are completed at midterm and the end of the semester.  The midterm evaluations are used to provide students initial feedback so they are able to work on things for the rest of the semester. Students and preceptors are required to meet and discuss the end of semester evaluation and each signs the evaluation sheet.

The preceptors use a 3-point scale to evaluate students:

Strengths and weaknesses are provided by the preceptor and goals and objectives are discussed for subsequent semesters.  The clinical coordinator for Athletic Training Ms. Amy Hand is responsible for collecting all preceptor evaluations.  They are turned in to her mailbox on the last day of class each semester. Ms. Hand is responsible for interpreting and sharing data with the athletic training program director, Athletic Training faculty, as well as all preceptors.  The data are sent to preceptors as de-identified data that is collected over the course of several semesters.  Preceptor data are also shared (de-identified) at preceptor training, which occurs every three years. The program director and clinical coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Research Article Abstracts

As part of each clinical class, students are required to complete research article abstracts on specific patient injuries/patient care.  The student is responsible for finding a scholarly article, writing an abstract, and presenting it in class.  The assignment is worth 50 pts and is a part of the following clinical classes:

1^st year students: ATEP 292/293
2^nd year students: ATEP 392/393
3^rd year students: ATEP 492

The program director and clinical coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Research Project (ATEP 494)

This assignment is specific to ATEP 494 and is designed to teach students to use and integrate research into their clinical decision making.  The total project has 275 points.

Develop Research Topic (10 Points)
Develop Research Questions and Hypothesis & Complete Prospectus (40 Points)
Develop a review of literature chart with 10 peer-reviewed articles (50 Points)

Title of article
Purpose of study
Participants
Brief methods
Results
Conclusions

FINAL PAPER: Must include abstract, introduction, review of literature, methods, reference sections, and all instruments (surveys or protocols) in an appendix. (Written in APA or AMA) (100 Points)

RESEARCH PROPOSAL (75 Points)

Assessment data from this assignment will be collected at the end of the spring semester by the clinical coordinator and placed in each student file.  The program director and clinical coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

PRECEPTOR EVALS

1^st Year students in ATEP 293: 82.6% received higher than 2 on preceptor evaluation for questions 11-13.
2^nd Year students in ATEP 393: 94% received higher than 2 on preceptor evaluation for questions 11-13.
3^rd Year students in ATEP 494: 89.9% received higher than 2 on preceptor evaluation for questions 11-13

RESEARCH ABSTRACTS

1^st year students in ATEP 293: 100% of students scored higher than 80% on article abstract assignment
2^nd year students in ATEP 392 100% of students scored higher than 80% on article abstract assignment
2^nd year students in ATEP 393: 89.3% of students scored higher than 80% on article abstract assignment
3^rd year students in ATEP 492: 100% of the students scored higher than 80% on the article abstract assignment

RESEARCH PROJECT FOR ATEP 494

All criteria for this learning outcome were except for the following:

• For the Abstract assignment, the low class average and low percentage of students above 80% is due to 5 students failing to submit the assignment which resulted in a grade of 0/10 which therefore drove down the class average. It was also a result of not understanding the assignment requirements and submission of below average work.
• For the Introduction assignment, the rationale behind the low percentage of students above 80% for the assignment is 6 students failed to turn in the assignment which resulted in a grade of 0/10.

This learning outcome was not addressed in the preceptor evaluations in ATEP 292, 392, or 492.  The preceptor evaluations where the content is addressed reflected good scores.  We will consider adding a few more questions related to this learning outcome—as we currently have only 3 questions used to measure this outcome.

We are trying to integrate evidenced-based medicine into all aspects of our AT program.  Part of the requirement for the research abstract assignments is for students to discuss the article with their clinical site preceptors.  The success of these assignments has resulted in us integrating more “research” into our preceptor training modules.  When our program trains preceptors each year, integrating research will be included as well as feedback from preceptors on how to increase the quality of the assignment. The key is to link the research articles (discussed in class) with students’ clinical sites where they actually see patients.  We are considering including clinical preceptors in an “online” discussion as part of the assignment. We need to do a better job of linking some of these assignments to the clinical sites where students apply the information.

Oral Practical Midterm and Final Examination

At least 80% of students will receive a 70% or higher on a clinical proficiency based oral practical exam section covering Domain I in ATEP 292 and ATEP 267.

BOC Examination

The Athletic Training Program average for Domain I (Injury/Illness Prevention and Wellness Protection) of the BOC will be within one standard deviation of the average for all candidates during the reporting period for the BOC (April to February). Domain I represents 25% of the questions on the exam.

Preceptor Evaluations – see methods in learning outcome 1

Oral Practical Midterm and Final Examination

Students take a midterm and final oral practical examination as a part of all their clinical classes during each semester they are in the athletic training program.  These exams have multiple modules that assess student outcomes in specific competencies related to implementing strategies to prevent the incidence and/or severity of injuries and illnesses and optimize a patient’s overall health and quality of life.  Each exam is worth 100pts and is included in clinical course syllabi.

Data from oral practical exams are collected by the Athletic Training program director and placed in students’ files at the end of each semester. The program director and clinical coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

BOC Examination

The BOC examination determines if students meet the minimum criteria to be certified as an athletic trainer.  It is the culminating exam that credentials athletic trainers.  The exam is taken in the last semester of a student’s program and consists of 5 specific domains.   Scores for each domain are provided to program directors in January of each year.  The passing point used for the BOC certification exam is a criterion-referenced approach called the Angoff Modified Technique. The average of the proportions, or probabilities, is multiplied by the total number of questions on the exam. The result then represents the "minimally acceptable" score.

BOC scores are sent directly to the Athletic Training program director each January.  The program director and clinical coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

ORAL PRACTICAL MIDTERM AND FINAL EXAM

Students in ATEP 292:
       Midterm exam:    94.3% scored higher than 70%
       Final exam:         100% scored higher than 70%

Students in ATEP 267
        Midterm exam:    92.8% scored higher than 70%
        Final exam:         97.1% scored higher than 70%

BOC RESULTS FOR DOMAIN I
        2015/16              75.3% (USC)       74.6% (National Average)
        2016/17              84.1% (USC)       80.8% (National Average)

All criteria for this learning outcome were met.

We are happy with the results for this learning outcome.  Many of the educational competencies related to this outcome are taught early in our AT program.  But despite the early classes, seniors are still doing well in this domain as evident by the BOC results.  We set the criteria of 70% because of how early we introduce material related to this outcome, but we will now consider raising the acceptable criteria to 80%.  In addition, our improvement on BOC results for this domain from last year to this year is a clear indicator that our targeted strategies have worked.

Preceptor Evaluations

1st year students:  At least 60% of students in ATEP 293 will receive a score of two (2) or higher on questions 20-29 on the End of Semester Evaluation by Preceptors examining clinical examination skills and diagnosis.

2nd year students: At least 70% of students in ATEP 392 will receive a score of two (2) or higher on questions 20-29 on the End of Semester Evaluation by Preceptors examining clinical examination skills and diagnosis.

2nd year students: At least 70% of students in ATEP 393 will receive a score of two (2) or higher on questions 20-25 on the End of Semester Evaluation by Preceptors examining clinical examination skills and diagnosis.

BOC Examination

The Athletic Training Program average for Domain II (Clinical Evaluation and Diagnosis) of the BOC will be within the standard deviation of the average for all candidates during the reporting period for the BOC (April to February).Domain II represents 22% of the questions on the exam.

Preceptor Evaluations – see methods in learning outcome 1

BOC Examination – see methods in learning outcome 2

PRECEPTOR EVALS

1^st Year students in ATEP 293: 85.6% received higher than 2 on preceptor evaluation for questions 20-29.
2^nd Year students in ATEP 393: 91.5% received higher than 2 on preceptor evaluation for questions 20-29.
3^rd Year students in ATEP 494: 93% received higher than 2 on preceptor evaluation for questions 20-25.

BOC RESULTS FOR DOMAIN II

2015/16          72.8% (USC)       71.3% (National Average)
2016/17          78.2% (USC)       73.7% (National Average)

All criteria for this learning outcome were met.

The preceptor evaluations for our clinical evaluation and diagnosis competencies are completed in spring in ATEP 393 and not in ATEP 392.  So we did not report preceptor evaluations for ATEP 392. They are also completed in ATEP 494 and not in ATEP 492---so they are done once a year. Clinical evaluation and diagnosis has been a strength of our program for many years and we continue to look for clinical sites where students are given the opportunity to work on their skills. We will also consider raising the standard in our junior/senior level classes (ATEP 393 and ATEP 494).

Since our AT program has a large number of clinical sites where we can send students, it is important to continually evaluate them for quality.  We will look at specific clinical sites where students are not scoring as high in this domain and see if it is the result of opportunities to practice these skills.  We have a large number of quality clinical sites and can certainly move students where they are given more of a chance to work on these skills—such as the orthopedic office and physical therapy clinics. We also saw another increase in our BOC results for this domain from 2015-16.  We will continue to re-visit how we can integrate evaluation/diagnosis skills across the entire curriculum.

Preceptor Evaluations

3rd year students: At least 80% of students in ATEP 492 will receive a score of two (2) or higher on questions 20-29 on the End of Semester Evaluation by Preceptors examining skills and knowledge in immediate and emergency care.

BOC Examination

The Athletic Training Program average for Domain III (Immediate and Emergency Care) of the BOC will be within the standard deviation of the average for all candidates during the reporting period for the BOC (April to February). Domain III represents 19% of the questions on the exam.

Preceptor Evaluations – see methods in learning outcome 1

BOC Examination – see methods in learning outcome 2

PRECEPTOR EVALS

3^rd Year students in ATEP 492: 88.6% received higher than 2 on preceptor evaluation for questions 20-29.

BOC RESULTS FOR DOMAIN III

2015/16      76.8% (USC)       77.5% (National Average)
2016/17      82.5% (USC)       78.4% (National Average)

All criteria for this learning outcome were met.

Immediate management content was an issue for our AT program as 2015-16 BOC results for this domain indicated our scores below the national average.  We hired Stephanie Rosehart (emergency care/CPR instructor) to oversee how we integrate this content into our curriculum.  She has always been used as an adjunct to teach the emergency care courses (ATEP 300 and ATEP 310), but now she has more responsibilities for course sequencing and integration of content and skills.  I think integrating Ms. Rosehart into the AT Program and curriculum was beneficial for the program and could be the main factor in our improved scores in this area. We moved from 1% behind the national average to 4% above the national average on the immediate/emergency care domain of the BOC exam.

We were looking at targeting some new measures and criteria for immediate emergency care content directly from our ATEP 310 and ATEP 310L classes, but since these are freshmen level courses and we are terminating the BS in Athletic Training Degree there will be no more athletic training students taking these courses.  We will look to possible re-visit this content in SO/JR classes before ATEP 492 which is a senior level class. And this content will certainly be included in the new athletic training masters program.

Preceptor Evaluations

3rd year students (spring):  At least 80% of students in ATEP 494 will receive a score of two (2) or higher on questions 20-29 on the End of Semester Evaluation by Preceptors examining skills and knowledge in treatment and rehabilitation.

BOC Examination

The Athletic Training Program average for Domain IV (Treatment and Rehabilitation) of the BOC will be within the standard deviation of the average for all candidates during the reporting period for the BOC (April to February). Domain IV represents 22% of the questions on the exam.

Preceptor Evaluations – see methods in learning outcome 1

BOC Examination – see methods in learning outcome 2

PRECEPTOR EVALS

3^rd Year students in ATEP 494: 92.3% received higher than 2 on preceptor evaluation for questions 20-29.

BOC RESULTS FOR DOMAIN IV

2015/16        69.4% (USC)       70.2% (National Average)
2016/17        77.6% (USC)       74.6% (National Average)

All criteria for this learning outcome were met.

Therapeutic intervention content was another area where we demonstrated significant improvement over the past year.  We were below the national average in this domain for the BOC in 2015-16 and this past year jumped to 3% above the national average.  Our ATEP 466 class has had 3 different instructors over the past 3 years.  This past year we hired Dr. Kari Odland-Wolff who has expertise in the area of therapeutic interventions, including manual therapy and modalities.  She is a perfect fit for both ATEP 366 and ATEP 466—which are the primary courses that reflect the content and skills within this learning outcome.  We believe the continuity of instructors for these two courses will continue to help us meet our learning outcomes in this area.

These results confirm our commitment or enhance student learning in the area of therapeutic interventions (based on last year’s assessment).  We will look upon the expertise of Dr. Odland-Wolff to advise us on new measures and criteria that we can integrate directly in to ATEP 366 and ATEP 466.

Preceptor Evaluations

1st year students:  At least 60% of students in ATEP 292/293 will receive a score of two (2) or higher on questions 14-15 on the End of Semester Evaluation by Preceptors examining cultural competence.

2nd year students: At least 70% of students in ATEP 392/393 will receive a score of two (2) or higher on questions 14-15 on the End of Semester Evaluation by Preceptors examining cultural competence.

3rd year students: At least 80% of students in ATEP 492/494 will receive a score of two (2) or higher on questions 14-15 on the End of Semester Evaluation by Preceptors examining cultural competence.

Psychosocial Referral Project (ATEP 494)

At least 80% of students will score higher than 80% on the psychosocial referral assignments in ATEP 494.

Clinical Integrated Proficiencies for ATEP 494

At least 95% of all students will score a two (2) on the Clinical Integrated Proficiencies for ATEP 494.

Oral/Practical Midterm and Final Examination

At least 80% of students will receive a 70% or higher on a clinical proficiency based oral practical exam section covering psychosocial issues and referral in athletic training in ATEP 492 and 494.

Preceptor Evaluations – see methods in learning outcome 1

Psychosocial Referral Project (ATEP 494)

This assignment is specific to ATEP 494 and is designed to teach students to develop a protocol for an appropriate referral of an individual with a suspected mental health issue.  Protocol includes:

*detection of case
*activation of protocol
*referral process
*follow-up procedures

Each group is required to present their referral process to the class and complete two (2) peer assessment of an article critique that is related to the designated topic area. Project is worth 150 pts.

The program director and clinical coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Clinical Integrated Proficiencies for ATEP 494

This assessment is specific to clinical class ATEP 494 and includes scenarios related to psychosocial issues in athletic training.  Students are required to go over scenarios with their clinical preceptor during the course of a semester.  The rubric for Clinical Integrated Proficiencies is as follows:

The program director and clinical coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Oral/Practical Midterm and Final Examination – see methods in learning outcome 2

PRECEPTOR EVALS

1^st Year students in ATEP 293: 96.8% received higher than 2 on preceptor evaluation for questions 14-15.

2^nd Year students in ATEP 393: 100% received higher than 2 on preceptor evaluation for questions 14-15.

3^rd Year students in ATEP 494: 95.6% received higher than 2 on preceptor evaluation for questions 14-15.

PSYCHOSOCIAL PROJECT and CLINICAL INTEGRATED PROFICIENCIES

OP MIDTERM AND FINAL EXAM FOR ATEP 492

Midterm Oral Practical Exam:

20/23 (87%) of students received score of 80% or better on this exam
100% of students received a score of 70% or better on this exam

Final Oral Practical Exam:

18/23 (78%) of students received score of 80% or better on this exam
100% of students received a score of 70% or better on this exam

All criteria for this learning outcome were met except for the Clinical Integrated Proficiencies. The low percentage of students above 95% on these proficiencies is a result of 6 students scoring a two (2) or one (1) for the required skills. In addition, of those that did score a two (2) or one (1) drove down the class average to 94%, just below the acceptable threshold.

Learning outcomes related to psychosocial issues needs to be better integrated across the entire program and not just examined in specific assignments and exams within classes. This learning outcome was not addressed in the preceptor evaluations in ATEP 292, 392, or 492.  The preceptor evaluations where the content is addressed reflected good scores.  We will consider adding a few more questions related to this learning outcome—as we currently have only 2 questions used to measure this outcome.

We are also happy with the integrated skills and class project for ATEP 494.  This was the first year we used these projects and it went very well. We will re-visit additional psychosocial measures for the ATEP 494 class in discussion with the instructor of the class.

It appears we may have set the criteria for the OP examinations too low (70%) as 100% of all students were able to meet this goal.  I posted additional data pertaining to students meeting a higher threshold of 80%. We also need to consider breaking up this section into specific questions related to psychosocial issues—as much of this exam is a complete assessment of skill and some of the exam might not be relevant. We still believe this is an important measure for students in this area as they are often required to do a complete assessment of a patient, which includes psychosocial issues.  As we indicated—we might be able to measure something more specific using additional class projects/assignments in ATEP 494.

There is no OP exam in ATEP 494, so we did not have data.  It will be taken out for next year’s assessment.

Preceptor Evaluations

1st year students: At least 60% of students in ATEP 292/293 will receive a score of two (2) or higher on questions 1-2, 8-10 on the End of Semester Evaluation by Preceptors examining organizational issues in athletic training.

2nd year students: At least 70% of students in ATEP 392/393 will receive a score of two (2) or higher on questions 1-2, 8-10 on the End of Semester Evaluation by Preceptors examining organizational issues in athletic training.

3rd year students: At least 80% of students in ATEP 492/494 will receive a score of two (2) or higher on questions 1-2, 8-10 on the End of Semester Evaluation by Preceptors examining organizational issues in athletic training.

BOC Examination

The Athletic Training Program average for Domain V (Organizational and Professional Health and Well-Being) of the BOC will be within the standard deviation of the average for all candidates during the reporting period for the BOC (April to February). Domain V represents 12% of the questions on the exam.

ATEP 496 Class Projects

At least 80% of students will score higher than 80% on all of the organizational and administration assignments in ATEP 496.

Preceptor Evaluations – see methods in learning outcome 1

BOC Examination – see methods in learning outcome 2

ATEP 496 Class Projects

This assessment is specific to ATEP 496 and is designed to teach students about risk management and organizational issues in athletic training (each assignment is worth 100 pts):

#1: Creation of bid sheet and budget development
#2: Creation of policy and procedure manual for a sports medicine department
#3: Risk assessment using AT Your Own Risk: A Safer Approach to Work Life and Sport
#4: Critique of Electronic Medical Recordkeeping Systems
#5: Safe School Evaluation of their clinical site

The program director and clinical coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

PRECEPTOR EVALS

1^st Year students in ATEP 293: 95.7% received higher than 2 on preceptor evaluation for questions 1-2, 8-10.
2^nd Year students in ATEP 393: 100% received higher than 2 on preceptor evaluation for questions 1-2, 8-10.
3^rd Year students in ATEP 494: 92.2% received higher than 2 on preceptor evaluation for questions 1-2, 8-10.

BOC RESULTS FOR DOMAIN V

2015/16     69.8% (USC)       72.4% (National Average)
2016/17     74.8% (USC)       71.6% (National Average)

ATEP 496 CLASS ASSIGNMENTS  

All criteria for this learning outcome were met.

This learning outcome was not addressed in the preceptor evaluations in ATEP 292, 392, or 492.  The preceptor evaluations where the content is addressed reflected good scores.  The program covers a great deal of this content across the curriculum but it is primarily addressed in ATEP 496. We plan to integrate specific questions related to administration based on the level of student (SO, JR, SR), rather than same questions throughout each year of preceptor evaluations.  It is worth exploring.

We are also pleased with our BOC scores for the Organization and Administration domain as last year we were behind the national average.  We are happy to now be 3% above the national average for this domain on the BOC. This is another course where we had multiple instructors over the past several years.  We plan to keep the assignments outlined in this assessment plan, as opposed to allowing individual instructors create their own assignments.  These assignments were worked on collaboratively and reflect the content needed to be successful on the BOC exam as well as practice as a strong athletic trainer.  We plan to use the same instructor again next year to maintain the progress we made.

Preceptor Evaluations

1st year students:  At least 60% of students in ATEP 292/293 will receive a score of two (2) or higher on questions 3-7 on the End of Semester Evaluation by Preceptors examining a teamed approach to healthcare practice.

2nd year students: At least 70% of students in ATEP 392/393 will receive a score of two (2) or higher on questions 3-7 on the End of Semester Evaluation by Preceptors examining a teamed approach to healthcare practice.

3rd year students: At least 80% of students in ATEP 492/494 will receive a score of two (2) or higher on questions 3-7 on the End of Semester Evaluation by Preceptors examining a teamed approach to healthcare practice.

Preceptor Evaluations – see methods in learning outcome 1

PRECEPTOR EVALS

1^st Year students in ATEP 293: 92% received higher than 2 on preceptor evaluation for questions 3-7.
2^nd Year students in ATEP 393: 99.3% received higher than 2 on preceptor evaluation for questions 3-7.
3^rd Year students in ATEP 494: 95.6% received higher than 2 on preceptor evaluation for questions 3-7.

These criteria were met.

This learning outcome was not addressed in the preceptor evaluations in ATEP 292, 392, or 492.  The preceptor evaluations where the content is addressed reflected good scores.  This learning outcome covers a number of ethical and practical behaviors that we want all students to learn.  This type of content is covered throughout the entire 4-year program and integrated into student learning both formally and informally.  We might consider being more specific in the learning outcome.  We currently measure this information in 5 questions on preceptor evaluations each year in the program.  Although we are pleased with the current results, we might consider more specific learning outcomes that target specific behaviors we are looking for.  This will certainly be addressed as we phase out the BS degree and move towards a masters-level AT program.

Oral/Practical Midterm and Final Examination

At least 80% of students will receive 80% or higher on a clinical proficiency based oral practical exam section covering all content learned in previous domains in ATEP 494.

BOC Examination

The Athletic Training Program will have 1st passing rate on the BOC of 90% or higher for all candidates during the reporting period for the BOC (April to February). Programs with a 3-year aggregate 1st time passing rate of less than 70% are automatically put on probation by CAATE.

Oral/Practical Midterm and Final Examination – see methods in learning outcome 2

BOC Examination – see methods in learning outcome 2

OP MIDTERM AND FINAL FOR ATEP 492

Midterm Oral Practical Exam

20/23 (87%) of students received score of 80% or better on this exam
100% of students received a score of 70% or better on this exam

Final Oral Practical Exam

18/23 (78%) of students received score of 80% or better on this exam
100% of students received a score of 70% or better on this exam

BOC EXAMINATION RESULTS - 1^st time passing rate

2014-15                95% (USC)           80.6% (National Average)
2015-16                94.2% (USC)        82.7% (National Average)
2016-17                100% (USC)         83.6% (National Average)

All measures except the scores on the final oral practical exam met the criteria for this learning outcome.

This learning outcome characterizes the culmination of a student's didactic and clinical educational experiences as an athletic trainer in providing quality patient care. We think it is best measured by student outcomes on the BOC exam and our final clinical exams.

The scores for the midterm oral practical met our established criteria for this learning outcome. However, we fell just short of our 80% mark for the final oral practical.  The OP exams use several clinical preceptors to help with the assessment.  Although we meet with them for a few minutes prior to exam, we can do a better job of communicating scoring rubrics and expectations for certain parts of the exam.  Feedback from students on this exam indicates that they feel there is often discrepancies in how certain preceptors grade certain skills.  We plan to address this in our preceptor training and work to make each exam more standardized.  This past year we also had several instructors teaching multiple sections of a clinical class and this resulted in different questions/scenarios used in the OP exams.  This will also be addressed and we plan to have the same instructor for each clinical class.

We believe the best measure of meeting this content is student scores on our national certification exam.  We are please this past year to have a 100% passing rate on the BOC exam. We continue to stay significantly above the national average.  This data validates much of what our program has done over the past few years.  We will use this as momentum to seek even better ways to design didactic and clinical educational experiences for our students.

Preceptor Evaluations

1^st year students:  At least 60% of students in ATEP 293 will receive a score of two (2) or higher on questions 11-13 on the End of Semester Evaluation by Preceptors examining advanced knowledge and use of Evidence-based medicine.

2^nd year students: At least 70% of students in ATEP 393 will receive a score of two (2) or higher on questions 11-13 on the End of Semester Evaluation by Preceptors examining advanced knowledge and use of Evidence-based medicine.

3^rd year students: At least 80% of students in ATEP 494 will receive a score of two (2) or higher on questions 11-13 on the End of Semester Evaluation by Preceptors examining advanced knowledge and use of Evidence-based medicine.

Research Article Abstracts

At least 80% of students will score higher than 80% on the research article abstract assignments in all clinical classes.

Research Project (ATEP 494)

At least 80% of students will score higher than 80% on the research project assignment in ATEP 494.

Preceptor Evaluations

Students are evaluated each semester by their assigned clinical preceptor as part of their required series of clinical classes.  Preceptor evaluations are completed at midterm and the end of the semester.  The midterm evaluations are used to provide students initial feedback so they are able to work on things for the rest of the semester. Students and preceptors are required to meet and discuss the end of semester evaluation and each signs the evaluation sheet.

The preceptors use a 3-point scale to evaluate students:

Strengths and weaknesses are provided by the preceptor and goals and objectives are discussed for subsequent semesters.  The clinical education coordinator for Athletic Training Ms. Amy Hand is responsible for collecting all preceptor evaluations.  They are turned in to her mailbox on the last day of class each semester. Ms. Hand is responsible for interpreting and sharing data with the athletic training program director, Athletic Training faculty, as well as all preceptors.  The data are sent to preceptors as de-identified data that is collected over the course of several semesters.  Preceptor data are also shared (de-identified) at preceptor training, which occurs every three years. The program director and clinical education coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Research Article Abstracts

As part of each clinical class, students are required to complete research article abstracts on specific patient injuries/patient care.  The student is responsible for finding a scholarly article, writing an abstract, and presenting it in class.  Part of the assignment will include interaction and feedback from the student’s clinical preceptor.

1^st year students: ATEP 292/293
2^nd year students: ATEP 392/393
3^rd year students: ATEP 492

The program director and clinical education coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Research Project (ATEP 494)

This assignment is specific to ATEP 494 and is designed to teach students to use and integrate research into their clinical decision making.  Suggested components of the research project are listed below.

Develop Research Topic
Develop Research Questions and Hypothesis & Complete Prospectus
Develop a review of literature chart with 10 peer-reviewed articles

Title of article
Purpose of study
Participants
Brief methods
Results
Conclusions

FINAL PAPER: Must include abstract, introduction, review of literature, methods, reference sections, and all instruments (surveys or protocols) in an appendix. (Written in APA or AMA)

RESEARCH PROPOSAL

Assessment data from this assignment will be collected at the end of the spring semester by the clinical education coordinator and placed in each student file.  The program director and clinical education coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Oral Practical Midterm and Final Examination

At least 80% of students will receive a 70% or higher on a clinical proficiency based oral practical exam section covering Domain I in ATEP 292 and ATEP 267.

BOC Examination

The Athletic Training Program average for Domain I (Injury/Illness Prevention and Wellness Protection) of the BOC will be within one standard deviation of the average for all candidates during the reporting period for the BOC (April to February). Domain I represents 25% of the questions on the exam.

Preceptor Evaluations – see methods in learning outcome 1

Oral Practical Midterm and Final Examination

Students take a midterm and final oral practical examination as a part of all their clinical classes during each semester they are in the athletic training program.  These exams have multiple modules that assess student outcomes in specific competencies related to implementing strategies to prevent the incidence and/or severity of injuries and illnesses and optimize a patient’s overall health and quality of life.

Data from oral practical exams are collected by the Athletic Training program director and placed in students’ files at the end of each semester. The program director and clinical education coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

BOC Examination

The BOC examination determines if students meet the minimum criteria to be certified as an athletic trainer.  It is the culminating exam that credentials athletic trainers.  The exam is taken in the last semester of a student’s program and consists of 5 specific domains.   Scores for each domain are provided to program directors in January of each year.  The passing point used for the BOC certification exam is a criterion-referenced approach called the Angoff Modified Technique. The average of the proportions, or probabilities, is multiplied by the total number of questions on the exam. The result then represents the "minimally acceptable" score.

BOC scores are sent directly to the Athletic Training program director each January.  The program director and clinical education coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Preceptor Evaluations

1st year students:  At least 60% of students in ATEP 293 will receive a score of two (2) or higher on questions 20-29 on the End of Semester Evaluation by Preceptors examining clinical examination skills and diagnosis.

2nd year students: At least 70% of students in ATEP 393 will receive a score of two (2) or higher on questions 20-29 on the End of Semester Evaluation by Preceptors examining clinical examination skills and diagnosis.

2nd year students: At least 70% of students in ATEP 494 will receive a score of two (2) or higher on questions 20-25 on the End of Semester Evaluation by Preceptors examining clinical examination skills and diagnosis.

BOC Examination

The Athletic Training Program average for Domain II (Clinical Evaluation and Diagnosis) of the BOC will be within the standard deviation of the average for all candidates during the reporting period for the BOC (April to February). In the last testing period, Domain II represented 22% of the questions on the exam.

Preceptor Evaluations – see methods in learning outcome 1

BOC Examination – see methods in learning outcome 2

Preceptor Evaluations

3rd year students: At least 80% of students in ATEP 492 will receive a score of two (2) or higher on questions 20-29 on the End of Semester Evaluation by Preceptors examining skills and knowledge in immediate and emergency care.

BOC Examination

The Athletic Training Program average for Domain III (Immediate and Emergency Care) of the BOC will be within the standard deviation of the average for all candidates during the reporting period for the BOC (April to February). In the last testing period, Domain III represented 19% of the questions on the exam.

Preceptor Evaluations – see methods in learning outcome 1

BOC Examination – see methods in learning outcome 2

Preceptor Evaluations

3rd year students (spring):  At least 80% of students in ATEP 494 will receive a score of two (2) or higher on questions 20-29 on the End of Semester Evaluation by Preceptors examining skills and knowledge in treatment and rehabilitation.

BOC Examination

The Athletic Training Program average for Domain IV (Treatment and Rehabilitation) of the BOC will be within the standard deviation of the average for all candidates during the reporting period for the BOC (April to February). In the last testing period, Domain IV represented 22% of the questions on the exam.

Preceptor Evaluations – see methods in learning outcome 1

BOC Examination – see methods in learning outcome 2

Preceptor Evaluations

1st year students:  At least 60% of students in ATEP 293 will receive a score of two (2) or higher on questions 14-15 on the End of Semester Evaluation by Preceptors examining cultural competence.

2nd year students: At least 70% of students in ATEP 393 will receive a score of two (2) or higher on questions 14-15 on the End of Semester Evaluation by Preceptors examining cultural competence.

3rd year students: At least 80% of students in ATEP 494 will receive a score of two (2) or higher on questions 14-15 on the End of Semester Evaluation by Preceptors examining cultural competence.

Psychosocial Referral Project (ATEP 494)

At least 80% of students will score higher than 80% on the psychosocial referral assignments in ATEP 494.

Clinical Integrated Proficiencies for ATEP 494

At least 95% of all students will score a two (2) on the Clinical Integrated Proficiencies for ATEP 494.

Oral/Practical Midterm and Final Examination

At least 80% of students will receive a 80% or higher on a clinical proficiency based oral practical exam section covering psychosocial issues and referral in athletic training in ATEP 492 .

Preceptor Evaluations – see methods in learning outcome 1

Psychosocial Referral Project (ATEP 494)

This assignment is specific to ATEP 494 and is designed to teach students to develop a protocol for an appropriate referral of an individual with a suspected mental health issue.  Protocol includes:

*detection of case
*activation of protocol
*referral process
*follow-up procedures

Each group is required to present their referral process to the class and complete two (2) peer assessment of an article critique that is related to the designated topic area. Project is worth 150 pts.

The program director and clinical education coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Clinical Integrated Proficiencies for ATEP 494

This assessment is specific to clinical class ATEP 494 and includes scenarios related to psychosocial issues in athletic training.  Students are required to go over scenarios with their clinical preceptor during the course of a semester.  The rubric for Clinical Integrated Proficiencies is as follows:

The program director and clinical education coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Oral/Practical Midterm and Final Examination – see methods in learning outcome 2

Preceptor Evaluations

1st year students: At least 60% of students in ATEP 293 will receive a score of two (2) or higher on questions 1-2, 8-10 on the End of Semester Evaluation by Preceptors examining organizational issues in athletic training.

2nd year students: At least 70% of students in ATEP 393 will receive a score of two (2) or higher on questions 1-2, 8-10 on the End of Semester Evaluation by Preceptors examining organizational issues in athletic training.

3rd year students: At least 80% of students in ATEP 494 will receive a score of two (2) or higher on questions 1-2, 8-10 on the End of Semester Evaluation by Preceptors examining organizational issues in athletic training.

BOC Examination

The Athletic Training Program average for Domain V (Organizational and Professional Health and Well-Being) of the BOC will be within the standard deviation of the average for all candidates during the reporting period for the BOC (April to February). In the last testint period, Domain V represented 12% of the questions on the exam.

ATEP 496 Class Projects

At least 80% of students will score higher than 80% on all of the organizational and administration assignments in ATEP 496.

Preceptor Evaluations – see methods in learning outcome 1

BOC Examination – see methods in learning outcome 2

ATEP 496 Class Projects

This assessment is specific to ATEP 496 and is designed to teach students about risk management and organizational issues in athletic training.  Components of the assignment included the following:

#1: Creation of bid sheet and budget development
#2: Creation of policy and procedure manual for a sports medicine department
#3: Risk assessment using AT Your Own Risk: A Safer Approach to Work Life and Sport
#4: Critique of Electronic Medical Recordkeeping Systems
#5: Safe School Evaluation of their clinical site

The program director and clinical education coordinator meet at the end of spring semester to review data for the academic assessment plans and determine if any changes are suggested by the data. They also meet at the end of each year to discuss clinical preceptors and clinical sites. Data are included as part of our CAATE-accreditation report which is due each October.

Preceptor Evaluations

1st year students:  At least 60% of students in ATEP 293 will receive a score of two (2) or higher on questions 3-7 on the End of Semester Evaluation by Preceptors examining a teamed approach to healthcare practice.

2nd year students: At least 70% of students in ATEP 393 will receive a score of two (2) or higher on questions 3-7 on the End of Semester Evaluation by Preceptors examining a teamed approach to healthcare practice.

3rd year students: At least 80% of students in ATEP 494 will receive a score of two (2) or higher on questions 3-7 on the End of Semester Evaluation by Preceptors examining a teamed approach to healthcare practice.

Preceptor Evaluations – see methods in learning outcome 1

Oral/Practical Midterm and Final Examination

At least 80% of students will receive 80% or higher on a clinical proficiency based oral practical exam section covering all content learned in previous domains in ATEP 492.

BOC Examination

The Athletic Training Program will have 1st passing rate on the BOC of 90% or higher for all candidates during the reporting period for the BOC (April to February). Programs with a 3-year aggregate 1st time passing rate of less than 70% are automatically put on probation by CAATE.

Oral/Practical Midterm and Final Examination – see methods in learning outcome 2

BOC Examination – see methods in learning outcome 2