Faculty Info

Licensing

Penn State has a University-wide license for CATME. The College of Engineering will pay the entire cost of the yearly license to Purdue and then will bill individual Colleges/Campuses for their costs of use. It is recommended that you confirm the expenditure with your College/department before using this for-fee tool. These conditions are subject to change.

For more information, please go to the CATME Licensing Webpage.

Procedure for Implementing CATME in a Canvas Course

Currently, CATME does NOT integrate with Canvas. This means that everyone will need a separate CATME account in order to distribute and submit evaluations. Students will need to complete the evaluations outside of Canvas. CATME does provide email notifications that include reminders and links to evaluations once faculty has configured the assignments on the CATME website. Assignments in Canvas are still needed assuming that the evaluations will eventually end up as individual grades in the Canvas grade book.

Faculty will need to do the following in order to use CATME in their classes regardless of the delivery format (resident, online, etc.) of the course:

Some students express concerns about how peer evaluations might affect their grades. While all assessments and their related grading rubrics should align with the learning outcomes, there are some practices to consider when using peer evaluations of teamwork.

The recommended practice is that faculty and their teaching assistants are assigning the grades, not the students. Student-submitted evaluation scores in CATME, or any other peer evaluation tool for that matter, should be used as the basis for a grade assigned by the faculty. It may be best to consider the evaluation scores as part of a larger picture when determining grades. Other forms of evidence should be considered as well. For example, emails from students earlier in the semester, observed group dynamics, and CATME's feedback may contribute towards the final peer evaluation grade.

For other practices, please visit the Schreyer Institute Self-Paced Module: Working with Student Teams

To create an account, please watch the following tutorial

2/19/19 Update: All CATME TA accounts are now Dual Student/Instructor accounts.

Dual Account use is described in this video:

What is a dual access account?

A dual access account gives users both student and instructor CATME access.

My teaching assistants have dual access accounts, do they have to purchase a CATME license?

NO, teaching assistants that have dual access accounts are covered by your instructor’s CATME license If they are being delegated into your surveys.
If they create their own surveys they will need a CATME license.

How can you switch between instructor and student with a dual access account?

On your home activity page, go the top right hand corner of the screen to find the drop down button to switch between instructor and student access.

To add your class to CATME, please watch the following tutorial.

At this point, you have two options to set up teams in CATME:

  1. CATME Team-Maker – allows for criteria-based team* making that is driven by student input; use option "4a. Using CATME Team-Maker"
  2. Canvas Groups – allows for flexible group* formation based on random, self-selected, or assigned groups; use option "4b. Using Canvas Groups"

* The terms "team" and "group" are used to refer to groups of students working together collaboratively on class assignments. CATME and Canvas use different terminology and this information mirrors the use respective to each platform. CATME uses "team(s)" and Canvas uses "group(s)". In cases where the reference is either general or platform-agnostic, "team" will be used.

You may want to chose the CATME Team-Maker tool if you would like to:

  • base team formation off various criteria, the criteria includes, but is not limited to:
    • preferred times to work (schedules)
    • writing, software, and hands-on skills
    • leadership preferences
    • discipline
    • and more…
  • use a tool that builds teams that are easily referenced in CATME Peer Evaluations

Creating a Team Maker Survey

Timezone Scheduling for Instructors

Importing Teams into Peer Evaluations

This step is done during step "5. Create a CATME activity" when you create your Peer Evaluations.

You may want to chose to export your Canvas Roster if you would like to:

1. Assign students to Canvas Groups

Use one of the methods above to create random or self-selected groups or use these directions to manually create groups in Canvas: How do I manually create groups in a group set?

2. Export the Canvas Roster

There isn't a way to directly export Group membership information from Canvas. There is a Canvas Feature Request that needs your votes to be accepted. Please consider voting this idea up.

Lucy Bamwo at the bottom of the comments provides a creative workaround to export Group membership information from within Canvas.

Currently, there is no way of exporting groups out of Canvas. However, you could set up a group assignment (assign marks to whole group) 0 point, non-submission assignment, place in 0% weighting group and make sure that it has no due date and the grade posting policy is set to manual. You can then score the group with a number in speedgrader and then export the csv from the gradebook. You will then be able to sort them into groups by the group score given.

3. Prepare the Roster for use with CATME

Once you download your Excel file from Canvas, you'll need to make some adjustments to it before uploading it to CATME.

The file-import action looks for a single "header" line at the top of the input file that maps out where the various required fields are. After the "header" line comes the individual student records. For example:

Go through your file and delete all unnecessary information, such as grades, scores, points possible, Login ID, etc.

For a complete guide to formatting your CSV file, please see CATME Instructor FAQ: General and Technical Questions.

4. Import your roster into CATME

This step is done during step "5. Create a CATME activity" when you create your Peer Evaluations. You will be provided the opportunity to import your CSV file into CATME via your web browser.

Alternative Groups Export Option

Instead of waiting for Canvas to create the functionality to export Group membership information or using workarounds, the IST Office of Learning Design has created a Python script that you can use to export a CSV file that has been formatted for use with CATME.

Creating a Peer Evaluation Survey

There are a number of ways students can receive notifications about upcoming surveys:

FAQ

No. The CATME Peer Evaluations provides a number of questions that faculty can use to build a survey in just a few minutes.

The 7 Default Peer Evaluation Criteria and Questions in CATME:

Click to download a PDF of the first five questions

  1. Contributing to the Team's Work
    A rubric with sample students names along the top with various descriptions of students' performance
  2. Interacting with Teammates
    A rubric with sample students names along the top with various descriptions of students' performance
  3. Keeping the Team on Track
    A rubric with sample students names along the top with various descriptions of students' performance
  4. Expecting Quality
    A rubric with sample students names along the top with various descriptions of students' performance
  5. Having Related Knowledge, Skills, and Abilities
    A rubric with sample students names along the top with various descriptions of students' performance
  6. Team Satisfaction
    Three questions related to team satisfaction with corresponding pull-down menus with various responses
  7. Confidential Comments to Instructor
    A form field that allows students to enter opened-ended comments that are sent to the faculty

For more information, please watch this series of short videos: Five CATME Dimensions Video Training Modules

No. While there are a variety of questions available from the configuration tool, there are no options to add your own questions or edit existing ones.

CATME provides access to both summary and raw data for each survey. You may notice that CATME does not calculate the sum of average scores for each of the questions. Here are some instructions how you can calculate these sums using a spreadsheet application.

  1. log into CATME
  2. click on the "View Results" button in the right-hand column
  3. select the table contents
  4. position your cursor before "Student" in the upper, left-hand corner
  5. click-and-drag down to the bottom, right-hand most cell
  6. copy the table contents
  7. switch to MS Excel (or other spreadsheet application)
  8. paste contents into Excel (try using "Paste special…" to experiment with different ways of adding the content)
  9. insert a new column after "Having KSAs"
  10. label new column "SUM"
  11. use the new column to sum up the scores from the previous five columns
  12. the max score is 25

Rater Practice [RP] is an assignable "game-like simulation" designed to improve student's understanding of the CATME rating scale prior to students doing peer evaluations. RP is assigned individually to a student activity so it can be used multiple times in a term. RP has been assigned to over 7,000 surveys and played over 300,000 times in the past 12 months.

View the Rater Practice video playlist on Youtube

If you'll be working with an instructional designer or a similar specialist that will assist you with the initial setup of CATME in your section, please be prepared to share the following since setting up CATME is still a manual process:

  • How many evaluations you’ll be using
  • How many points (if any) each of the evaluations are worth, and if students get a separate grade just for submitting (optional)
  • When access to each evaluation will be opened and when each evaluation will be due
  • Notification to the designer when the groups have formed and if there are any changes to the groups since those have to be updated in CATME manually before upcoming evaluations are opened
  • Where to find the group compositions, usually under the People > Project Groups tab in Canvas
  • Do you expect anyone with accessibility needs? CATME is not yet accessible (as of early January 2019). There is an alternative form of assessment for those individuals that requires manual data merging after-the-fact.

Situation: "I created a roster and an activity but I made a mistake with my sort and the names are mismatched with the email addresses. Can I just over-write the roster and it will fix the issue? Some students have already logged in. Will it reset their accounts?"

Response: Nothing the students entered into the activity will be lost if you are not changing their email addresses. Click on the activity and reload a corrected file. All entered data will be saved by email address.

The billing should also not be affected since costs are related to the number of student email addresses enrolled with CATME activities.

Addressing Issues with Student Teams and Peer Evaluations

The sections below are from a 2015 Leonhard Center workshop, presented by Dr. Richard Layton, one of the creators of CATME. During his presentation, he discusses how he uses CATME to address issues with student teams and peer evaluations.

Forming Student Teams

Peer Evaluations

Additional resources on CATME’s Teamwork Model

CATME currently has three modules on SMARTER Teamwork that can help faculty to better understand how they can improve teamwork in their classes.

  • CATME Teamwork Dimension Modules
  • CATME Rating Scale
  • Rater Practice