How do I create juried assessments in Insights?

What is Juried Assessment?

Insights allows Coordinators to facilitate Juried Assessments—an evaluation process where student work or academic achievements are reviewed independently of their collection context by a panel of experts (a jury of assessors). Assessors review student-submitted work independently, evaluating it against established criteria and standards, and provide feedback and scores.

Juried assessment gathers artifacts from learners and evaluates learning outcomes through the jury. Learners' personal information is anonymized to support more holistic determinations of student progress on learning outcomes and to conduct rubric norming.

Rubric Norming qualifies what the group agrees represents:

• Mastery of the artifact
• A middle range of mastery
• A lack of mastery according to the rubric criteria

Artifacts are then distributed to the group to be rated on the rubric, and ratings are averaged to determine final scores. Like course-embedded assessment, results are aggregated and used to improve student experiences and to satisfy accreditation requirements.

Complete the Canvas Assessment Modal

In the Add Canvas Assessment modal window, complete the following fields:

Complete Juried Assessment Details

Scroll down to the Juried Assessment Details section and complete the following:

A. Academic Years and Assessing Terms

• Select the academic year(s) and assessing term(s) for this juried assessment.
• Multiple terms may be selected to track progress toward a goal over time.
• Artifacts assessed in a given Juried Assessment term are collected prior to the current term, and—if previous Juried Assessment terms exist—after and inclusive of the previous term.

B. Select Mastery Rating Scales

• Select the rating scale that aligns with this assessment. Once selected, it will display below.
• You may edit the rubric criteria for the ILOs or PLOs being juried assessed.

C. Assessors

• Enter the email address of each assessor, clicking + Add after each entry to display an additional field.
• Upload Assessors (CSV): Click the upload link to download the Excel template and upload a CSV file containing multiple assessors' email addresses.

D. Save Rubric Criteria

• Update any of the rubric criteria by clicking the criteria description and editing it in the modal window. When done editing click save.
  
• Once you have updated and saved the descriptions, click "Save Criteria" for the rubric. (This is critical before completeing the Juried assessment).

When all fields are complete, click Add Assessment.

Once a Juried Assessment has been deployed

When students submit assignments created for Juried Assessment, Insights collects those submissions to enable evaluation of each submission independent of the course grading context. This occurs in addition to collecting outcome results for that assignment once faculty grade the Canvas rubric.

Notify Assessors to Complete Juried Assessment

Notify Assessors to Complete Juried Assessment

At this stage:

• ✅ Assessments have been planned in Insights
• ✅ Assignments have been deployed to Canvas
• ✅ Faculty have published assignments to learners
• ✅ Students have submitted assignments for grading/scoring

Now, when Coordinators reach a Juried Assessment term, they will notify the assessors.

Steps to Notify Assessors

1. Navigate to Insights > Plan > Assessment Plans.
2. Locate the Juried Assessment and click the pencil icon to edit the Assessment Plan.
3. On the Create Plan page, scroll to the Manage Juried Assessment section.
4. Click the chevron to expand the Manage Juried Assessment view.
5. Click the email icon to Send Out Juried Assessment to assessors (jurors).
   

Assessors (whose email addresses were entered in the Juried Assessment Details during plan creation) will receive an email containing a Private URL (PURL) to access the juried assessment scoring screen. This link is accessible to assessors with or without Canvas access.

Execute the Assessor Workflow of Juried Assessment

Access the Scoring Screen

Assessors receive an email notification containing their Private URL. Upon clicking the link, they will be prompted to confirm their email address.

Once inside the scoring interface, assessors can:

• Download the student evidence file (note: student identity may still be visible within the assignment content itself)

• Assess the evidence against the rubric by clicking the appropriate score cell

• Navigate between students using the navigation at the top of the page

• Complete scoring for each student submission

Finalize and Submit

When all submissions have been scored, click Finalize and Submit.

Upon page refresh or re-login, assessors will have access to an updated view that includes:

• Results compared to the average across all raters
• An area to record findings and recommendations relative to the Juried Assessment instance

Analyze Juried Assessment Results

Coordinators can view and analyze juried assessment results by cycle directly from the Insights Dashboard.

Coordinators: Access the Dashboard

1. In Insights, navigate to the Dashboard from the main menu.

2. In the Cycle field, select the cycle of the Juried Assessment to be reviewed.

3. Click Search.

Apply Filters (Optional)

• Update the Subject Code filter and click Apply to display overall course-embedded assessment data for additional context.

• Scroll down to the Assessment Plan section.

• Click View Cycle Results to see Juried Assessment data specifically.

Faculty and Coordinators: Review Results

The Juried Assessment results are displayed in the second graph on the results page.

From this view, Coordinators can:

Summary

Insights expands on the traditional Canvas Outcomes experience by supporting both course-embedded and juried assessments. With juried assessment, Insights enables assessors (jurors) to efficiently view artifacts and evaluate submissions both individually and collectively—supporting holistic, normed evaluation of student learning outcomes at the program and institutional level.