Plan > Assessment Plans
What is Juried Assessment?
Create a Juried Assessment Plan.
Deploy a Juried Assessment Plan.
Notify Assessors to complete Juried Assessment.
Execute Assessor Workflow of Juried Assessment.
Analyze Juried Assessment results.
Juried Assessment
Insights allows Coordinators to facilitate Juried Assessments—an evaluation process where student work or academic achievements are agnostic to their collection context by a panel of experts (jury of assessors). Assessors review the student submitted work independently, assessing it based on established criteria and standards and provide feedback and assign scores.
Juried assessment gathers artifacts from learners, and the learning outcome is assessed by the jury. Learners’ personal information is removed (anonymized), in order to make more wholistic determinations of student progress on learning outcomes and conduct rubric norming.
Norming qualifies what the group agrees on is an artifact that represents mastery, one that represents middle range of mastery, and one that lacks mastery according to the rubric criteria. Artifacts are then distributed to the group to be rated on the rubric, and the ratings averaged to determine the final points. Like course-embedded assessment, the results are aggregated and used to improve students' experiences and to satisfy accreditation requirements.
Create a Juried Assessment Plan
To create a Juried Assessment Plan, go into Insights and go to Plan > Assessment Plan. Click the blue button to + Create Assessment Plan.
PREREQUISITE: Juried assessments in Insights must be part of a continuous Assessment Plan process (and therefore belong to a specific Cycle)—so a prerequisite to creating a Juried Assessment Plan is to ensure a Cycle has been created in Insights before proceeding with the creation of a plan. Edit an existing cycle listed in the Assessment Plan screen to add a juried assessment to that cycle; or, create a new cycle. See article: How do I create Cycles and Goals for Assessment Plan creation?
In the modal window Plan Assessments, click the Cycle dropdown and select the cycle for this juried assessment; then click + Add Assessment.
NOTE: Only cycles that are not yet associated with an assessment plan will be listed in the Cycle dropdown.
To create a juried assessment within a cycle where an assessment plan already exists, locate the planned assessment from the list on screen, click the pencil icon to Edit; then choose + Add Assessment to create an additional assessment plan.
In the modal window Add Assessment, complete the fields according to the steps defined below:
- Type of Measure: select Canvas Assessment because this will be an assessment where an assignment is embedded into the Canvas course; and click the toggle to Collect Submissions for Juried Assessment (green check mark indicates activated). Once selected, the modal window updates with necessary fields for creating a juried assessment plan.
- Assignment Description: (optional) enter a description or purpose of the juried assessment being created.
- Collect results for: check the button if this juried assessment is for Program or Institutional Assessment.
- Programs: if Program Assessment was selected in Step 3, click the Programs dropdown to choose the Program (more than one program may be selected).
- Learning Outcome Type: choose ILO or PLO—if Institutional Assessment was selected in Step 3 (Program is not applicable) and ILO (for Institutional Learning Outcome) auto populates in the Learning Outcome Type dropdown.
- For Mastery of: Category and/or Learning Outcome (optional): select an outcome category to auto populate the Learning Outcome dropdown (to the right). Select the Learning Outcome you want the assessment to measure. Multiple Learning Outcomes may be selected.
- Collection Terms: click the dropdown to select the desired term(s) for collection of data.
- Goal: select a goal to be assessed (these were added to the cycle when it was created from Setup > Assessment > Cycles & Goals).
- Objective: (optional) select an objective from the dropdown.
- Departments: select the subject codes for this assessment, more than one code can be selected.
- Course Levels: select the course levels for this assessment.
- Assignment Type: choose the assignment type that should be assessed.
- Canvas Submission Type: File Upload is the default method of assignment submission for juried assessment.
- Course Preview: Click the right chevron (>) icon to expand the view to see courses related to criteria selected above. Scroll down to review the list of courses and uncheck any that should not be included in this juried assessment.
-
- Assessing Terms: select the terms for this juried assessment. (There may be multiple terms selected if you would like to juried assess at different time-points in order to track progress towards a goal. The artifacts assessed in a given Juried Assessment are all of those collected prior to the current Juried Assessment term, and, in the case of previous Juried Assessment terms, after and inclusive of the previous term.)
- Select Mastery Rating Scales: select the rating scale that should align to this assessment. Once selected, it will display below. You have the option of editing the rubric criteria for the ILOs or PLOs that will be juried assessed.
- This criteria change will apply to all other Insights assessments of that Learning Outcome moving forward (in Canvas or in Juried Assessment).
- We currently only support Juried Assessment of ILOs and PLOs.
- Assessors: Enter the email address of your first assessor, then click + Add, to display an additional field to continue adding email addresses of additional assessors.*
*Upload Assessors: this is a link to upload a CSV file of multiple assessors' email addresses. When clicked, be sure to download the excel template and then upload your CSV file with the email addresses.
When done, click Add Assessment.
In the modal window for Plan Assessments, the Juried Assessment details are displayed for review and the plan is flagged as a Juried Assessment. Click the blue button Create New Plan and the plan will then be listed on screen under Plan Assessments.
A green confirmation message will display: Assessment plan successfully created.
Deploy Assessment Plan to Canvas
Once assessments are planned, Coordinators can view the plan details on the Plan Assessments. Click the up arrow icon in the assessment plan to deploy to Canvas.
- Click the right chevron (>) icon to expand the assessment plan specifics and view the current status.
- Click the pencil icon to edit/view plan details.
- Click the Trash icon to delete the assessment plan.
- Click the up arrow icon to deploy the assessment to Canvas.
Deploying a juried assessment plan creates an unpublished assignment shell in the Faculty’s Canvas course, for faculty to edit and publish according to their expertise and needs.
When students make submissions on an assignment that was created for Juried Assessment, Insights collects the submissions to enable Juried Assessment of each submission independent of the course grading context (this is in addition to collecting outcome results for that assignment once faculty grade the Canvas rubric). See next step to notify assessors to complete Juried Assessment.
Notify Assessors to Complete Juried Assessment
Assessments have been planned in Insights; assignments have been deployed to Canvas; Faculty have published those assignments to learners; and, students have submitted assignments for grading/scoring.
Now, Coordinators, have reached a Juried Assessment term will notify the assessors (from email addresses that were added to the Juried Assessment Details). Go to Plan Assessments (Insights > Plan > Assessment Plans) and locate the Juried Assessment. Click the pencil icon to edit the Assessment Plan.
In the modal window for Plan Assessments, the Juried Assessment details are displayed and the plan is flagged as a Juried Assessment. Under the section Terms to be Assessed, click the email icon (highlighted below) to Send out Juried Assessment to assessors (jurors).
Assessors (the email addresses entered into the Juried Assessment Details when the plan was created) will receive an email with a private URL (PURL*) to access the juried assessment scoring screen for assessors (including those without Canvas access). See the Assessor workflow outlined below.
*PURL = a private URL designed for preservation of specific resource. In this case, it provides assessors access to the juried assessment scoring.
Execute Assessor Workflow of Juried Assessment
Assessors will receive an email notification like this one shown below. They will click the private URL link in the email.
- After being prompted to confirm their email address, the PURL will open to a page that looks like Insights (but can be accessed without being logged in as an Insights or Canvas user).
- Assessors can download the unidentifiable student evidence file (identity may be still be visible within the assignment) and assess the evidence against the rubric by clicking the associated score’s cell.
- Using the navigation at the top, click through each student using and complete scoring accordingly.
- When done, click Finalize and Submit. Upon page refresh or re-login, a new version of the rating page, including results as compared to the average across all raters, as well as an area to record findings and recommendations relative to the Juried Assessment instance will be available. These results, findings, and recommendations will also be available once assessment is complete on the Results page of the coordinator dashboard for the Cycle in question.
Analyze Juried Assessment Results
Coordinators can view and analyze juried assessment results by cycle on their Insights dashboard. Go into Insights and select the Dashboard from the menu. In the Cycle field select the cycle of the Juried Assessment to be reviewed, and click Search.
Update Subject Code filter and click Apply to show overall course-embedded assessment data if you desire that context; then scroll down to the Assessment Plan section and click View Cycle Results to specifically see Juried Assessment data. 
Juried Assessment results display in the second graph.
- Coordinators can analyze this data compared to other course-embedded assessment results (the graph above Juried Assessment Results)
- Hover over any section of the graph to see specific details about the results.
- To view/edit findings and recommendations made on the specific Juried Assessment select Show Filters and toggle to display findings and recommendations for each objective.
Insights expands on the traditional Canvas Outcomes experience by supporting both course-embedded and juried assessment. With juried assessment, Insights allows assessors (jurors) to easily and efficiently view artifacts (evidence) and evaluate both individually and as a group per their preference.