Beta or Prod? What should we consider as we plan our testing and piloting of Insights?

Turning on Insights is easy, like a flip of a switch. What can be challenging with any new technology is the actual rollout and adoption of a new tool. We want to provide you with the tools necessary to succeed with your Insights rollout.

One of the best ways to overcome common technology implementation challenges is a clear plan for testing and piloting before rollout. This enables you to include all of your stakeholders, faculty and staff in a way that is intentional, designed to your existing process or desired process, and safe in regards to your live data.

Your testing of Insights will happen in Canvas beta, an isolated area that will not impact your live data or events.

As you test in Canvas Beta, you can try features out without disrupting your current processes and, at the same time, have deeper internal planning sessions on when and how to roll out the features in Canvas production.

Even as you transition to your production environment, you can determine who you initially want to have using Insights, when, and for what purpose. You are in the driver's seat.

Three key aspects to consider as you plan

Once you reach this stage, it is important to maintain momentum during your testing and piloting phases. Here are three main aspects that will need to stay at the forefront of your preparation.

Your environment requirements. Always follow your institution's requirements and recommendations first and foremost. Know your institution's IT requirements that will impact your strategy for implementation and adoption, especially when it comes to testing and piloting tools that access Canvas Live Events.
Canvas beta refresh. Canvas beta refreshes every Saturday and features you are testing may be impacted or lost by that refresh. Read below for specifics on what is effected with each Canvas beta refresh. Here is a link to the Canvas release schedule to help you plan specific dates to decide what and when to test when and to help you set expectations with your staff.
Insights Canvas Intake. This is the initial set up that forms the connection through which Insights will pull live data from Canvas. Depending on how you use Canvas today and how your data is organized, it may need to be tested more than once ensure accuracy. Read below for specifics on what is effected with each Insights Canvas Intake. Once this functions as expected, the Canvas sync will be in place and will not need to be reset again.

Pros and Cons to testing in each respective location:
Canvas beta, Canvas production

Let's start with what to test in your beta environment as you progress in your implementation process. Some of these recommendations may be immediately applicable to the work your faculty and staff are doing; some of these may be adopted a year or more later after your initial pilot. Regardless of the timeline, it is crucial to thoroughly test all aspects to ensure that a small team within your institution comprehends the complete capabilities and can assist in determining when staff should start utilizing these features, at a pace that suits your institution.

The following are recommended to be tested/confirmed:

Under the Setup Area:

Users & Roles (Assessment Coordinator)

Users - Add roles (Click on the roles to access the dropdown menu) and assign Academic Subjects to users as an Assessment Coordinator.

Data Interpretation

Course Code Pattern (Course Levels) - Confirm your course codes and levels.
Term management - Confirm you have the terms in insights set up correctly.

Institutional Organization

Institutional Hierarchy - Is your institutional hierarchy organized properly?
Academic Subject Code - Do all academic subjects have the correct code assigned to them?
Programs - Are your programs named correctly and in the correct organizational area?
Course Group Type - Do you have your course group types defined?
Course Groups - Are your course groups set up for your programs?

Curriculum Map (Assessment Coordinator - For PLOs Program Coordinator)

ILOs (including Categories) - Have your ILOs been entered and approved?
PLOs (including Categories) - Have your PLOs been entered and approved?

Assessment (Assessment Coordinator and Program Coordinator)

Canvas Default Rating Scale (Mastery Levels) - Have you aligned your rating scales to your Institutional Mastery Levels?
Cycles & Goals - Ensure you can add an assessment cycle with goals.
Cycle Type - Define your Cycle types.
Assignment Types - Define your assignment types.
Plan Templates - Be sure you can create an assessment plan and an assessment plan template.

Under the Outcomes Area (Assessment Coordinators and Program Coordinators)

CSLOs - Make sure all CSLOs are approved.
Test Updating CSLOs.

Under the Mapping Area (Assessment Coordinators and Program Coordinators)

Ensure that mapping is complete or updated if necessary.

Under the Plan Area (Assessment Coordinators and Program Coordinators)

Ensure you can create an assessment plan (if you have them from a template)
Ensure you can deploy your plan.
Make sure you can enter Findings and Recommendations.

For Faculty

Are there Outcomes in Canvas?
Is there at least one Rubric in Canvas?
Are assessments deployed from Assessment Plans?
Be sure they are seeing their assessment results.
Be sure they can see the outcomes and see unapproved outcomes.
Make sure you can enter reflections and actions.
Make sure you have the correct action types.