Let's start with a distinction between testing (Canvas beta) and piloting (Canvas production). In both cases, you will want to test and train on functions in the list below.
Note: When you move to production, your data collection begins. This is flipping the switch on. Your data dashboards will start at this moment, moving forward. So if your goal is to capture longitudinal data, consider turning on Insights sooner and have it running in the background as you prepare faculty and staff to integrate it into your process as appropriate, at your pace. Some of these functions may be immediately applicable to the work your faculty and staff are doing; some of these may be adopted a year or more later after the pilot. Some of these things you may not pursue at your institution.
Whatever the case may be, consider testing as much as you can to make sure that at least a small team at your institution knows the full capability and can help identify when and where staff can start utilizing it more, at your pace. *And so that you can avoid purchasing duplicate capabilities down the road.
The functions you will want to test include:
- Academic Subject Code
- Terms
- Cycles & Goals
- Cycle Type
- Assignment Types
- Institutional Hierarchy
- Course Code Pattern (Course Levels)
- Users
- Canvas Default Rating Scale (Mastery Levels)
- Assessment Plan
- Assessment Results (Chart)
- Findings and Recommendations
- Reflections and Actions
- Action Type
- ILOs (including Categories)
- Programs
- PLOs (including Categories)
- Course Groups
- Course Group Type
- Outcomes on Analyze Page (CLOs)
- Curriculum Mappings
The actions you will want to test by user role include:
- User roles and assigned subjects
- Action type
- Course group type
- Assignment type
- Cycle type
- Course code pattern setting
- Institutional hierarchy
- Programs and course groups
- ILOs
- PLOs
- Canvas default rating scale
- Assessment
- Approving course learning outcomes
- Curriculum mapping
- Reflections and actions
Testing in Canvas beta
Most likely, your first experiences with Insights will be in your Canvas beta. If you are part of a cohort at your institution, you will stay in beta for at least the 8 weeks of the cohort experience. We recommend not rushing your beta testing and stay there until you have tested the features and completed planning your rollout and adoption, regardless of where you are in your cohort experience.
Pros: In Canvas beta, you can test out all of the functions and actions, within your environment, and with your Canvas data. Your Insights settings can be established without interruption by the Canvas beta refresh. Additionally, all data that was retrieved from Canvas to populate Insights will stay in Insights even during Canvas beta refreshes.
Cons: Once you begin, you will have until Saturday of that week to test before Canvas will refresh your beta. When Canvas refreshes everyone's beta environment, you will lose the data that was pushed into Canvas beta by Insights; for example, any Signature Assignments you created in an Insights Assessment Plan will be lost. You will also lose any changes or edits you made in Canvas beta that do not exist in your Canvas production.
Takeaway: Plan the window in which you want to test ahead of time. If you need to do the testing across multiple Canvas beta cycles, try to split your testing tasks into logical categories to minimize any feelings of re-doing work in Canvas.
Who can access this? The team involved with onboarding Insights and optionally, if you decide it is appropriate, a very small subset of faculty. Consider limiting the size and scope of people involved because of the nature of Canvas' weekly beta refresh schedule.
Continue phasing and pilot in Canvas production
Pros: In Canvas production, you can continue rolling out the implementation in phases around your planned training and adoption. Here, you can pilot, practice, and train on all functions and actions, within your environment, and with your Canvas data and live events. Proceed with caution if you are doing anything completely new or for the first time. Consider doing it first in beta if you haven’t already done so, even after moving to production.
Note: When you move to production, your data collection begins. This is flipping the switch on. Your data dashboards will start at this moment, moving forward.
If your goal is to capture longitudinal data, consider turning on Insights sooner and have it running in the background as you prepare faculty and staff to integrate it into your process as appropriate, at your pace.
Cons: There are two things to keep in mind. These two items have a high impact on mastery data and on configuring the data dashboards.
First, after you start using Insights in production, even just for piloting, the course code pattern cannot be modified later.
Second, this isn’t a con, it is simply a caution to be aware of: taxonomy. You can add additional taxonomies, but you cannot modify an existing taxonomy. This is specifically to retain the integrity of your longitudinal data.
You can add additional taxonomies, knowing that your longitudinal data will reflect multiple taxonomies from the point of addition, forward.
For this reason, we highly recommend spending at least one full week testing in your Canvas beta.
Who can access this? You have the flexibility to grant access to whomever you choose, whenever you choose. Your LMS administrator will be able to adjust the javascript at the account or subaccount levels to show or hide the Insights button in the navigation bar. When you are ready, you have the flexibility to add coordinator roles as you see fit. If someone is not currently utilizing Insights, a message will be displayed prompting the user to request access when they select "Insights".
What about an Insights sandbox?
Ideally, you will never need to use our sandbox and can go right into testing in your beta. In an emergency, know that this option can be available for a fee if you need it.
Pros: In the Insights sandbox, you can test out all of the actions with Insights demo data. This is simply a demo space to play around with settings and views and practice doing the actions.
Cons: Your ability to see the functionality will be limited by virtue of your own use of Canvas and the customizations you may have within your own Canvas environment. In other words, your instance of Canvas may look slightly or significantly different than ours here at Insights.
Who can access this? The team involved with onboarding Insights can request access to an Insights sandbox. This might look like the Canvas LMS administrator alone, or it might look like the LMS admin and 1-2 assessment / curriculum staff. We recommend not adding faculty to the Insights sandbox because it will not look like the customized environment your faculty are used to experiencing in Canvas.
Related articles:
How do we move Insights from beta to production?
Beta or Prod? What should we keep in mind as we plan our testing and piloting?