Click here to download the Quizzical Instructor Guide
1. Activating your Course
To enable Quizzical in Quercus, please follow this guide.
Quizzical is integrated into Portal. Every course can have one Quizzical instance. Student and TA information is passed on from Portal to the Quizzical course.
Click on the Quizzical link in your Portal Account to get into the Quizzical system. Click Activate Course if you are launching Quizzical for the first time.
To begin engaging students in Quizzical activities, you need to complete your course setup. This includes:
- Scheduling Course Lectures
- Configuring the Teaching Assistant’s Load
- Adjusting your Course Settings
- Finalizing Course Section
Scheduling Course Lectures
To start using Quizzical you first need to schedule your lectures. Scheduling allows you to select the course lecture dates for which students need to submit assignments, and set the number of assignments that each student needs to submit throughout the course. As a rough guide, it takes on average about 5 minutes for you/TA to grade questions, so this time commitment should be considered when deciding on the number of questions your students will author, and it will help you calculate TA contracts.
1. Click the Scheduling tab
Update the following fields to configure your Quizzical course schedule.
i) Date Range: The start and end dates of your Quizzical course. These might be the same as the session start/end dates, or some time frame in between during which you want to engage your students in Quizzical activities.
ii) Week Days: The days of the week you have your lectures.
iii) Time: The time of the day your lecture starts.
Once you enter this information, the calendar months for the selected date range will appear with the week days highlighted. You can deselect any days on which your class/lecture will be cancelled, which might include holidays, or other irregularities when a lecture will not occur (e.g. days on which you are at a conference).
In the screenshot above, the instructor scheduled his lectures to be every Monday and Wednesday between March 5 to June 1. She knows that on March 23 to March 25 she will be away on a conference, so she deselected the two days.
Note: Make sure to deselect lectures that fall on Public or University holidays. These will not be applied automatically.
iv) Assignments per Student: The number of assignments each student should submit throughout the course. In the example above, the course has 24 different lectures and the number of assignment per student is 4. Scheduling will randomly assign 4 of the 24 lectures to each student.
Note: The maximum number of assignments a student can have in Quizzical course is 10.
v) Questions per Assignment: The number of questions a student should submit in each assignment. In this example, the student needs to submit 2 questions per assignment. Generally, you would have students submit one question per assignment, and assign them to write a number of questions throughout the term (they will be randomly assigned, and spread out). In some cases you may wish to have students assigned each lecture date with multiple questions per assignment. Be sure to keep in mind the TA hours and grading load to provide effective and timely feedback on these questions. The maximum number of questions a student can submit in each assignment is 5.
vi) Days Allocated for Question Composition: The number of days after a lecture date during which the student should submit an assignment. The maximum number of days that can be allocated for an assignment is 24 days. Consider setting the days limit to around 3 or 4 in order to get submissions soon after each lecture and thus a consistently updated quiz pool that matches the participation marks (see below) for quiz taking.
2. Click Setup Lectures
The Schedule Details shows
- The number of students enrolled in your Quizzical course
- Total number of lectures you have throughout the course
- The total number of assignments (number of students x number of assignments per student)
The Lecture View shows the lecture dates, time, and the number of students assigned to submit questions for the lecture. In the action column you can
- Click (x) to remove a lecture from the list.
- Click (+) to add another lecture to the list. The new lecture is added with the same date as the lecture row you were at but with no students assigned. Adjust the lecture date as needed. This may be useful if you have two classes on one day (e.g. a lecture and a tutorial and you want questions for each of these).
After adding/removing lectures, click Distribute Students to redistribute students to lectures according to the configuration values you entered.
The Student View tab shows the student list and the lecture assignment of each student.
Click Reset Calendar on the top right-hand corner of the page to clear your selections and start over.
Click Save the Schedule to finish configuring your lectures and assignments.
Configuring the Teaching Assistant’s Load
Click the Teaching Assistants tab to view the Teaching Assistants (TAs) enrolled in your course.
The page displays the total number of assignments that should be submitted throughout the Quizzical course. The page also displays the names and UTORids of the TAs enrolled in your course.
You can assign different loads to the different TAs, but the sum of all loads should be 100%. The load percentage is the percentage of assignments that will assigned to each TA to review and grade.
The TA load is calculated on a rolling weekly basis. This allows you to dynamically adjust the load percentage throughout the term, and Quizzical will apply it as submitted questions on forthcoming lectures are distributed for grading.
Adjusting your Course Settings
The course settings are available under the Settings tab.
i) Enable Equation Editor: Enabling the equation editor will display the equation editor to students in the question box to allow them to add equations to their questions
ii) Question must contain figure: If set to “No” the student may choose whether or not to select a figure to include in a question. If set to “Yes” the student must select a figure to include in a question. Figures must be added first by the instructor. See Adding Figuresfor guidelines about how to add figures to your Quizzical course. Keep in mind that the figures are not shown with the question, but rather are shown after the quiz taker has submitted their answer, to help them to visualize why they were correct or incorrect
iii) Question must contain reference: If set to “No” the student may choose whether or not to include a reference to a page number or figure number in a question. If set to “Yes” the student must include a reference in question.
iv) Answer/Distractor must contain complete justification: If set to “No” the student may choose whether or not to include a justification for correct and incorrect answer choices. If set to “Yes” the student must include a justification for all the answer choices given in a question. Including justifications was a founding principal of Quizzical and it is suggested that you have the students articulate why answers are correct and distractors are not. When students see the answers to their quiz, these justifications (and any reference and/or image) are shown, enabling them to better understand the topic
v) Enable direct rejection by TA: If set to “No” a TA can only either accept a question or sent it back to the student to review. If set to “Yes” a TA can also choose to reject a question.
vi) Days Allotted for resubmission: The number of days given to a student to resubmit a question that was returned back for review by the instructor or the TA. If the number of days passed without resubmitting the question, the question will be in an expired state. It is suggested to make this number 2 or 3, so that if the revised question is approved it will enter the quiz pool in a timely fashion.
vii) Maximum score for each student authored question: The maximum mark that a student can receive for a submitted question. You may wish to consider awarding marks for submitting on time, for correctly classifying their question as recall/application, and/or for associating a relevant figure or reference with their question. The rubric you develop will need to be rigorous to ensure that all of your TAs are following the same logic.
viii) Minimum grade for questions to be selected in a quiz: The minimum grade that qualifies a student-submitted question to be added to the quiz question pool. For example, if the minimum grade is set to 2.5, only questions that scored 2.5 and above will be added to the quiz pool. We suggest a cut off of 3.5 for questions worth 4 marks (~75% score). This will ensure that only high-quality questions move into the quiz pool.
ix) Questions per quiz: The number of questions included in a quiz. We recommend 4 or 5.
x) Time allowed for each question during quiz: The number of minutes/seconds given for each question. This is only used as an indicator for the time a student should spend on the question. A warning message will be displayed once the time elapses, but the student will still be able to answer the question. You may wish to set this to mimic the average time that they would have to answer a question in a formal exam situation.
xi) Total number of participation marks available: The marks a student receives if she has participated in a quiz. Participation marks are received if a student answers the minimum number of questions required (see xiii), receives a minimum score for the answered questions (see xiv), and attempts the quiz within the set time frame (see xii).
xii) Number of days after a lecture to attempt quizzes for participation marks: The number of days during which a student can attempt a quiz in order receive the participation marks. If the number of days is set to 5 and the student attempts the quiz 6 days after the lecture, she will not receive any participation marks, even if she answers all questions correctly. This is designed as a participation incentive to encourage students to engage in active self-study. Keep in mind: if students are allotted 4 days to submit, and it takes the grader 2-3 days to grade the question, there will be no or few questions in the quiz bank for 6-7 days after the lecture. We typically set this value to 14, so that on average, students have about a week after the lecture to attempt questions for the participation marks.
xiii) Minimum number of questions to attempt for full participation marks per lecture: The number of questions a student should answer in a quiz in order to receive the participation marks for the lecture.
xiv) Minimum percentage score for questions to count for quiz participation marks: The overall minimum percentage a student should receive on a quiz in order to receive the quiz participation marks. Historically we have used 60%.
Finalizing Course Activation
Now that you have setup your course, click the Overview tab.
Click Activate the Section to finalize your course activation
Your Quizzical course is now active and accessible to the students.
2. Editing Course Schedule
Now that your course is active, the Scheduling tab will display your lecture dates and the number of students assigned to each lecture.
Click the wrench icon beside a lecture to change the lecture date/time.
Click the trash icon beside a lecture to cancel the lecture.
Note: a cancelled lecture cannot be added back.
Click the arrow beside the Number of Students in a lecture to display the student names. Click the wrench icon beside a student’s name to re-assign the student to another lecture.
3. Adding Figures
An instructor can require students to include figures in any question they submit (see course settings).
The instructor is responsible for uploading all figures that students might need to include in their questions. Typically, if there is a textbook for the course, the publisher will make available all jpg images and these can be easily added to the appropriate folders (e.g. lecture 4).
Click the Figures tab to add figures to your Quizzical course.
All figures are uploaded into folders. Click Create New Folder to create a new folder, or select an existing folder to drop in your selected file. Click Delete to delete a selected figure or folder.
4. Viewing Student Submissions
Click the Students tab to view the list of students enrolled in your Quizzical course.
The list displays the UTORid, name, and the number of assignments submitted by every student enrolled in the course.
Click on a student’s name to view a summary of the student’s activity in the Quizzical course. This includes
i) Submitted Questions
Submitted questions are those submitted by the student but which have not yet been accepted or rejected by the TA or instructor.
Click Review to go to the question details page where you accept, grade, or reject a question, or send it back to the student for review.
- You should only provide a grade if you are accepting a question.
- Only instructors can reject a question on 1st submission. TAs should provide constructive feedback and send it back for review
- Only questions that with a grade equal to or greater than the minimum grade set in the course settings will be added to the quiz pool
ii) Accepted Questions
Accepted questions are those that received a score equal to or greater than the minimum score set in the course settings. They are available in the quiz pool and can appear in quizzes.
Click Review to review the question, or Remove to remove the question from the quiz pool.
iii) Rejected Questions
Rejected questions are those that have been either been rejected (Rejected Status) by the TA or instructor, or sent back to the student to review (Requires Editing Status).
Click Review to review the question, or Add to directly add the question to the quiz pool.
iv) Expired Questions
Expired questions are questions that have not been submitted by the student during the assigned time frame.
If you wish to allow the student more days to submit to question, enter the number of Extend Days and click Submit. The number of extend days is calculated starting from the current date.
5. Reviewing Course Questions
The Questions tab displays a list of questions that were submitted by students in your Quizzical course. Questions are classified as Submitted, Accepted, Rejected, or Expired.
6. Reviewing the Question Pool
The Question Pool tab displays the questions that are in the quiz pool, grouped by lecture.
Click View to view the question details. You can choose to override the grade, reject the question, or send it back for review.
Click Remove to remove the question from the quiz pool.
7. Viewing Quiz Attempt Reports
The Reports tab displays a summary of all questions in the quiz pool that have been attempted by students in quizzes.
Click Export Grade Data to download a csv file containing data about the grades received by students for assignment submission and participation
Note: Currently, the Export in BB Format button does not provide any useful function.
8. Viewing Course Details and Progress (Course Overview)
The Overview tab displays
i. Scheduling details
- Number of lectures
- Number of assignments/lecture.
ii. Alerts: for example, a live course will display
A course that has NOT yet been configured will display
iii. Class Statistics
- Question Pool Statistics for specific lectures
Note that these can be filtered by date and one can click on/off the graphics with regard to valid participation marks, percent correct, and number of questions attempted.
The icon in the upper right allows downloads of the graph as a jpg, pdf, svg or png file.
- Teaching Assistant Progress for specific dates
The instructor can view the number of questions in progress, finished, or requiring editing. Hovering over the bars identifies TA by name.
Note that this allows the instructor to monitor TA activity and promote communication to ensure timely grading.
- Class Progress for specific dates
Note: Hover over bar graph elements for more details.