Multiple Choice Analyzer

How many instructors revise their multiple-choice tests based on their students’ responses? If one wishes to make a better multiple-choice test, a statistical analysis of these responses could be helpful.

We have developed a freely available web-based program to analyze a multiple-choice test, and it requires no programming experience – just your student response data and the answer key in two excel spreadsheets.

The program outputs a pdf file that goes beyond the usual scoring but includes many other metrics, which would help the instructor to revise the test for improving its reliability.

The outputs include difficulty index, discrimination index, Cronbach alpha, item response theory, and distractor analysis. It also outputs if a question should be kept or removed based on four different criteria. The source code is open-access, and we invite others to improve it.

The description is given here at https://www.garrickadenbuie.com/project/mc-test-analysis/

The program can be accessed at https://apps.garrickadenbuie.com/mctestanalysis/

How to make the input excel files is at http://www.eng.usf.edu/~kaw/MCTestAnalysis/MCTestAnalysis_input.pdf

An example of a hypothetical output report is here at https://www.garrickadenbuie.com/project/2017-07-06-mc-test-analysis_files/MCTestAnalysis_Example-Report.pdf

The source code for developers is at https://github.com/gadenbuie/mctestanalysis

_________________

This post is brought to you by

An Example of Doing Learner Introductions in an Online Class

An Example of Doing Learner Introductions in an Online Class

Getting coherent, complete, and thoughtful learner introductions in an online class requires intentional prompting and guidance.

By Joanna Bartell and Autar Kaw

September 2, 2020

Whether we are teaching online, face-to-face, or somewhere in between, having students introduce themselves is essential to the community building process for a class. When students know something about other students, it is easier for students to communicate with each other as well as with the instructor.

I have been teaching a core lecture class in Numerical Methods for three decades now. Although I am well-versed in several pedagogies, including hybrid, blended, and flipped learning, Numerical Methods had never been taught entirely online before the Summer 2020 semester. I come to know students by remembering their names, asking for their name when a student asks questions, or while conducting in-class active learning exercises in a flipped classroom.

In summer 2020, because of the COVID19 crisis, I had to teach the course online. I chose to give the “learner introduction” assignment to students to get to know them better as well to get students to know their peers.

The prompt of the assignment read as follows.

Since Jed and I have introduced ourselves, we would like you to do the same now by the end of the first week of classes. It will make it easier to communicate with you and your classmates. Say something about yourself (nothing personal), and what you expect from the course. About 50-100 words would be sufficient.

and the grading rubric was as follows.

You already have two samples of introduction, and you are not asked to give a lengthy introduction.
5 points for a reasonable introduction
2.5 points for less than a reasonable introduction.
0 points for an irrelevant introduction.

Although all students participated in this low stakes assignment (about 0.25% points), several responses were rich and gave an extended introduction full of anecdotes, but many of them were not uniform, lacked coherency, and some used instant messaging dialect and included several spelling and grammar mistakes.

Dr. Joanna Bartell is an instructor of Communication in the College of Engineering (CoE) at the University of South Florida, working to develop the CoE’s integrated communication program. Before joining the CoE in 2018, Dr. Bartell taught core communication courses for over a decade, including Interpersonal Communication and Persuasion courses. She is also well-versed in several pedagogies, including writing center and feminist pedagogy, which take highly individual, student-centered approaches. Who better to ask than Joanna about what can be done to elicit a better response from students.

I contacted Joanna, and she shared some of her experiences and suggestions, which she has outlined below:

Consider your request:
The learner introduction is a situating assignment, where you’re asking students to 1) talk a little bit about themselves, and 2) both understand and articulate their expectations. These are significant assignments and can help students understand and manage their expectations and involvement in the class. The reciprocity mechanism you include at the beginning of the prompt is helpful (“We opened up, now it’s your turn”) for getting students to respond to and mimic your introductory offering.

In your learner introduction assignment, students are asked to answer the following:

      1. Who are you?
      2. What do you expect from this course?

The rest of the assignment should be based on this request and what you hope students will gain from it.

Know your desired length and word count thresholds:
In my experience, while most students excel at writing as few words as possible, they’re not generally very good at getting the necessary information into those few words – concise writing is a skill that takes practice. Sentence lengths vary, with very long, complex sentences using up to 80 words (possibly more, but not typically), and very short sentences using as few as three words (e.g., “The dog ran.”), so knowing what you want from your students is key to writing the prompt.

For a skilled, concise writer, an adequate answer to the learner introduction prompts will probably be a minimum of 3 full-length, descriptive sentences (~20-25 words each), while an average writer will need more.

Define your preferred boundaries:
Most students like defined boundaries, which I have to balance with my desire to let students interpret clear assignment instructions and respond thoughtfully and creatively. Engineering students in particular, really want word counts. I find a balance between my pedagogical approach and students’ desires for rigid boundaries by locating a reasonable minimum threshold, as described above, and then doubling it (or more). This balance of expectations communicates to students that there is a general range for an acceptable response, but that they have agency and independence in how they respond. In my experience, this approach seems to satisfy students’ needs for boundaries and makes them feel more comfortable to connect and engage in a way that suits them.

 In your learner introduction prompt, the instructors’ introductions model style and boundaries.

Define the audience:
This isn’t always necessary, but for assignments like this learner introduction assignment, I like to remind students that their writing is going to be seen by an audience of their peers. To take this a step further, I like to define what “an audience of their peers” is, exactly, and what kind of expectations that the audience has.

This is the prompt structure and wording that Joanna suggested, which we used for the class this Fall:

Rafael, Nicole, and I introduced ourselves, and we would like all of you to do the same. Introducing yourselves to us and your classmates will make communicating with one another easier. As part of your introduction, include the following:

W [e.g., "where you're from," or, for distance learning, "where you're living right now"]
X [e.g., "your field of study"]
Y [e.g., "your desired profession after graduation"]
Z [e.g., "your favorite pastime or hobby"]

Your introduction must be clear, organized, and professional; you are introducing yourselves to learning colleagues who expect appropriate professionalism and personal engagement. Write 75-125 words or more.

About the rubrics, Joanna suggested defining the rubric points a little more. She noted that she prefers to grade assignments like the learner introduction assignment using the general notions of “thoughtful and complete.”

The existing rubric point of “a reasonable introduction” is highly debatable and may signal to students that this assignment is low expectations and low stakes. Although it might not be particularly high stakes, I like to use early assignments to set expectations for students that, even in low stakes assignments, students are expected to meaningfully engage in the course content, tasks, and assignments.

The grading rubric we used was as follows:

You already have three samples of introduction, and you are not asked to give a lengthy introduction.

10 points for a clear, organized, and professional introduction.
5 points for mostly missing one of the above three requirements.
0 points for mostly missing two or more of the above three requirements.

So I tried it in the Fall 2020 semester, and I noticed that almost all the learner introductions were coherent, complete, and had anecdotal components.

This semester, I left voice feedback for all the learner introductions. Many students were pleasantly surprised and thanked me for the feedback on possible courses to take and career paths to explore.

It may be “small stuff,” but it is the accumulation of “small stuff” that makes up the quality of the classroom environment.

How do I solve an initial value ODE problem in MATLAB? Updated for MATLAB 2020a

Many students ask me how do I do this or that in MATLAB.  So I thought why not have a small series of my next few blogs do that.  In this blog, I show you how to solve an initial value ordinary differential equation

  • The MATLAB program link is here.
  • The HTML version of the MATLAB program is here.
  • DO NOT COPY AND PASTE THE PROGRAM BELOW BECAUSE THE SINGLE QUOTES DO NOT TRANSLATE TO THE CORRECT SINGLE QUOTES IN MATLAB EDITOR.  DOWNLOAD THE MATLAB PROGRAM INSTEAD

%% HOW DO I DO THAT IN MATLAB SERIES?
% In this series, I am answering questions that students have asked
% me about MATLAB. Most of the questions relate to a mathematical
% procedure.

%% TOPIC
% How do I solve an initial value ordinary differential equation?

%% SUMMARY

% Language : Matlab 2008a;
% Authors : Autar Kaw;
% Mfile available at
% http://numericalmethods.eng.usf.edu/blog/ode_initial.m;
% Last Revised : May 14, 2009;
% Abstract: This program shows you how to solve an
% initial value ordinary differential equation.
clc
clear all

%% INTRODUCTION

disp(‘ABSTRACT’)
disp(‘ This program shows you how to solve’)
disp(‘ an initial value ordinary differential equation’)
disp(‘ ‘)
disp(‘AUTHOR’)
disp(‘ Autar K Kaw of https://autarkaw.wordpress.com’)
disp(‘ ‘)
disp(‘MFILE SOURCE’)
disp(‘ http://numericalmethods.eng.usf.edu/blog/ode_initial.m’)
disp(‘ ‘)
disp(‘LAST REVISED’)
disp(‘ May 14, 2009’)
disp(‘ ‘)

%% INPUTS
% Solve the ordinary differential equation 3y”+5y’+7y=11exp(-x)
% Define x as a symbol
% Define y(x)n as a symbol also
syms x y(x)
%The ODE
Dy=diff(y);
ode_eqn=3*diff(y,x,2)+5*diff(y,x,1)+7*y == 11*exp(-13*x)’;
% The initial conditions
iv_1=Dy(0)==17;
iv_2=y(0)==19;
% The value at which y is sought at
xval=2.0;
%% DISPLAYING INPUTS

disp(‘INPUTS’)
func=[‘ The ODE to be solved is ‘ char(ode_eqn)];
disp(func)
iv_explain=[‘ The initial conditions are %s’ char(iv_1) ‘ ‘ char(iv_2)];
disp(iv_explain)
fprintf(‘ The value of y is sought at x=%g’,xval)
disp(‘ ‘)

%% THE CODE
conds=[iv_1 iv_2];
% Finding the solution of the ordinary differential equation
soln=dsolve(ode_eqn,conds);
soln=simplify(soln);
% vpa below uses variable-precision arithmetic (VPA) to compute each
% element of soln to 5 decimal digits of accuracy
soln=vpa(soln,5);

%% DISPLAYING OUTPUTS
disp(‘ ‘)
disp(‘OUTPUTS’)
output=[‘ The solution to the ODE is ‘ char(soln)];
disp(output)
value=subs(soln,x,xval);
fprintf(‘ The value of y at x=%g is %g’,xval,value)
disp(‘ ‘)

This post is brought to you by

Using Microsoft Forms as a Personal Response System

While the pandemic has shifted our face-to-face classes to remote learning, finding new tools that are simple to use for the student as well as the instructor is quite a task.

One of the tools I am using to create the initial discussion in my weekly sessions for my Numerical Methods course at the University of South Florida is the  Microsoft Forms Quiz.  The quiz replaces the personal response system I used to use as that now costs students more than $30 a semester.  Microsoft Forms Pro is free to use.  To get it, just go to forms.office.com.

The quiz question options include the multiple-choice, fill-in-the-blank, and multiple-answers type of questions.  I do not use the quiz for grading or attendance. Hence, I do not have to worry about LMS compatibility or academic integrity – the quiz is just a plain way for the student to gauge their understanding so far and for the instructor to know what they know well and on what they may need help.

To avoid repeating what others have written succinctly about how to make a quiz, go here to learn how to develop a quiz and add a question, and if you are a STEM educator and want to add equations or math symbols, see an example here.

Mobile Preview

The pallete of equations is limited but does serve my purpose mostly for a Numerical Methods course.

Equation Pallete

For example, the pallete does not have options for matrices and integrals, but you can get around that by using an image.  However, only one image can be added to a question stem.  On top of that, an image is not an available feature at all for the options in a multiple-choice question.  To get around the limitation of one image in the question stem and none in the options, you can save the whole question and the options as an image (in MS word, make your text with equations, cut and paste them as a picture, right-click to save the picture as an image; alternatively, take a screenshot and crop it), and then use the options within MS Forms to direct to the choices.  See the figure below for an example.  However, do make images of small size in width so that they fit well on a smartphone without having to magnify the image.  A width of fewer than 600 pixels and keeping the height to be of similar magnitude as the width works best.

Using an image to present a question with many equations

The choices for previews are for a computer and a mobile phone; themes are available, but it is best to use simple school colors; the quiz can be shared with anyone through a link or QR code or embedded link or email.

Share Options

Here is an example of a quiz I made for the course.  Try it out by clicking on the link or by using your smartphone camera app to scan the QR code below. No one is grading, and I will not know who you are.

QR Code

Options for responses include accepting responses or not, and the start and end time.

Settings

How do I use a quiz in the class?

I use a tablet for the synchronous online sessions. I use Blackboard Collaborate Ultra as the live-streaming app (video conferencing tool) to teach the class.  It would be similar if you are using Zoom, MS Teams, or YouTube live-streaming.

While streaming the class, I paste the QR code image in an OneNote file and also put the link to the quiz in the chatbox of the streaming app as an alternative.

I open the quiz on a separate computer (a dual monitor connected to your tablet will do as well) so that I can see the responses getting updated in real-time. This gauges when the discussion about the quiz should resume, and how it should be conducted.

How do I refer to the quiz during the discussion so that I can write on the quiz itself as well? I just make a PDF file of the quiz and insert it into an OneNote page. Creating the PDF file is tricky because only the first page of the MS Forms Quiz can be printed.  To get all pages of the quiz printed as a PDF file, open a browser and go to the shareable link you gave to the students.  Now you can print all the pages of the quiz.

While discussing the quiz, students communicate verbally via their microphone or the chatbox of the streaming app. You can step it up and use break-out rooms of your streaming app to practice using the think-pair-share technique.

___________________

This post is brought to you by

How do I do that in MATLAB

HOW DO I DO THAT IN MATLAB SERIES?

A short sample programming project to exhibit how to submit one

Here is a short sample of the submission expectations for a programming project in numerical methods.  I show you this through an example.

See the sample project assignment [PDF].

See the corresponding mfile [m].  It shows you how to format the mfile with comments, write in sections, display tables, use print statements, formatting, etc.

See the expected complete submission report [PDF] as it involves typed pages done in a word processor as well as the published version of the mfile.

Learn how to make a single pdf file from various parts of the sample project [BLOG].  Did you know as a USF student you can download Adobe Acrobat for free?  It is a tool to make PDF files from wordprocessor docs as well as to merge them with other PDFs.

COVID19 Regression Model and Other Thoughts

March 28, 2020
I am not an epidemiologist but do regression modeling for a living. I have done regression modeling to predict student grades just after the first test, and now we will be using adaptive learning metrics to improve our identification schemes even earlier in the semester. It is not to spell boom or doom to a student but only to intervene early with personalized recommendations.
Looking at whatever data I can use and have time to scrape, four things are reasonably clear at this time to me about COVID19.
1) First, the rate of infection is exponential but it does not stay like that forever.  The logistic function of the infection rate is analogous to how the mass of a moving rocket decreases as it burns up its fuel. F=ma, but m is not a constant.
2) Second, President Trump is finally thinking about quarantining NY, NJ, CT area. A little late but it will definitely decrease the power of the exponent.
3) Third, we have to get more testing done but one which is totally random. We could have found the effect of the spring breakers coming to FL and of the college kids being sent home to parents who are having kids late in life. Please do not send your grandkids to grandpa/grandma’s retirement home. They may be the children of the corn.
4) Fourth, Florida and Louisiana need to get their head straightened out and use tougher rules to keep people inside and a method to keep outsiders out. They are the next hot zone.

How to Make a PDF file

Before COVID19 hit our lives, it was so easy for the student to submit a hard copy.  Now with soft copies being asked for, they and the instructors have to learn a bit.

So, I had this programming assignment before COVID19 closed our face-to-face classes. It involved handwritten pages, a published file, and writing out a short description of conclusions with equations in a word processor.  The student would print them all out at home or at the university, and collate them.  But now it has to be scanned and uploaded to the CANVAS learning management system.

So, if these three types of documents were to be uploaded as one pdf file, what would you do.  I am assuming that everyone has a smartphone (an iPhone or an android phone)

If you have a printer and a scanner

Print all the documents and scan the document as a pdf file

If you have a printer but no scanner

Print all the documents.  Download the CamScanner app on your iPhone or Android and follow the instructions of this YouTube video.

Follow the directions up to the time it tells you how to email the document as I do not want you to email it to me.

If you have neither a printer nor a scanner

a) Take the handwritten documents. Download the CamScanner app on your iPhone or Android and follow the instructions of this

Follow the directions up to the time it tells you how to email the document as I want you to upload it to CANVAS.

b) For typing the pages that include equations, use MS 365 Word, and develop your document.  Save it as a .docx file first, and only then save it as .pdf file.  You can save a document as a pdf file within MS 365 Word itself.

c) a MATLAB file can be published as a pdf file. See https://autarkaw.org/2020/03/25/how-to-publish-in-matlab/

d) Merge pdf documents with Acrobat Pro. Did you know as a USF student you can download Adobe Acrobat for free?  It is a tool to make PDF files from wordprocessor docs as well as to merge them with other PDFs.  If you are a reader outside USF and do not have Acrobat Pro as it costs money, use https://smallpdf.com/merge-pdf. Just drag the pdf files, and it will merge them for you.

____________________

The brute force way

(not recommended though but desperate times require desperate measures)

a) Take photos of each handwritten page.  Send the images to your email and save them on your computer.   Insert images (Insert -> Pictures in Word Menu) of the handwritten work into an MS 365 Word doc. But check your file size. It can get too big for CANVAS to handle and your internet to upload.  The limit may be 100MB.

b) For typing the pages that include equations, continue with the MS Word doc.

c) a MATLAB file can be published as a doc file. See https://autarkaw.org/2020/03/25/how-to-publish-in-matlab/  Cut and paste it into the word doc.

d) Save it as a .docx file first, and only then save it as .pdf file.  You can save a document as a pdf file within MS 365 Word itself.

______

This post is brought to you by

How to PUBLISH in MATLAB

This is how you can get your mfile published.

Click anywhere in the mfile (if your cursor is not in the mfile, you may not get the proper menu open)

-> Go to PUBLISH
-> Click on Down Black Arrow under Publish
-> Choose Edit Publishing Option
-> click on Output file format and the right column entry will show html, but if you click on html, you will get a drop-down box as shown in the figure below.  Choose your option.

You can click on the figure below to see a larger version of the figure.

____________________

This post is brought to you by