Multiple Choice Analyzer

How many instructors revise their multiple-choice tests based on their students’ responses? If one wishes to make a better multiple-choice test, a statistical analysis of these responses could be helpful.

We have developed a freely available web-based program to analyze a multiple-choice test, and it requires no programming experience – just your student response data and the answer key in two excel spreadsheets.

The program outputs a pdf file that goes beyond the usual scoring but includes many other metrics, which would help the instructor to revise the test for improving its reliability.

The outputs include difficulty index, discrimination index, Cronbach alpha, item response theory, and distractor analysis. It also outputs if a question should be kept or removed based on four different criteria. The source code is open-access, and we invite others to improve it.

The description is given here at

The program can be accessed at

How to make the input excel files is at

An example of a hypothetical output report is here at

The source code for developers is at


This post is brought to you by

An Example of Doing Learner Introductions in an Online Class

An Example of Doing Learner Introductions in an Online Class

Getting coherent, complete, and thoughtful learner introductions in an online class requires intentional prompting and guidance.

By Joanna Bartell and Autar Kaw

September 2, 2020

Whether we are teaching online, face-to-face, or somewhere in between, having students introduce themselves is essential to the community building process for a class. When students know something about other students, it is easier for students to communicate with each other as well as with the instructor.

I have been teaching a core lecture class in Numerical Methods for three decades now. Although I am well-versed in several pedagogies, including hybrid, blended, and flipped learning, Numerical Methods had never been taught entirely online before the Summer 2020 semester. I come to know students by remembering their names, asking for their name when a student asks questions, or while conducting in-class active learning exercises in a flipped classroom.

In summer 2020, because of the COVID19 crisis, I had to teach the course online. I chose to give the “learner introduction” assignment to students to get to know them better as well to get students to know their peers.

The prompt of the assignment read as follows.

Since Jed and I have introduced ourselves, we would like you to do the same now by the end of the first week of classes. It will make it easier to communicate with you and your classmates. Say something about yourself (nothing personal), and what you expect from the course. About 50-100 words would be sufficient.

and the grading rubric was as follows.

You already have two samples of introduction, and you are not asked to give a lengthy introduction.
5 points for a reasonable introduction
2.5 points for less than a reasonable introduction.
0 points for an irrelevant introduction.

Although all students participated in this low stakes assignment (about 0.25% points), several responses were rich and gave an extended introduction full of anecdotes, but many of them were not uniform, lacked coherency, and some used instant messaging dialect and included several spelling and grammar mistakes.

Dr. Joanna Bartell is an instructor of Communication in the College of Engineering (CoE) at the University of South Florida, working to develop the CoE’s integrated communication program. Before joining the CoE in 2018, Dr. Bartell taught core communication courses for over a decade, including Interpersonal Communication and Persuasion courses. She is also well-versed in several pedagogies, including writing center and feminist pedagogy, which take highly individual, student-centered approaches. Who better to ask than Joanna about what can be done to elicit a better response from students.

I contacted Joanna, and she shared some of her experiences and suggestions, which she has outlined below:

Consider your request:
The learner introduction is a situating assignment, where you’re asking students to 1) talk a little bit about themselves, and 2) both understand and articulate their expectations. These are significant assignments and can help students understand and manage their expectations and involvement in the class. The reciprocity mechanism you include at the beginning of the prompt is helpful (“We opened up, now it’s your turn”) for getting students to respond to and mimic your introductory offering.

In your learner introduction assignment, students are asked to answer the following:

      1. Who are you?
      2. What do you expect from this course?

The rest of the assignment should be based on this request and what you hope students will gain from it.

Know your desired length and word count thresholds:
In my experience, while most students excel at writing as few words as possible, they’re not generally very good at getting the necessary information into those few words – concise writing is a skill that takes practice. Sentence lengths vary, with very long, complex sentences using up to 80 words (possibly more, but not typically), and very short sentences using as few as three words (e.g., “The dog ran.”), so knowing what you want from your students is key to writing the prompt.

For a skilled, concise writer, an adequate answer to the learner introduction prompts will probably be a minimum of 3 full-length, descriptive sentences (~20-25 words each), while an average writer will need more.

Define your preferred boundaries:
Most students like defined boundaries, which I have to balance with my desire to let students interpret clear assignment instructions and respond thoughtfully and creatively. Engineering students in particular, really want word counts. I find a balance between my pedagogical approach and students’ desires for rigid boundaries by locating a reasonable minimum threshold, as described above, and then doubling it (or more). This balance of expectations communicates to students that there is a general range for an acceptable response, but that they have agency and independence in how they respond. In my experience, this approach seems to satisfy students’ needs for boundaries and makes them feel more comfortable to connect and engage in a way that suits them.

 In your learner introduction prompt, the instructors’ introductions model style and boundaries.

Define the audience:
This isn’t always necessary, but for assignments like this learner introduction assignment, I like to remind students that their writing is going to be seen by an audience of their peers. To take this a step further, I like to define what “an audience of their peers” is, exactly, and what kind of expectations that the audience has.

This is the prompt structure and wording that Joanna suggested, which we used for the class this Fall:

Rafael, Nicole, and I introduced ourselves, and we would like all of you to do the same. Introducing yourselves to us and your classmates will make communicating with one another easier. As part of your introduction, include the following:

W [e.g., "where you're from," or, for distance learning, "where you're living right now"]
X [e.g., "your field of study"]
Y [e.g., "your desired profession after graduation"]
Z [e.g., "your favorite pastime or hobby"]

Your introduction must be clear, organized, and professional; you are introducing yourselves to learning colleagues who expect appropriate professionalism and personal engagement. Write 75-125 words or more.

About the rubrics, Joanna suggested defining the rubric points a little more. She noted that she prefers to grade assignments like the learner introduction assignment using the general notions of “thoughtful and complete.”

The existing rubric point of “a reasonable introduction” is highly debatable and may signal to students that this assignment is low expectations and low stakes. Although it might not be particularly high stakes, I like to use early assignments to set expectations for students that, even in low stakes assignments, students are expected to meaningfully engage in the course content, tasks, and assignments.

The grading rubric we used was as follows:

You already have three samples of introduction, and you are not asked to give a lengthy introduction.

10 points for a clear, organized, and professional introduction.
5 points for mostly missing one of the above three requirements.
0 points for mostly missing two or more of the above three requirements.

So I tried it in the Fall 2020 semester, and I noticed that almost all the learner introductions were coherent, complete, and had anecdotal components.

This semester, I left voice feedback for all the learner introductions. Many students were pleasantly surprised and thanked me for the feedback on possible courses to take and career paths to explore.

It may be “small stuff,” but it is the accumulation of “small stuff” that makes up the quality of the classroom environment.

How do I solve an initial value ODE problem in MATLAB? Updated for MATLAB 2020a

Many students ask me how do I do this or that in MATLAB.  So I thought why not have a small series of my next few blogs do that.  In this blog, I show you how to solve an initial value ordinary differential equation

  • The MATLAB program link is here.
  • The HTML version of the MATLAB program is here.

% In this series, I am answering questions that students have asked
% me about MATLAB. Most of the questions relate to a mathematical
% procedure.

% How do I solve an initial value ordinary differential equation?


% Language : Matlab 2008a;
% Authors : Autar Kaw;
% Mfile available at
% Last Revised : May 14, 2009;
% Abstract: This program shows you how to solve an
% initial value ordinary differential equation.
clear all


disp(‘ This program shows you how to solve’)
disp(‘ an initial value ordinary differential equation’)
disp(‘ ‘)
disp(‘ Autar K Kaw of’)
disp(‘ ‘)
disp(‘ ‘)
disp(‘ May 14, 2009’)
disp(‘ ‘)

% Solve the ordinary differential equation 3y”+5y’+7y=11exp(-x)
% Define x as a symbol
% Define y(x)n as a symbol also
syms x y(x)
%The ODE
ode_eqn=3*diff(y,x,2)+5*diff(y,x,1)+7*y == 11*exp(-13*x)’;
% The initial conditions
% The value at which y is sought at

func=[‘ The ODE to be solved is ‘ char(ode_eqn)];
iv_explain=[‘ The initial conditions are %s’ char(iv_1) ‘ ‘ char(iv_2)];
fprintf(‘ The value of y is sought at x=%g’,xval)
disp(‘ ‘)

conds=[iv_1 iv_2];
% Finding the solution of the ordinary differential equation
% vpa below uses variable-precision arithmetic (VPA) to compute each
% element of soln to 5 decimal digits of accuracy

disp(‘ ‘)
output=[‘ The solution to the ODE is ‘ char(soln)];
fprintf(‘ The value of y at x=%g is %g’,xval,value)
disp(‘ ‘)

This post is brought to you by

Using Microsoft Forms as a Personal Response System

While the pandemic has shifted our face-to-face classes to remote learning, finding new tools that are simple to use for the student as well as the instructor is quite a task.

One of the tools I am using to create the initial discussion in my weekly sessions for my Numerical Methods course at the University of South Florida is the  Microsoft Forms Quiz.  The quiz replaces the personal response system I used to use as that now costs students more than $30 a semester.  Microsoft Forms Pro is free to use.  To get it, just go to

The quiz question options include the multiple-choice, fill-in-the-blank, and multiple-answers type of questions.  I do not use the quiz for grading or attendance. Hence, I do not have to worry about LMS compatibility or academic integrity – the quiz is just a plain way for the student to gauge their understanding so far and for the instructor to know what they know well and on what they may need help.

To avoid repeating what others have written succinctly about how to make a quiz, go here to learn how to develop a quiz and add a question, and if you are a STEM educator and want to add equations or math symbols, see an example here.

Mobile Preview

The pallete of equations is limited but does serve my purpose mostly for a Numerical Methods course.

Equation Pallete

For example, the pallete does not have options for matrices and integrals, but you can get around that by using an image.  However, only one image can be added to a question stem.  On top of that, an image is not an available feature at all for the options in a multiple-choice question.  To get around the limitation of one image in the question stem and none in the options, you can save the whole question and the options as an image (in MS word, make your text with equations, cut and paste them as a picture, right-click to save the picture as an image; alternatively, take a screenshot and crop it), and then use the options within MS Forms to direct to the choices.  See the figure below for an example.  However, do make images of small size in width so that they fit well on a smartphone without having to magnify the image.  A width of fewer than 600 pixels and keeping the height to be of similar magnitude as the width works best.

Using an image to present a question with many equations

The choices for previews are for a computer and a mobile phone; themes are available, but it is best to use simple school colors; the quiz can be shared with anyone through a link or QR code or embedded link or email.

Share Options

Here is an example of a quiz I made for the course.  Try it out by clicking on the link or by using your smartphone camera app to scan the QR code below. No one is grading, and I will not know who you are.

QR Code

Options for responses include accepting responses or not, and the start and end time.


How do I use a quiz in the class?

I use a tablet for the synchronous online sessions. I use Blackboard Collaborate Ultra as the live-streaming app (video conferencing tool) to teach the class.  It would be similar if you are using Zoom, MS Teams, or YouTube live-streaming.

While streaming the class, I paste the QR code image in an OneNote file and also put the link to the quiz in the chatbox of the streaming app as an alternative.

I open the quiz on a separate computer (a dual monitor connected to your tablet will do as well) so that I can see the responses getting updated in real-time. This gauges when the discussion about the quiz should resume, and how it should be conducted.

How do I refer to the quiz during the discussion so that I can write on the quiz itself as well? I just make a PDF file of the quiz and insert it into an OneNote page. Creating the PDF file is tricky because only the first page of the MS Forms Quiz can be printed.  To get all pages of the quiz printed as a PDF file, open a browser and go to the shareable link you gave to the students.  Now you can print all the pages of the quiz.

While discussing the quiz, students communicate verbally via their microphone or the chatbox of the streaming app. You can step it up and use break-out rooms of your streaming app to practice using the think-pair-share technique.


This post is brought to you by

A short sample programming project to exhibit how to submit one

Here is a short sample of the submission expectations for a programming project in numerical methods.  I show you this through an example.

See the sample project assignment [PDF].

See the corresponding mfile [m].  It shows you how to format the mfile with comments, write in sections, display tables, use print statements, formatting, etc.

See the expected complete submission report [PDF] as it involves typed pages done in a word processor as well as the published version of the mfile.

Learn how to make a single pdf file from various parts of the sample project [BLOG].  Did you know as a USF student you can download Adobe Acrobat for free?  It is a tool to make PDF files from wordprocessor docs as well as to merge them with other PDFs.

COVID19 Regression Model and Other Thoughts

March 28, 2020
I am not an epidemiologist but do regression modeling for a living. I have done regression modeling to predict student grades just after the first test, and now we will be using adaptive learning metrics to improve our identification schemes even earlier in the semester. It is not to spell boom or doom to a student but only to intervene early with personalized recommendations.
Looking at whatever data I can use and have time to scrape, four things are reasonably clear at this time to me about COVID19.
1) First, the rate of infection is exponential but it does not stay like that forever.  The logistic function of the infection rate is analogous to how the mass of a moving rocket decreases as it burns up its fuel. F=ma, but m is not a constant.
2) Second, President Trump is finally thinking about quarantining NY, NJ, CT area. A little late but it will definitely decrease the power of the exponent.
3) Third, we have to get more testing done but one which is totally random. We could have found the effect of the spring breakers coming to FL and of the college kids being sent home to parents who are having kids late in life. Please do not send your grandkids to grandpa/grandma’s retirement home. They may be the children of the corn.
4) Fourth, Florida and Louisiana need to get their head straightened out and use tougher rules to keep people inside and a method to keep outsiders out. They are the next hot zone.

On Making A Video Lecture on iPad and Uploading to YouTube

On Making A Video Lecture on iPad and Uploading to YouTube

Guest Blogger: Rasim Guldiken

Date: March 20, 2020

While we are scrambling to take the courses we teach from face-to-face format to remote format, we all could use simple tools to make the process simple and easy to implement.  I have created two extensive tutorials.

The first tutorial  shows step-by-step instructions on screencasting your lecture with an iPad using any stylus, editing the video in the Imovie app of iPad, uploading it to YouTube, or directly to your LMS.

The second tutorial illustrates on how to embed any YouTube video to your Canvas LMS course content.

I have been recording and uploading the videos to my Canvas course since Spring 2019, but following the same procedure has been frustrating as it takes several hours for videos to process. I really can’t blame anyone; there is an unprecedented load to the servers.

That is the reason I created this alternative method tutorial on using 1) YouTube as a platform to upload and store your content  and then 2) embedding the videos to Canvas.

This process works flawlessly as the videos are not stored on the LMS servers.

About the guest blogger

Dr. Rasim Guldiken is an Associate Professor of Mechanical Engineering at the University of South Florida.  His engineering education interests lie in open courseware for a course in Fluid Mechanics, metacognitive activities, and flipped learning.

Why Do We Use Numerical Methods?

Numerical methods are techniques to approximate mathematical procedures (an example of a mathematical procedure is an integral).

Approximations are needed because we either cannot solve the procedure analytically (an example is the standard normal cumulative distribution function)

or because the analytical method is intractable ( an example is solving a set of a thousand simultaneous linear equations for a thousand unknowns for finding forces in a truss)


This post is brought to you by

Why Don’t I Allow (Not Ban) Use of Cell Phones in Class – An Open Letter to Students?

Dear Students:

You are not allowed to use cell phones in this class — it is considered academic disruption. If you are expecting an important call or text, please let me know in advance and I will advise to sit at desks close to the door. You can quickly go outside to take the call or reply to your text. In a class of over a hundred, only a few people take me up on this offer, but the offer is available to you. “Loved ones though should call 911 in case of an emergency — the local responders will be there faster than you can.”

If you have an approved request for accommodation through the Student Disability Services to use your cell phone, please use it as per the allowed guidelines — just let me know in the first week of classes. You are being an exception to the policy is nobody’s business including mine. Do not be concerned about being “outed” — we all are outed every day – some are in a wheel chair while others use a skateboard, some can only take the elevator while others only take the stairs; some of us are not good looking while others are blessed with a head full of hair; some wear a head dress, while others have a skullcap on.

You are allowed to use flat laptops and tablets only if they are used to take notes with a stylus. A student cannot claim they want to type notes as that is virtually impossible to do in an engineering course lecture that is full of equations and sketches. Anyway, taking notes by hand is shown to be cognitively better than typing.

About those students who may mention personal responsibility and that disallowing the cell phone is an ego trip for me, how many studies do I need to show these students about negative effects of multitasking and working memory when one is learning something new.

The student use of cell phones distracts others and there are studies on that too.

Since we are using active learning in class, questions are encouraged even more so. It is harder to discern if a student is asking a question because of inattention or lack of understanding.  The former is a waste of precious class time for the rest of us.

And more importantly, removing temptation is much better than self-control. We only have so much self-control quota during the day – so having a no cell phone policy is helping the student to preserve some self-control for more important issues of the day.

Some may say, “Be more interesting than the incoming text”. I just cannot compete, as learning something new is hard and looking at a “Facebook like” is an immediate high.

Control the machine; do not let the machine control you.

Best of Learning
Autar Kaw
Your Instructor for Numerical Methods


This post is brought to you by