Jekyll2021-07-30T21:02:01+00:00https://wjonasreger.github.io/website/feed.xmlWJRW. Jonas Reger's website for professional, academic, and developer interests.W. Jonas RegerStudent’s Randomly Guessing on a Multiple Choice Exam2021-03-29T00:00:00+00:002021-03-29T00:00:00+00:00https://wjonasreger.github.io/website/Student's-Randomly-Guessing<p><strong>How Well Does Randomly Guessing Work?</strong></p>
<hr />
<p>Every student has been told that whenever they get stuck on a multiple choice question, they should make a guess since the probability of getting the right answer when guessing is higher than when not answering at all. This is certainly true! However, how well does this guessing method work? How does it change in response to different exams? How well will their score compare to another student who had a better idea of what answer to pick? Here is an analysis of how this works in different scenarios. (Note: Assume all exams to be completely multiple choice and have the same number of choices for each question).</p>
<p>Let’s say there’s a teacher of an advanced college course with <strong>100</strong> students. The students are given an exam with <strong>28</strong> multiple choice questions with <strong>3</strong> choices each. After calculating the test scores, the teacher finds there to be too many students doing well on the exam. So the teacher decides that the exam is too easy for the students. However, the exam can’t be made too hard, so the teacher rationalizes that making the exam very short with only <strong>4</strong> questions with <strong>5</strong> choices each and increasing the question difficulty will make it more difficult for too many students to get A’s. After calculating the test scores, the teacher finds that there are less students getting A’s and concludes that the exam difficulty and format is appropriate for the course.</p>
<p>How does randomly guessing on these two exams compare? It may feel intuitive to say that a student will be more likely to score higher in the second exam, but in fact it actually decreased for the second exam as there are more choices per question. However, how would randomly guessing on these two exams compare if they both had the same number of choices per question instead? On average, there isn’t much difference in what students could score on either exams… But the variation in possible test scores decreases as the number of test questions increase (i.e. Central Limit Theorem). So, if the whole class guessed their way through both exams, then we should see roughly similar average scores for the whole class. This tells us that if a lot of students got A’s on the first exam, then there is some combination of it either being a little too easy, and students studied and were fairly prepared for it. Since the scores dropped in exam 2 this would mean that either students did not study as well and weren’t prepared, or the exam difficulty was too high. In either case, it is more likely that students had to resort to randomly guessing due to not knowing the material well enough.</p>
<p>One important change between the exams is the number of questions coupled with increasing content difficulty. The reduction in test questions introduced more variability in possible test scores for students who may have to resort to random guessing. For the student, the increased variability brings the opportunity to score very well or very poorly or somewhere in between (i.e. if you are asked one question and randomly guess, then you can either score 0% or 100%. If you are asked 100 questions and randomly guess, then there are many more possible scores but within a smaller interval around the expected value of the average score). However, how would random guessing scores compare to the scores of students who prepared for the exam? If a student put in 30 hours of studying and scored 50% (2 out of 4) on the second exam, then how would this compare to the potential score of a student who didn’t study at all and randomly guessed? The increased variability in possible random guessing scores, will actually make this situation more common especially with higher difficulty level questions.</p>
<p>Here are some plots and a statistics table created from a monte carlo estimation of these potential random guessing scores in different scenarios (i.e. number of questions, number of choices per question). You can see the effect of the Central Limit Theorem as the plotted lines approach the theoretical average value. Designing a multiple choice exam with this effect in mind will likely separate the probabilities of higher scores between student’s randomly guessing and studying sufficiently. This separation in chance can manipulated alongside the adjustment of the exam difficulty level to reward students more for those studied for it rather than those who simply guess randomly.</p>
<figure class="">
<img src="/website/assets/images/mathstat-posts/100students-randomly-guessing.png" alt="This is a plot comparing random guessing scores for 100 students." />
<figcaption>
Cumulative distribution of scores for students who randomly guess on a multiple choice exam for various numbers of questions and choices per question.
</figcaption>
</figure>
<p>This table is drawn from the simulation of <strong>100</strong> students answering questions with <strong>3</strong> choices. As we can see here like in the plots, the chance of a student randomly guessing and getting a high score drops as the number of questions increases. This happens because of the Central Limit Theorem, which states that if there’s a population (i.e. score of test questions) with a mean and standard deviation, then with increasingly large sample sizes (i.e. number of test questions), the sample means will approximate to the normal distribution. We see the effect of the CLT in this simulation through the decrease of the SD as the number of test questions increase.</p>
<p><strong>Note: These statistics are monte carlo estimates, so we would observe these values on average. For example, if we gave the same test of 4 questions to 10 different classes, we would observe an average maximum (i.e. the average of the maximum scores from each class), which would approximate the maximum value shown in the table (92.96%).</strong></p>
<table>
<thead>
<tr>
<th>Number of Questions</th>
<th>Minimum</th>
<th>25%</th>
<th>Median</th>
<th>75%</th>
<th>Maximum</th>
<th>Mean</th>
<th>SD</th>
</tr>
</thead>
<tbody>
<tr>
<td>4 Questions</td>
<td>0.00</td>
<td>22.87</td>
<td>25.77</td>
<td>49.98</td>
<td>92.96</td>
<td>33.29</td>
<td>23.52</td>
</tr>
<tr>
<td>12 Questions</td>
<td>3.90</td>
<td>24.73</td>
<td>33.25</td>
<td>41.87</td>
<td>68.67</td>
<td>33.34</td>
<td>13.57</td>
</tr>
<tr>
<td>20 Questions</td>
<td>9.30</td>
<td>25.80</td>
<td>33.28</td>
<td>40.12</td>
<td>60.60</td>
<td>33.32</td>
<td>10.50</td>
</tr>
<tr>
<td>28 Questions</td>
<td>12.57</td>
<td>27.41</td>
<td>32.99</td>
<td>39.21</td>
<td>56.39</td>
<td>33.35</td>
<td>8.89</td>
</tr>
<tr>
<td>36 Questions</td>
<td>14.77</td>
<td>27.97</td>
<td>33.20</td>
<td>38.47</td>
<td>53.65</td>
<td>33.31</td>
<td>7.84</td>
</tr>
<tr>
<td>44 Questions</td>
<td>16.42</td>
<td>28.56</td>
<td>33.23</td>
<td>38.01</td>
<td>51.63</td>
<td>33.33</td>
<td>7.09</td>
</tr>
<tr>
<td>52 Questions</td>
<td>17.64</td>
<td>28.91</td>
<td>33.21</td>
<td>37.63</td>
<td>50.18</td>
<td>33.33</td>
<td>6.53</td>
</tr>
<tr>
<td>60 Questions</td>
<td>18.73</td>
<td>29.24</td>
<td>33.24</td>
<td>37.33</td>
<td>48.92</td>
<td>33.33</td>
<td>6.06</td>
</tr>
</tbody>
</table>
<p>So, if you are a student, then try your best to study your material as much as possible because more often than not, the chances of getting a better score is higher when you know the answer:) However, sometimes you just have to guess, and the risks/rewards are greater the less questions there are.</p>
<p>If you are a teacher, then you can use this phenomenon to design your exams to the appropriate difficulty level, while rewarding the students who study for your exams rather than those who randomly guess their way through your exams!:)</p>
<hr />
<p>If you are interested in viewing the code I used to generate these simulations, you can view them <a href="https://github.com/wjonasreger/stats-is-fun">here</a>.</p>W. Jonas RegerHow Well Does Randomly Guessing Work?Regression & Residual Visualization App2021-03-21T00:00:00+00:002021-03-21T00:00:00+00:00https://wjonasreger.github.io/website/Regression-&-Residual-Visualization<p><strong>Visualizing Residuals About the Regression Line</strong></p>
<hr />
<p>Students in Foundational Statistics courses often find the concept and behaviours of prediction errors (residuals) to be confusing and intimidating (Math!). I created this <a href="https://h550e6-wjonasreger.shinyapps.io/rrvisapp/" target="_blank">Interactive Shiny App</a> in order to help students <em>visualize</em> the balancing of prediction errors about the regression line through the use of a ranked color gradient (i.e. the color changes for each data value of increasing prediction error). Other interesting features to explore in this app are the relationships that exist between the regression line, prediction errors, correlation, sample size, and outliers (for y and/or x).</p>W. Jonas RegerVisualizing Residuals About the Regression LineLaTex Test (Laplace Distribution)2021-03-03T00:00:00+00:002021-03-03T00:00:00+00:00https://wjonasreger.github.io/website/latex/LaTex-Test<p><strong>Inverse CDF Sampling of the Laplace Distribution</strong></p>
<hr />
<p>The Laplace distribution has location $\mu \in \mathbb{R}$ and scale $\sigma \in \mathbb{R}^+$ with:</p>
<p>Probability Density Function:</p>
\[f(x \mid \mu, \sigma) =
\frac{1}{2\sigma}\text{exp}\left(-\frac{\mid x - \mu \mid}{\sigma}\right),
\quad x \in \mathbb{R}\]
<p>Cumulative Density Function:</p>
\[F(x \mid \mu, \sigma) =
\frac12 + \frac12 \text{sign}(x-\mu)
\left(1-\text{exp}\left(-\frac{\mid x - \mu \mid}{\sigma}\right)\right),
\quad x \in \mathbb{R}\]
<p>Inverse CDF:</p>
\[F^{-1}(u \mid \mu, \sigma) =
\mu - \sigma \cdot \text{sign}(u - 0.5)\ln(1-2 \mid u - 0.5 \mid),
\quad u \in [0,1]\]
<p>LaTex Code:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PDF:
$$f(x \mid \mu, \sigma) =
\frac{1}{2\sigma}\text{exp}\left(-\frac{\mid x - \mu \mid}{\sigma}\right),
\quad x \in \mathbb{R}$$
CDF:
$$F(x \mid \mu, \sigma) =
\frac12 + \frac12 \text{sign}(x-\mu)
\left(1-\text{exp}\left(-\frac{\mid x - \mu \mid}{\sigma}\right)\right),
\quad x \in \mathbb{R}$$
ICDF:
$$F^{-1}(u \mid \mu, \sigma) =
\mu - \sigma \cdot \text{sign}(u - 0.5)\ln(1-2 \mid u - 0.5 \mid),
\quad u \in [0,1]$$
</code></pre></div></div>
<p>R Code:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># PDF
dlaplace = function(x, mu, sigma) {
density = (1/(2*sigma))*exp(-abs(x-mu)/sigma)
return(density)
}
# CDF
qlaplace = function(u, mu, sigma) {
FuInv = mu - sigma * sign(u - 0.5) * log(1-2*abs(u - 0.5))
return(FuInv)
}
# ICDF sampling
set.seed(1)
U = runif(1000)
mu = 1; sigma = 2
X = qlaplace(U, mu, sigma)
# Empirical vs. Theoretical density
xx = seq(-15, 15, 0.1)
fx = dlaplace(xx, mu, sigma)
hist(X,
main = "Empirical vs. Theoretical density",
ylim = c(0, max(fx)),
col = rgb(232, 74, 39, max = 255, alpha = 200),
freq = F)
lines(xx, fx, type = 'l', lwd = 2, col = '#13294b')
</code></pre></div></div>
<figure class="">
<img src="/website/assets/images/mathstat-posts/laplace_EvT_densityplot.png" alt="This is a plot comparing the empirical (histogram) and theoretical (curve) densities of the Laplace distribution." />
<figcaption>
This is a plot comparing the empirical (histogram) and theoretical (curve) densities of the Laplace distribution.
</figcaption>
</figure>
<p>Liquid Code:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{ %
include figure image_path="/assets/images/mathstat-posts/laplace_EvT_densityplot.png"
alt="This is a plot comparing the empirical (histogram) and theoretical (curve) densities of the Laplace distribution."
caption="This is a plot comparing the empirical (histogram) and theoretical (curve) densities of the Laplace distribution."
% }
</code></pre></div></div>W. Jonas RegerInverse CDF Sampling of the Laplace DistributionLet’s Get Motivating!2021-02-22T00:00:00+00:002021-02-22T00:00:00+00:00https://wjonasreger.github.io/website/fun/Motivation<p>Motivation for later.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/-TDyceWAhBI" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>W. Jonas RegerMotivation for later.Epidemic, Endemic, and Eradication Simulations by Primer2020-05-21T00:00:00+00:002020-05-21T00:00:00+00:00https://wjonasreger.github.io/website/primer/epidemic-endemic-eradication-primer<iframe width="560" height="315" src="https://www.youtube.com/embed/7OLpKqTriio" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>W. Jonas Reger