Statistics

Statistics is a science of collecting, summarizing, representing, and
analyzing data in order to answer certain research questions. Also, for any
conclusions reached, a measure of reliability must be provided.

In this tutorial, we cover the following topics:

• mode
• median, percentiles, and quartiles
• mean
• standard deviation
• Hypothesis testing

Mode

The mode is the value in the data which occurs most often. The mode is often used with categorical data – data whose values are labels for different possible categories. The mode can also be used with continuous quantitative data, in which case we use the methods calculus to obtain it.

Example:

Consider the data in the Table below.

What is the mode of this data?

Solution: the mode is the phone type with the highest number sold, viz: iPhone.
Nb. The data in the table are categorical (i.e., qualitative).

Median, percentiles, and quartiles

A median of a dataset is a number which divides it into two equal halves: lower half and upper half.
When data is skewed, such as with income data, a better measure of center to report is the median. The median and the mean are two of three measures of center (the third being the mode); they are used with quantitative data, making them suited for a different purpose than the mode, which is often the reported measure of center for categorical data. When data is symmetric, then a better measure of center to report is the mean (or average) because it uses all the values in the data for its calculation: the median only uses the middle value(s) for its computation, hence making it more resistant to outliers than the mean, and thus making it a better measure of center when data is skewed. An outlier is an observation or value that does not follow the general pattern in the data, such as a value which is much larger or much smaller than the rest of the data values.
So, the presence of an outlier can pull the mean down or up than it would have been without the outlier.

Example:

A quiz consisting of five questions was taken by 100 students and the following Table displays the results. Find the median score.

Solution: The median is the average of the two middle values: the values in the following
positions: 50th and 51st: obtained from the index formula:

$\frac{1}{2}(n+1)$

, where 

$n=100=$

 size of the sample. 

$\frac{1}{2}(n+1)=\frac{101}{2}$

 = 50.5, which when rounded down
and up gives 50 and 51, respectively. We now create the following cumulative frequency table to help with the locations of
the 50th and 51st values in the data (after the data has been sorted from smallest to largest, or vice-versa (which was already the case for this question)).
Cumulative frequency is the running total of all the frequency up to and including the current frequency.
The first cumulative frequency is the same as the first frequency. Nb. Frequency here is the number of students
at each score level.

From the cumulative frequency column, we see that the 50th value is a score of 3 and
the 51st value is a score of 4. So, the median score is the average of 3 and 4: $\frac{3+4}{2}=3.5$.

$Percentiles$, when clearly specified, are numbers which divide the data into one hundred parts,
with the parts containing roughly the same number of observations. For example, a 70th percentile score on an exam
is a score which 70% of observations are below or equal to and 30% of observations are above it or equal to it.
The relative position of the $k$th percentile is at the index: $\frac{k}{n}(n+1)$. If this index
is an exact integer, then the $k$th percentile is the observation at that index. Otherwise, the index is
rounded down and up, and the average of the values at these indices would be the $k$th percentile.

$Quartiles$ are a special case of percentiles: they divide the dataset into four parts, with each
part containing approximately the same number of observations. There are three quartiles, viz:

1. first quartile, $Q_1$
2. second quartile, $Q_2$
3. third quartile, $Q_3$

The first quartile, $Q_1$, is the same as the 25th percentile.
The second quartile, $Q_2$, is the same as the 50th percentile or median.
The third quartile, $Q_3$, is the same as the 75th percentile.
The $k$th quartile is located at the index $\frac{k}{4}(n+1)$.

Example:

Find the five-number summary for the following data:$$20,4,0,-6,100,6,9,3,10,20,72$$

Solution: Sort the data from smallest to largest as follows:
$$-6,0,3,4,6,9,10,20,20,72,100$$
$\text{minimum}=-6$
lower half: -6, 0, 3, 4, 6; $Q_1=\text{median of lower half}=3$
$\text{median}=Q_2=9$
upper half: 10, 20, 20, 72, 100; $Q_3=\text{median of upper half}=20$
$\text{maximum}=100$
So, the five-number summary is: $(\text{min},Q_1,Q_2,Q_3,\text{max})=(-6,3,9,20,100)$.

Mean

The mean or average is a measure of center. It is calculated by adding up the observations in the data and then
dividing by the number of observations, if the variable is a discrete random variable.
The mean is also called the expected value. For a continuous random variable with probability density function (pdf) $f(x)$,$$E(X)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}xf(x)dx$$

Example:

A random variable $X$ has the following pdf.
$f(x)=\{\begin{array}{ll}{x}^2& \text{if }0<x<1\\ \frac{1}{9}& \text{if }x=1\\ \frac{8}{9}x& \text{if }1<x<1.5\\ 0& \text{otherwise}\end{array}$
Compute $E(X)$, the expected value of $X$.

Solution: This is mixed random variable because there is probaboility mass at $x=1$ (this is the discrete part of
$X$) and the random variable takes values in the intervals: [0, 1] and [1, 1.5] (this is the continuous part of $X$)
$\begin{array}{rl}E(X)& ={\int }_0^{1}xf(x)dx+1.f(1)+{\int }_1^{1.5}xf(x)dx\\ & ={\int }_0^{1}x\cdot x^{2}dx+1\cdot \frac{1}{9}+{\int }_1^{1.5}x\cdot \frac{8}{9}xdx\\ & =\frac{1}{4}{x}^4{|}_0^1+\frac{1}{9}+\frac{8}{27}{x}^3{|}_1^{1.5}\\ & =\frac{1}{4}+\frac{1}{9}+\frac{8}{27}({1.5}^3-1)\\ & =\frac{115}{108}=1.06\end{array}$

Standard Deviation

The standard deviation of a random variable, $X$, is how far, on average, a randomly sampled observation
falls below or above the mean. There are a number of ways to find the standard deviation, ${\sigma }_X$, for a random variable, $X$,
but we will use the following definition:$$\text{variance}={\sigma }_X^2=E[X^2]-E(X{)}^2;{\sigma }_X=\sqrt{\text{variance}}$$

Example:

A random variable $X$ has the following frequency distribution.
$$\overline{)\begin{array}{cc}X:\text{Score}/5& \text{Number of students}\\ 0& 5\\ 1& 6\\ 2& 14\\ 3& 25\\ 4& 40\\ 5& 10\end{array}}$$Compute ${\sigma }_X$. What percentage of the observations are within one standard deviation of the mean?

Solution: We obtain the following additional column of relative frequencies (i.e., probabilities).$$\overline{)\begin{array}{ccc}X:\text{Score}/5& \text{Number of students}& \text{Relative frequency},p(x)\\ 0& 5& 5/100=0.05\\ 1& 6& 6/100=0.06\\ 2& 14& 14/100=0.14\\ 3& 25& 25/100=0.25\\ 4& 40& 40/100=0.40\\ 5& 10& 10/100=0.10\\ \text{Total}& 100& 1\end{array}}$$$E[X]=\sum x\cdot p(x)=0(.05)+1(.06)+2(.14)+3(.25)+4(.4)+5(.1)=3.19$
$E[X^2]=\sum x^2\cdot p(x)=0(.05)+1(.06)+4(.14)+3^2(.25)+4^2(.4)+25(.1)=11.77$
$\text{variance}={\sigma }_X^2=11.77-{3.19}^2=1.5939⟹{\sigma }_X=\sqrt{1.5939}=1.26$
bounds for one std. dev. of the mean are: $[\text{mean}-1{\sigma }_X,\text{mean}+1{\sigma }_X]=[3.19-1.26,3.19+1.26]=[1.93,4.45]$,
so that the scores between 1.93 and 4.45 are: 2, 3, 4. Therefore, the percentage of observations with scores of
2, 3, or 4 is: 0.14 + .25 + .4 = 79%

Test of Hypothesis

Consider the table above, consisting five explanatory variables, $x_1$, $x_2$, $x_3$, $x_4$, and $x_5$, and one response variable $y$.
There are 12 observations (obs) in the data. An analyst claims that the variables $x_3$ and $x_4$ do not contribute significantly to predicting the
value of $y$. Use the partial F-test to investigate this claim, with $\alpha =0.05$ significance level.

Solution: We want to test whether the reduced model given by$$\text{Model 1:}{y}_i={\beta }_0+{\beta }_1{x}_{1i}+{\beta }_2{x}_{2i}+{\beta }_5{x}_{5i}+{ϵ}_i$$is better than the full model given by$$\text{Model 2:}{y}_i={\beta }_0+{\beta }_1{x}_{1i}+{\beta }_2{x}_{2i}+{\beta }_3{x}_{3i}+{\beta }_4{x}_{4i}+{\beta }_5{x}_{5i}+{ϵ}_i$$So, the null and alternative hypoyheses are respectively:$$H_0:{\beta }_3={\beta }_4=0$$$$H_1:\text{at least one of}{\beta }_3\text{or}{\beta }_4\text{is not zero}$$The test statistic is given by:$$F_0=\frac{\frac{\text{SSE(reduced)}-\text{SSE(full)}}{p}}{\text{MSE(full)}}$$where $\text{MSE(full)}=\frac{\text{SSE(full)}}{n-(k+1)}$ $\text{(MSE: mean squared error; SSE: sum of squared error)}$,
$p=$ number of explanatory variables removed from the full model = 2 (${\beta }_{3}and{\beta }_4$),
$k=$ number of explanatory variables in the full model (= 5),
$n=$ number of observations in the data set (= 12),
Using R (full result from R is shown below), we get the least-squares regression equations
for the reduced and full model to be respectively:$$\text{Model 1}:y=.4276+.1666x_1-.2946x_2+.3798x_5$$$$\text{Model 2}:y=.4309+.1690x_1-.2885x_2+.0172x_3-.0318x_4+.3799x_5$$ $$ 
From the R output above, we see that the coefficients ${\beta }_3$ and ${\beta }_4$ are not significant, supporting
the analyst’s claim, hence we do not reject the null hypothesis.
Nb. If we had done this by hand, then we would have needed to compute the test statistic, $F_0$,
and then compared it to the critical value $F_{\alpha }$, which can be obtained from R using the following
F-distribution quantile command, with two degrees of freedom, one for the numerator ($p=2$), and the other
for the denominator ($d{f}_{denom}=n-(k+1)=12-6=6$): $qf(0.95,2,6)=5.14$. It is a good
exercise to verify that $F_0<F_{\alpha }$, so that the we do not reject $H_0$, and the explanatory
variables $x_3$ and $x_4$ do not signicantly contribute to the prediction of $y$.