Confidence intervals

Concept of Confidence Intervals

Let’s suppose that you were the campaign manager for Hillary Clinton during the 2016 presidential election. 

In this context, a sample showed that 63% of voters would vote for Hillary (after the third debate) But you know that sampling isn't reliable, so you choose to take a new random sample as a precaution. This sample shows that only 48% of voters supports Clinton.

Which estimate could you trust? How could you know for sure if it would be sensible to spend more campaign funds, when one estimate indicated that you would win and the second indicated the opposite? These are the types of questions a confidence interval help answer.

Definition


A confidence interval is an interval that contains a population parameter, such as a population proportion with a given probability. 

Example: The proportion of Americans who had plan to vote for Hillary Clinton is between 51 to 57%, with a 95% probability.

In the introductory section about descriptive statistics, we learned the difference between point estimates and population parameters. Population parameters are used when working with population data when all data for our population is available.

Usually, it is extremely time and resource consuming to collect data for an entire population. Therefore, sampling is used instead. The purpose of sampling is to create an accurate picture of the entire population from a (small) segment of the population.

Point estimates are based on a sample and are provided as an estimate or an approximation of the corresponding population parameter.


As a point estimate is based on a sample, we expect that it will deviate slightly from the given population parameter - the question is just how much the estimate may differ.  This is what a confidence interval tell us.


For example, if we want to examine the average household income in Denmark through a random selection of 100 subjects (n), it is intuitive that the point estimate for the average income (x) will not be completely identical to the real average income (m), i.e., the true average. To obtain the true average (also called the population average), we would have to question the entire population of Danish households (N).

With this background, we can reason that point estimates contain some uncertainty and that the accuracy of a point estimate depends on the size of the sample.

The larger the sample, the more it will resemble the population, and therefore, the more accurate the point estimate will be.

Let’s take a simple example. Suppose you throw a coin 5 times and observe only one outcome in which the result was tails.

Based on this sample, the point estimate for the proportion of (pˆ) tails would be only 20% (1 out of 5), even though the population proportion (p) is 50%.

The large deviation between the point estimate and population proportion can be attributed to the relatively small size of the sample. If you increased the sample size from 5 flips of the coin to 1,000, you would see that the proportion of heads to tails would be approximately 50% each.

Now that we have reasoned that the size of a sample has a decisive impact on the precision of a point estimate, a new problem pops up.

Although we know that a point estimate based on 1,000 observations will be more accurate than one based on 10, we do not know how much more ac- curate it will be. Regardless of sample size, we do not know how close the point estimate will be to a given population parameter.

A confidence interval compensates for this problem. With a confidence interval, we can calculate an interval that identifies where the true population parameter will be for a certain probability.

Confidence Intervals as a Concept

Let us examine the concept of a confidence interval. To begin, imagine a car manufacturer that has only produced 10 cars (N = 10). All 10 cars have been tested for how far they can run on 1 liter of petrol. You can see the results below:

Average consumption is 11.10 km /liter, with a standard deviation of 2.77. Notice that the notation for population parameters is used. This is because data include the total population, which consists of 10 cars.

Let us now assume that we take a sample of 3 cars (n 5 3). If we take all possible combinations of 3 cars out of the population of 10, we would get 12011 different samples and a similar number of different averages based on these point estimates. To get an overview of the many different values of point estimates, we can illustrate them in a frequency distribution.

Apart from the distribution of point estimates, we can see a drastic difference between the maximum and minimum values. Despite the fact that we previously calculated the population average (m) as 11.1 km/l, point estimates of the average based on 3 observations vary from about 10 to 14.33 km/l!

Another very important observation is that the distribution of point estimates seems to follow a normal distribution, i.e., a symmetrical bell-shaped distribution.

Another important detail is that, when it comes to point estimates of proportions and averages, the data will only follow a normal distribution when the sample size is relatively large. In context, that means at least 30 observations.

The Central Limit Theorem

If you draw a large sample (n . 30) from a population with a random distribution, the distribution of point estimates (for example averages and proportions) will be approximately normally distributed. 

The interesting thing about this is, no matter whether the distribution of a single observation is right-or left-skewed, the distribution of its point estimate will be normally distributed. This is of crucial importance for the use of hypothesis tests and confidence intervals, which are primarily based on normal distributions.

Let’s use incomes in Hollywood as an example. These incomes are probably right skewed, with many people having mediocre incomes and a few having (very) high incomes.

As shown in the figure above, the distribution of the point estimates is normally distributed, despite the fact that per capita income in Hollywood follows a right skewed distribution. This relationship fits the central limit theorem. It always holds true for distributions of the point estimates of pro- portions and averages, as long as they are based on samples that are larger than 30 observations.

Let’s briefly summarize the basic aspects of confidence intervals:

  • Point estimates are calculated on a sampling basis, using just a sample of a population. As a point estimate is not calculated based on all the elements of a population, there can be great variation in the values of different point estimates, and the in the value of the population parameter being sought.
  • To address the uncertainty of a point estimate, we can calculate a corresponding confidence interval. A confidence interval assigns a certain probability to go with a given population parameter.
  • Confidence intervals are based on a normal distribution 12. The distributions of point estimates of proportions and averages follow the central limit theorem. When the sample exceeds 30 observations, the result will always be a normal distribution.

The Role of the Normal Distribution

Now that we have outlined the basic concept of confidence intervals, we can proceed with a less theoretical example. Suppose you had run for president in 2008 had taken a sample of 100 random. Americans, of whom 52 had planned to vote for Obama (pˆ 5 0,52).

You would have wanted to know how accurately the point estimate (pˆ ) was compared to the true proportion (p), meaning the proportion you would have gotten if you would have asked all 200 million American voters.

The central limit theorem showed that point estimates follow a normal distribution. As mentioned, the normal distribution is convenient in the sense that there is a fixed relationship bet- ween the number of standard deviations from the average and the area of normal distribution.

This relationship is the cornerstone of a confidence interval. For example, let us say we consider the area spanning 2 standard deviations on either side of the average, which covers 95% of the distribution, as shown in the figure to the right.

This must necessarily mean that an arbitrary point estimate could be deducted and then ad- ded within the range of 2 standard deviations, resulting in a range that crosses the middle of the distribution with 95% probability.

Since the middle of the distribution represents the population average, it means that we have created an interval that has a 95% probability of containing the population average—so, we have created a confidence interval.

The point of a confidence level is that, by subtracting and adding a number of standard deviations from the point estimate, we thus create an interval in which the population parameter will lie with an associated probability.

The Importance of the Confidence Level

The confidence level is the probability or certainty that our interval will contain the given population parameter. The higher the confidence level, the more certain is it that the interval contains the population parameter. Please note that the confidence level and the width of the confidence interval increase together.

As a rule, you should always try to use a 95% confidence level, but you are free to use other levels. Just be aware that the width of the confidence interval increases as the confidence level increases. The higher the confidence level, the wider and more inaccurate the confidence interval becomes.

The correlation between confidence level and confidence interval width can be illustrated by the following example. A meteorologist can estimate with 100% certainty that the temperature on a midsummer night is going to be between -100 and 100 degrees Celsius. 

Alternatively, the weather- person could estimate with 80% certainty that the temperature on a midsummer night is going to be between 16 and 32 degrees Celsius.

Despite the fact that the last interval has a lower confidence level, and is thus less safe, it is far more accurate and useful.

The Basic Elements of the Confidence Interval

So far, we have discussed the concept of confidence intervals. This section outlines the construction of the confidence interval.

There are different types of confidence intervals, but their very basic foundation is the same:

The point estimate and its standard deviation are based on the sample and are thus fixed values. We can adjust the confidence level as previously explained.

Regarding the standard deviation, it is important to note that we are not talking about the standard deviation of any isolated observation, but the standard deviation of the point estimate.

Example


Suppose you wanted to find the average age of the students in your class. You would need to obtain the ages of all your fellow students and then calculate the average and standard deviation. In this example, the variable would be defined as “the age of the individual students in your class.”

Let’s say you then wanted to extend the study to find the average age of the students in the entire school. With several hundred students, it would be too time consuming to gather information from all of them. Instead, you could take 10 random samples, with 20 students in each, and then calculate the average age for each sample.

In this context, our observation is no longer the age of a single student, but the average age for a complete sample of 20 students. The standard deviation must accordingly be calculated for the point estimate (which includes 20 students) and not a single observation (one student).

The standard deviation for a point estimate is affected by the sample size.

The relationship between the sample size and the standard deviation can be illustrated relatively simply. If you were to roll a die an infinite number of times, the average of the sum of the pips would be 3.5 (µ).

Now suppose that you instead threw a die twice and got two 1s and then two 6s. The average of both trials would be 1 and 6.

If we increased the sample size to, for example, 1,000 rolls of the die, we would be unlikely to get 1,000 identical results. Instead, we would expect a more even spread of high and low results, which would move the average toward the center (µ = 3.5). After performing several experiments with 1,000 die rolls, the respective averages would thus deviate much less than in a similar experiment in which you only threw a die twice.

Calculating the standard deviation of the point estimate for the confidence interval takes place automatically when you use the Statlearn program. See the end of this chapter for confidence interval formulas.

Summary

Point estimates are calculated based on samples and, consequently, contain some uncertainty. Sample size has an important impact on this uncertainty. The larger the sample size the more ac- curate the point estimate. Although the point estimate may be relatively accurate, we can never know exactly how close the estimate is to an estimated population parameter. It is in this context that a confidence interval may be inaccurate.

A confidence interval is an interval estimate that contains the true value of a population parameter with a certain probability. We can, therefore, confidently quantify the precision of a point estimate.

A confidence interval is calculated based on three elements:

Point estimate ± confidence level * standard deviation

The width of the confidence interval is determined by the standard deviation and confidence level. 

Where standard deviation is a value calculated from the sample, the confidence level can be adjusted depending on the probability desired for the interval that will contain the population parameter. When the confidence level increases, the confidence interval becomes wider.

This increases the probability that the interval will contain the given population parameter. The disadvantage of increasing the confidence level is that the interval will become wider and, therefore, less accurate.

Determining the Width of the Confidence Interval

In the section on the basic elements of the confidence interval, we showed that the width of the confidence interval is affected by the confidence level and the standard deviation of the point estimate, respectively. As we previously mentioned, the standard deviation of the point estimate and the sample size are also important (n).

Since the sample size affects the standard deviation, this necessarily means that the width of the confidence interval must also be affected. Refer to the chart below to see how the width of a confidence interval can be calculated from the sample size.

Where the Z-value is interpreted as the number of standard deviations that correspond to the confidence level and Lo is the confidence interval margin.

α (alpha) is the probability of error. At a confidence level of 95%, the chances of making an error are 5%.

Example


Determination of sample size for the proportion.

A scientist at Novo Nordisk wants to calculate the proportion of adverse reactions to a new insulin preparation with a 95% confidence level. It is an additional requirement that the interval must have a maximum width of 0.04 (i.e., the distance between the lower and upper limits of the interval should be a maximum of 0.04). How big should the sample be in order to meet the requirement for interval width?

Overview of Confidence Intervals based on 1 Sample

Video examples

Confidence interval for the proportion, one sample

Confidence interval for average, Poisson variable, one sample

Confidence interval for the average, known variance, one sample

Confidence interval for the two average (two samples)

Confidence Interval Exercises

The first problem set tests your comprehension of the terms covered in this chapter. You will need to explain the significance of various terms related to confidence intervals in your own words. The second problem set requires you to make calculations.

Interpretation exercises:

  • What is the difference between a point estimate and a confidence interval?
  • What is the purpose of using a confidence interval rather than a point estimate?
  • The larger the sample is, the more accurate the calculated point estimates will be. If you have selected a large sample, isn’t it sufficient to just calculate a point estimate, since we know this will be relatively accurate?
  • Are confidence intervals always used in an attempt to identify the value of a population parameter, or can you also calculate confidence intervals for point estimates?
  • What does “confidence level” mean?
  • How important is the confidence level in relation to the width of the confidence interval?
  • Why choose a 95% confidence level when we could come up with a safer result by choosing a confidence level of 99%?
  • What is the purpose of calculating a confidence interval for the difference between two averages?

Calculation exercises:

Problem 1.

Danske Bank has a growing feeling that some customers are highly dissatisfied with the mortgage loan advice they have been given. To get an overview of the issue, a sample of 193 customers known to have mortgages was taken, which included 17 who were strongly dissatisfied with the advice they had received.

Calculate the percentage of highly dissatisfied customers with a 95% confidence level.

Problem 2.

A survey among 338 men and 254 women who use Apple’s iPhone showed that the average male user was 23.2 years old, while the average female user was 20.6 years old. Assume that the popu- lation’s standard deviation is 5 years for both sexes.

Question 1. Set the variable and calculate the mean age for male and female iPhone users with a 95% confidence level.

Question 2. In light of the previous question, assess whether we can assume there is a difference in the average age of male and female iPhone users.

Question 3. Calculate the difference in the average age of male and female iPhone users with a 95% confidence level.

Question 4. Suppose that 45% of iPhone users are male. If a sample is taken of 300 iPhone users, what is the probability that more than half will be male?

Problem 3.

Novo Nordisk has been having quality control problems related to the production of insulin syringes. Production has temporarily been stopped to put all efforts toward getting an overview of how many of the 12,000 syringes produced don’t meet quality standards. In a sample of 300 insulin syringes, the company found 10 that should be discarded.

On this basis, calculate the interval for the proportion of the total number of insulin syringes that need to be thrown out, with a 99% confidence level.

Problem 4.

During the preparation of a large advertising campaign for a new type of car insurance, Admiral prepared a feasibility study. Among the 200 test subjects, there were 38 who indicated an interest in the new type of car insurance.

Question 1. To get an overview of the market potential, you want to estimate the upper and lower limits of the expected number of people interested in this new type of car insurance.

Question 2. Along with the feasibility study, Admiral also conducted a study gauging interest in a new type of life insurance. Out of the 200 test subjects, 47 people were interested in the new life insurance policy. Does this give us reason to believe that the new life insurance policy is more popular than the new car insurance policy?

Problem 5.

You are responsible for coordinating logistics for Coca-Cola Denmark. One of your tasks is ordering all the syrup needed to produce Coca-Cola products in the upcoming months. Therefore, you re- ally want to obtain accurate sales forecasts. From experience, you know that the forecasts for the month of June are usually too high.

You have just received the latest sales forecast for June, which calls for 68 barrels of syrup. Based on the past 10 years of sales history for the month of June, you have calculated that, on average, 57 barrels of syrup are actually required.

Question 1. Define the variable and distribution.

Question 2. Calculate the required amount of syrup for June with a 95% confidence level.

Question 3. How large is the probability that next week’s sales will equal at least 550,000 liters of Coca-Cola, assuming that weekly average sales equal 500,000 liters, with a standard deviation of 45,000 liters?

Problem 6.

Porsche has launched a major market study to pinpoint which models are driven by consumers from different age segments.

Question 1. Determine the proportion of 26 to 30-year-old Porsche owners driving a Boxter model, with a 95% confidence level.

Question 2. Calculate the proportion of Boxter models represented among all Porche models, with a 99% confidence level.

Question 3. Use a confidence level of 95% to determine how much larger the proportion of Cayenne drivers is than Cayman drivers among the 31 to 35-year-old age segment.

Problem 7.

Publicis, a global advertising agency, has developed a method to ensure that only the most effective advertisements are broadcast. For each client, Publicis develops at least two different commercials, which are then assessed by the appropriate people within the target audience.

Publicis has been working on a new advertising campaign for Nokia for a long time and has finally narrowed the choices down to two ads. Each advertisement was tested on a 10 point scale and was rated by different people. Here are the results:

Advertisement number 1: Among the 31 viewers, the average rating was 7.1, with a standard deviation of 1.7.

Advertisement number 2: Among the 31 viewers, the average rating was 7.9, with a standard deviation of 2.4.

As advertisement number 1 would be considerably cheaper to produce than advertisement number 2, Publicis requires a statistical assessment of whether there is a significant difference between the two advertisements.

In this light, calculate the interval for the difference between the two average scores, with a 95% confidence level, and comment on which advertisement would be more appropriate to use.

Problem 8.

You are employed by Savills’ marketing department and would like to know how effective your site is, since the company is thinking of improving its layout. Therefore, you survey 215 customers, asking them what they think about the layout of the website. Out of the 215 customers, 21 answered that they think the layout is boring.

Calculate the proportion of customers who think the layout is boring, with a 95% confidence level.

Problem 9.

IT manufacturer Acer conducted a customer satisfaction survey that found many customers are dissatisfied with the time it takes to get a computer repaired. A sample of 389 customers who needed computer repairs showed that it takes an average of 2.9 weeks, with a standard deviation of 1 week, before the customer got his or her computer back.

Calculate the average amount of time it takes to repair a customer’s computer, in weeks, with a 95% confidence level.

Problem 10.

Google performed a usability survey for its Android Market, which is used to download mobile phone applications. The company asked 350 users how easy they thought the Android Market was to use. Out of the 350 users surveyed, 214 said they found the Android Market very easy to use.

Calculate, with a 95% confidence level, the proportion of users who say that the Android Market is very easy to use.

Solutions for Confidence Interval Problems

The following shows solutions to the exercises for confidence intervals. For the sake of simplicity, the solutions have been reduced to just the results. You will need to get the other information on your own. For an exam, it is important that you show your work, including all of the information listed under the given procedures for this section. Please see: the section titled Steps for Calculating Confidence Intervals earlier in this chapter.

Interpretation exercises:

  • A point estimate can be perceived as a simple estimate based on a sample. A point estimate is used as an indicator of the value of the given population parameter, such as the average for the population. A confidence interval can be seen as a point estimate with an added layer of information. A confidence interval is an interval that says where the estimated population parameter will lie within a given probability.
  • A confidence interval is more informative than a simple point estimate.
  • No matter how large the sample, you can never tell where a point estimate lies along an estimated population parameter. You cannot quantify the precision of a point estimate without using a confidence interval.
  • Confidence intervals are calculated based on point estimates, but always for population parameters.
  • The confidence level represents the probability that an interval contains a given population parameter. The higher the confidence level, the wider the confidence interval.
  • Like the confidence level, the sample size is important to the interval width. The larger the sample, the narrower the interval, ceteris paribus. The reason is that the sample used to calculate the standard deviation is part of the interval calculation.
  • Because a 99% interval is wider and thus less accurate than the interval that corresponds to a 95% confidence level.
  • To find whether there is a difference in the value of the two population averages.

Calculation Exercises:

Problem 1.

X: Number of strongly dissatisfied customers.

X ~ b(n = 193, p = 0,088)​

Confidence interval for the proportion.

Lower limit: 0,04809829

Upper limit: 0,12806752​

Problem 2.

Question 1.

X: Age (in years) of men who use an iPhone.

X ~ N(Ẍ = 23,2 σ = 5 )

95% confidence interval for the average.

Lower limit: 22,6669601

Upper limit: 23,7330399

Y: Age (in years) of women who use an iPhone.

Y ~ N(Ẍ = 20,6 σ = 5 )​

95% confidence interval for the average.

Lower limit: 19,9851046

Upper limit: 21,2148954

Question 2.

The upper limit of the confidence interval for the average age of female iPhone users seems to be lower than the corresponding upper limit for the confidence interval of male iPhone users. This suggests that the average age of female iPhone users is lower than the average age of male iPhone users. When comparing two averages, it is statistically more correct to use a confidence interval to calculate the difference between the two averages—see next question (Question 3).

Question 3.

95% confidence interval for the difference between 2 averages (µx - µy).

X: Age (in years) of male iPhone users.

Y: Age (in years) of female iPhone users.

Lower limit: 1,78622613

Upper limit: 3,41377387

The positive lower and upper limits of the confidence intervals indicate that the average age of female iPhone users is lower than the average age of male iPhone users.

Question 4.

X: Number of men who use the iPhone.

X ~ b(p = 0, 45 n = 300)

P(X ≥ 151) = 0,03627756

Problem 3.

X: Number of insulin syringes that should be discarded.

X ~ b(p = 0,0333, n = 300)

99% confidence interval for the proportion.

Lower limit: 0,0066

Upper limit: 0,06

Conclusion: Overall, there is a 99% probability that between 79 and 720 insulin syringes out of 12,000 syringes will have to be discarded.

Problem 4.

Question 1.

X: Number of people interested in the new car insurance.

X ~ b(p = 0,19, n = 200)

95% confidence interval for the proportion.

Lower limit: 0,13563087

Upper limit: 0,24436913

Question 2.

X: Number of people interested in the new car insurance.

Y: Number of people interested in the new life insurance.

95% confidence level for the difference between 2 proportions (px - py).

Lower limit: -0,1250562

Upper limit: 0,03505619

Conclusion: Since the interval overlaps, equaling 0, the possibility that both types of insurance are equally popular cannot be excluded.

Problem 5.

Question 1.

X: Number of barrels of Coca-Cola syrup to be used in June.

X ~ Ps(λ = 57) when the variable is discrete and based on a time interval

Question 2.

95% confidence interval for the intensity.

​Lower limit: 52,32065

Upper limit: 61,67935​

Question 3.

P(x ≥ 550.000) = 0,13326

Problem 6.

Question 1.

X: Number of 26 to 30-year-old drivers who have a Porsche Boxter.

X ~ b(p = 0,3158, n = 190) when the variable is discrete and the events are independent.

95% confidence interval for the proportion

Lower limit: 0,24969502

Upper limit: 0,38188392

Question 2.

X: Number of motorists who drive a Porsche Boxter.

X ~ b(p = 0,3123, n = 855) when the variable is discrete and the events are independent.

99% confidence interval for the proportion.

Lower limit: 0,27145702

Upper limit: 0,35310439

Question 3.

X: Number of motorists who drive a Porsche Cayenne.

Y: Number of motorists who drive a Porsche Cayman.

95% confidence interval for the difference between 2 proportions (px - py).

Lower limit: 0,0076326

Upper limit: 0,0775409

Problem 7.

X: Rating for advertisement no. 1 (10 point scale, where 10 is the best).

Y: Rating for advertisement no. 2 (10 point scale where 10 is best).

X and Y ~ N(µ, σ) when both samples are greater than 30, as per the central limit theorem.​

95% confidence interval for the difference between 2 averages (µx - µy)​.

Lower limit: 21,7407187

Upper limit: 0,14071873​

Problem 8.

X: Number who think that the layout is boring.

X ~ b(p = 0,0977, n = 215) 95% confidence interval for the proportion.

Lower limit: 0,05799175

Upper limit: 0,13735709

With a probability of 95%, we can conclude that the proportion of customers who think the website’s layout is boring is between about 6% and 14%.

Problem 9.

X: Repair length (in weeks).

X ~ N(Ẍ = 2,9 s = 1)​

95% confidence interval, unknown population variance.

Lower limit: 2,80031493

Upper limit: 2,99968507​

Problem 10.

X: Number of customers who see Android as easy-to-use.

X ~ b(p =0,6114, n 5= 350) 95% confidence interval for the proportion.

Lower limit: 0,56036369

Upper limit: 0,66249346

We can, with a 95% probability, conclude that the proportion of customers who see the Android Market as very easy to use is between about 56% and 66%.