Use of Statistics in Data Science Notes – Class 10 Data Science (419)

Use of Statistics in Data Science Notes for Class 10 DS covering subsets, two-way frequency tables, mean, median, MAD, and standard deviation for easy learning and quick revision.

What are Subsets?

• A subset is a smaller portion of data taken from a larger data set.
• Subsetting is used to access only the required data.
• It helps in selecting and filtering specific variables and observations.
• Example: If a table has 100 rows and 100 columns, and you need only the first 5 rows and first 5 columns, that smaller table is called a subset.

How do we Subset the Data?

• The three main types of data subsetting are:
  • Row-based subsetting
  • Column-based subsetting
  • Data-based subsetting

Row-based Subsetting

• In row-based subsetting, only selected rows are taken from the data table.
• Rows can be selected from the top or bottom of the table.
• Example: From a table of 6 rows and 4 columns, selecting the top 3 rows creates a row-based subset.

Column-based Subsetting

• In column-based subsetting, only the required columns are selected from the data set.
• It is used when all columns are not needed for analysis.
• This helps in working only with relevant information.

Data-based Subsetting

• In data-based subsetting, rows are selected based on specific data or conditions.
• Only the rows that match the required criteria are included in the subset.

Two-Way Frequency Table

• A two-way frequency table is a statistical table that shows the frequency (number of observations) for two variables.
• One variable is represented by the rows, and the other variable is represented by the columns.
• Each cell in the table shows the number (frequency) of data points that belong to both categories.

Chocolate Survey Example

Consider the following table, which contains the results of a poll on people’s favourite chocolates.

• Now break down data into age categories (row categories) and choices (column categories) as given in image below
• The row categories are the age groups: 5–10 years, 10–15 years, and 15–20 years.
• The column categories are the choices: Like chocolates and Do not like chocolates.
• From the table, we can find:
  • Total number of people surveyed.
  • Total number of people who like or do not like chocolates.
  • Which age group likes chocolates the most.

Interpreting Two-Way Tables

Look at the following Two-way table.

Based on the data, we can make following observations:

• The entries in a two-way table represent the count (frequency) of observations.
• The left column contains the categories of one variable.
• The top row contains the categories of the second variable.
• The center cells show the count (frequency) for each combination of categories.
• The last row and last column show the total count for each category.
• The bottom-right cell shows the grand total (sum of all observations).
• A two-way table helps us understand and compare the relationship between two variables.
• As shown in the above example, if a cell contains 40 at the intersection of Female and Owns a Car, it means that 40 females own a car.

Two-Way Relative Frequency Table

• A two-way relative frequency table is a two-way table that displays relative frequencies (percentages) for each category instead of actual frequencies (counts).
• It is especially useful when different groups have different sample sizes.
• Using percentages makes it easier to compare data and identify patterns between different categories.
• To create a two-way relative frequency table, convert the frequency of each cell into a percentage.
• Two-way relative frequency tables help in better understanding and comparing the data.

Mean

• In data science, the mean is also called the simple average.
• It represents the average value of a data set.
• The mean indicates the central value around which the data is spread.
• While calculating the mean, every value is given equal importance (equal weight).
• Formula to calculate mean is:
  Mean = Sum of all values ÷ Number of values
• Example: For the data set {18, 25, 12, 30, 15, 22, 10, 28, 20, 20}:
  • Sum of all values = 200
  • Mean = 200 ÷ 10 = 20
• Real-life Application: Mean can be used to find the
  • average travel time from home to school or office.
  • average runs scored by a cricket player in a tournament.
• The mean represents the center of the data set, which is why it is called a measure of central tendency.

Median

• Median is another measure of central tendency.
• It is the middle value of a sorted data set.
• Before finding the median, the data must be arranged in ascending or descending order.
• If the data set has an odd number of values, the middle value is the median.
• Example:
  • Data: [12, 34, 56, 89, 32]
  • Sorted data: [12, 32, 34, 56, 89]
  • The middle value is 34, so the median is 34.
• If the data set has an even number of values, there are two middle values.
• In this case, the median is the average of the two middle values.

Mean vs Median

Mean	Median
Mean is the average of all values in a data set.	Median is the middle value of a sorted data set.
It is calculated by adding all values and dividing by the total number of values.	It is found by identifying the middle value after arranging the data in ascending or descending order.

Mean Absolute Deviation (MAD)

• Mean Absolute Deviation (MAD) is the average distance of all data values from the mean.
• It measures how far the values are spread from the mean.
• While calculating MAD, negative distances are ignored by taking the absolute value.
• Steps to calculate MAD:
  • Calculate the mean of the data set.
  • Find the distance of each value from the mean.
  • Take the absolute value of each distance.
  • Find the average of all the absolute distances.
• The MAD value shows the variability or spread of the data.

Example: Calculating Mean Absolute Deviation (MAD)

Consider the following data set:
Data Set: {15, 18, 20, 22, 25, 30}

Step 1: Calculate the Mean
Add all the values and divide by the total number of values.

Mean = (15 + 18 + 20 + 22 + 25 + 30) ÷ 6
Mean = 130 ÷ 6 = 21.67 ≈ 22 (rounded off)

Step 2: Find the Absolute Distance of Each Value from the Mean
Subtract the mean (22) from each value and take the absolute value (ignore the negative sign).

Data Value	Distance from Mean (22)	Absolute Distance
15	15 − 22 = -7	7
18	18 − 22 = -4	4
20	20 − 22 = -2	2
22	22 − 22 = 0	0
25	25 − 22 = 3	3
30	30 − 22 = 8	8

Step 3: Calculate the Mean of the Absolute Distances
Add all the absolute distances and divide by the total number of values.

MAD = (7 + 4 + 2 + 0 + 3 + 8) ÷ 6
MAD = 24 ÷ 6 = 4

Final Answer

• Mean = 22 (rounded off)
• Mean Absolute Deviation (MAD) = 4

Standard Deviation

• Standard Deviation (SD) measures how spread out the data values are from the mean.
• It shows whether the data values are close to the average or far from it.

Steps to Calculate Standard Deviation

• Calculate the mean of the data set.
• Subtract the mean from each data value.
• Square each difference.
• Find the average of the squared differences (this is called variance).
• Find the square root of the variance to get the standard deviation.

Real-Life Applications of Standard Deviation

• Weather Forecasting: Helps predict how consistent and reliable weather forecasts are. A low standard deviation indicates more reliable forecasts.
• Business: Helps companies analyze the consistency of monthly sales or profits.

Example: Calculating Standard Deviation

Consider the following data set:
Data Set: {4, 6, 8, 10, 12}

Step 1: Calculate the Mean
Add all the values and divide by the total number of values.

Mean = (4 + 6 + 8 + 10 + 12) ÷ 5
Mean = 40 ÷ 5 = 8

Step 2: Find the Difference Between Each Value and the Mean
Subtract the mean (8) from each value.

Data Value	Difference (Value − Mean)
4	4 − 8 = -4
6	6 − 8 = -2
8	8 − 8 = 0
10	10 − 8 = 2
12	12 − 8 = 4

Step 3: Square Each Difference
Square each difference to make all values positive.

Difference	Squared Difference
-4	16
-2	4
0	0
2	4
4	16

Step 4: Calculate the Variance
Add all the squared differences and divide by the total number of values.

Variance = (16 + 4 + 0 + 4 + 16) ÷ 5
Variance = 40 ÷ 5 = 8

Step 5: Calculate the Standard Deviation
Find the square root of the variance.

Standard Deviation = √8 ≈ 2.83

Final Answer

• Mean = 8
• Variance = 8
• Standard Deviation = 2.83