CS325 Topic 5, Lectures 1,2,3,4,5,6

A way of reasoning and a collection of tools and methods to understand the world and data.

Quantities calculated or derived from raw data, like facts, figures, and measurements.

It helps make sense of our complex world, especially when dealing with big data.

A table consisting of variables and data elements used for analysis.

Sort the raw dataset, create equal-length bins, count occurrences, and tally them under a 'Frequency' column.

To show the distribution of a quantitative variable using bars that represent the frequency of values in intervals.

The difference between the maximum and minimum values in a dataset.

A bell-shaped curve where most students are in the middle quartiles (Q2 and Q3).

The strength of the linear association between two quantitative variables.

Correlation lies between -1 (perfect negative correlation) and 1 (perfect positive correlation).

Even if two variables are strongly correlated, it doesn’t imply one causes the other; there could be a third variable affecting both.

The average value, calculated by dividing the sum of all data points by their count.

The middle value in a dataset, or the average of the two middle values if the dataset has an even number of data points.

A distribution divided into four equal parts, with each quartile representing 25% of the data.

The average spread of data points from the mean, showing how spread out the values are.

It measures how two variables deviate from their respective means.

The range where 50% of the data lies, usually between the second and third quartiles.

A data point that significantly deviates from the other values in the dataset.

A hidden variable that may affect the two variables being studied, leading to misleading correlations.

Research shows a strong positive association, with more study time leading to higher exam scores.

Research shows a strong negative association, with more leisure time leading to lower exam scores.

A Python library used for working with datasets, derived from "Panel Data" and "Python Data Analysis." What is a DataFrame in Pandas?

" and each card is separated by a new line as requested.

Probability is the measure of the likelihood that an event will occur.

A probability of 0 indicates impossibility.

A probability of 1 indicates certainty.

A trial is the process of performing a random experiment or selection.

An outcome is the value measured, observed, or reported in an experiment.

An event is a set of outcomes from an experiment.

The probability of flipping heads or tails is 1/2 or 0.5.

Independent events are events where the outcome of one event does not influence the outcome of the other.

Dependent events are events where knowing the outcome of one event gives information about the other.

Conditional probability is the probability of an event occurring given that another event has already occurred.

Conditional probability is expressed as P(B|A), meaning the probability of event B given event A.

P(E|F) is calculated as P(E) * P(F) when E and F are independent.

Bayes' theorem is used to calculate conditional probabilities.

The formula for P(A|B) is P(A and B) / P(B).

Rolling a die and flipping a coin are independent events because the outcome of one does not affect the other.

The mean of a random sample has a sampling distribution that can be approximated by a Normal model, especially when the sample size is large enough.

A minimum sample size of 30 is considered sufficient.

It describes the population mean (µ), standard deviation (σ), and sample mean (ӯ).

The sample mean becomes more and more similar to the true population mean.

A continuous probability distribution that is symmetrical on both sides of the mean, often called a bell-shaped curve.

The distribution becomes more centered around the mean (3.5), with a shape that tightens and approaches a bell-shaped curve.

It allows drawing conclusions and estimations about the entire population based on data samples.

When averaging the results of multiple dice tosses, the distribution of sampling means tends to be centered around the expected mean, forming a normal curve as sample size increases.

It allows estimating the average annual income of a town’s population by using sample data from a group of individuals.

The sample must be random and large enough to accurately reflect the population’s characteristics.

It is the use of data analysis and statistics to make conclusions about a population.

Estimation and Hypothesis testing.

Estimation is concerned with inference about the numerical value of unknown population values from incomplete data.

Hypothesis testing is a method to check if a claim about a population is true or not, based on sample data.

The null hypothesis is the default position or value for a parameter, assuming no change, effect, difference, or relationship.

The alternative hypothesis is what we are trying to test, suggesting a change, effect, difference, or relationship.

Reject the null hypothesis (Ho) or fail to reject the null hypothesis (Ho).

A p-value measures the probability that an observed difference occurred by random chance.

It means the observed difference is statistically significant, and we can reject Ho and accept H1.

It indicates that the observed difference is not statistically significant, and we cannot reject Ho.

A confidence interval displays the probability that an estimate will fall between a pair of values around the mean.

It means that 95% of the time, the estimate will fall between the upper and lower values of the confidence interval.

P-hacking is the misuse of data analysis to find patterns that can be presented as statistically significant when they are not.

P-hacking leads to biased reporting, misleading statistical inferences, and incorrect data used in machine learning models, resulting in inaccurate predictions.

SciPy is a Python library built on top of NumPy arrays, used for scientific and technical computing applications.

SciPy provides optimized and integrated functionalities with NumPy and Matplotlib for data analysis and visualization.

Sparse data refers to datasets where most features contain zero values, with only a few attributes containing useful data.

A CSR (Compressed Sparse Row) matrix is used for faster access and slicing on row-wise sparse data.

A CSC (Compressed Sparse Column) matrix is used for faster access and slicing on column-wise sparse data.

Euclidean distance measures the shortest line between two points in any dimension.

A t-test determines whether there is a significant difference between the means of two variables.

A p-value is the probability that the results from the sample data occurred by chance, typically ranging from 0.0 to 1.0.

A p-value is considered statistically significant when it is less than 0.05.

A p-value greater than 0.05 indicates that the observed difference occurred by chance and is not statistically significant.

A p-value less than 0.05 indicates that the observed difference is genuine and statistically significant.