lecture 3 *diagnostics *

theory of true and error score

if replicated , a stydy , would it give the same results

X= T + e

x = observed test score

T = true score

e = measurment error

*scores between 0 and 1

the cor(relation) between test scores after repeated assessments or between iteams within test

1. consistency of our score
2. ration between actual behaviour (T) and test score (X)

classical test theory

T = consistency of construct

e = inconsistancy of error

X > T → e is positive

positive

X < T → e is negative

underestimation : perosn was stressed / anxious

1. if behaviour is actually different
2. measurment errors

*item selection

*test administration

*test scoring

*systematic errors of measurment

sample of behaviour

examinee/examinator/environment is different

more objective = more reliable
more subjective = less reliable

your measurements of the same thing will vary in predictable ways

either negative or positive

the averege cannot be 0

in the test costruction or assesment → measurment of validity

randomn errors that occur

both positive and negative

reliability → affect consisteny of scores

the similarity between true score (T)← → and assesd score (X)

in order to estimate "e" → should be able to avoid sytematic errors

determine confidence intervals

1. they can be positive and negative
2. averege is 0 in a large sample
3. not realted to someones true score
4. not related to eachother
5. they are normally distributed

population averege

sample averege : **M**

popualtion standard deviation

sample standard deviation

population variance

sample variance

**μ = 0**

sd = **σ**

how far from the averege is "e"

68% of the X scores have "e" max 1SD above T

95% of the X scores have "e" max 2 SD above T

99% of the X scores have max 3 SD above T

X = T +e → **μx = μT + 0**

averege test score X = averege true score T

*how different scores are from one another

*not going to averege out

\$r = T/X\$

bigger than 1

less than 1

the closer R to X → closer to 1

X = T +e → \$σ^2 x = σ^2 T +σ^2 e\$

variance X = variance T + variance e

\$rxx = σ^2T / σ^2 X\$

rxx = reliability test score X

σT2 = veriance true score T

σX2= variance test scoare X

\$rxx = σ^2T / σ^2 T+σ^2 e\$

1. variance in true score
2. variance in measurment error

68 95 98%

X have max 1 / 2 /3 SD above or below T

the smaller measurment error is (more narrow confidence is )

between _ 1.0 and +1.0

1. Test retes
2. Alterate forms

1. split half
2. coefficient alpha
3. KR20

1. Inter rater score

realtion between socres of group on a repeated assesment

calculate pearson's \$r\$ between T0 and T1

*true scores are correlated

*measuremnt errors are not corraleted

*measurment scores are not correlated with true scores

randomn fluctuations within individual
randomn fluctuation due to environment
NOT due to fluctuations due to sample ( same familiarity , wrong interpretation both times )

practice effects (people remeber )
underestimation of reliability of aspect with natural fluctuations

relation between score of a group on a test with two alternate versions

e.g. version A and vesion B

1. not alweys alternate form available
2. alternate might asses something totally different

randomn flcutuations within individual
randomn fluctuations due to environment
randomn fluctuations due to sample of items

*especially useful wehrn you expect parctice effect

relation between scores of a group on two test halves

calculate correlation between 2 split group

one half might be harder than other
split test have reduced reliability
underestimation
what estimation does split half give us ?
randomn fluctuation due to smaple of items

relation between scores on all possible split halves, with Spearmen Brown correction

estimation of internal consistency



ra = reliability in terms of coefficent alpha
N = number of items test
σj2 = variance of item
∑σj2 = sum of variances all items
σ2= varince of the total test score (vareince = SD2)

only for questions for yes

when we have binominal variable

more items
vareince decreases
consistency between items increases

not much variance as there are only one type of "no"

relation between scores of different examiners

especially imoprtant when using subjective methods