Reliability (visit the concept map that shows the various types of reliability) A test is reliable to the extent that whatever it measures, it measures it consistently. If I were to stand on a scale and the scale read 15 pounds, I might
wonder. Suppose I were to step off the scale and stand on it again, and again it read 15 pounds. The scale is producing consistent results. From a research point of view, the scale seems to be reliable because whatever it is measuring, it is measuring it consistently. Whether those consistent results are valid is another question. However, an instrument cannot be
valid if it is not reliable. There are three major categories of reliability for most instruments: test-retest, equivalent form, and internal consistency. Each measures consistency a bit differently and a given instrument need not meet the requirements of each. Test-retest measures consistency from one time to the next. Equivalent-form measures consistency between two versions of an
instrument. Internal-consistency measures consistency within the instrument (consistency among the questions). A fourth category (scorer agreement) is often used with performance and product assessments. Scorer agreement is consistency of rating a performance or product among different judges who are rating the performance or product. Generally speaking, the longer a test is, the more reliable it tends to be (up to a point). For research purposes, a minimum reliability of .70 is required
for attitude instruments. Some researchers feel that it should be higher. A reliability of .70 indicates 70% consistency in the scores that are produced by the instrument. Many tests, such as achievement tests, strive for .90 or higher reliabilities. Relationship of Test Forms and Testing Sessions Required for Reliability Procedures Test-Retest Method (stability: measures error because of changes over time) If one were investigating the reliability of a test measuring mathematics skills, it would not be wise to wait two months. The subjects probably would have gained additional mathematics skills during the two months and thus would have scored differently the second time they completed the test. We would not want their knowledge to have changed between the first and second testing. Equivalent-Form (Parallel or Alternate-Form) Method (measures error because of differences in test forms) Internal-Consistency Method (measures error because of idiosyncrasies of the test items) – Split-Half – Kuder-Richardson Formula 20 (K-R 20)
and Kuder-Richardson Formula 21 (K-R 21) – Cronbach’s Alpha I have created an Excel spreadsheet that will calculate Spearman-Brown, KR-20, KR-21, and Cronbach’s alpha. The spreadsheet will handle data for a maximum 1000 subjects with a maximum of 100 responses for each. Scoring Agreement (measures error because
of the scorer) – Interrater Reliability – Percentage Agreement ——— All scores contain error. The error is what lowers an instrument’s reliability. ———- There could be a number of reasons why the reliability estimate for a measure is low. Four common sources of inconsistencies of test scores are listed below: Test Taker — perhaps
the subject is having a bad day ———- Del Siegle, Ph.D. Created 9/24/2002 Which three attributes are used to determine the reliability of an instrument?There are three major categories of reliability for most instruments: test-retest, equivalent form, and internal consistency. Each measures consistency a bit differently and a given instrument need not meet the requirements of each.
What are the 3 ways of measuring reliability?Reliability refers to the consistency of a measure. Psychologists consider three types of consistency: over time (test-retest reliability), across items (internal consistency), and across different researchers (inter-rater reliability).
How will you determine the reliability of your instruments?Reliability can be assessed with the test-retest method, alternative form method, internal consistency method, the split-halves method, and inter-rater reliability. Test-retest is a method that administers the same instrument to the same sample at two different points in time, perhaps one year intervals.
What determines the reliability?Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same methods under the same circumstances, the measurement is considered reliable.
|