Varel2001

MATERI PENUNJANG KULIAH

METODE PENELITIAN KOMUNIKASI

BAGIAN III VALIDITAS DAN RELIABILITAS PENGUKURAN

Copy ini hanya dimaksudkan sebagai penunjang perkuliahan tatap muka dalam kelas. Peserta kuliah tidak akan memahami materi dalam copy ini tanpa mengikuti perkuliahan serta membaca buku teks yang diwajibkan ataupun dianjurkan

PROGRAM PASCASARJANA KEKHUSUSAN MANAJEMEN KOMUNIKASI POLITIK DEPARTEMEN I. KOMUNIKASI – FISIP-UI SEMESTER GANJIL 2001 SEPTEMBER2001/10/28/2008

MEASUREMENT RESULTS: variances/scores differences SOURCES OF VARIANCES/ SCORES DIFFERENCES (Singleton, 1988; p.112; Selltiz, et.al., pp. 164-169)

TRUE DIFFERENCES

RANDOM ERRORS

SYSTEMATIC ERRORS

differences in the concept the measure intended to measure (Singleton, 1988); differences in the characteristics we are attempting to measure (Selltiz, 1976).

measurement errors due to random or chance factors: (Singleton, 1988) or transient aspects of the person, of the situation of measurement, or of the measurement procedures that are likely to vary from one act of measurement to the next, even though the characteristics

biases inherent in the method or operational definition (Singleton,1988) ; is an error introduced into the measurement by some factor that systematically affects the characteristics being measured or the process of measurement (Selltiz, et el., 1976)

SEPTEMBER2001/10/28/2008

we are trying to measure has not changed. (Selltiz, 1976)

Measurement errors

MEASUREMENT ERRORS

RANDOM ERRORS

SYSTEMATIC ERRORS

Kesalahan yang terjadi secara sistematis, antara lain bersumber dari faktor-faktor yang inherent dalam alat ukur atau definisi operasional konsep yang diukur.

• Kesalahan yang terjadi secara random/acak, akibat kondisi, proses, atau variasi prosedur pengukuran yang dilakukan.

•

• Membias secara acak ke berbagai

• Membias ke satu arah kemungkinan

SEPTEMBER2001/10/28/2008

arah kemungkinan

RELIABILITY Tingkatan sejauh mana pengukuran yang dilakukan memperoleh hasil yang konsisten (antar waktu, antar pengamat, antar indikator, dsb.) The consistency of a measure (Bailey, 1987)

SEPTEMBER2001/10/28/2008

tertentu

VALIDITY Tingkatan sejauh mana pengukuran yang dilakukan benarbenar mengukur konsep yang semula akan diukur The degree to which a test measures what it purports to measure (Borg & Gall; 1971)

RELATIONSHIP BETWEEN RELIABILITY AND VALIDITY

• A MEASURE CAN BE RELIABLE BUT INVALID • RELIABILITY IS A NECESSARY CONDITION FOR VALIDITY

LOW RELIABILITY LOW VALIDITY •

•

•

• HIGH RELIABILITY • • LOW VALIDITY • •

• • •• •• • HIGH RELIABILITY HIGH VALIDITY

SEPTEMBER2001/10/28/2008

LOW RELIABILITY HIGH VALIDITY???

RELIABILITY IS A NECESSARY BUT NOT SUFFICIENT CONDITION FOR VALIDITY

FACE - in the judgment of others

CONTENT Pre-data Validity

SAMPLING - capture the entire dimensions

CONCURRENT preexisting criterion VALIDITY

MEASUREMENT QUALITY

SEPTEMBER2001/10/28/2008

CRITERION Data-based -Validity

PREDICTIVE criterion

- predicted

CONVERGENT - similar construct CONSTRUCT Data-based Validity

DIVERGENT construct

- opposing

RELIABILITY

STABILITY

OVER TIME

EQUIVALEN CE

OVER ALTERNATE / RATER internal consistency

HOMOGENE OVER INDICATORS ITY Sources: Allen and Yen (1979); Seltiz et al.(1986); Sekaran (1992)

SEPTEMBER2001/10/28/2008

unidimensionality

VALIDITY ESTIMATES CONTENT/ APPARENT VALIDITY

FACE VALIDITY

. . . in the judgment of others -

Kesepakatan pakar tentang sejauh mana definisi operasional/ indikator yang dipergunakan suatu instrumen benar-benar mengukur konsep yang ingin diukur LOGICAL/SAMPLING VALIDITY . . . capture the entire dimensions Kesepakatan pakar tentang sejauh mana definisi operasional/ indikator-indikator suatu instrumen mewakili keseluruhan dimensi konsep yang diukur

CRITERIONRELATED VALIDITY Pragmatic validity

CONCURRENT VALIDITY

correlated with a preexisting criterion distinguish objects that differ in their present status - tingkat sejauh mana hasil pengukuran berkorelasi dengan pengukuran konsep lain atau kondisi tertentu yang diasumsikan sebagai kriteria (e.g.: Kemampuan Index of Democracy menempatkan AS dan Korut dalam kategori berbeda) PREDICTIVE VALIDITY

correlated with future/predicted condition - distinguish objects that will differ in the future . Tingkat sejauh mana hasil pengukuran suatu konsep mampu memprediksi keadaan di masa datang (Contoh: Indeks Liberalisasi Ekonomi 1980 dan Indeks Demokratisasi 1990) CONSTRUC T VALIDITY Theoryrelated validity

CONVERGENT VALIDITY

positively correlated with similar constructs Tingkat sejauh mana hasil pengukuran suatu konsep ber-hubungan positif dengan hasil

SEPTEMBER2001/10/28/2008

pengukuran konsep lain yang secara teoretis harus sama (e.g.: Index Human Freedom Index dan Civil Liberties Index) DISCRIMINANT VALIDITY

uncorrelated or negatively correlated with opposing construct - Tingkat sejauh

mana hasil pengukuran suatu konsep berbeda (tidak berkorelasi / berkorelasi negatif) dengan pengukuran konsep lain yang secara teoretis harus berbeda (Contoh: Human Freedom Index dan Totalitarianism Index)

SEPTEMBER2001/10/28/2008

RELIABILITY The consistency of a measure (Bailey, 1987)

STABILITY

RELIABILITY RELIABILITY:

OVERTIME RELIABILITY: Konsistensi hasil pengukuran dari waktu ke waktu

EQUIVALENCE

OVER-RATERS/ALTERNATE

Konsistensi hasil suatu pengukuran dng. pengukuran lain yang serupa, atau dng. pengukuran yang dilakukan pengamat lain (memakai instrumen yang sama) internal

consistency (summative rating scales)

HOMOGENEITY

OVER-INDICATORS:

Konsistensi hasil pengukuran antar indikator dalam suatu instrumen pengukuran

unidimensionality SEPTEMBER2001/10/28/2008

(cumulative rating scales)

SEPTEMBER2001/10/28/2008

STABILITY ESTIMATES: OVERTIME RELIABILITY Pearson’s r Correlation Coefficient

SUBJECTS A

TEST-1 ( t1 )

TEST-2 ( t2 )

X2

Y2

4.0

3.75

1600

14.06

3.0

3.00

900

9.00

3.5

3.25

1225

10.56

.5

1.75

25

3.06

1.0

2.00

100

4.00

1.5

2.25

225

5.06

2.5

3.00

625

9.00

XY

150.00 B 90.00 C 113.75 D 8.75 E 20.00 F 33.75 G 75.00 N=7 491.25

∑X= 160

∑Y= 19.00 ∑X2= 4700 ∑X2= 54.74

XY=

Pearson’s r Correlation Coefficient N∑ xy rxy =

(∑x) (∑y)

= 0.98 [ N∑x2 - (∑x)2 ] [ N∑y2 - (∑y)2 ]

Statistical significance

SEPTEMBER2001/10/28/2008

∝ = 0.05 df = n - 2 = 5 rc0.05;5 = 0.74 r xy = 0.98 > rc0.05;5 ( reject H 0)

critical value significant

HOMEGENEITY ESTIMATES: INTERNAL CONSISTENCY

(summative rating scales ) • Spearman-Brown coefficient Split-half reliability coefficients rX1X2 =

2 rY1Y2 _____________ 1 + rY1Y2

rX1X2 = reliabilitas pengukuran seluruhnya rY1Y2 = korelasi antara skor belahan 1 dan 2

•

Cronbach's α

rX1X2 =

2 [ s2 - ( s2Y1 + s2Y2 ) ] ______________________ s2Y1Y2

rX1X2 = reliabilitas pengukuran seluruhnya s2Y1 = varians skor belahan 1 s2Y2 = varians skor belahan 2 SEPTEMBER2001/10/28/2008

HOMEGENEITY ESTIMATES: INTERNAL CONSISTENCY HUMAN FREEDOM INDEX Pearson’s r Correlation Coefficient

Spearman-Brown coefficient 2 rY1Y2

N∑ xy - (∑x) (∑y) rxy =

rxy =

[ N∑x2 - (∑x)2 ] [ N∑

1

y2 - (∑y)2 ]

NEGARA A B C D E F G H I J K L

Skor HFI belahan 1 (X) 14 15 4 13 12 3 14 15 4 13 12 3

Skor HFI belahan 2 ( Y)

X2

Y2

XY

14 14 5 12 14 4 14 14 5 12 14 4 ∑X2

SEPTEMBER2001/10/28/2008

+ rY1Y2

∑Y2

∑XY

HOMEGENEITY ESTIMATES UNIDIMENSIONALITY

(cumulative rating scales ) COEFFICIENT OF REPRODUCIBILITY: . . . the percentage of original responses that could be reproduced by knowing the scale scores to summarize them (Babbie, 1992; p.186). Contoh penghitungan coefficient of reproducibility skala Freedom of Expression INDICATORS:

1.All dissent is forbidden and effectively supressed. Citizens are wary of criticizing the government even privately 2.Dissent is discouraged, whether by informal pressure or by systematic censorship, but control is incomplete. There is some freedom of private discussion 3.Citizens express their views on all topics without fear of punishment Observatio n pattern Scale types

Mixed types

No. of countr ies

Index score s

Scale scores

YE S 1

NO

1

0

1

0

0

Total scale errors

1

0

0

10

3

1

0

0 1 0 1

1 0 0 0

0 1 0 0

7 2 8 1

2 1 0 1

2 3 0 2

0 0 0 18

1 0 1

1 1 0

0 0 1

4 5 7 44

2 1 2

1 1 1

14 5 7

number of CoR =

errors

1number of cases x

item

SEPTEMBER2001/10/28/2008

number of

16 Sep 98 SPSS for MS WINDOWS Release 6.0

Page 6

RANKING INDONESIA : CIVIL LIBERTIES - POLITICAL RIGHTS - POLYARCHY HUMAN FREEDOM CNTRY

RCIVLIB8

RPOLRIG8

RPOLY85

RHDI87

RHFI85

43.0 33.5 33.5 33.5 52.0 43.0 33.5 52.0 43.0 52.0 69.0 52.0 33.5 62.0 62.0

41.0 31.5 41.0 31.5 45.5 50.5 31.5 45.5 50.5 45.5 61.5 50.5 41.0 61.5 61.5

18.0 18.0 8.5 8.5 18.0 18.0 18.0 18.0 25.0 25.0 43.5 25.0 28.5 43.5 43.5

29.0 24.0 38.0 61.0 25.5 55.0 45.0 34.0 22.0 67.0 46.0 52.0 62.0 44.0 43.0

36.0 38.5 38.5 38.5 45.0 45.0 49.0 na 53.5 58.0 60.0 62.5 62.5 66.5 68.0

Mexico SKorea Thailand India Singapore Egypt Philippine Malaysia Chile Bangladesh Saudi Indonesia Pakistan China Libya N

=

71.0

71.0

45.0

71.0

71.0

Number of cases read:

71

Number of cases listed:

71

HUMAN FREEDOM INDEX HFI disusun sebagai suatu Summative Rating Scale (Likert-type Scale) berdasarkan 40 indikator, a.l., kebebasan untuk menyatakan pendapat, melakukan oposisi, menentukan pasangan, kebebasan dari penganiayaan, dsb. oleh Charles Humana (1986). INDICATORS

Rights to: 03. To assembly Freedom from: 07. Unlawful detention Freedom for: 19. Political SEPTEMBER2001/10/28/2008

4 Respect for rights or freedom

3 Some violations or infringement s

2 Substantial violation

1 Continuous or total denial

opposition Personal Rights 40. Homosexuality HUMAN FREEDOM INDEX (HFI): Dalam laporan tahunan UNDP, Human Development Report 1991, mempergunakan data Human Rights Rating yang disusun oleh Charles Humana (World Human Rughts Guide 1986).

SEPTEMBER2001/10/28/2008

POLYARCHY INDEX Indeks Polyarchy dikembangkan sebagai Cummulative Rating Scale (Guttman-type Scale). Mengukur pluralisme politik berdasarkan keberadaan perangkat tatanan kelembagaan yang memungkinkan dan menjamin adanya oposisi publik dan hak untuk berpatisipasi dalam proses-proses politik . . . the set of institutional arrangements that permits public opposition and establishes the right to participate in politics (Coppedge and Reinicke, 1993; p.47).

CONCEPT

POLYARC HY

DIMENSI ONS

INDICATORS

FREE & FAIR ELECTION

1.No meaningful election are held 2.Marred by fraud and coercion 3.Meaningful fair election

FREEDOM OF ORGANIZA TION

1.All organizations are banned or controlled 2.Only nonpolitical organizations are allowed 3. Some independent political organizations are banned 4.Full freedom for political organization

FREEDOM OF EXPRESSIO N

1.All public dissent is suppressed 2.Some public dissent is supressed 3.Full freedom of expression

AVAILABILI

1.No public alternative to

TY OF ALTERNATI VE INFO SOURCES

official information 2.Alternative sources exist only for nonpolitical issues 3.There is preferential presentation of official views in the media 4.No preferential presentation of official views in the media

Disusun berdasarkan: Coppedge and Reinicke (1993) “Measuring Polyarchy”. In Inkeles, Inkeles, Alex (Ed.), On Measuring Democracy: Its Cosequences and Concomitants. New Brunswick, London: Transaction Publishers; pp. 47-68.

POLITICAL RIGHTS - CIVIL LIBERTIES Dua Indeks yang sebagai kombinasi dimaksudkan untuk menentukan tingkat kehidupan demokrasi. Suatu Summative Rating Scale (Likert-type Scale) yang dikembangkan oleh Raymond Gastil, berdasarkan 11 indikator Political Rights dan 14 indikator Civil Liberties ( lihat “The Comparative Survey of Freedom: Experiences and Suggestions”. In Inkeles, Inkeles, Alex (Ed.), On Measuring Democracy: Its Cosequences and Concomitants. New Brunswick, London: Transaction Publishers; pp. 21-46.).

Sample Checklist for Political Rights 1. Chief authority recently elected by a meaningful process 2. Legislature recently elected by a meaningful process Alternatives for 1 and 2: a. No choice and possibility of rejection b. No choice but some possibility of rejection c. Government or single-party selected candidates d. Choice possible only among governmentapproved candidates e. Relatively open choices possible only in local elections

f. Open choice possible within a restricted range g. Relatively open choices possible in all elections Sample Checklist for Civil Liberties 17. Free from unjustified political terror or imprisonment 18. Free trade unions, peasant organizations, or equivalent 19. Free businesses or cooperatives

- -

Correlation Coefficients - HFI85 HDI87 HFI85 -

CIVLIB88

HDI87

.6130 ( 68) P= .000

CIVLIB88

.8142 .7174 ( 68) ( 71) P= .000 P= .000

POLRIG88

.7612 ( 68) P= .000

.6706 ( 71) P= .000

.9284 ( 71) P= .000

POLY85

.7102 ( 42) P= .000

.4652 ( 45) P= .001

.8295 ( 45) P= .000

GNP6588

.1118 ( 68) P= .364 R E L I A B I L I T Y N of Cases = 86.0 Guttman Split-half = 20 Items in part 1 Alpha for part 1 =

Improve98/10/28/2008

POLRIG88

POLY85

GNP6588

-

-

-

.7830 ( 45) P= .000

-

.3144 .1674 .2306 .2514 ( 71) ( 71) ( 71) ( 45) P= .008 P= .163 P= .053 P= .096 A N A L Y S I S S C A L E (S P L I T) N of Items = 40 .9791 Unequal-length Spearman-Brown = 20 Items in part 2 .9583 Alpha for part 2 =

.9820 .9454

23

FACTORIAL VALIDITY KONFIRMASI VALIDITAS KONSTRUK PENGUKURAN MELALUI FACTOR ANALYSIS (CONFIRMATORY FACTOR ANALYSIS)

LIBERALISM – CONSERVATISM A DUALISM CONCEPT ? LIBERALISM CONSERVATISM DUALITY

CON1 LIBERALISM

CONSERVATISM DUALISM

CON2 CON3 LIB1

FACTOR . . . a construct, a hypothetical entity, a latent variable that is assumed to underlie

Improve98/10/28/2008

LIB2 LIB3

FACTOR I FACTOR II .03513 .9476 6 .09007 .9408 2 .05609 .8541 0 .22722 .8109 1 .9434 .05173 8 .9458 .10260 4

24

tests, scales, items, and indeed, measures of almost any kind (Kerlinger, 1986; p. 569).

PRINCIPLES TO INCREASE THE RELIABILITY OF MEASURES (Neuman, 1997; pp. 147-148)

• • • •

Improve98/10/28/2008

Clearly conceptualize all constructs Increase the level of measurements Use multiple indicators of a variable Use pretest, pilot studies, and replication

25

JOB SATISFACTION SCALE (Adopted from Schuessler’s Job Satisfaction Scale. In Miller, 1991; p.453)

SD A 5

Statements

DA 4

N 3

A 2

SA 1

1. There is too little variety in my job 2. I tend to get bored on the job 3. There must be better places to work 4. I would like more freedom on the job 5. I have to small a share in deciding matters that affect my work 6. My job means more to me than just money* 7. I am not satisfied with the the work I do 8. My job gives me a chance to do what I do best* 9. People feel like they belong where I work* *Positive statements 12 Oct 98 SPSS for MS WINDOWS Release 6.0

Statistics for SCALE Improve98/10/28/2008

Mean 31.4733

Variance 4.2743

Std Dev 2.0674

Page 1

N of Variables 9 26

N of Cases =

129.0

JOB SATISFACTION SCALE R E L I A B I L I T Y

Reliability Coefficients N of Cases = 129.0

A N A L Y S I S

Alpha =

-

S C A L E

N of Items =

(A L P H A) 9

.6198

Item-total Statistics

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8

Scale Mean if Item Deleted

Scale Variance if Item Deleted

28.0275 27.9570 27.9612 27.9756 27.9054 27.9399 28.1163 27.9008

3.5993 3.3573 3.2213 2.4755 3.2929 3.2890 3.2524 4.5462

Improve98/10/28/2008

Corrected ItemTotal Correlation .1503 .5014 .5466 .7012 .3719 .5468 .5300 -.3015

Alpha if Item Deleted .6419 .5462 .5301 .4419 .5719 .5347 .5350 .6782 27

Q9

28.0023

4.8057

-.4374

12 Oct 98 SPSS for MS WINDOWS Release 6.0

.7132 Page 5

13 Oct 98 SPSS for MS WINDOWS Release 6.0

Page 2

JOB SATISFACTION SCALE R E L I A B I L I T Y

A N A L Y S I S

-

S C A L E

(A L P H A)

Item-total Statistics

Q2 Q3 Q4 Q5 Q6 Q7

Scale Mean if Item Deleted

Scale Variance if Item Deleted

17.4627 17.4658 17.4873 17.4181 17.4485 17.6254

3.6783 3.5845 2.4132 2.9834 3.1811 3.2917

Corrected ItemTotal Correlation

Alpha if Item Deleted

.3779 .3960 .8262 .6503 .7433 .6072

.8343 .8326 .7372 .7829 .7700 .7938

Reliability Coefficients N of Cases = Improve98/10/28/2008

130.0

N of Items =

6 28

Alpha =

.8241

13 Oct 98 SPSS for MS WINDOWS Release 6.0

Page 3

WORDING EFFECTS IN REFERENDUM % Differences b/w cons and pros Common Market Should the United Kingdom come out of the Common Market?

The Government recommends the acceptance of the renegotiated terms of British membership of the Common Market. Should the United Kingdom stay in the Common Market? Her Mayesty’s government believes that the nation’s best interests would be served by accepting the favourably Improve98/10/28/2008

+ 10.8 - 11.2

- 16.2 29

negotiated terms of our continued membership of the Common Market. Should the United Kingdom stay in the Common Market Source: Butler and Kitzinger (1976; p.60)

Improve98/10/28/2008

30

BEBERAPA PRINSIP PENYUSUNAN INSTRUMEN PERTANYAAN (Sumber: Neuman, 1997; pp.223-237)

• Avoid jargon, slang, and abbreviations “Apakah suami anda memperlihatkan gejala Oedipus complex?”

• Avoid ambiguity, confusion, and vagueness “Apakah anda berolahraga secara teratur?”

• Avoid emotional language and prestige bias “Kalangan berpendidikan umumnya setuju agar film-film televisi bertema kekerasan dan sex dilarang. Apakah anda setuju dengan pelarangan film-film televisi seperti itu?”

• Avoid double-barreled questions “Apakah anda setuju dengan rencana untuk menyelenggarakan Sidang MPR bulan Oktober 1998 dan Pemilihan Umum bulan Mei 1999?”

• Avoid leading questions “Haruskah pemerintah mengucurkan dana lebih banyak lagi untuk mengamankan jalanan ibukota yang semakin rawan kejahatan?”

• Avoid asking questions that are beyond respondents’ capabilities “Berapa menitkah rata-rata anda menonton televisi selama tiga bulan terakhir ini?”

• Avoid false premises “Pemda telah terlalu banyak memberi layanan kepada masyarakat. Apakah anda setuju layanan tersebut dikurangi demi menghemat biaya?” Improve98/10/28/2008

31

• Avoid overlapping or unbalanced “Bagaimanakah penilaian anda terhadap kinerja pimpinan anda? Luar biasa, istimewa, sangat bagus, bagus, atau memuaskan?

Improve98/10/28/2008

32