Jumat, 20 April 2012

THE ASSIGNMENT OF ITEM ANALYSIS


I  just wanna share my last assignment 

THE ASSIGNMENT OF ITEM ANALYSIS
By :
Hikhmah Oky Pravitasari

ENGLISH DEPARTMENT
LANGUAGES AND ARTS FACULTY
STATE UNIVERSITY OF SURABAYA
2008

ITEM ANALYSIS

  1. INTRODUCTION
Testing has an important role in teaching learning process. It is used as an instrument for measuring and evaluating activities. Brown (2004: 03) states that a test is needed to measure the student’s abilities or performance which focuses on specific competencies or objectives. Furthermore, a teacher must have a good or qualified test for the students to get correct information about student’s abilities. A qualified test means that a test must fulfill some criteria of a good test.
In this paper I would like to analyze the final English test items for the ninth year students of SMPN 1 NGORO MOJOKERTO 2008. The format of the test is multiple choices which consist of 20 items number. The analyses are about some criteria of good test. They are validity of the test (face, content, construct), reliability of the test, item analysis and also the revision.

  1. VALIDITY
Validity refers to whether a test measures what we intend it to measures (tucker, 1975:229). It means that validity of a test measure that it is supposed to measure. If a test is able to measure what it its purposes, then the test has high validity. In this case I’ll discuss the validity of a test into three types; face validity, content validity, and construct validity.

1.    Face Validity
Face validity of a test is that the way of a test is done by looking at the way of the test is written, whether the test is constructed based on the step of how to construct a good or not. Harris (1969:20) stated that face validity can never be permitted is the way the test looks’.
Based on Haris opinion above, the researcher concludes that the face validity of a test refers to:
  1. The way the test ‘looks’ based on the tastes, the test administrator, and the educators’ point of views.
  2. The way the test is organized. It consists of test and test directions.
After identifying all of the test items, the writer makes an analysis the validity of the English summative test items for the ninth year students of SMPN 1 NGORO MOJOKERTO 2008. The test look is enough, because the test has no cover and it has specific object clearly such as class, subject lesson, time. The size is of letter is 12, good and can be read easily. The test direction is enough because it has general instruction and it still there are many instructions which is not clears enough. A test item type is multiple choices. It consists of 20 item test. They are presented following the level of difficulty.
 The Result of Analysis of Face Validity
No
Aspect of Test and List of Question
Characteristic
Explanations
Good
Enough
Poor
1.
Test Looks
·         How is the test’s cover?
·         How is the size of letter used?

·         How is the size of paper used?



x


x



x



o   The test has no cover.
o   The size is 12, good and can be read easily.
o   The paper used is legal size paper
2.
Test Directions
·         How is the general instruction of the test?
·         How is the specific instruction of every part?





·         How is the instruction for going on for the next section or to next page?

x







x


















x

o   test has no general instruction
o   The specific instructions are clear because the test only consist one part.

o   There is no instruction as an ending of the test. For example. This is the last page.


3
Test Items Types
·         How are the types of the test item chosen?
x


o   Test Item types is multiple choice.





2. Content Validity
As we know that the analysis of content validity of a test is mean to find out the relationship of the test items with the content of materials based on the curriculum being used. The most important question here is whether the test item is measuring what should be measured. Heaton (1975:154) states that content validity is extended to which a test measure a representative sample of the subject-matter content. A test can be said to have high content validity if the test items represent almost all of the content of materials stated in the subject matter.

The Example of analyzing the Content Validity

No
learning content
Learning objectives
Number of test items
The sum of the items
%
1.











Membaca
-Siswa dapat menjawab pertanyaan berdasarkan informasi dalam text

-Siswa dapat mengidentifikasi makna kata yang terdapat dalam teks bacaan.

6,8,10,22,26,
28,34





2,4,12,14,
24,36,40
7






7
70%







2.

Kosakata
-          -Siswa menguasai kosa kata:

# body
# insect
# honey
# eggs

# Useful
# Good
# Important
# Anywhere

# Somewhere
# Anywhere
# Nowhere
# Everywhere



16




18




20



1




1




1



15%
3.
Tata Bahasa dan penggunaan unsur bahasa
-          -Siswa dapat menggunakan Question tag.
-           
-          -Siswa dapat menggunakan  Oresent perfect.

30



38
1



1
10%



4.
Pengetahuan umum
-siswa dapat mengungkapkan tujuan teks dialog
32
1
5%

Based on the analysis above, the percentage of content validity is concluded as follows:
1.    There is no item for listening and speaking skill since the test is given in the written form (0%).
2.    There are fourteen items for reading skill. It is number 2, 4, 6, 8, 10, 12, 14, 22, 24, 26, 28, 34, 36, 40. (70%)
3.    There are three items for vocabulary usage. They are number 16, 18, 20 (15%).
4.    There are three items for Grammar. They are number 30, 38 (10%).
5.    There is one general knowledge. Its number 32 (5%).

3. Construct Validity
Beside considering the content validity. A good test must have construct validity. According to Heaton (1975:154) if a test has construct validity, it is capable of measuring certain specific characteristics in accordance with a theory of language behavior and learning. This type of validity assumes the existence of certain learning theories or constructs underlying the acquisition of abilities and skills.

Item
Kompetensi Dasar
Indikator
Soal
Construct
Yes
No

2.





4.







6.


















8.





10.
























12.





14.





16.







18.




20.





22.




















24.







26.













28.


30.






32.




34.












36.






38.









40.


MEMBACA
Tidak sesuai dengan kurikulum.



Idem







11.3 Merespon makna dan langkah retorika dalam esei pendek sederhana secara akurat, lancar, dan berterima untuk berinteraksi dalam konteks kehidupan sehari-hari dalam teks berbentuk report.

Idem


.


Idem
























Idem          





Idem





Idem







Idem             




Idem





11.3 Merespon makna dan langkah retorika dalam esei pendek sederhana  secara akurat, lancar, dan berterima untuk berinteraksi dalam konteks kehidupan sehari-hari dalam teks berbentuk narrative.

Idem







Merespon makna dalam teks fungsional pendek secara akurat, lancar dan berterima untuk berinteraksi dalam konteks kehidupan sehari-hari


Idem


Idem






Tidak sesuai dengan kurikulum.


Merespon makna dalam teks fungsional pendek secara akurat, lancar dan berterima untuk berinteraksi dalam konteks kehidupan sehari-hari.

Idem






Idem









Idem






Mengidentifikasi makna kata yang terdapat dalam teks descriptive.


Menentukan makna kata/ sinonim yang terdapat dalam teks report.



Mengidentifikasi informasi tertentu berdasarkan teks report.















Mentukan tujuan/fungsi social dari teks report dengan benar.

Menyusun sebuah paragraph menjadi sebuah teks report yang padu dan benar.




















Menentukan makna kata/ Antonim yang terdapat dalam teks report.

Menentukan makna kata/ sinonim yang terdapat dalam teks report.

Melengkapi dengan kosakata yang benar dari sebuah teks rumpang sederhana berbentuk report.

Idem




Idem





Mengidentifikasi informasi tertentu berdasarkan teks narrative.

















Mengidentifikasi makna kata yang digarisbawahi berdasarkan parahraaf narrative.


Mengidentifikasi informasi tertentu berdasarkan teks fungsional pendek berbentuk pengumuman.








Idem


Menjawab pertanyaan tentang question tag.



Menentukan tujuan umum dari teks dialogue.


Mengidentifikasi informasi tertentu berdasarkan teks fungsional pendek surat pribadi.







Menentukan makna kata yang berdasarkan teks fungsional pendek surat pribadi.

Menentukan past perfect dari paragraph rumpang.






Mengidentifikasi informasi tertentu berdasarkan teks fungsional pendek tentang pengumuman.

The word “gallop” in the last sentence means…



The word” this coloring” in the second paragraph refer to…




From the text above the correct statement is….
















The purpose of the writer of the text is…




Chose the right order of the following sentences
  1. It takes place within three or days.
  2. The more important the person who dies, the more buffaloes they kill.
  3. It is carried out in honour to the death person.
  4. One of the Torajan traditions is the funeral.
  5.  It is done by slaughtering a water buffalo.


…Indonesia Ocean whose huge




It is well-known for its long beautiful” in the first paragraph. The synonym of the underlined word is…

Bees give us ….(16) and beeswax.






Some insect are…. (18) Because they spread disease and destroy crops.

If you don’t subscribe newspaper, you can have it…in the town there are kiosk selling them.

What lesson can we learn from the story?



















“When all the animals saw them coming” in the third paragraph, the underlined word refers to…


“What does the text tell us about?












What is the aircraft design for?

Joko: Did you watch the film last night?
Joyo: Yes, I do
Joko: Lovely……
Joyo: yes it was.


What is the purpose of the dialogue?



In Sue’s opinion, what is the resort like?










I can’t wait to pack my bag and go home.





Rina: it is hot news. Have you heard it?
Sita: Not yet. What news?
Rina: Susi ….. a new dress for the party.
Sita: Really.
Rina: Yes I saw it yesterday.

Where this announcement would be put?

x





x







x


















x





x























x











x







x




x





x




















x







x













x









x




x












x
















x













































































































































x































x


















Index of Mean and Calculation Reliability of the Objective Test

No
X
f
fX
X
d
d2
1.
2.
3.
4.
5.
6.
7.
8.
9.

17
16
15
14
13
12
11
10
9
1
2
2
3
2
1
1
2
2
17
32
30
28
26
24
22
20
18
17
16
16
15
15
14
14
14
13
13
12
11
10
10
9
9
3,44
2,44
2,44
1,44
1,44
0,44
0,44
0,44
-0,56
-0,56
-1,56
-2,56
-3,56
-3,56
-4,56
-4,56
11,82
5,96
5,96
2,1
2,1
0,2
0,2
0,2
0,31
0,31
2,43
6,55
12,67
12,67
20,8
20,8



Σx=16
Σfx=217


Σd2=105,17

Mean :                           Deviation
x = ΣfX                               d = X - x
        n
x = 217                               Where: d = deviation
        16                                         X = real score
x = 13,56                                               x = mean


  1. RELIABILITY

Beside we learn the important the importance of validity. We have to know the other a necessary characteristic any a good test. It is reliability. Reliable means that the result of the test can be trusted or can be relied. Reliability of the test is the stability of the test scores (Haris, 1969:14).
Tuckman (1975:157) states that the reliability of the whole test can be estimated by using the whole KR-21 formula.
KR-21 formula:



Reliability= 1 – x (n-x)
                          n.sd2
 
 




Note:
X = mean
n = The sum of the students
Sd = Standard Deviation

KR-21 = 1 – x(n-x)
                  n. sd2
                 =   1 – 13,56 ( 16-13,56)
                    16(6,6)
            = 1- 33,0864
                   105, 7
          = 1-0,313
          = 0, 68
                

Σd2
105,17
Sd2
6,6
KR-21 (Reliability
0,68

D.     ITEM ANALYSIS

Item analysis refers to the process of collecting, summarizing, and using information about individual test item, especially information about student’s responses to items. Brown (1996:50) states” item analysis is performed simply to investigate how well the items or a test are working with a particular group of students.
According to (Heaton, 1975:196) “an item analysis is done in order to identify the best items. It means that the teacher have to know whether the item are good or not in order to get the better result.

1.    Item of Difficulty
Item difficulty is also called facility value (the index og difficulty). Facility value (F.V) is generally expressed as the fraction of the students who answered correctly. It is calculated by using the formula:

 F.V = Correct U + Correct L
                                            2n                                  
Where:
F.V                   = the facility value or the index of difficulty
Correct U        = the number of the students in Upper group who answered the item correctly.
Correct L        = the number of the students in lower group who answered the item correctly.

2.    Item Discrimination
According to Heaton (1975:179) “the discrimination index of an item indicates the extent to which the item discriminates between the tastes, separating the more able from the less able”. The index of discrimination (D) tells us whether those students who performed well on the whole test tended to do well or badly on each items in the test. It is calculated by using the formula:

 F.V = Correct U - Correct L
                                        n                                    
Where:
F.V                   = the facility value or the index of difficulty
Correct U        = the number of the students in Upper group who answered the item correctly.
Correct L        = the number of the students in lower group who answered the item correctly

.
Upper Group
1
2
3
4
5
6
7
8
Lower Group
1
2
3
4
5
6
7
8


17
16
16
15
15
14
14
14

13
13
12
11
10
10
9
9




Data for Analyzing the Index of Discrimination


Item
U
L
U+L
F.V
U-L
    D.P
COMMENT
F.V
D.P
2
6
6
12
0,8
0
0
E
P
4
8
6
14
0,9
2
0.3
E
S
6
4
3
7
0,4
1
0,1
M
P
8
8
2
10
0,6
6
0,8
M
E
10
8
5
15
0,8
3
0,4
E
G
12
8
4
12
0,8
4
0,5
E
G
14
6
7
13
0,8
-1
-0,1
E
VP
16
7
7
14
0,9
0
0
E
P
18
6
4
10
0,6
2
0,3
M
S
20
5
6
11
0,7
-1
-0,1
M
VP
22
6
3
9
0,6
3
0,4
M
G
24
7
4
11
0,7
3
0,4
M
G
26
6
5
11
0,7
1
0,1
M
P
28
6
4
10
0,6
2
0,3
M
S
30
5
6
11
0,7
-1
-0,1
M
VP
32
3
3
6
0,4
0
0
M
P
34
6
3
9
0,7
1
0,1
M
P
36
3
2
5
0,3
1
0,1
D
P
38
7
5
12
0,8
2
0,3
E
S
40
7
2
9
0,6
5
0,6
M
G

NOTES:

             F.V
       The standard value of the index of difficulty is:
0.00-0.30       = difficult
0.30-0.70       = good (average)
0.70-1.00.       = easy     (Arikunto, 90: 212).
According   to   Arikunto (1990:   221),   the   criteria   of the   index   of discrimination are as follows:


        D.P
E    : Excellent        (0, 7 – 1, 00)
G   : Good (0, 4 – 0, 7)               
S    : Satisfactory   (0, 2 – 0, 4)
P    : Poor               (0, 0 – 0, 2)
VP  : Very Poor      (Negative)






3. The effectiveness of Distracter

"Distracter efficiency is one more important measure of multiple-choice items value in test and, one that is related to the item discrimination" (Brown, 2004; 60). It is really important to know the item is good, not good or non function. From the table below, it can be seen that the test is good when the correct answer in upper group is higher than the lower group. Vise versa, when the uncorrected answer in lower group is higher than the upper group. The analysis the test item can be said not good if the correct answer in upper group is lower than the lower group and also the uncorrected answer in lower group is lower than the upper group. The last, non function means if the result of upper and lower group are the same.







Table of Function of Key

Item
Options
Upper
Lower
Comment
2



  A*
B
C
D
6
1
0
0
6
0
2
0
NF
NG
G
NF
4
A
B
 C*
D

0
0
8
0
2
0
6
0
G
NF
G
NF
6
A
B
 C*
D
1
0
4
3
1
0
3
4
NF
NF
G
G
8
A
B
 C*
D

0
0
8
4
0
1
2
0
NF
G
G
NG
10
A
B
 C*
D

0
0
8
0
1
1
5
1
G
G
G
G
12
A
B
 C*

D

0
0
8
0
1
3
4
0
G
G
G
G
14
 A*
B
C
D

6
1
1
0
7
0
1
0
NG
NG
NF
NF

16

A
B
 C*
D

0
1
7
0

0
0
7
1
NF
NG
NF
G
18
A
B
C
 D*
0
0
2
0
1
1
1
4
G
G
NF
G
20




 A
  B*
C
D

0
5
1
2
0
6
0
2
NF
NG
NG
NF
22
 A
  B*
C
D

2
6
0
0
4
3
0
1
G
G
NF
G
24
A
B
C
  D*

0
1
0
7
2
1
1
4
G
NF
G
G

26
   A*
B
C
D

6
2
0
0
5
3
0
0
G
G
NF
NF
28
A
B
  C*
D

1
0
6
1
1
0
4
3
NF
NF
G
G
30



A
B
  C*
D
1
2
5
0
0
2
6
0
NG
NF
NG
NF
32.




   A*
B
C
D
3
2
2
1
3
1
1
3
NF
NG
NG
G
34




   A*
B
C
D
6
1
1
3
5
1
0
1
G
NF
NG
NG
36.

A
  B*
C
D
1
3
2
2

4
2
2
0
G
G
NF
NF

38.
A
B
C
   D*
0
0
1
7
2
1
0
5
G
G
NG
G
40.
A
B
C
   D*
0
0
1
7
2
2
2
2

G
G
G
G











Where:
G = good
NG = Not Good
        NF = Non Function

The analysis and the revision of the items

No
Question + Option
The analysis
Revision

2
The word “gallop” in the last sentence means….
a.       run
b.       protect
c.       fight
d.       walk
This question is clear; most students can answer the question correctly. The correct answer between upper and lower group is the same. But from the content it is not include the curriculum
No revision
4
The word “this coloring” in the second paragraph refer to…
a.       The big brown eyes
b.       I have no doubt
c.       I’m absolutely certain
d.       I doubt that

This item is valid in constructs validity, but in content it is not include the curriculum. most of students can answer correctly. And index pf discrimination is satisfactory.
No revision
6.








From the text above the correct statement is….
  1. Earth is the only planet containing water.
  2. All planets containing water.
  3. There is living in the earth.
  4. To identify the character of earth.

This item is valid in content and construct, but it is not in index discrimination, this item is poor (it can’t determinate).
My suggestion, the choices of the answer must be changed.

From the text above the correct statement is….
  1.  only planet containing water.
  2.  Planets containing water.
  3. There is living in the earth.
  4. To identify the character of earth.

8.


The purpose of the writer of the text is…
  1. to tell about the general truth
  2. To tell the history of the earth.
  3. To give description about the earth.
  4. To identify the character of the earth
This item is valid in construct and content. It is moderate in faculty value and excellent in DP. It can determinate between upper and lower class.
No revision
10



Chose the right order of the following sentences
  1. It takes place within three or days.
  2. The more important the person who dies, the more buffaloes they kill.
  3. It is carried out in honor to the death person.
  4. One of the Torajan traditions is the funeral.
  5.  It is done by slaughtering a water buffalo.
  1. 2-5-4-3-1
  2. 2-3-5-4-1
  3. 4-3-1-5-2
  4. 4-3-5-2-1
This item is valid in content and construct. It includes the curriculum and it includes the reading skill. FV is moderate, and DP is good, it means that it can determinate between upper and lower class students.
No revision
12.



…Indonesia Ocean whose huge
  1. enormous
  2. giant
  3. tiny
  4. massive

This item is valid. It can be seen in content and construct. It is reading skills that concern the vocabulary. It can determinate between upper and lower class.
No revision
14
It is well-known for its long beautiful” in the first paragraph. The synonym of the underlined word is…
  1. famous
  2. important
  3. interesting
  4. beautiful
This item is valid. It can be seen in content and construct. It is reading skills that concern the vocabulary. All students can answer it correctly, But it can’t determinate between upper and lower class, this item must be changed.
It is well-known for its long beautiful” in the first paragraph. The synonym of the underlined word is…
a.       famous
b.       popular
c.       important
d.       interesting

16
Bees give us ….(16) and beeswax.
  1. body
  2. insect
  3. honey
  4. eggs

This item is valid. It can be seen in content and construct. It is reading skill that concern in vocabulary. All students can answer it correctly. But it can’t determinate between upper and lower class. So this item must be changed.
Bees produce….(16) and beeswax.
a.       milk
b.       honey
c.       eggs
d.       silk
18


Some insect are…. (18) Because they spread disease and destroy crops.
  1. useful
  2. good
  3. important
  4. harmful

This item is valid. It can be seen in content and construct. It is reading skill that concern in vocabulary. Most of students can answer it correctly. It can determinate between upper and lower group. We can see it from the faculty value and index of discrimination.
No revision
20

If you don’t subscribe newspaper, you can have it…in the town there are kiosk selling them.
  1. somewhere
  2. anywhere
  3. nowhere
  4. everywhere

This item is valid. It can be seen in content and construct. It is reading skill that concern in vocabulary. The index of discrimination is very poor. It can determinate between upper and lower class. So the item must be changed. The word anywhere and everywhere is ambiguous
If you want to look for newspaper, you can have it…in the town there are kiosk selling them.
  1. Somewhere
  2. Anywhere
  3. Nowhere
  4. there

.
22


What lesson can we learn from the story?
  1. be yourself
  2. don’t easily be tricked
  3. don’t be greedy
  4. believe in your self


This item is valid. It can be seen in content and construct. It is reading skill that based on the paragraph narrative. It can determine between upper and lower group. The question is not too easy and not too difficult.
No revision
24
“When all the animals saw them coming” in the third paragraph, the underlined word refers to…
  1. descriptive text
  2. narrative text
  3. procedure text
  4. recount text

This item is for reading skill. The students answer it correctly. The item is already good. So, no revision for this item.
No revision
26
“What does the text tell us about?
  1. The newest aircraft Boeing 747.
  2. An economical air craft.
  3. Boeing planes
  4. Paris airways
This item is for reading skill. It can’t determinate between the upper and lower class. The distracters make the students confused because the answers are similar. It means that the students still confuse to the question.
“What is the main idea of this paragraph?
  1. The newest aircraft Boeing 747.
  2. Paris airways.
  3. An efficient aircraft.
  4. Carry the passenger.

28



What is the aircraft design for?
  1. Space transportation
  2. Space shuttle
  3. Commercial air transportation
  4. Aircraft carrier

This item is reading skill based on the paragraph that concern in the curriculum. The students can answer it correctly. It doesn’t need to be revised.
No revision
30


Joko: Did you watch the film last night?
Joyo: Yes, I do
Joko: Lovely……
Joyo: yes it was.
  1. didn’t
  2. isn’t
  3. wasn’t
  4. doesn’t
This item is for grammar. The result of the index discrimination is very poor. It can’t be determinate between the upper and lower class. The students still confused about the option.  It needs to be revised.
Joko: Did you watch the film last night?
Joyo: Yes, I do
Joko: Lovely……
Joyo: yes it was
a.       isn’t
b.       wasn’t
c.       was
d.       doesn’t
32

What is the purpose of the dialogue?
  1. to present a point of view
  2. to persuade the reader
  3. to describe something
  4. to inform the weather tonight

This item is about the general knowledge. The students still confused about the purpose of dialogue.
No revision
34


In Sue’s opinion, what is the resort like?
  1. awful
  2. nice
  3. enjoyful
  4. sunny

This item is for vocabulary mastery based on the short functional text The students still confused about the vocabulary.
No revision

36


I can’t wait to pack my bag and go home.
The underline phrase means……
a. ask someone to bring it.
b. put her clothes in
c. Put her belonging into
d. carry things with

This item is for vocabulary mastery based on the short functional text. This item is too difficult because most of the students can’t answer correctly. The students still confuse about the 2 option which is similar. It must be changed.
I can’t wait to pack my bag and go home
The underline phrase means……
a. ask someone to bring it.
b. put her clothes in
c. take in.
d. carry things with

38

Rina: it is hot news. Have you heard it?
Sita: Not yet. What news?
Rina: Susi ….. a new dress for the party.
Sita: Really.
Rina: Yes I saw it yesterday
  1. will make
  2. is making
  3. can make
  4. has made
This item is for grammar and language usage, particularly in present perfect. Most of the students can answer correctly. It doesn’t need to be revised.
No revision
40
Where this announcement would be put?
  1. in all school
  2. in all parks
  3. at the shopping complex
  4. at the glory super market.
This item is reading skill. Most of the students can answer correctly. This item is also valid based on the content or construct. The difference between upper and lower class is clear.
No revision






































CONCLUSION

Based on the result of the analysis, it can be concluded that the final English test for the ninth grade made by the teacher of Junior high School SMPN 1 NGORO MOJOKERTO has a high content validity. 90% of content validity of the test based on “standard Isi” curriculum. Almost test items is reading skill which is including grammar and vocabulary. The item analysis shows that the test is not good enough since 35% of the test need revising. The reliability of the objective test of the final English test is 0.68. In conclusion, it is considered good because the reliability is more than 0,6. Thus if the the test is given to other students. They will get the score nearly the same


























References
Arikunto, S.1985. Dasar-Dasar Evaluasi Pendidikan. Jakarta: Bumi Aksara.

Brown, H. D. 2004. Language Assessment Principles and Classroom Testing. New York: Pearson Education, Inc.
Harris, D. P. 1969. Testing English as a Second Language. New York : Mc Graw-Hill, Inc.
Heaton, J.B. 1975. Writing English Language Tests. London: Longman Group Limited.
Tuckman, W. B. 1975. Measuring Educational Outcomes: Fundamentals of Testing. New York: Harcourt Brace Jovanovich, Inc.
Weir, C. J. 1993. Understanding and Developing Language Test. London: Prentice Hall International (UK) Ltd.  









Tidak ada komentar:

Posting Komentar