Developed by Peter Pappas
K - 12 Coordinator for Social Studies Standards
Pittsford Central Schools
Sources: NYS Social Studies Resource Guide, National Assessment of
Educational Progress,
"Designing and Collecting Data for School" by Paul Brandon and Terry Higa
(University of Hawaia)
Foundations:
A. Identify the topics to be addressed in the test. Remember that assessments should
always be in alignment with district standards:
Standard: What should the students understand, know and be able to do?
Assessment: How will we know if students have achieved the desired results? What
will we accept as evidence of student understanding and proficiency?
B. Assess at a variety of cognitive levels. Examples:
Knowing . Tasks in this area can measure students' ability to answer questions
by recalling information. "What is the worlds largest tropical rainforest?"
Understanding . In this area, students can be asked to attribute meaning to
what has been observed and to explain events. "Why are tropical rainforests
located near the equator?"
Applying . This area of thinking can call on students to use many tools and
skills as they attempt to develop a comprehensive understanding of a problem en route to
proposing viable solutions. "Identify two developments that are threatening the
tropical rainforests."
C. Use a wide range of authentic materials as stimuli for assessment questions.
These include an atlas, maps, charts, graphs, tables, text-based primary source documents
and literary works, and various art forms, including photographs, paintings, cartoons, and
posters.
D. Assess a range of skills related to these stimuli. Students can be asked to
synthesize information from multiple stimuli or to use outside knowledge in order to
interpret a given stimulus. They might, for example, be asked both to describe the data in
a table and to draw on outside knowledge to give factually accurate explanations for the
patterns revealed. Include performance tasks that, for example, require students to
create maps or graphs based on narratives or tables of quantitative data.
Writing Multiple-choice Test Items
A. Basic principles
The question is called an item.
Each item should be should be concerned with a single problem and presented in
clear and simple language. Avoid vague terms.
Extraneous material should be avoided. Students should be told as much as is needed to
understand the question, but no more.
Write items at the appropriate level of difficulty.
B. Writing item stems
The beginning of the multiple-choice item is called the stem.
It should be either a direct question ending in question mark or an incomplete
statement ending with a colon. It should include words that otherwise would be repeated in
each response.
Items asking for definitions should put the term to be defined in the stem and
alternative definitions in the responses.
It is not necessary to include the phrase "of the following" in the stem.
C. Writing item responses
Ideally, each item should have four possible responses.
The correct response should clearly be the best of all the possible responses.
The responses "all of the above" or "none of the above" cannot be
used.
Make all responses plausible and attractive to examinees that lack the information or
ability necessary to correctly answer the item.
Each response should be distinct from the others; avoid overlapping responses.
Arrange the responses in a logical order, if one exists. Responses consisting of
numbers should be in ascending or descending order.
Insure that the correct responses are equally distributed in responses a, b. c, d.
Avoid irrelevant clues to correct answers, such as, when the correct answers are stated
more precisely or with more words than the alternative responses.
D. Mechanics
Provide clear instructions.
If the stem is an incomplete statement, each response should begin with a lower case
letter and end with a period.
Responses following stems that are complete sentences should begin with an uppercase
letter, and either end without a period if they are phrases, or end with a period if they
are complete sentences.
Responses should be grammatically consistent with the stem and parallel with one
another in form. Sentence structure should be as simple as possible.
When possible, avoid negatively stated items. When writing items stated in the
negative, draw the respondents attention to the negative form by printing negative
words in italics.
Constructed Response Items
A. Constructed response questions:
are open ended, short answer questions that measure application-level cognitive skills
as well as content knowledge. No opinion questions.
use a range of primary and secondary stimuli and authentic "real world"
examples including time lines, maps, graphs, cartoons, charts, and short readings.
are graded against specific criterion (employ a scoring rubric).
B. Constructed response questions should be based on a given stimulus and include a
series of questions which build from simple to complex.
First question: A general or specific question with the answer found in the stimulus.
"What was the population in the year?"
Second question: Make connections between and among the different parts of the
stimulus. "In which year was the immigration the greatest?"
Third question: Require the student to respond with information related to (but not
included in) the stimulus. "State a historic trend that explains the data?"
C. Constructed response questions can assess higher level thinking:
make comparisons, contrasts, causes, effects, changes
identify patterns or conflicting points of view
categorize or summarize information
construct graphs or charts from data
state a generalization, conclusion, explanation or prediction
D. Scoring: Each constructed-response question should be scored according to a rubric
(scoring guide) that gives varying degrees of credit for correct or partially correct
answers. The rubric should include enough information or examples to allow different
raters to at arrive the same score for a given student response.
1. A rubric for a short constructed-response question can be:
"Complete" score (2) represented a complete and appropriate answer.
"Partial" score (1) indicated that the response had some, but not all, of the
components of an appropriate response.
"Inappropriate" score (0) represented an answer that had none of the
components of an appropriate response.
2. Extended constructed-response questions are lengthier and more complex exercises
that allow for a finer level of discrimination in scoring the responses. Their rubric can
be:
"Complete" score (3) was assigned to a response that was complete and
appropriate.
"Essential" response (2) was less complete but included the most important
components of an appropriate response.
"Partial" response (1) included some appropriate components, but fewer or
less central ones than those required for an "Essential" score.
"Inappropriate" (0) response included only inappropriate material.
Thematic Essays
Thematic essays will require students to write a coherent essay that has an
introduction, supporting paragraphs and a conclusion. Note that the current SED social
studies essay is divided into multiple sections, which invite independent responses.
Currently, students can receive full credit without writing an introduction or conclusion.
Thematic essays:
focus on concepts and themes in the Social Studies Learning Standards.
go beyond recall and reiteration of facts.
emphasize critical thinking skills.
ask students to make connections and linkages.
are criterion referenced and employ a 5 point scoring rubric.
require students to move beyond responding to the separate tasks in the question, and
consider the connections between sections of the question.
Sample thematic essay question Global History:
Theme: Nationalism - "Throughout global history, nationalism
has
had positive and negative effects."
Task: Define the term nationalism.
Select one country you have studied and give specific historic examples
of nationalism within that country.
Describe a situation where nationalism was either a positive
or a negative force within the countrys history.
To receive full credit in a five point rubric scale, the essay would need to:
show a clear understanding of the theme.
address all aspects of the task.
show an ability to analyze issues and events.
write a cohesive, cogent essay that uses a rich variety of details.
summarize key arguments and points made in the essay.
include a strong introduction and conclusion.
Credit would be reduced if the essay:
failed to address all aspects of the task.
lacked depth or accuracy.
used faulty analysis of the theme or lacked any analysis of the theme.
lacked focus, concrete examples or historic details.
lacked an introduction or conclusion.
Document-Based Questions
Document Based Questions (DBQ) assess the ability of each student to work with
historical sources in multiple forms. The secondary DBQ will have a maximum of 8
documents; at least 2 of which will be visuals. The 7/8 assessment will have a DBQ with 4
documents (including 2 visuals). The DBQ requires many of the same skills used in
developing a research paper - interpreting primary and secondary sources, evaluating
sources, considering multiple points of view, using historic evidence, developing and
supporting a thesis.
Document-based questions:
are based on the Social Studies Learning Standards, themes and concepts.
focus on critical thinking skills and ask students to make comparisons, draw analogies,
apply knowledge to the given data, and require students to apply historic analysis.
ask students to take positions on issues or problems and support their conclusions.
require students to look at issues from multiple perspectives.
require students to apply skills they use as adults
are criterion referenced and employ a scoring rubric.
The DBQ will consist of two parts. In the first part, students will be required to
answer a key question on each document. This generally involves interpreting the main idea
or point of view expressed in the document. This section will be "scaffolded" to
the second part in which the student will write an essay using the documents to respond to
a specific question. In this second part, students are asked to utilize all the sources,
plus their understanding of history to respond to a question. This second part will
require them to move beyond simply quoting all the documents. They will need to use the
documents as evidence in support of a thesis that responds to the assigned (part two)
question. The student will be required to use all documents and incorporate outside
information. The student will have no choice on the DBQ.
To receive full credit in a five point rubric the essay will need to:
thoroughly address all aspects of the task by accurately interpreting the documents
plus incorporate outside information related to the documents.
discuss all aspects of the task and support with accurate facts, examples and details.
weigh the importance, reliability and validity of the evidence.
analyze conflicting perspectives presented in the documents. Weave the documents into
the body of the essay.
include a strong introduction and conclusion.
Credit would be reduced if the response:
did not recognize the reliability, validity, or perspectives of the documents.
reiterated the content of the documents with little or no use of outside information.
discussed the documents in a descriptive rather than analytic manner.
showed little recognition of the tasks, lacked an introduction or conclusion.
|