外教社博学文库:大规模英语考试作文评分信度与网上联机阅卷实证研究电子书

出版说明

序言

前言

致谢

Chapter 1 Introduction

1.1 Rationale for the study

1.2 Objectives of the study

1.3 Organization of the thesis

1.4 Def inition of terms

1.4.1 Online

1.4.2 Marking

1.4.3 Online marking

1.4.4 Online Marking System (OMS)

1.4.5 Local Area Network (LAN)

Chapter 2 Research Questions and Methodology of the Study

Chapter 3 Issues in the Direct Testing of EFL/ESL Writing Ability

3.1 Introduction

3.2 What is a direct writing test?

3.3 EFL/ESL writing ability:What shall we test?

3.4 Issues in validity

3.4.1 What is validity?

3.4.2 Types of validity

3.5 Issues in reliability

3.5.1 What is reliability?

3.5.2 Methods of judging reliability of writing assessments

3.6 The relationship between validity and reliability

3.7 Four components of a direct writing test

3.7.1 The task

3.7.2 The writer

3.7.3 The scoring procedure

3.7.4 The rater

3.8 Washback

3.8.1 Washback in general

3.8.2 Washback of direct tests of writing

3.9 Practicality

3.10 Summary

Chapter 4 The CET Writing Test

4.1 Introduction

4.2 The writing test required by the CET

4.2.1 A direct test

4.2.2 Positive washback

4.3 The scoring of CET compositions

4.3.1 The scoring approach currently adopted

4.3.2 Procedures involved in scoring CET essays

4.4 Computer-aided adjustment of writing scores

4.5 Discussion

Chapter 5 The First Experimental Study

5.1 Introduction

5.2 Compositions

5.3 Participants

5.4 Data collection procedure

5.5 The introspection and retrospection studies

5.5.1 Introduction

5.5.2 Data elicitation

5.5.3 Tape transcription

5.5.4 Data analysis

5.6 The questionnaire studies

5.6.1 Design of the questionnaires

5.6.2 Analysis of questionnaire responses

5.7 Findings from the introspection, retrospection and questionnaire studies

5.7.1 Issues and problems in rating CET essays online

5.7.2 Decision-making behaviors while rating CET—4 essays

5.7.3 Summary of comments made by the raters on essays

5.7.4 Essay elements’ inf luences on raters’ decision-making

5.7.5 Elements of good CET essays in the raters’ eyes

5.8 Analysis of writing scores

5.9 Summary and discussion

5.9.1 About the issues and problems involved

5.9.2 About the raters’ scoring decisions

5.9.3 About the writing scores

Chapter 6 The Second Experimental Study

6.1 Introduction

6.2 Compositions

6.3 Participants

6.4 Data collection procedure

6.5 Problems encountered

6.6 Data analysis

6.7 Results

6.8 Summary

Chapter 7 Design of the OMS

7.1 Introduction

7.2 Literature review on online marking of compositions

7.2.1 Automated scoring of essays

7.2.2 Online scoring of essays by human raters

7.3 A preliminary model of marking essays online

7.4 Overview of the CET Online Marking System (OMS)

7.4.1 The data management module

7.4.2 The training module

7.4.3 The rating module

7.4.4 The monitoring module

7.5 Operation of the OMS and the rater interface

7.5.1 Overview of the operation of the OMS

7.5.2 The OMS rater interface

7.6 Main features of the CET OMS

7.6.1 Random distribution of scripts

7.6.2 Eff icient score recording

7.6.3 Online real-time monitoring of scoring

7.6.4 Quality control of raters

7.7 Advantages of the CET OMS

7.7.1 Real and eff icient random distribution of scripts at the national level

7.7.2 Real-time online monitoring of raters

7.7.3 Assured quality control of scoring

7.7.4 Overall eff iciency

7.7.5 Eff icient and economical storage of scripts

7.7.6 Express retrieval of scripts and scores

7.7.7 Eff icient management and potential utilization of test data for research

7.8 Limitations of online scoring and solutions

7.9 Summary

Chapter 8 The Third Experimental Study

8.1 Context of the experiment

8.2 Participants

8.3 Compositions

8.4 Data collection

8.4.1 Step 1: Online marking

8.4.2 Step 2: Conference marking

8.5 Data analysis

8.6 Results

8.7 Summary and discussion

Chapter 9 Data Analysis Using FACETS

9.1 FACETS and method

9.2 The f irst approach: comparison of rater severity and consistency from the online setting and the conference setting

9.2.1 Rater severity and consistency: the online setting

9.2.2 Rater severity and consistency: the conference setting

9.2.3 Comparison of rater severity and consistency in two settings

9.2.4 Comparison of rater severity change between two settings

9.3 The second approach: bias analysis

9.3.1 Bias analysis: rater by essay interactions

9.3.2 Bias analysis: rater by setting interactions

9.4 Conclusion

9.5 Discussion

Chapter 10 Summaries, Discussions,Implications and Recommendations

10.1 A ref ined model of online scoring of CET essays and its main features

10.2 Benef its proceeding from online scoring

10.3 Practicality

10.4 Scoring quality

10.5 Raters’ comments

10.6 Suggestions for the improvement of the Online Marking System

10.7 Implications for other writing tests

10.8 Suggestions and recommendations for future research

10.8.1 Suggestions for future research in online marking of compositions

10.8.2 Recommendations for future research in EFL writing assessment

10.9 Theoretical and practical signif icance of the study

References

Appendices

后记