The Intelligent Essay Assessor (IEA) is a set of software tools for scoring the quality of essay content. The IEA uses Latent Semantic Analysis (LSA), which is both a computational model of human knowledge representation and a method for extracting semantic similarity of words and passages from text. Simulations of psycholinguistic phenomena show that LSA reflects similarities of human meaning effectively. To assess essay quality, LSA is first trained on domain-representative text. Then student essays are characterized by LSA representations of the meaning of their contained words and compared with essays of known quality on degree of conceptual relevance and amount of relevant content. Over many diverse topics, the IEA scores agreed with human experts as accurately as expert scores agreed with each other. Implications are discussed for incorporating automatic essay scoring in more general forms of educational technology.
Questar declined a request to visit their facilities, but reached by phone, Susan Trent, vice president of assessment services, said that the essays are scored as objectively as is possible.
Automated Essay Scoring Versus Human Scoring: A Correlational Study
AES programs can't read in the way that humans. They can't parse the meaning of words; they can't understand whether an analogy is silly or apt. AES programs don't read; they compare. They examine a sample of essays scored by humans (the "training set") and then they compare the unscored essays to the scored subsample.