5 Tips for Grading Student Writing with AI
Artificial intelligence (AI) is starting to transform the way we grade student work. Yet, while AI can grade true-false and multiple-choice questions quickly and efficiently, it can struggle to understand and evaluate human writing. That’s because human writing is often nuanced, creative and context-dependent, making it difficult for AI to assess its quality.
That said, AI is very useful for grading student writing when teachers provide it with appropriate contextual information. With such information, AI grading and feedback can help teachers identify areas of weakness and identify common mistakes. AI can also save teacher’s considerable time and help them create a healthier work-life balance.
Here are five tips to help you leverage AI tools for effective grading of student writing:
AI Grading Works Best with Structured Writing
AI is typically programmed to recognize and evaluate specific patterns and criteria, such as those found in a five-paragraph essay. According to experts from Georgia State University and Vanderbilt University, AI models “can identify and evaluate the lead, position statement, supporting claims and evidence [in a five-paragraph essay] as well as a human.” AI is well suited to evaluating “logic and persuasion,” including how well a five-paragraph essay is organized and how well the arguments are developed.
2. Provide a Graded Sample of Student Writing
Include at least one sample of student writing that you have graded and includes your written feedback. Ideally, you will provide a sample from the specific assignment you want AI to grade and one that emphasizes your most important grading criteria. Remember also to include your rubric for that assignment.
Armed with this information, AI can better understand your pedagogical focus and grading criteria. Also, by seeing your written feedback, AI can analyze not only feedback content, but also your written feedback style.
3. Include the Grading Criteria for a Standardized Course
AI has access to a lot of information on standardized courses. So, if you teach one, leverage that information. For example, if you teach United States History and are grading a DBQ, include the APUSH Rubric for grading a documents-based-question. If you teach ELA, consider including the Common Core Standards for English Language Arts & Literacy. In addition, include any appropriate state guides. For example, the Florida DOE has a series of writing scoring samplers that can help humanities educators.
4. Combine Your Information into a Single Document
In theory, you can upload multiple documents to ChatGPT, Microsoft Co-Pilot and various AI grading tools. But that doesn’t always work and you might exceed their character limits. So, combine your information succinctly — sample graded writing, rubric, standardized grading criteria — into a single document, preferably a PDF. AI tools can read, analyze, and summarize PDF documents effectively, so it’s a handy format to use.
Keep in mind that you may only need a part of a document, such as a few pages of a long standardized course guide. For instance, you may only need “Writing Standards 6-12” of the Common Core Standards for ELA — five pages of a sixty-six page document — to upload to your AI grader. So, download the course guide and make a copy with only the pages you need. Then combine them with the other contextual information you’ll provide to your AI grader.
5. Write a Clear Prompt
Finally, write clear instructions regarding exactly how you want your AI tool to grade and comment on student writing and what you don’t want it to do. And don’t forget basic contextual information, such as the title of your course, the grade level, as well as the specific curriculum topic, to ensure that the grading and feedback are appropriate for the level and abilities of your students. In addition, include any specific criteria that you want the AI tools to focus on in grading student writing.
In all, human oversight is key to effective use of AI grading systems. AI will continue to struggle to understand and evaluate human writing, so teachers must analyze their results wih an eye on identifying areas where students are struggling and how teachers can adjust their pedagogy accordingly
