Writing Moderation vs Calibration and creating AI Model Texts

AI in the Classroom: Demystifying Moderation vs. Calibration

As educators, we're all exploring how AI tools like ChatGPT, Gemini, and Co-pilot can support our work. From creating lesson plans to generating quizzes, the potential is huge. But when it comes to assessing student writing, there's a crucial distinction we need to understand: the difference between moderation and calibration.

These terms aren't just jargon; they define how we can ethically and effectively use AI while protecting the integrity of our professional judgment.

Why You Shouldn't Use AI for Writing Moderation

Let's start with a clear, non-negotiable point: AI has no place in a writing moderation meeting.

Think of moderation as the heart of our professional practice. It’s where we, as a team, gather to review a selection of student writing samples. We bring our agreed-upon rubric and have a rich, professional discussion. We debate whether a text demonstrates a "developing" or "mastering" skill. We challenge each other's assumptions and, through this collaborative process, build a shared understanding of what our standards look like in practice.

Moderation is a deeply human, collaborative, and interpretive process. It's not about applying a checklist; it's about building a common professional language. Using an AI to "grade" a student's essay and then comparing it to a teacher's grade would completely short-circuit this essential conversation. The AI's result is a final judgment, but our goal is to reach a collective understanding.

The Unspoken Problem with AI Models

So, if AI isn't for moderation, where does it fit in? Many teachers are using it to create model texts—and this is where we run into a second, more subtle issue.

I recently asked an AI to write an essay about simple machines from the perspective of a 10-to-12-year-old. The result was a perfectly structured, grammatically flawless text. It was so "perfect" that my colleague and I assessed it as a level above what I had requested.

This happens because AI models are designed to generate the ideal, flawless output. They are not built to replicate the messy, nuanced, and very real process of human learning. A real 10-year-old’s writing includes:

• Developmental inconsistencies: A strong point mixed with a weaker one.

• A unique voice: The personality and experiences of the student shine through.

• Common errors: Run-on sentences, spelling mistakes, and other quirks that are a natural part of the learning journey.

The AI’s output lacks these human elements. It gives us the polished final product, not a realistic example of the learning process.

So, How Can We Actually Use AI? Calibration

This brings us to calibration, where AI can be a powerful tool.

Instead of seeing the AI's "perfect" text as a direct model for our students, let's see it as a tool to calibrate our professional understanding. Here's how you can use it:

1. Generate a Text: Ask the AI for a text based on a specific prompt and level, just as you've been doing.

2. Assess It Yourself: Use your professional judgment to assess the AI-generated text against your rubric. Don't be surprised if it's a level too high.

3. Use It as an Aspirational Example: Now you have an exemplary text that you can use to show your students what "mastery" looks like. It becomes a goal to work toward, not a realistic starting point.

4. Manually Adjust It: To create a truly useful model text for your classroom, you'll need to "un-perfect" the AI's output. Add in some developmental errors, simplify the vocabulary, or adjust the sentence structure to make it a more realistic and attainable example for your students.

Ultimately, AI is a powerful assistant, but it can't replace our professional judgment. By understanding the difference between moderation and calibration, we can use these tools to inform our practice without sacrificing the human element that makes our work so meaningful.