AI Activity 5: Correlation, Regression, and Overinterpretation: When Association Is Not Explanation

Student habits and exam outcomes

AI activity
Modified

March 31, 2026

Correlation and regression can summarize patterns, but those summaries are easy to overinterpret. AI may generate polished interpretations that sound persuasive while quietly implying causation, ignoring subgroup structure, overlooking influential points, or treating a fitted line as the full story.

You will use an AI tool as a learning partner to investigate a core data science reality: statistical associations require careful wording, visual checking, and human judgment. Even when AI gives technically correct output, people still need to decide whether the language is justified, whether the model matches the data pattern, and what conclusions are actually supported by the evidence.

You are graded on how you think, evaluate, validate, interpret, and explain, not on whether AI makes mistakes.

Learning something new is allowed

You are not expected to already know every concept that might come up in this activity. AI may introduce an idea such as:

  • lurking variable
  • confounding
  • Simpson’s paradox
  • influential observation
  • leverage
  • omitted variable
  • nonlinear relationship

If AI introduces a new idea, you may use it as a learning tool, but you must still explain your reasoning in plain language and connect it back to evidence from the dataset.

Auditing means evaluation, not error hunting

In this activity, auditing does not mean trying to prove that AI is wrong.

It means evaluating AI output as a statistical work product:

  • Does it answer the actual question?
  • Is the wording statistically appropriate?
  • Does the interpretation go beyond the evidence?
  • What assumptions, limitations, or missing checks matter here?
  • What should be accepted, revised, or extended before the result is trustworthy?

Sometimes AI will overstate. Sometimes AI will be mostly careful. In either case, your job is to evaluate the output and explain why your final interpretation is more trustworthy.

Goal of this activity

By the end of this activity, your group will do five things:

  1. Measure and visualize an overall association between two quantitative variables.
  2. Use AI to generate an interpretation of correlation and regression output.
  3. Audit the interpretation for wording, assumptions, subgroup structure, and influential observations.
  4. Rewrite the interpretation using precise, statistically appropriate language.
  5. Explain what can and cannot be concluded from observational data.
To Do
  1. Download the dataset from the Dataset section below.
  2. Write your investigation question (2 to 3 sentences). Step 1 in Tasks.
  3. Run 3 to 5 AI prompts that cover interpretation, caution, subgroup structure, and one new idea if needed. Step 2 in Tasks.
  4. Complete checklist items 1 to 6 first. Step 4
  5. Finish the language audit, revised interpretation, and final judgment in checklist items 7 to 10. Step 4
  6. Draft your synthesis, then build slides using the required 5 slide structure. Step 5 and Step 6

Assigned Roles

[NOTE:] Each student has a designated role for accountability. However, teammates are encouraged to collaborate, support one another, and learn together in order to produce high quality, cohesive work.

Prompt Engineer

Responsibilities

  • Create 3 to 5 purposeful AI prompts
  • Save AI responses and build the AI Interaction Log
  • Write annotations for each prompt and response pair

Data Science Auditor

Responsibilities

  • Evaluate whether the AI output matches the statistical question
  • Check whether AI interpretation is too strong, too causal, or missing needed cautions
  • Design and run visual and statistical checks for subgroup structure, influential observations, and model fit
  • Explain what the group accepted, revised, or extended, and why

Synthesizer

Responsibilities

  • Write the Human Authored Synthesis in clear course language
  • Build slides using the required structure
  • Ensure the final work is consistent and concise

Dataset

Required file

Dataset notes

For this activity, your main required variables are:

  • Response variable: final_exam_score
  • Predictor variable: study_hours_week
  • Context variable for grouped checking: section

You may also use either sleep_hours or caffeine_mg_day for one secondary check.

What you will submit

  • Group submissions
    • AI Interaction Log
    • Human Authored Synthesis
    • Slides for a 12 to 15 minute presentation
  • Individual submission
    • Individual Reflection (150 to 200 words)

Step by step tasks

Step 1 Define your investigation question (before using AI)

Write 2 to 3 sentences answering:

  • What overall relationship are we studying
  • What does the slope or correlation try to summarize
  • Why might an observational relationship be easy to overinterpret
  • What human checks are needed before trusting the interpretation

For this activity, your main analysis must focus on:

  • final_exam_score as the response
  • study_hours_week as the predictor

Your group must also use section to check whether one overall trend hides meaningful subgroup structure.

Example investigation question you may use or adapt:

“We want to study the relationship between weekly study hours and final exam score, and we want to see how easily AI might overinterpret that relationship. We will compute a correlation and fit a simple regression line, but we will also check whether the data are observational, whether section differences matter, and whether unusual points or nonlinearity change the story.”

Step 2 Use AI strategically

[Note:] You may run several prompts, but keep 5 most useful and meaningful prompts for reporting.

Your prompts must be purposeful and iterative.

Prompt requirements

  • At least 1 prompt asking AI to interpret a correlation and regression line in context
  • At least 1 prompt asking AI whether the result supports a causal claim
  • At least 1 prompt asking AI what visual or model checks are needed before trusting the interpretation
  • At least 1 prompt asking AI how subgroup structure or a lurking variable could change the story
  • At least 1 prompt asking AI how influential observations or nonlinearity can affect correlation or regression
  • At least 1 prompt asking AI to explain one relevant idea in plain language if your group meets a concept that is new

Suggested prompt starters (you may adapt)

  • “I fit a simple linear regression of final_exam_score on study_hours_week. How should I interpret the slope, intercept, and \(R^2\) in plain language?”

  • “Can I say that studying more causes a higher final exam score from this dataset? Why or why not?”

  • “What kinds of language would overstate the evidence when describing a correlation between study hours and exam score?”

  • “How could section differences or a lurking variable change the interpretation of one overall regression line?”

  • “How can an influential observation change a correlation or regression interpretation, and what checks should I run?”

  • “If the scatterplot is not perfectly linear, how should I describe the relationship without overstating what a simple line means?”

  • “Teach me one relevant idea, such as confounding, Simpson’s paradox, or influential point, but explain it in plain language first.”

Step 3 Create the AI Interaction Log

For each prompt, include:

  • Prompt goal
  • The AI response excerpt you used
  • Your annotation:
    • What AI got right
    • What language or claims needed caution
    • What needed evidence or validation
    • What your group accepted, revised, or extended

At least one entry in your AI Interaction Log must show how your group evaluated AI generated wording and made it more statistically appropriate.

Important rule

  • You may not paste AI text into your final synthesis verbatim.

Step 4 Analyze the data and complete the required validation checks

Import the required file, then complete the validation checklist below. Your evidence can be screenshots, printed outputs, or short summaries of what you observed. A short summary without numbers or outputs does not count as evidence.

1. Rows and columns

  • Confirm the dataset has 210 rows and 10 columns after import.

2. Observational context

  • State clearly why this dataset is observational rather than experimental.
  • Explain why that matters when interpreting a correlation or regression line.

3. Overall association

Using study_hours_week and final_exam_score:

  • create a scatterplot
  • compute the correlation
  • fit a simple linear regression model
  • report the regression equation
  • report the slope and (R^2)

Then answer:

  • What does the slope mean in context?
  • Is the intercept meaningful in context?
  • What does the correlation summarize, and what does it not tell you?

4. Visual check of the line

  • Add the fitted line to your scatterplot.
  • Describe whether the straight line seems like a reasonable first summary.
  • State one thing the line captures well and one thing it may miss.

5. Grouped check using section

  • Create a scatterplot of study_hours_week versus final_exam_score colored by section.
  • Compare the overall pattern to the within-section patterns.
  • Explain whether one overall line may hide meaningful section related structure.

6. Influential or unusual observations

  • Identify at least one observation that looks unusual in the scatterplot or regression diagnostics.
  • Provide evidence using a diagnostic such as residual size, leverage, Cook’s distance, or a refit after removing one point.
  • Explain how the interpretation changes, or does not change, when the unusual observation is considered.

7. Secondary check for shape or interpretation

Choose one of these pairs:

  • sleep_hours and final_exam_score
  • caffeine_mg_day and final_exam_score

For your chosen pair:

  • create a scatterplot
  • describe the apparent form of the relationship
  • explain why a single number such as (r) may not tell the whole story
  • state whether a simple linear interpretation seems fully adequate

8. Language audit of AI output

Select one AI generated interpretation and audit the wording.

You must identify:

  • one phrase that is statistically appropriate
  • one phrase that is too strong, too causal, or too confident
  • one phrase that needs more evidence or more precise wording

Then rewrite the interpretation in better language.

9. Final revised interpretation

Write a short interpretation, in your own words, that includes all of the following:

  • what the overall association suggests
  • what cautions are needed because the data are observational
  • what section or unusual point checks added to your understanding
  • what you can and cannot conclude

10. Judgment and caution

Write 4 to 6 sentences answering:

  • Is the simple correlation and regression line a useful first summary
  • What important limitations or cautions must accompany it
  • What human checks mattered most in your group
  • What additional data or design would be needed to support a stronger causal claim

Step 5 Write the Human Authored Synthesis (group)

Length target: 400 to 600 words.

Your synthesis must include:

  • your investigation question
  • at least two reasons AI generated interpretation can become too strong in correlation or regression tasks
  • evidence from your checks on observational context, subgroup structure, or influential observations
  • a revised interpretation written in precise statistical language
  • a short recommended routine for checking AI generated regression or correlation interpretations before trusting them

Your synthesis must be written in your own words.

Step 6 Presentation slides (group)

Use this exact slide structure:

  • Our question
  • What AI suggested
  • What we audited and checked
  • Our final interpretation, with evidence
  • One takeaway for future data science work

Time: 12 to 15 minutes.

Individual Reflection (each student)

Write 150 to 200 words answering:

  • What did AI help you learn
  • What did AI oversimplify or phrase too strongly
  • What did you contribute as a human thinker
  • How will you change your AI use in future data work

Your reflection must match your assigned role.