Close

All MCPL branches will be closed on May 27 for Memorial Day. Our Virtual Branch is available 24/7.

Back to top

Are AI Chatbots Spot On?

Are AI Chatbots Spot On?

April 5, 2024

Recently, our resident first grader brought home his first-ever report based on facts he learned from Pebble Go, an app he uses on his school-assigned Chromebook. After celebrating “101 Days of First Grade” with his Dalmatian shirt, it was not surprising that he chose Dalmatians as his first report topic. We all learned quite a bit about those dogs, but there’s always room to learn more. As someone who has recently jumped into exploring chatbots, I thought it might be fun to compare what our resident “expert” and the free versions of ChatGPT, CoPilot, and Claude have to say about Dalmatians.

Immediately noticeable is that while our first grader used rudimentary citing techniques by recording page numbers for his report, none of the AI-generated reports cited their sources. I could add a prompt to list the sources, but that information was not automatically shared. The generated reports may supply a general outline and help introduce ideas of what to include, but the shared facts need to be double-checked, and proper credit is given.

When requested to write a short report, the structure of that result varied. With ChatGPT,  each paragraph starts with a topic heading followed by a colon. CoPilot created a bulleted outline that used complete sentences. It also included an image of a Dalmatian with a litter of puppies and a link to the American Kennel Club. Claude provided a “bare bones” five-paragraph summary.

When I prompted the chatbots to write as a first grader, the results were roughly equivalent to an adult writing for a first grader. Using our first grader as a test reader, the CoPilot report included only one unknown word. With ChatGPT and Claude, the vocabulary was above first-grade level and included more advanced - and unknown - words.

There were a few differences in the facts presented in the reports. Although this isn’t a comprehensive list, it shows that using one AI source limits the results when researching a topic. Here are some of the differences:

  • CoPilot did not mention Dalmatia, now known as Croatia, as the potential origin of these dogs.
  • ChatGPT did not include their size (medium size). CoPilot mentioned their approximate height, while Claude mentioned what they typically weigh.
  • Claude did not mention fire trucks, which you might assume would be an obvious main fact.
  • Our first grader discovered that Dalmatians are also called plum pudding dogs, which none of the chatbots mentioned.

On the other hand, some of the sentences found in one report were fairly close to sentences from other AI sources. One standout example is CoPilot’s: “They have spots all over their bodies like they’re wearing polka dot pajamas.” ChatGPT provided, “It's like they're wearing their own polka-dot pajamas!”

AI can be a great starting point for writing reports. It suggests main topics, facts, and possible report structure in very little time, but final reports still require human interaction and effort.

Terri M.
Digital Innovation Specialist

View All Blogs

Read Similar Blogs:
Current Issues and Topics
Technology
Writing

Resources You May Also Like

Indie Author Project

Submit and share your eBook in libraries statewide.
More Info

Explora for Kids

Get info for papers, class projects, or homework.
More Info

Digital Literacy

Build your knowledge of digital tools for today's world.
More Info

Events You May Also Like

Blogs You May Also Like

Gamer Geek on a Budget
Read More

Gamer Geek on a Budget

If you didn’t get the game of your dreams for the holidays and don’t have the cash to drop on it right n
Outrage Machine
Read More

Outrage Machine

Outrage Machine by Tobias Rose-Stockwell examines how tech and media affect public discourse and society. 
What Is DNS?
Read More

What Is DNS?

There’s a chance that you haven’t used the term “DNS” in a regular conversation unless you’re
Stay in Touch with Your Reading
Read More

Stay in Touch with Your Reading

One perk of clicking My Account on MCPL’s homepage is your ability to keep track of your reading.

Was this page helpful? Yes No