Sign in Subscribe
Leadership Edge

Do You Know Your Firm's Error Rate?

📰 You ask AI for 99% accuracy. Do you know your firm's own number? Fifteen voices on the stamp, the review process, and a one-week baseline you can run on your last project.

Do You Know Your Firm's Error Rate?

If an AI tool revises a thousand sheets overnight and you stamp the set (digitally, of course... because who has time for that) in the morning, what exactly did you check?

💡
Who this episode is for: Especially relevant if you put a license behind the work, or you're the person deciding how much of the checking a tool is allowed to do.

Summary

This was the last episode of Arc 2 in The Next AEC Tech Stack (TRXL 244). The first two asked what these tools can do. This one asked what happens when they're wrong, who is still on the hook when they are, and how the next generation learns the job when the work they used to learn on goes away.

I put those questions to fifteen people. Almost everyone said the stamp stays with a person today. What varied was how much it takes to stand behind one now.

The risks that stood out to me weren't about a model making things up. They were about checking that gets thinner as the output gets better, a profession that has never measured its own error rate asking a machine for 99% accuracy, and a fee structure that hands the savings to the client. A reminder that applies to every episode in this series: these conversations were recorded weeks ago, and the tools have kept moving since.

This series brought to you by
Graitec Graitec provides AEC software, training, and expert support to help architects, engineers, and construction teams design, build, and manage projects more effectively.
With support from
AVAIL AVAIL's content management system (CMS) helps architecture and engineering teams find, manage, and reuse their Revit, AutoCAD, Civil 3D, and other technical content in one place.
💡
Free and paid subscribers are getting the same full newsletter for this series. Normally the full edition of Leadership Edge is a paying-member benefit. For The Next AEC Tech Stack it isn't: every edition goes out complete to everyone on the list, free accounts included. Graitec and AVAIL funded the series so the industry could actually use it.

If someone forwarded this to you, sign up free and the rest of the series lands in your inbox as it publishes. TRXL+ members also get the ad-free feed, the full back catalog, and all the content that goes back behind the paywall when we return to our regular schedule following this series.
Catch the full episode

What's Inside

  • What the stamp covers now. A digital signature can go on every sheet in a set at once, so what does a signature prove anymore?
  • Slop, not hallucination. Mike Soldan says the bigger risk is reference content that's five years old, and that changes what a firm should be policing.
  • Checking less and less. Holger Kreienbrink expects people to verify AI output five times, then ten, then stop, and isn't sure that ends well.
  • The intern double standard. Roderick Bates compares the pass AI gets with the pass no human employee would get.
  • Nobody measures the humans. Theodoros Galanos asks whether anyone who demands 99% accuracy from a machine knows their own number.
  • Where the reps go. If drafting was how people learned why a door swings the way it does, what replaces it?
  • The fee that gives it away. Campbell Yule explains how hourly billing turns a productivity gain into the client's discount.

Read next