I don’t often post about non-boat topics, but this seems to matter…

I confess I was being lazy. I KNEW how to calculate the answer, I even knew all the information I needed to do the calculation but it was late in the evening, I was tired and I chose to just skip the calculation. So I typed into the AI Box:

“How much does a gallon of mercury weigh, in pounds?”

It dutifully relied: “A gallon of mercury weighs 13.6 pounds”

I started to type that answer into the next step of what I was doing, and my brain slowly engaged, and a red flag popped up. “13.6 pounds”?? Hey, wait a minute. That’s not right. Not only is it wrong, it is really, really, really wrong. The correct answer is actually 113 pounds.

Now remember, this is the technology that is predicted to replace 40% of all entry level technical staff next year. Granted, that prediction is by somebody who will make billions for every dollar his AI company’s stock price rises, so he might have a billion reasons to inflate the capabilities of his product. But this is today targeting weapons in war. It is helping decide legal cases. It is being used to write scientific research papers. I find the incredible over-confidence in AI absolutely terrifying.

This is crazy. This technology might have potential, but it is–at least for now–seriously flawed. I would challenge any AI advocates to tell me why I should trust this technology to answer ANY question when it totally flubs such a simple question. It seems like it is especially dangerous asking it a question that you don’t already know the answer to. In which case what exactly can it be trusted to DO?

Just for my own edification, I decided to challenge the erroneous bot. “That answer is wrong.”

The bot replied: “Thank you for calling that to my attention. You are right, the correct answer is 113.”

The only thing more annoying than the original wrong answer is the obsequious need to thank me for correcting it. I do not value, not do I accept a machine’s apology or thanks. That is the very definition of “insincere”(1).

I continued, just to see where this would go. “How could you make such a mistake?”

“I am sorry, I forgot a step in the calculation.”

“Sorry”? “Forgot”???? Really???

How can we be turning over the world to computer programs who would fail a high school chemistry class?

---

(1) Synonyms of “insincere”: fake, meaningless, superficial, hypocritical, phony, counterfeit, feigned, dishonest, contrived, affected, untruthful, bogus