/w=3840,quality=90,fit=scale-down)
Raphael Thys
This week brought the opening salvos of a new battle of the bots.
“It’s a new day in search. In fact, a race starts today,” said Microsoft CEO Satya Nadella when he introduced the new and improved Bing, now bolstered by ChatGPT, the artificial intelligence chatbot the internet can’t shut up about. Just a day earlier, Google announced Bard, its own AI chatbot-powered search tool.
Marketplace’s Meghan McCarty Carino spoke with Chirag Shah, a computer science professor at the University of Washington. He said AI chatbots process language more intuitively than older search models and could transform how people access information. The following is an edited transcript of their conversation.
Chirag Shah: This is something that researchers in this field for decades have asked for, that it would be great if these systems could process a whole sentence, questions and more contextualized inquiry. So that’s definitely now starting to happen as people get accustomed to entering not just a bunch of keywords, but their full question with a lot of details. I mean, it would take some time. I don’t see keyword-based search going away anytime soon. But that would be a big shift in user behavior.
Meghan McCarty Carino: Now, let’s talk about some of the problems, one of which was made pretty obvious earlier this week when Google circulated some demonstrations of its AI chatbot called Bard. And it made a factual error in at least one of the responses. What does that tell us about the limitations of this technology, especially when we’re using it for search?
Shah: So this is a danger that these things, while they’re very effective 90% of the time, we don’t know what are those 10% of the times when it fails. And in most cases, you’re not going to have a well-informed user on the other end to be able to question this. The difference here, compared to a traditional search engine, is if you get some bad or wrong answer, chances are there are other documents, other sites, other sources giving you a better answer or right answer. Here, you don’t have the ability to do that. These things are often hallucinating answers, so they’re making up answers that don’t exist.
McCarty Carino: A lot of discussion of some of the misinformation or hallucinations, I guess, as we’re calling it when it comes to ChatGPT. But also this week, there’s been a lot of commentary about some of the things that ChatGPT is omitting in answers, whether it’s because it’s sort of been programmed to omit certain things or the learning model is magnifying these things — people asking questions about [former President Donald] Trump and the chatbot sort of returning blank answers. I mean, what kind of implications does that have?
Shah: Yeah, this is problematic. So they have done this, of course, as a guardrail. So ChatGPT, one of the things I would credit them is they have done a really good job putting up the guardrails, except that these guardrails are pretty shaky, they’re pretty superficial. And this stems from the fact that the large language model being used, or ChatGPT, doesn’t really understand things. So if you and I are talking, and if I don’t want to get into politics, I can carefully deny your request that I don’t want to get into that. But you would understand my position. Here, ChatGPT is not doing it out of understanding, it’s doing it out of this very superficial, programmatic barrier, just because they don’t want to get into these controversial topics. But then if you think about it, this is a big limitation because so much that goes on around us, they’re not black and white, they’re not clear-cut, where you can just give a factual answer. There are possibilities. Now, one good thing with the traditional search engine is, instead of saying that this is the answer, it gives you a bunch of different answers, some of them are contradicting to others, and that’s fine. The user then has the ability to go through them, make up their own mind. Whereas here, either it gives you an answer or it doesn’t give you an answer, it doesn’t have this ability to say, “Well, here are different factors, here are different, and so-and-so says this, and so-and-so says this other thing.” So it’s not doing that. So the solution they’ve come up with, at least for the short term, is simply not answer it. That’s a huge limitation because people come to expect finding at least something. It doesn’t happen often that you go to Google or Bing, you search for something and you get absolutely nothing in return.
McCarty Carino: As an expert who’s immersed in AI and search engines, what are you going to be watching as companies compete for dominance in this space?
Shah: When it comes to web search, it reaches billions of people. It covers all kinds of languages. People are able to search in their native languages, they’re able to translate things. So we’re currently only looking at a very small aspect of it. It’s a very Westernized, very English-driven, very much targeted to somebody who’s kind of information literate. So it has a very small demographic footprint compared to the web search, which reaches much, much broader audience. And so one of the things that I’ll be looking at is how these things get expanded to accommodate those demographics. So I’ll be looking at developments that hopefully look at what users want and how to support them in different situations, different contexts, different cultures, different languages and so on. But that’s going to take, take a while.
McCarty Carino: What would winning this race look like, and who stands to lose out as things narrow down?
Shah: Maybe Google and Microsoft, neither of them will clearly win or lose, they will just be a duopoly in this market. But there is a good chance for a lot of small players to lose out. Currently, what happens with search engines is people go to search engines, they search for something. And then they’re directed to these other websites where they can find their information. But now, as Google and Microsoft start giving answers right there on their page, they could potentially cut off all these other places where users could go. So there is a real threat to that layer. And then of course, users may be losers here because they would start losing the ability to validate things. And I hope regulators are also looking into the anti-competitive behavior here because I think those are real dangers.
Related links: More insight from Meghan McCarty Carino
If you missed the controversy about ChatGPT being politically biased, it seemed to originate with a tweet showing that ChatGPT refused to write a poem about President Donald Trump but would write a poem about President Joe Biden. The fact-checking site Snopes looked into it and found it to be true, and many users have found similar limitations.
As Chirag Shah said, this is likely because of the chatbot’s guardrails, meant to prevent it from spewing hateful, offensive or misinformed statements.
And I mentioned that Google’s Bard committed a factual error when it was posed the question: “What new discoveries from the James Webb Space Telescope can I tell my 9-year-old about?” Reuters reported Bard mistakenly answered that the telescope had taken the first photo of a planet outside our solar system. But NASA confirmed that the honor actually goes to the European Southern Observatory’s Very Large Telescope, which achieved that feat in 2004.
To be fair, ChatGPT has also been caught making up things. When Washington Post columnist Geoffrey Fowler tested the new version of Bing this week, the chatbot seemed to confuse art and life, concocting a conspiracy theory about actor Tom Hanks’ involvement in the Watergate scandal.