First impression of ChatGPT Agent, and apps on ChatGPT

OpenAI has been at the forefront of AI narrative these days to the extent it is difficult to keep track of their press releases and their implications. So, I first want to go back and discuss something that they launched almost three months ago: ChatGPT Agent.

In their launch post, the first use case OpenAI showed for “ChatGPT Agent” is booking flights. So, I asked my ChatGPT the following query: “Find me cheapest flight from SFO to Dhaka. Travel date December 6 to February 6”

The agent then worked for ten minutes, and provided me the below wall of texts.

If you asked me to guess what the agent might do to respond to my query, I would have said it would find the cheapest flight (and perhaps a couple other options), and give me a clickable link in case I decide to book the flight. To my surprise, they not only gave me the wall of texts, but when I clicked the link shown in ChatGPT’s response, it just led me to…screenshots from flight comparison websites, including Google! There was actually no way for me to click to get to the actual website and book the flight. You can see my query and ChatGPT’s response here.

I would have been more forgiving if it were a recently launched feature that needs more finetuning, but the fact that it was launched three months ago and yet remains a less than a half baked idea today does not bode well. In contrast, it took Google less than 30 seconds to give me a clickable result.

Now, let’s try something that OpenAI indeed launched just a couple of days ago: integrating other apps on ChatGPT within the prompt. This is OpenAI’s third attempt to essentially graduate from just a chatbot to a platform that aggregates other apps in it. ChatGPT plugins was their first attempt in March 2023 which was later deprecated. Then they launched GPT Store in January 2024. Sam Altman in an interview with Ben Thompson suggested yesterday that “GPTs actually did work, GPTs get a surprising amount of usage, but inside a company or someone for their own workflows or whatever.”

Now, with “Apps in ChatGPT” launch, some selected apps are directly integrated with ChatGPT that you can summon just by typing their name on the query box. One such selected app is Booking.com

So, I tried to book a hotel through ChatGPT. I typed “Booking” in the query box and nothing showed up. Then I tried “Booking.com” and it did appear then. I finished my query: “Booking.com find me an accommodation near Pacifica, CA on October 30-31 for two people”

This time, the agent was reasonably fast and showed me five results that I can actually click and go to Booking’s website to book the hotel. You can see my query and ChatGPT’s response here.

Google too shows five results at the top, but you can scroll down to see many more results, or just click the map on the right to select a hotel, both of which you cannot do on ChatGPT.

Despite the integration with ChatGPT, I still need to enter all my details on Booking.com if I try to book the hotel. That is also the case if you try to book a hotel through Google and go to Booking.com website from there. Ultimately, Booking is deeply interested in not just another transaction, but a clear way to potentially own a direct relationship with the customer. If you just do this via Booking’s app, you won’t need to enter all the details again after the first time.

Booking a hotel was a much better experience than trying to book a flight through ChatGPT (of course, that’s an incredibly low bar and it was still slightly worse than booking through Google or directly on Booking’s app). It is worth highlighting why booking the hotel was a relatively better experience. Sam Altman seems to think he’s doing almost a charity to these apps by not abstracting them away from the user experience completely. See the interaction between Ben Thompson and Sam Altman from the aforementioned interview below:

Ben Thompson (BT): Well, with these apps in there, is there a sense on your side it’s like, just to go back to the, “We have all this usage, it’s going to be better if I can just use Zillow in the app”? This idea of going somewhere else and Zillow says, “Oh, I’d rather them be in our app, we spent so much time on it”, do you feel you have the power to dictate, “For the user, it’s a better experience and because the users are here, if you’re not there, someone else will be” — and you’re sort of able to, dictate sounds bad, but if it’s a better user experience, that’s better for everyone else?

Sam Altman (SA): No. Here’s another place that I think my early career training was useful. There was a version of this we could have done where it was a better user experience, but terrible for the partners.

BT: What would that look like?

SA: Well, I mean, on that Zillow example, what if you just said like, “Hey, ChatGPT, find me all of the houses that meet these things”, and we said we’re going to control the UI.

BT: Oh, right. So there’s no even a presentation layer of the Zillow app, you’re just getting the results.

SA: Yeah, yeah. But I felt really strongly that when we do this, it’s something that the whole ecosystem benefits from, and specifically that new startups can rocket into existence because of it. So we did this in a way where you very much have the relationship with the other site. You’re calling them by name, we’re suggesting them by name, they’re taking over the UI, they’re linking their account. So I think there was something we could have done that was maybe slightly more user-friendly, but not good for the other companies, and I really didn’t want us to do that.

I hope my examples made it clear to you that if OpenAI did try to completely abstract away the apps, booking a hotel would probably feel similar to booking a flight: it would probably work for 10 minutes and then give me screenshot of booking websites!! Of course, if they did try to abstract away the apps completely, the apps could simply block OpenAI (or any AI crawlers). OpenAI is still somewhat early in aggregating demand, so the apps are still not as powerless as they are in Google’s properties today.

I am skeptical that even these integrations will lead to any material shift in consumer behavior in booking a hotel (definitely not booking a flight as it currently stands), so how these integrations will evolve remains to be seen. Altman did indicate in that interview that “hopefully this works better and if it doesn’t, we’ll keep trying.”

In addition to “Daily Dose” (yes, DAILY) like this, MBI Deep Dives publishes one Deep Dive on a publicly listed company every month. You can find all the 63 Deep Dives here.

Current Portfolio:

Please note that these are NOT my recommendation to buy/sell these securities, but just disclosure from my end so that you can assess potential biases that I may have because of my own personal portfolio holdings. Always consider my write-up my personal investing journal and never forget my objectives, risk tolerance, and constraints may have no resemblance to yours.

My current portfolio is disclosed below: