My Honest Review of ChatGPT Agent

OpenAI just released ChatGPT agent, a new tool designed to actively complete work for you rather than just answer questions.

ChatGPT agent can take actions on your behalf. It can browse websites and complete multi-step tasks from start to finish. Unlike OpenAI’s Operator, it’s more reliable and incorporates the best aspects of Deep Research.

As a programmer who has been working in web automation for the past few years, I couldn’t resist the temptation to put it to the test—and here’s my honest review.

ChatGPT Agent Use Cases

ChatGPT agent is an evolution of Operator and Deep Research. Operator is an AI tool designed to browse the web and visually interact with websites, while Deep Research focuses on analyzing information deeply to generate comprehensive reports.

ChatGPT agent can do both. It can seamlessly switch between reasoning and action—it knows when to stop and think and when to click or type something to move a task forward.

The main use cases of ChatGPT agent are:

• Use the web: It can do different tasks for you, such as making bookings/appointments, comparison, analyzing sites, etc
• Generate reports: It can produce a PowerPoint deck, a formatted Excel spreadsheet, or a PDF report as the end result of a task
• Connect to your apps and take action: It can connect to apps such as Google Calendar, Gmail, Maps, and take action on your behalf

I’m going on vacation soon, so I asked ChatGPT agent to find the best restaurants in the city I’m visiting and create a new saved list on Google Maps, including notes about the signature dishes to try at each place

The video’s sped up. It took the ChatGPT agent about 23 minutes to finish the task.

First, ChatGPT agent searches for the best restaurants in the city, selects 10 of them, and collects information about their signature dishes. Then it pauses and asks me to take control to enter my Google login credentials. After that, it creates a list on Google Maps with notes for each restaurant and saves it.

All of this is amazing! That said, the ChatGPT Agent isn’t ideal for every type of task. Let’s take a look at what it does well — and where it falls short.

Performance Benchmarks: Setting New Standards in AI

The ChatGPT Agent has demonstrated exceptional performance across several industry benchmarks, showcasing its advanced capabilities in diverse applications. Key achievements include:

• Humanity’s Last Exam: Excelling in complex problem-solving and reasoning tasks, reflecting its ability to tackle intellectually demanding challenges.
• Frontier Math: Mastering intricate mathematical computations with precision and reliability.
• DSBench: Delivering top-tier performance in data science tasks, including analysis, visualization, and predictive modeling.
• Spreadsheet Bench: Demonstrating expertise in creating, managing, and manipulating spreadsheets for various professional needs.

These benchmarks highlight the agent’s ability to handle specialized tasks with accuracy and efficiency, making it an indispensable tool for professionals in fields such as finance, research, education, and data analysis.

Customization and Workflow Optimization

The ChatGPT Agent is designed to adapt to your unique workflow requirements, offering extensive customization options to enhance productivity. You can automate recurring tasks, such as daily updates, weekly reports, or monthly reviews, making sure consistency and accuracy in your operations. By automating routine processes, the tool allows you to redirect your energy toward higher-value activities, fostering innovation and creativity.

Whether you’re managing a team, conducting research, or overseeing complex projects, the ChatGPT Agent’s flexibility ensures it can be tailored to meet your specific needs. Its ability to integrate seamlessly into existing workflows makes it a practical solution for professionals seeking to optimize their time and resources.

Pricing and Accessibility: Flexible Plans for Every User

The ChatGPT Agent is available through tiered pricing plans, making sure accessibility for users with varying needs and budgets:

• Pro Plan: Priced at $200 per month, this plan offers 400 messages per month, catering to high-volume users and businesses requiring extensive automation.
• Plus Plan: Available at $20 per month, this plan provides 40 messages per month, making it an affordable option for individuals or smaller-scale operations.

This flexible pricing structure ensures that you can select a plan that aligns with your usage requirements, allowing you to use the tool’s capabilities without exceeding your budget. Both plans provide access to the full suite of features, making sure you can maximize the value of the ChatGPT Agent regardless of your chosen tier.

Empowering Productivity Across Industries

The ChatGPT Agent is more than just a tool; it’s a comprehensive solution designed to enhance productivity, streamline workflows, and simplify complex tasks. By integrating advanced AI with practical automation, it enables you to achieve greater efficiency and focus on strategic objectives. Whether you’re managing financial models, creating detailed presentations, or analyzing intricate datasets, the ChatGPT Agent delivers precision, reliability, and value.

Its emphasis on user control, coupled with its exceptional performance across benchmarks, makes it a versatile asset for professionals in diverse fields. By automating repetitive tasks and optimizing workflows, the ChatGPT Agent enables you to unlock new levels of productivity and creativity, making sure you stay ahead in an increasingly competitive landscape.

ChatGPT Agent: Where it shines and disappoints

OpenAI’s demos emphasize the idea of users giving a task to the agent, then stepping away to focus on other tasks while the agent completes the work independently.

While ChatGPT agent can often operate independently, it occasionally requires the user to be present at their computer. Whether it’s to enter login credentials, solve captchas manually, or confirm a critical action, user involvement is sometimes necessary.

Other times, the agent doesn’t need you to take control, but it may take so long to perform a simple action—like failing to select an item from a dropdown after several minutes—that you end up stepping in to help move the automation forward.

Below is an example of this. I asked ChatGPT agent to go to a real estate site and search for homes to buy in Marbella, Spain, within a price range of €100k to €400k. It was doing a good job, but suddenly it failed to locate the maximum price in the dropdown. After 14 minutes, I took control of the site and did it myself.

I have to say, though, that was the only issue. It went through 13 homes, collected the data I asked for, and exported it to an Excel file. I double-checked the site and confirmed that the data was accurate—no hallucinations!

ChatGPT agent amazes at how well it handles tasks on the web. You can even see it pause to think before clicking or moving a task forward. However, all of this comes with one downside—it’s slow.

It took ChatGPT agent around 20 minutes (with one intervention from me) to get the job done. That makes it well-suited for some tasks, but not so great for others.

If you need a tool to perform urgent web tasks, it’s better not to use ChatGPT agent. Even simple actions like clicking, selecting elements, and searching can take the agent several seconds—or even minutes

However, if you need a tool for non-time-sensitive tasks, ChatGPT agent is your best friend. You can use it to do research, take action, generate spreadsheets and slides, collect data, and more.

In the past, I had to write Python code to build a bot that could handle all of this for me. But now, I can just give instructions in plain English to the agent and get similar results.

Just remember that ChatGPT agent isn’t fully autonomous, so don’t close your laptop, grab a coffee, and expect everything to be done when you come back.

I think the number of credits you get is a bit low for what you actually get. Especially considering that one credit is used for every interruption, whether it’s you intervening (to solve a captcha or help when it gets stuck), or ChatGPT agent pausing to ask you to enter login credentials or confirm an action.

Let me know in the comments what you think about OpenAI’s agent.