“You should eat at least one small rock per day,” Google’s AI Overviews search feature infamously told a user in the days after it was released.

In another instance, it suggested adding ⅛ cup of non-toxic glue to the sauce after a user asked how to make cheese stick to pizza. (At least it specified the glue should be non-toxic.)

The blunder from Google ricocheted through the tech world and is perhaps the most notable troubled product launch of the rapidly unfolding AI boom, but it's hardly an outlier. MSN published an AI-generated obituary for a celebrity who was still alive and once recommended an Ottawa food bank as one of the city’s top tourist attractions. Employee management software provider Lattice recently announced it would offer official employee records for AI “workers” — only to roll back the product within days after massive backlash. Even Google’s AI troubles stretch far beyond its search function. The company’s stock tumbled by $100 billion in 2023 after its Bard chatbot erroneously said during a demo that the James Webb Space Telescope took the first-ever image of a planet outside our solar system. Earlier this year, its image generator also came under fire for producing historically inaccurate images — for example, including people of color in images of the U.S. founding fathers.

The promises of the generative AI boom are big, from supercharged AI copilots to unprecedented productivity and a new world of creativity. But as everyone rushes to integrate AI into their products and internal processes, there’s no denying companies are building the plane while flying it. For the most part, implementing AI has been a big mess, or at the very least, an extraordinary challenge.

Companies are struggling to wade through the hype, figure out which tools to integrate, how to develop AI quickly without locking themselves out of future breakthroughs, and how to navigate fast-moving regulation. There’s also a variety of intricate copyright, security, privacy, and compliance concerns, as well as lingering kinks with the technology and its penchant for making up information. In a 2024 survey from Boston Consulting Group, 66% of C-suite executives from 50 markets said they’re ambivalent or outright dissatisfied with their organization’s progress on AI and generative AI so far. They cited an unclear AI roadmap, lack of necessary talent and skills, and an absence of strategy around how to implement AI responsibly. Gartner predicts 30% of generative AI projects will be abandoned after proof of concept by the end of next year.

At the same time, the pressure is on. “Adopt AI or get left behind” has emerged as a common message, and shareholders want to hear about companies’ AI strategies as urgency has become the north star of the technology. James Robinson, chief information security officer at cybersecurity provider Netskope, said the generative AI boom is unique from past technological revolutions in how the pressure is coming from all directions, especially top-down. It’s also unique in its speed, broadness, potential for how AI products can be used for harm, and the rush of new regulations, which he said is “probably faster than we've seen in any other kind of technology adoption.”

“There’s innovation, there’s speed, and then there’s the rigor of the governance structures we have to adhere to,” he said.

At the same time, developing the technology was also an uphill battle. Just to integrate AI functions into the sticky notes on its virtual whiteboarding software, for example, Miro had to completely refactor its entire canvas — from how objects land on the canvas to adding metadata and enabling objects to broadcast to each other. Basically, they had to make the sticky notes, which previously operated like static objects, aware of and able to talk to each other.

“It was really challenging, the core technology, to get this done,” Chow said.

There’s one major technical hurdle every company tapping generative AI is facing: hallucinations. One company that’s attempted to measure how often large language models hallucinate (the cheeky term that’s stuck to describe how they fabricate information) put estimates at between 3% and 27% of the time, depending on the model. Looking at specific domains, those numbers get even more concerning. A study published in the Cureus Journal of Medical Science that investigated the accuracy and authenticity of references in medical articles written by ChatGPT found that out of 115 references, 47% were fabricated and 46% were authentic but inaccurate. Only a meager 7% were both accurate and authentic. A Stanford study that looked at how leading models respond to legal queries found hallucination rates ranging from 69% to 88%.

“We're used to computers being accurate, but generative AI can hallucinate and make up answers. We’re used to computers being consistent, but generative AI has an element of randomness,” said Briana Brownell, founder and CEO of Pure Strategy, who helps companies develop their AI and data strategies.

For a company like Crunchbase, for example, nothing is more important than accuracy. As a go-to database for information about businesses and the latest venture capital deals, accurate information is quite literally what the company sells.

“We have about 75 million people coming who trust our data. We can't be wrong,” said Megh Gautam, the company’s chief product officer. “Hallucinations are just unacceptable.”

The company has still cautiously experimented with generative AI, using it to more efficiently write descriptions for new companies and surface new kinds of insights for customers. It’s also in the early stages of updating its search function to leverage AI. Still, the team cannot fully trust the AI and has to validate data before pushing it to the site, according to Gautam.

The most difficult part of the entire process, however, was figuring out how to communicate the new AI-powered features to users, Gautam said. In the case of offering a new data point, such as the new Growth Insight feature designed to indicate how much a specific company is growing, Crunchbase had to balance offering enough information needed to render trust without bombarding the user with an overwhelming amount of text. This included information about the new feature, the data sources, and how the growth number was determined. It was an exercise in bringing together a lot of moving pieces and impacted information architecture, UI, and UX.

While traditional computer programming is largely understood, so much about AI is still up in the air. Companies are left grappling with questions about what’s an acceptable use case for AI, how to test use cases, and whether to tap off-the-shelf models from companies like OpenAI or attempt to build their own. The rapidly evolving nature of the technology also poses major challenges in terms of how to develop without structurally locking yourself out of future progress.

“If there's an amazing breakthrough two months from now, we do not want to get backed into an architectural choice we made,” Gautam said.

Companies looking to integrate AI into their workflows are also having to navigate “AI washing,” wherein providers overinflate the degree to which their offerings are actually based on AI to capitalize on the hype, as well as “AI sprawl.” In a 2024 Canva survey on deploying AI tools, 71% of CIOs said they expect to adopt between 30 and 60 new apps in 2024, with 72% expressing concerns about sprawl. Tapping too many tools can increase the complexity of a company’s tech stack and open up new security risks, not to mention drive up the IT budget. The challenge comes from the bombardment of AI tools that are available, many with redundant capabilities, which makes it difficult to determine which ones to use. In the survey, 84% of CIOs said there are already too many AI tools available. Not only are the vendors that companies already work with rushing to adopt AI, but there are a slew of new offerings from startups, too.

Whether deploying AI into internal workflows or building products, cost is also a major factor. The costs associated with generative AI — from specialized GPUs and cloud resources to top talent and compliance work — can be astronomical. An increasing number of Wall Street analysts and Silicon Valley investors are beginning to sound the alarm that all the spending might be premature, according to The Washington Post. In June, Goldman Sachs published a report arguing it’s hype-driven and that there’s “little to show for” the huge amount of spending on generative AI.

“The big tech companies have no choice but to engage in the AI arms race right now given the hype around the space and FOMO,” wrote Jim Covello, head of global equity research at Goldman Sachs, in the paper, adding that “over-building things the world doesn’t have use for, or is not ready for, typically ends badly.”

When OpenAI released ChatGPT in November 2022, it launched a tidal wave toward generative AI. Leaders across sectors looked toward the problems it could solve and opportunities it could unlock, but many jumped on it simply because it was the hottest new tech in town — or because there was immense pressure to do so. Generative AI-products are still new and time will tell how successful they are, but across the board, tech leaders implementing AI stress that it has to offer value and not just be AI for AI’s sake.

“I have met zero customers who have said, ‘I wish I had more AI.’ But they have asked me, ‘Could it tell me more? Could it do more for me?’’’ Chow said.

Governance, regulations, hallucinations, complicated architectural challenges, AI sprawl, new cybersecurity vulnerabilities, and massive costs still represent just a snapshot of the hurdles companies are facing to embrace AI. There are companies getting creative to overcome them: Getty, for example, avoided the contentious copyright issues around text-to-image models by creating its own, trained only on data it owns outright. With such a wide range of challenges surrounding the technology, however, one thing is clear: AI has to be an all-hands-on-deck effort.

“You have to bring your whole team into it. The product managers, the designers, the marketing team, engineers,” Chow said. “Everyone has to be part of the process, otherwise some process will break.”