Dwindling start-up investments, relentless headcount pressures (read layoffs), and an obsession with productivity metrics clearly demonstrate that there’s no running away from our current ‘era of efficiency.’ As organizations continue to grapple with their largest expense, it’s time to talk about the silver bullet that is supposedly solving this big bulging bill i.e., Agents and Agentic AI. While executives across industries are openly admitting that this once-in-a-lifetime innovation will help shrink their workforce, there is one conversation that is being highly ignored – What does this ‘solution’ actually cost?
Whenever I attempt to venture into a conversation on the ‘cost of AI’, the room is usually quick to jump into one of two assumptions; that I am either attempting to convince them that AI will destroy the planet with its energy consumption or that AI will eventually take over the human race. Now while I truly believe that both of these are worthwhile debates, there are other angles to the conversation; angles that aren’t necessarily cynical but definitely practical.
Let’s start with a question: How much do you think saying ‘Please’ and ‘Thank you’ to ChatGPT costs OpenAI?
Per Sam Altman, OpenAI CEO, these niceties cost OpenAI “Tens of millions of dollars” in computational costs. I have no doubts that maintaining good manners with our GenAI tools has its benefits, but I can’t help wondering how much each transaction costs if these three words alone cost tens of millions. How do these costs break down between computing power, training, storage, and infrastructure? And how will these expenses ultimately affect organizations implementing AI solutions?
While it’s been challenging to get accurate data on model training costs, estimates suggest it takes between 50-100 million dollars to train the largest LLM models, involving hundreds of highly paid people working for up to 6 months. OpenAI’s GPT-4, according to the estimates, required approximately $78 million in compute costs alone. Even more staggering is Google’s Gemini Ultra, which commanded an estimated $191 million investment in training resources. Even running these models is expensive – ChatGPT’s daily operational costs is around $700,000, translating to somewhere between 4 to 36 cents per query, depending on who you ask and how they’re calculating it. While this might seem negligible, it adds up quickly at scale. Even for mid-sized applications integrating ChatGPT, costs can run between $3,000 and $7,000 per month for just 100,000 monthly queries. Consider this: Google’s search engine, financed through advertising, has an estimated cost per query of 2.5 cents. ChatGPT’s estimated cost per query is around 36 cents – roughly 14 times more expensive. And that’s before we factor in the salaries of AI researchers and developers, which average $150,000 at OpenAI and $127,000 at Microsoft.
The infrastructure costs are equally staggering. AI data centers consume massive amounts of energy – a single AI training run can emit as much carbon as five cars over their entire lifetimes, according to a 2022 study from the University of Massachusetts Amherst. Microsoft, OpenAI’s primary infrastructure partner, reportedly allocated over $50 billion for AI infrastructure in 2023. For enterprises, the costs become more tangible: running an AI model similar to GPT-3 internally could cost $10-15 million annually in computing resources alone, not including development and maintenance costs.
The irony isn’t lost here – in our rush to cut costs through AI implementation, we’re creating one of the most expensive computing workloads in history.Companies like Google and Microsoft are building specialized AI chips and data centres to reduce these costs. Some say that these costs are likely to increase vs decrease but only time will tell.
So, what does this mean for us as human resources professionals?
First, we’ll probably get relabeled to depict the expansion from humans to humans + agents + multiple technologies. It’s no wonder that Moderna went ahead and merged its HR and technology teams. I expect more similar moves in the future. However, this is the most inconsequential of changes.
More importantly, we need to develop a deep understanding of how our workforce planning will evolve. Agent and AI costs will be managed similar to our workforce cost with us determining where agents are worth the cost and where they aren’t. This includes getting savvy with a variety of new metrics such as determining the appropriate human: agent ratio for tasks in addition to the regular manager, job level and other ratios that we currently enjoy determining. And lastly, we need to model usage frameworks, training and monitoring to ensure employees aren’t utilizing agents for ‘frivolous’ or unnecessary tasks (yes, that includes using it for personal meal prep!).
There is no denying that transformative innovations like the advent of electricity or telephones come by once in a lifetime. Agentic AI most definitely falls into this category. As with any breakthrough innovation, adoption is key. Thus, the first order of action is to encourage adoption of GenAI and agents irrespective of how frivolous the use. However, as we push for adoption, we must also start considering the boundaries of AI usage. This isn’t just about responsible AI use; it’s about strategic resource and agent budget management. With 65% of businesses now integrating generative AI into their workflows, the question of how to optimize its use becomes increasingly pertinent.
With 82% of companies planning to integrate AI agents within the next one to three years, we’re on the brink of a significant transformation in how work is done. AI is already making substantial impacts in areas like content creation, customer support automation, and personalized communications. As these tools become more sophisticated, their role in enhancing productivity and driving innovation will only grow
So, while we don’t need to stop being polite to our AI assistants just yet, we do need to start thinking critically about how we use these resources. The era of unlimited, consequence-free AI usage is ending before it really began – and that’s something every organization needs to prepare for.
The HR Business Partner Story 