Leadership CONNECT: 27-FEB-2025 (Y25W10)

Background

Ethan Mollic, a leading voice in AI, recently (24-Feb-2025) shared key insights in his post “A new generation of AIs: Claude 3.7 and Grok 3.” [1] Among other things, it includes the above quality/cost visualization.

On the Y-axis: Quality, as defined by the Graduate-Level Google-Proof Q&A test (GPQA) is a series of very hard multiple-choice problems designed to test advanced knowledge. PhDs with access to the internet get 34% right on this test outside their specialty and 81% inside their specialty (marked approximately as “Human PhD in their field”).

On the X-axis, the cost per million tokens is the cost of using the model (Gemini Flash Thinking Costs are estimated). The data is based on Mollic’s research, but Epoch and Artificial Analysis were good sources, and Latent Space offers a more comprehensive graph of costs across many models.

“This graph shows how quickly this trend has advanced, mapping the capability of AI on the y-axis and the logarithmically decreasing costs on the x-axis.”

Specifically, when GPT-4 came out, it was around $50 per million tokens (each token is roughly a word). Now, it costs around 12 cents per million tokens to use Gemini 1.5 Flash, an even more capable model than the original GPT-4.

A Few Things to Note:

• This is a map from June of 2023, the release of GPT-4, a relatively short time.
• The cost Y axis is logarithmic—meaning prices drop a lot — 99.7% by one measure.
• On the other hand, we need many more tokens these days. For example, research engines generate answers with 10,000 tokens and more.
• Soon (as is already the case with the new Anthropic 3.7), the engine will decide if the user wants a short and quick answer or more profound research (this feature was planned for GPT-5.0, and Anthropic has already released it). In the API, you can state if you want deeper thinking — meaning more tokens).
• If you are technical, Special technologies for users or corporate, like RAG, may not be needed, as you can simply send the entire content to be processed.

Top Five Lessons for Leaders

1. AI is Becoming Ubiquitous: As costs drop, AI will be integrated into nearly every business function, making strategic adoption crucial.
2. Competitive Edge Depends on Execution: As AI becomes more accessible, organizations’ success in implementing and optimizing AI solutions will determine their differentiation.
3. The Cost of AI is No Longer a Barrier: The decreasing price of AI means more companies, including startups, can leverage advanced AI capabilities without massive investments.
4. Increased AI Usage Requires Smarter Governance: More AI-generated content and automation demand strong oversight, ethical considerations, and security protocols.
5. Adaptability is Key. Organizations continuously updating their AI strategies and workflows will reap the most value from AI advancements.

My Tips for You

1. Are you paying the minimum $20 per OpenAi (ChatGPT), Anthropic (Claude), etc.? If not, please do so—it will help you use it and teach you a lot.
2. Much like mobile or cloud, which used to cost hundreds of dollars and now cost much less, major tech players like OpenAI, Google, and Anthropic will likely dominate AI as a service in a few years. On top of that, you will get specific services for various business functions, and then you will have your particular use—this is where you will shine and need to compete.
   

[1] See Mollic Post.
[2] See all the Sparks in Yesha on Human Thinking Collection.