Marco Abis - personal
.
@me,
The Inference Cost Of Search Disruption – Large Language Model Cost Analysis by @dylan522p@birdsite.wilde.cloud
$30B Of Google Profit Evaporating Overnight, Performance Improvement With H100 TPUv4 TPUv5
"Google is playing defense on margins with this smaller model. They could have deployed their full-size LaMDA model or the far more capable and larger PaLM model, but instead, they went for something much skinnier.
This is out of necessity.
Google cannot deploy these massive models into search. It would erode their gross margins too much."
www.semianalysis.com/p/the-inference-cost-of-search-disruption
