This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.
These startups are chasing the next big thing in LLMs
Nine years after Google researchers introduced the transformer, this family of neural networks has become the engine inside every major large language model. But transformers are starting to show their age.
As LLMs get bigger and better, transformers have become a bottleneck. Their dense attention mechanism becomes increasingly expensive as the amount of text grows, and they’re not great at keeping track of a lot of information at once.
Here are four new ideas for how to solve the transformer problem—innovations that could change LLMs for good, making them faster, far more efficient, and (maybe) even smarter.
—Will Douglas Heaven
This story is from MIT Technology Review’s What’s Next series, which looks across industries, trends,
...
This post was originally published on this site.
