
Been experimenting with local LLMs lately. The idea of having unlimited access to a model running right on your own machine is such an appealing one.
The cost of compute is skyrocketing. I see local LLMs staying just slightly behind the cloud models.
But I also think at some point these models get so good that the small gap won’t matter as much. Depending on your use case, of course.
How are you using local LLMs? Are you investing in compute to run them?
English

