LLMs are super expensive to train but how much does it cost to run inference on them? Are there any known examples or guesses of the actual average monthly costs?

@codechips @a16z estimates LLM inference costs dropped 10x yearly e.g. GPT-3 went from $60/million tokens in 2021 to $0.06 today. Together.ai’s Llama 3.2 3B achieves similar results at minimal cost. For monthly costs, depends on usage: 1B tokens/month ≈ $60 (GPT-4-level). >>