Comment on Smaug-72B-v0.1: The New Open-Source LLM Roaring to the Top of the Leaderboard
Infiltrated_ad8271@kbin.social 4 months agoI tested it with a 16GB model and barely got 1 token per second. I don't want to imagine how long it would take if I used swap instead.