Comment on Smaug-72B-v0.1: The New Open-Source LLM Roaring to the Top of the Leaderboard
OutrageousUmpire@lemmy.world 4 months agoAny idea what 8Q requirements would be? Or 4 or 5?
Comment on Smaug-72B-v0.1: The New Open-Source LLM Roaring to the Top of the Leaderboard
OutrageousUmpire@lemmy.world 4 months agoAny idea what 8Q requirements would be? Or 4 or 5?
General_Effort@lemmy.world 4 months ago
huggingface.co/senseable/…/main
About 44GB and 50GB for the Q4 and 5. You’d need quite some extra to fully use the 32k context length.