Comment on Local LLM agents
How many concurrent users and what hardware if i may ask?
it’s an h100, I think, no idea about how many users
in my personal setup i use quantized versions on a 3080, which is not great, so I still lean a lot on APIs
eager_eagle@lemmy.world 6 days ago
it’s an h100, I think, no idea about how many users
in my personal setup i use quantized versions on a 3080, which is not great, so I still lean a lot on APIs