LLM System and Hardware Requirements – Running Large Language Models Locally #systemrequirements

This is a great 100% free Tool I developed after uploading this video, it will allow you to choose an LLM and see which GPUs could run it… : https://aifusion.company/gpu-llm/

Min Hardware requirements (up to 16b q4 models) (eg. Llama3.1 – 8b)
RTX 3060 12GB VRAM : https://amzn.to/3M0HvsL
Intel i5 or AMD Ryzen 5
Intel i5 : https://amzn.to/3WGZtp3
Ryzen 5 : https://amzn.to/46IigoC
36GB RAM
1TB SSD : https://amzn.to/4cBebEd

Recommended Hardware requirements (up to 70b q8 models) (eg. Llama3.1 – 70b)
RTX 4090 24GB VRAM : https://amzn.to/3AjIHow
Intel i9 or AMD Ryzen 9
Intel i9 : https://amzn.to/3YCeLxW
AMD Ryzen 9 : https://amzn.to/3YIaUiT
48GB RAM
2TB SSD : https://amzn.to/3YFQ83A

Professional Hardware requirements (up to 405b and more) (eg. Llama3.1 – 405b)
Stack of A100 GPUs or A6000 GPUs
https://amzn.to/3yojZ5T
Enterprise grade CPUs
https://amzn.to/3YDgByw
https://amzn.to/4dEbfY2