Why AI models do not work well?

If the computer does not meet the requirements, some AI models and softwares will not work well or run very slowly.

First of all, check the specifications of the computer, as most of AI softwares need at least 16GB RAM and most recent CPU.

Some AI with larger parameters need more than 16GB RAM, even up to 1024GB RAM.

Usually a AI model with 7B parameters needs 12GB RAM for FP16 computing, and needs 8GB RAM for 8bit computing, meanwhile the system needs several GB RAM for basic operations.

However, for better performance and faster token computing, double-sized RAM of the AI parameters plus 10GB RAM for system are recommended, as 24GB RAM is recommended for a 7B AI model.

And, above is same for the memory requirement of GPU.