Why local AI run slowly?

First of all, take a look at the requirements of the AI model. Usually the small RAM is the limitation for the processing speed.

If the computer does not have 16GB or more with bandwidth of higher than 100GB/s, it’s not recommended to run local AI models but better use online AI app.

For the computer with GPU, the CPU must have 8GB or more RAM with much higher bandwidth.

The second requirement is the performance of CPU. The CPUs released within 3 years with more than 4 cores are the best choices for local AI models.

Therefore, for acceptable speed of local AI, the recommendation is to upgrade the computer with much larger RAM, faster CPU and more powerful GPU.