192
edits
Difference between revisions of "Gpubox"
Jump to navigation
Jump to search
→Specific VM Configurations: add ufw rule for port 11434
(→Specific VM Configurations: add ufw rule for port 11434) |
|||
| Line 130: | Line 130: | ||
* Install nvidia drivers https://www.xda-developers.com/nvidia-stopped-supporting-my-gpu-so-i-started-self-hosting-llms-with-it/ | * Install nvidia drivers https://www.xda-developers.com/nvidia-stopped-supporting-my-gpu-so-i-started-self-hosting-llms-with-it/ | ||
** Pin the driver version so you don't have to re-run the nvidia installer every time the kernel gets updated | ** Pin the driver version so you don't have to re-run the nvidia installer every time the kernel gets updated | ||
* Install | * Install ollama with <pre>curl -fsSL https://ollama.com/install.sh | sh</pre> | ||
* Use ollama to pull and run deepseek-r1:8b | * Use ollama to pull and run deepseek-r1:8b | ||
* <pre>sudo ufw allow from 10.0.0.0/24 to any port 11434 proto tcp</pre> | |||
* Verify: http://ollama.local:11434/ should show the message <pre>Ollama is running.</pre> | * Verify: http://ollama.local:11434/ should show the message <pre>Ollama is running.</pre> | ||