3 Feb
2025
3 Feb
'25
4:11 p.m.
CSUA has a free-for-all "cluster" accessible to Berkeley affiliates
(including alums) which has 8x P100s.
On Mon, Feb 3, 2025 at 4:04 PM Miguel Vargas via sudo-discuss <
sudo-discuss(a)sudoroom.org> wrote:
What are you going to do with it? The full 670B param
Q8 quantization
model needs a monster server to run it on because of the required RAM. The
cheapest I've seen is this guy who says he got it running on a $6,000 build
but it only gets about 6 tokens/second,
https://x.com/carrigmat/status/1884244369907278106 . If you want to make
it fast you'd have to try to load the model into GPU memory which is going
to cost literally 100s of thousands of dollars.
On Mon, Feb 3, 2025 at 2:16 PM Autonomous via sudo-discuss <
sudo-discuss(a)sudoroom.org> wrote:
Would it be possible to obtain a copy of the Deep
Seek R1 base model and
then make it available to Sudo Room members to make additional copies?
Noisebridge has considerable bandwidth and perhaps they could download the
700GB model.
Downloading 700GB from my ISP would probably not work, but copying the
model to a 1TB SSD would work.
Here's the Deep Seek R1 base model on Huggingface:
https://huggingface.co/deepseek-ai/DeepSeek-R1/tree/main
_______________________________________________
sudo-discuss mailing list -- sudo-discuss(a)sudoroom.org
To unsubscribe send an email to sudo-discuss-leave(a)sudoroom.org
More options at
https://sudoroom.org/lists/postorius/lists/sudo-discuss.sudoroom.org/
_______________________________________________
sudo-discuss mailing list -- sudo-discuss(a)sudoroom.org
To unsubscribe send an email to sudo-discuss-leave(a)sudoroom.org
More options at
https://sudoroom.org/lists/postorius/lists/sudo-discuss.sudoroom.org/