3 Feb
2025
3 Feb
'25
4:03 p.m.
What are you going to do with it? The full 670B param Q8 quantization model
needs a monster server to run it on because of the required RAM. The
cheapest I've seen is this guy who says he got it running on a $6,000 build
but it only gets about 6 tokens/second,
https://x.com/carrigmat/status/1884244369907278106 . If you want to make it
fast you'd have to try to load the model into GPU memory which is going to
cost literally 100s of thousands of dollars.
On Mon, Feb 3, 2025 at 2:16 PM Autonomous via sudo-discuss <
sudo-discuss(a)sudoroom.org> wrote:
Would it be possible to obtain a copy of the Deep Seek
R1 base model and
then make it available to Sudo Room members to make additional copies?
Noisebridge has considerable bandwidth and perhaps they could download the
700GB model.
Downloading 700GB from my ISP would probably not work, but copying the
model to a 1TB SSD would work.
Here's the Deep Seek R1 base model on Huggingface:
https://huggingface.co/deepseek-ai/DeepSeek-R1/tree/main
_______________________________________________
sudo-discuss mailing list -- sudo-discuss(a)sudoroom.org
To unsubscribe send an email to sudo-discuss-leave(a)sudoroom.org
More options at
https://sudoroom.org/lists/postorius/lists/sudo-discuss.sudoroom.org/