What are you going to do with it? The full 670B param Q8 quantization model needs a monster server to run it on because of the required RAM. The cheapest I've seen is this guy who says he got it running on a $6,000 build but it only gets about 6 tokens/second, https://x.com/carrigmat/status/1884244369907278106 . If you want to make it fast you'd have to try to load the model into GPU memory which is going to cost literally 100s of thousands of dollars._______________________________________________On Mon, Feb 3, 2025 at 2:16 PM Autonomous via sudo-discuss <sudo-discuss@sudoroom.org> wrote:_______________________________________________Would it be possible to obtain a copy of the Deep Seek R1 base model and then make it available to Sudo Room members to make additional copies? Noisebridge has considerable bandwidth and perhaps they could download the 700GB model.
Downloading 700GB from my ISP would probably not work, but copying the model to a 1TB SSD would work.Here's the Deep Seek R1 base model on Huggingface:
sudo-discuss mailing list -- sudo-discuss@sudoroom.org
To unsubscribe send an email to sudo-discuss-leave@sudoroom.org
More options at https://sudoroom.org/lists/postorius/lists/sudo-discuss.sudoroom.org/
sudo-discuss mailing list -- sudo-discuss@sudoroom.org
To unsubscribe send an email to sudo-discuss-leave@sudoroom.org
More options at https://sudoroom.org/lists/postorius/lists/sudo-discuss.sudoroom.org/