lemmydividebyzero@reddthat.com to Technology@lemmy.worldEnglish · 3 months agoOpensource AI Must Winopensourceaimustwin.comexternal-linkmessage-square106linkfedilinkarrow-up1285arrow-down120cross-posted to: technology@midwest.socialtechnology@lemmy.ml
arrow-up1265arrow-down1external-linkOpensource AI Must Winopensourceaimustwin.comlemmydividebyzero@reddthat.com to Technology@lemmy.worldEnglish · 3 months agomessage-square106linkfedilinkcross-posted to: technology@midwest.socialtechnology@lemmy.ml
minus-squareshortwavesurfer@lemmy.ziplinkfedilinkEnglisharrow-up16arrow-down3·3 months agoOllama.com Install it and then “ollama run olmo-3:7b” gets you a local AI. If you want to run a smarter AI, then you’re going to need a bigger parameter model, which is going to take more hardware to run.
minus-squareshortwavesurfer@lemmy.ziplinkfedilinkEnglisharrow-up5·3 months agoNot the big ones, no. But I think almost everybody can run one with seven billion parameters.
minus-squaremabeledo@lemmy.worldlinkfedilinkEnglisharrow-up1arrow-down1·edit-23 days agodeleted by creator
minus-squareshortwavesurfer@lemmy.ziplinkfedilinkEnglisharrow-up4·3 months agoI ran it on my 8GB RAM desktop.
minus-squareZephyrXero@lemmy.worldlinkfedilinkEnglisharrow-up4arrow-down1·3 months agoI got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
minus-squareZephyrXero@lemmy.worldlinkfedilinkEnglisharrow-up4arrow-down1·3 months agoIf the model dumps over to system ram it gets super slow, you ideally want it to fit completely in your VRAM
Ollama.com
Install it and then “ollama run olmo-3:7b” gets you a local AI. If you want to run a smarter AI, then you’re going to need a bigger parameter model, which is going to take more hardware to run.
deleted by creator
Not the big ones, no. But I think almost everybody can run one with seven billion parameters.
deleted by creator
I ran it on my 8GB RAM desktop.
deleted by creator
I got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
deleted by creator
If the model dumps over to system ram it gets super slow, you ideally want it to fit completely in your VRAM