• 0 Posts
  • 53 Comments
Joined 1 year ago
cake
Cake day: January 17th, 2024

help-circle
  • Back in university, I studied basically all day long, which was tiresome after long sessions of study, even if with friends. My great superpower is that it used to just take me ~10 seconds of resting with my eyes closed to feel a huuuuge boost of energy that lasted for 1-2 hours. After that boost expired, I just did it again.

    Incredibly useful.






  • The 1.5B version that can be run basically on anything. My friend runs it in his shitty laptop with 512MB iGPU and 8GB of RAM (inference takes 30 seconds)

    You don’t even need a GPU with good VRAM, as you can offload it to RAM (slower inference, though)

    I’ve run the 14B version on my AMD 6700XT GPU and it only takes ~9GB of VRAM (inference over 1k tokens takes 20 seconds). The 8B version takes around 5-6GB of VRAM (inference over 1k tokens takes 5 seconds)

    The numbers in your second link are waaaaaay off.