technology

+++ date = "2023-03-24" +++ #decentralization #technology #artificial_intelligence at the moment, [huggingface](https://huggingface.co) is one of the main hubs for model distribution, they're basically a "free CDN", and that _DOES NOT SCALE_. from what i…

#artificial_intelligence #large_language_models #technology based on https://betterprogramming.pub/fine-tuning-gpt-j-6b-on-google-colab-or-equivalent-desktop-or-server-gpu-b6dc849cb205 ran on [RunPod], which provided the "cheapest" RTX A6000 GPU as …

https://www.runpod.io/ #artificial_intelligence #cloud_computing #technology provides gpu renting there are two models: - secure cloud: first party RunPod hardware - community cloud: decentralized compute, they vet and provide that at (supposedly) ch…

+++ date = "2023-04-14" +++ #virtual_reality #vrchat #technology I have been heavily using VRChat multiple times a week throughout the last year (majority of time on the Quest 2, until I was able to [buy a PC that could do VR](https://wf.lavatech.top/lu…

#llama #large_language_models #technology #performance | hostname | device | model | time per token | | - | - | - | - | | arae | cpu | 7b q4 (non-gptq) | 1000ms | | arae | cpu | 7b q4 (gptq) | TODO | | wall | cpu | 7b q4 (non-gptq) | 160ms | | wall | cpu…

#llama #large_language_models #technology #performance %at=2023-07-25T01:18:10 remember, the gold **SUBJECTIVE** target is 10tok/s (i think thats what chatgpt runs at, probably 40tok/s, i dont run those numbers yet), which is around 100ms per token. goi…

#llama #large_language_models #technology #performance timeline: - started project %at=2024-04-20T00:16:05.592Z - finished this version %at=2024-05-19T22:20:38.907Z to continue the tradition from [llama inference numbers] and [llama2 inference number…