large_language_models

+++ date = "2023-03-24" +++ #large_language_models #artificial_intelligence #musings it started with me reading this tweet: !twitter[https://twitter.com/spawning_/status/1639020027989962752] this is only tangential to the tweet's subject, but i wonder …

#artificial_intelligence #large_language_models #technology based on https://betterprogramming.pub/fine-tuning-gpt-j-6b-on-google-colab-or-equivalent-desktop-or-server-gpu-b6dc849cb205 ran on [RunPod], which provided the "cheapest" RTX A6000 GPU as …

+++ date = "2023-06-05" +++ *or, "a sequel to [quickly train gpt-j on some data] except the data is better and the model is Vicuna"* #large_language_models #artificial_intelligence #author_luna ## why > luna: ...now i need another cursed project > d…

#large_language_models #notes #author_luna i was able to use https://github.com/Lightning-AI/lit-llama to finetune llama-7b on a dual-3090 setup from a colleague of mine ~~however, the project does not support unstructured datasets, so i hacked together…

#llama #large_language_models #technology #performance | hostname | device | model | time per token | | - | - | - | - | | arae | cpu | 7b q4 (non-gptq) | 1000ms | | arae | cpu | 7b q4 (gptq) | TODO | | wall | cpu | 7b q4 (non-gptq) | 160ms | | wall | cpu…

#author_luna #artificial_intelligence #large_language_models #fine_tuning #peft tldr "because it didn't work with just the original lora layers", i dont know the exact reasons why https://f.l4.pm/notice/AXriEbvXG3SvzArR44 in [the original LoRA paper](ht…

#author_luna #large_language_models #peft #artificial_intelligence if i'm wrong about this email me. --- from what i gathered those two do not mix together very well. qlora it seems does their own quantization system on the NF4 datatype, which require…

see the others: - [llama inference numbers] - [llama2 inference numbers] i am not running perplexity scores on these (that's more important for a quantization comparison chart, rather than performance parameter tweaking), just want to find out how …

#author_luna #spam #artificial_intelligence #large_language_models this is a copy-paste of a long thread i wrote up on fedi mid december 2023. thought it had enough power to be its own article --- the more i look at spam the more i look at The Architec…

#author_luna #ai #fine_tuning #large_language_models see [petoriagpt](https://notnite.com/blog/petoriagpt/) for the first edition of this experiment ![Pasted image 20240121173928.png] *generated by me with [novelai diffusion anime v3](https://novelai…

#llama #large_language_models #technology #performance %at=2023-07-25T01:18:10 remember, the gold **SUBJECTIVE** target is 10tok/s (i think thats what chatgpt runs at, probably 40tok/s, i dont run those numbers yet), which is around 100ms per token. goi…

#llama #large_language_models #technology #performance timeline: - started project %at=2024-04-20T00:16:05.592Z - finished this version %at=2024-05-19T22:20:38.907Z to continue the tradition from [llama inference numbers] and [llama2 inference number…