large_language_models

(8 pages)

musings on copyright and datasets

+++ date = "2023-03-24" +++ #large_language_models #artificial_intelligence #musings it started with me reading this tweet: !twitter[https://twitter.com/spawning_/status/1639020027989962752] this is only tangential to the tweet's subject, but i wonder …

quickly train gpt-j on some data

#artificial_intelligence #large_language_models #technology based on https://betterprogramming.pub/fine-tuning-gpt-j-6b-on-google-colab-or-equivalent-desktop-or-server-gpu-b6dc849cb205 ran on [[RunPod]], which provided the "cheapest" RTX A6000 GPU as …

dril-instruct

+++ date = "2023-06-05" +++ *or, "a sequel to [[quickly train gpt-j on some data]] except the data is better and the model is Vicuna"* #large_language_models #artificial_intelligence #author_luna ## why > luna: ...now i need another cursed project > d…

quickly train llama-7b on some data

#large_language_models #notes #author_luna i was able to use https://github.com/Lightning-AI/lit-llama to finetune llama-7b on a dual-3090 setup from a colleague of mine ~~however, the project does not support unstructured datasets, so i hacked together…

llama inference numbers

#llama #large_language_models #technology #performance | hostname | device | model | time per token | | - | - | - | - | | arae | cpu | 7b q4 (non-gptq) | 1000ms | | arae | cpu | 7b q4 (gptq) | TODO | | wall | cpu | 7b q4 (non-gptq) | 160ms | | wall | cpu…

llama2 inference numbers

#llama #large_language_models #technology #performance %at=2023-07-25T01:18:10 remember, the gold **SUBJECTIVE** target is 10tok/s (i think thats what chatgpt runs at, probably 40tok/s, i dont run those numbers yet), which is around 100ms per token. goi…

why all linear layers?

#author_luna #artificial_intelligence #large_language_models #fine_tuning #peft tldr "because it didn't work with just the original lora layers", i dont know the exact reasons why https://f.l4.pm/notice/AXriEbvXG3SvzArR44 in [the original LoRA paper](ht…

qlora and gptq

#author_luna #large_language_models #peft #artificial_intelligence if i'm wrong about this email me. --- from what i gathered those two do not mix together very well. qlora it seems does their own quantization system on the NF4 datatype, which require…