my python packaging patterns
#author_luna #python #python-packaging #notes
updated
these are mine. i have been writing python software for 10 years (though Python to production is more like 5yrs ago), so i know how to deal with things when they go bad (or sometimes i think about the new shiny and then i become miserable and give up on the shiny)
see Relieving your Python packaging pain Why not tell people to "simply" use pyenv, poetry or anaconda. they condense my experience (and guidelines) pretty well.
as a developer of software #
venv + pip.
i have not used pip-tools but i might, as it doesn't break my workflow (allegedly! i haven't tried it!!!!)
i have used poetry and pipenv extensively to reach their edge cases, neither give me hope.
as a user of cli software made by others #
pipx is what i use, sometimes i have to hax together some dependency inside the venv, but that's easy (~/.local/pipx/venvs/something/bin/pip).
pip install --user, but i'd recommend pipx nowadays, especially in a post-PEP 668 world
as a user of cli software that i might want to use git versions of #
venv + pip.
as a machine learning researcher #
llm: mixture of conda (for cudatoolkit which needs gcc-6, not available on void linux. i need cuda toolkit for exllama or gptq) + venv (bad idea, should just be conda)
stable diffusion workloads: they don't require cuda toolkit afaik. venv-only in that case
there may be changes to this as i plan to assemble an ml system and i'll need something more organised for all the cuda installs, etc.