FS#1313 - git ports driver wipes built packages, downloaded sources, etc from ports tree
Hello, the git ports driver (/etc/ports/drivers/git) runs a 'git clean' stage during a ports -u that deletes downloaded sources, built packages, work dirs, and so on from the ports tree. The default crux configuration is to store all that stuff in the ports tree, and the other ports drivers don't behave in this manner. As a workaround, all that stuff can be stored outside the tree, but that's not the default setting...
I recommend rethinking the git clean stage (assuming the crux defaults for storing downloads/work/packages won't change). The git driver didn't always have this line in it, and I'm not sure why it was added.
Loading...
Available keyboard shortcuts
- Alt + ⇧ Shift + l Login Dialog / Logout
- Alt + ⇧ Shift + a Add new task
- Alt + ⇧ Shift + m My searches
- Alt + ⇧ Shift + t focus taskid search
Tasklist
- o open selected task
- j move cursor down
- k move cursor up
Task Details
- n Next task
- p Previous task
- Alt + ⇧ Shift + e ↵ Enter Edit this task
- Alt + ⇧ Shift + w watch task
- Alt + ⇧ Shift + y Close Task
Task Editing
- Alt + ⇧ Shift + s save task
Is this still happening?
Yes, it is.
It might do that if there is a some .git file without NAME=... in /etc/ports/.
Right?
If any of the git repos in /etc/ports were missing their $NAME settings, the driver would error out prior to running any of the git commands, since it performs a check to make sure $NAME is set.
The root of the problem is that the driver runs a 'git clean' on the repo, which will remove all untracked files (which includes built packages and sources). It didn't always do that, so someone must have intended it to work that way and I'm not sure why.
Maybe I'm missing something. For sure I've missed the NAME check.
I wasn't aware that ports will keep the packages and sources there, but in /usr/ports/{distfiles,packages}, like this: https://github.com/6c37/crux-ports-git/blob/3.3/bspwm/Pkgfile
I think we may be talking about two different things. My report was regarding the git ports driver at /etc/ports/drivers/git that is installed when the opt/git port is installed. These drivers are used for syncing the ports tree (with 'ports -u'), not for installing packages. The Pkgfile you linked is from a collection of third-party ports that use git repositories for downloading their sources rather than the usual tarball. Fetching sources from a git repo is not something handled by the git driver in question.
Furthermore, /usr/ports/{distfiles,packages} is not the default location for downloads and built packages - the port's directory is. Keeping distfiles and packages stored outside of the ports tree is one possible mitigation for the issue (because then it's outside the git repo that will get cleaned during ports -u), but it's not the default setting.
I didn't realized that default is
PKGMK_SOURCE_DIR="$PWD"
PKGMK_PACKAGE_DIR="$PWD"
PKGMK_WORK_DIR="$PWD/work"
Like a NOOB, I blindly followed the wiki instructions. It looks like most people use the default way (code) or the recommended way (comments/wiki/fake news). Most of them, judging by the popularity of this issue, follow the comments. It is psychological interesting.
LOL.
It is confusing to have the docs so different than the code/defaults.
And all the silly waves I did in the last days around here...
LOL.
Crux is so libertarian... Only one rule: everybody does what s/he wants with her/his installation. Defaults are just a starting point. Fight Club.
LOL
Anyway, I vote for you on this, but I also vote for defaults based on popularity, because this is the best way to avoid confusion. Of course, simplicity should be on top, but clarity should follow, because confusion makes the simple be less simple, at best.
> The git driver didn't always have this line in it, and I'm not sure why it was added.
I might be misreading the commit history, but according to https://crux.nu/gitweb/?p=ports/opt.git;a=commitdiff;h=bd86ca2544f8de84b455ad0e7e9b801abc51dc05 the "git clean" command was there from the very beginning. If you want to shield from deletion the downloaded sources, work directory, and built packages when running `ports -u`, there are two possibilities that come to mind:
1) have pkgmk detect when the repository is under git management, and create a .gitignore file based on the source array, the package name, and the setting for $PKGMK_WORK_DIR.
2) assemble a list of excluded files in the git driver itself, and pass this list to the -e flag of the git clean command.
Option 1 might encounter strong opposition because pkgmk is already more complicated than it was five years ago. Every new feature in pkgmk seems to need greater justification than the last feature added. Option 2 should be easier to persuade the core maintainers to adopt, and I'd be willing to put together a patch if there's enough interest.
On an unrelated note, does anyone actually use the LOCAL_REPOSITORY feature that closed Flyspray issue 1364? I can see when it was added in the commit history of the git driver, but there's no corresponding update to the ports(8) manpage illustrating how this feature might be used. It's a shame when a good idea fades from institutional memory just because nobody bothered to add some examples to the documentation.
I inserted a few lines in the git driver to shield pkgmk artifacts from deletion. See attached. After several weeks of testing, it looks like a viable solution to the original issue.
This updated driver does NOT inspect its environment to see whether pkgmk artifacts are actually saved in the ports tree. A more optimal refactoring would bundle the manipulations of .git/info/exclude into a separate function, which is only called when /etc/pkgmk.conf indicates an unchanged default location for downloads and built packages.