Categories: Ports
John McQuah
This article describes how to make your ports repository downloadable with git.
Read SettingUpAnHttpupRepo for an overview of the essential pieces of a ports repository. Apart from the sync and upload script, all the information there still applies to git repositories, because the official portdb performs its synchronization using httpup. In particular, you must remember to run httpup-repgen and git add REPO every time you want to push changes to your collection.
This step assumes you have created an account on github, gitlab, or some other host that responds to git requests.
Log in to your account through its web interface, and click the button that initializes a new project. (A typical icon to look for is a “+” near the top of the page.) Give it a descriptive name, and select the option that makes the project public.
After creating the bare repository, the first page you see will show a text box with the address of the clone file. Keep a record of this address, which will be needed when publishing your clone file in the subsequent steps.
It will be convenient to enable access to your git account using SSH key authentication. Navigate to the settings page, and find the button that lets you upload a public SSH key. You should create a dedicated SSH public/private keypair, and upload the public key to the web interface. Then write a stanza in ~/.ssh/config so that the correct key gets used for pushing changes to the server. Here is an example:
# ~/.ssh/config
Host git.mydomain.org
PreferredAuthentications publickey
IdentityFile ~/.ssh/git_id_ed25519
# User git # might be needed for some git servers
More detailed instructions (with screenshots) are available on the websites of the git host itself, in case ~/.ssh/config requires additional directives. For example:
In theory you could skip straight from step 1 to step 5 (publishing the clone file), relying solely on the web interface to upload any changes in your ports collection. Practically speaking, such a workflow does not scale well, especially when you maintain a large number of ports. Most maintainers will want to use the command-line interface, which entails putting your local ports collection under git version control. It is therefore assumed for the rest of this article that opt/git is installed on your CRUX system.
Let's assume you already have a ports collection in ~/myports. The following commands will set up the directory to be tracked by git.
$ cd ~/myports $ git init
At this point you have a bare repository; nothing has been added to the list of tracked files. Before adding any files or folders, you need to populate the file .git/config by declaring the intended connection between the local repository and the remote. You can establish this connection using the git pull command on the clone file whose address you saved in step 1.
$ git pull git@mydomain.org/path/to/clonefile.git
Another way to populate .git/config, if you had an existing repository (not bare) that you wanted to migrate from one server to another (e.g., github to gitlab), is the following sequence of commands:
$ git remote add <name> <url> $ git remote add upstream ssh://git@mydomain.org/path/to/clonefile.git $ git push upstream master
For the purposes of this article, let's continue under the assumption that you started from a bare repository.
Now that your ports directory is under version control, it's time to add the subdirectories. I suggest manually adding each port instead of doing "commit all", because "commit all" tends to include files that your audience should not be downloading (.httpup-repgen.ignore, for example). Note that by adding subdirectories manually you get to provide an informative commit message about each port.
For the convenience of users who synchronize via httpup, you should get in the habit of adding the REPO file (generated by httpup-repgen) along with the directory of the updated or newly-committed port. First create (or edit the existing) .httpup-repgen-ignore to prevent the inclusion of git version control info in the generated REPO file.
# ~/myports/.httpup-repgen-ignore .git .httpup README.md <shortname>.pub
Replace <shortname> with the name of your repository (as it appears on the portdb, if registered). Now you can run the following commands to add first_port to your git project:
$ httpup-repgen . $ git add -A first_port REPO $ git commit -m "first_port: initial commit"
If you type git commit on its own (not providing a message on the command line), git will become interactive and prompt you for a message. The message is then associated with every file or folder that was modified by git add, and will be shown on the web interface next to those files or folders. To ensure the relevance of each message, run git commit for each updated or newly-added port. All these locally-tracked changes can then be pushed to the server in one step:
$ git push
Next, to allow the ports(8) utility to easily sync your ports, create a config file that can be parsed by the git driver (/etc/ports/drivers/git). Here is a template for such a config:
# # /etc/ports/<shortname>.git # URL=https://git.mydomain.org/path/to/clonefile.git NAME=<shortname> BRANCH=master #LOCAL_REPOSITORY=/usr/pkgmk/git-ports/$NAME # End of file
Replace <shortname> with the nickname for your collection (the name that appears in the portdb), and edit the value of URL so that it points to the clonefile on your git server.
LOCAL_REPOSITORY (a setting on one of the comment lines in the template above) was added to the driver in response to Flyspray task 1364. This feature request made it possible to override the default target directory with a completely new path (even one outside /usr/ports, or with a name different from the value of NAME given in the config file). Despite the subsequent concerns expressed by Fun about letting external repositories change the target directory, the fact remains that <shortname>.git is sourced by a bash script running as root! Overriding any variable definitions declared in /usr/bin/ports was always a possibility (and not the most detrimental one). Because a 3- or 4-line config file can quickly be scanned for malicious code, the LOCAL_REPOSITORY feature was deemed harmless enough to survive Fun's critique. Customizing LOCAL_REPOSITORY might make sense if you synchronize a large number of user-contributed collections and want different mount points for the git collections versus the httpup/rsync collections, but for most users the default base directory /usr/ports never gets too cluttered, and this feature can be ignored.
Push the config file to your git server, so that it is visible to the public when they browse the web interface.
$ cp /etc/ports/<shortname>.git . $ git add <shortname>.git $ git commit -m "added a config file for ports -u" $ git push
The official portdb provides only an httpup sync file for each registered repository. Most visitors to the CRUX portdb will not dig too deeply into the individual repos to discover that yours is synchronizable via git. For those visitors who take the time to browse more carefully, you should advertise the git option on the landing page of your repository. On most git servers, the landing page is a Markdown file called README.md. You should edit README.md to include a link to <shortname>.git (which you uploaded in the previous step), as well as a link to your signify public key.
Since git synchronization of user-contributed repositories is not very common, your audience will need a reminder to activate only one of <shortname>.git or <shortname>.httpup (i.e., save the config file in /etc/ports). Having both files active would entail twice as much work for the server and for ports -u.
By now you should have a basic idea of the additional steps involved in setting up a git repository. After gaining some experience with the git workflow and port maintenance, you can apply to join the contrib team, touting your growing proficiency with the git suite and a willingness to learn more. As always, the man-pages for the various git commands are a good place to start, especially giteveryday(7), git-diff(1), git-commit(1), and git-restore(1). Another great resource is the book Head First Git by Raju Gandhi.