The Package System

Introduction

The package system (pkgutils) is made with simplicity in mind, where all packages are plain tar.gz files (i.e. without any kind of meta data). As a first-time CRUX user, you might have noticed package filenames appearing in the progress report of the setup script during installation. These packages were provided on the ISO, but soon you'll be building packages yourself.

Packages follow the naming convention <name>#<version>-<release>.pkg.tar.gz, where <name> is the name of the program, <version> is the version number of the program, and <release> is the version number of the package.
The pkg.tar.gz extension is used (instead of just tar.gz) to indicate that this is not just any tar.gz file, but a tar.gz that is meant to be installed using pkgadd. This helps distinguish packages from other tar.gz files. Note that pkgmk nowadays supports additional compression schemes like bzip2, lzip, xz, and zstd (with the extensions tar.bz2, tar.lz, tar.xz, tar.zst respectively).

pkgadd(8), pkgrm(8), pkginfo(8), and pkgmk(8) are the package management utilities. With these utilities you can install, uninstall, inspect, make packages, or query the package database.

When a package is installed using pkgadd a new record is added to the package database (stored in /var/lib/pkg/db). The basic package system does not have any kind of dependency checking, thus it will not warn you if you try to build a package that requires libraries or headers from other packages. Your mistake will only be revealed when the pkgmk build function exits with errors. The included prt-get tool, however, can be told to resolve dependencies, if called with prt-get depinst rather than simply prt-get install.

The following sections will in describe in short how to use the package utilities. Additional information about these utilities can be found on their respective man pages.

Using the Package System

Installing a Package

Installing a package is done by using pkgadd. This utility requires at least one argument, the package you want to install. Example:

 $ pkgadd bash#5.2.37-1.pkg.tar.gz

When installing a package the package manager will ensure that no existing files are overwritten. If conflicts are found, an error message will be printed and pkgadd will abort without installing the package. The error message will contain the names of the conflicting files. Example:

 $ pkgadd bash#5.2.37-1.pkg.tar.gz
 bin/sh
 usr/man/man1/sh.1.gz
 pkgadd error: listed file(s) already installed (use -f to ignore and overwrite)

To force the installation when conflicting files are reported, you can use the option -f (or --force). Example:

 $ pkgadd -f bash#5.2.37-1.pkg.tar.gz

The package system allows a file to be owned by exactly one package. When forcing an installation the ownership of the conflicting files will be transferred to the package that is currently being installed.

Warning

As stated earlier, the package file itself does not contain any meta data. Instead, the package manager uses the package filename to determine the package name and version. Thus, when installing a package file named bash#5.2.37-1.pkg.tar.gz, the package manager will interpret this as a package named bash at version 5.2.37-1. If pkgadd is unable to interpret the filename (e.g. # is missing or the filename does not end with .pkg.tar.gz) an error message will be printed and pkgadd will abort without installing the package.

Upgrading a Package

Upgrading a package is done using pkgadd with the -u option. Example:

 $ pkgadd -u bash#5.2.37-1.pkg.tar.gz

This will replace the previously installed bash package with the new one. If you have not previously installed bash, pkgadd will print an error message. The package system does not care about the version number of the package in that you can “upgrade” version 5.2.37-1 with version 5.2.35-1 (or even with version 5.2.37-1 itself). The installed package will be replaced with the specified package.

Upgrading a package is not simply a wrapper for pkgrm followed by pkgadd, because you usually want to preserve your customizations of the config and log files that are owned by the already-installed package. Therefore, pkgadd -u conducts some upgrade-specific checks of the filesystem and the package database to construct what is called a “keep list” (in addition to the “non-install list” which is initialized for every pkgadd transaction). The construction of both lists is governed by the file /etc/pkgadd.conf.

/etc/pkgadd.conf can contain rules describing how pkgadd should behave when installing or upgrading any package. A rule is built out of three fragments; event, pattern and action. The event names the kind of operation (INSTALL or UPGRADE) to which this rule will be applied. The pattern is a filename pattern expressed as a regular expression and the action applicable to the INSTALL or UPGRADE event is YES or NO. More than one rule of the same event type is allowed, in which case the first rule will have the lowest priority and the last rule will have the highest priority.
Example:

 # 
 # /etc/pkgadd.conf: pkgadd(8) configuration
 #

 UPGRADE         ^etc/.*$                NO
 UPGRADE         ^var/log/.*$            NO
 UPGRADE         ^etc/X11/.*$            YES
 UPGRADE         ^etc/X11/xorg.conf$     NO

 # End of file

The above example will cause pkgadd to never upgrade anything in /etc/ or /var/log/ (subdirectories included), except files in /etc/X11/ (subdirectories included), unless it is the file fn%/etc/X11/xorg.conf. The default rule is to install and upgrade everything, rules in this file are exceptions to that rule.

Note

If pkgadd finds that a specific file should not be upgraded, it will install it under /var/lib/pkg/rejected/. Files in this directory are never added to the package database. The user is then free to examine, use and/or remove that file manually. Another option is to use rejmerge. For each rejected file found in /var/lib/pkg/rejected/, rejmerge will display the difference between the installed version and the rejected version. The user can then choose to keep the installed version, upgrade to the rejected version or perform a merge of the two. Example (using the above /etc/pkgadd.conf):

 $ pkgadd -u ports#1.6-4.pkg.tar.gz
 pkgadd: rejecting etc/ports/compat-32.pub, keeping existing version
 pkgadd: rejecting etc/ports/compat-32.rsync.inactive, keeping existing version
 pkgadd: rejecting etc/ports/contrib.pub, keeping existing version
 pkgadd: rejecting etc/ports/contrib.rsync.inactive, keeping existing version
 pkgadd: rejecting etc/ports/core.pub, keeping existing version
 pkgadd: rejecting etc/ports/core.rsync, keeping existing version
 pkgadd: rejecting etc/ports/opt.pub, keeping existing version
 pkgadd: rejecting etc/ports/opt.rsync, keeping existing version
 pkgadd: rejecting etc/ports/xorg.pub, keeping existing version
 pkgadd: rejecting etc/ports/xorg.rsync, keeping existing version

 $ tree -L 3 /var/lib/pkg/rejected/
 /var/lib/pkg/rejected
   etc
      ports
         compat-32.pub
         compat-32.rsync.inactive
         contrib.pub
         contrib.rsync.inactive
         core.pub
         core.rsync
         drivers
         opt.pub
         opt.rsync
         xorg.pub
         xorg.rsync

If this pkgadd.conf had been in place during an upgrade from CRUX 3.7 to 3.8, and you neglected to check the contents of /var/lib/pkg/rejected/ (either manually or by running rejmerge), then ports -u would synchronize your ports tree with the older branch! Hence you should always pay attention to warnings and errors reported by pkgadd.

Removing a Package

Removing a package is done by using pkgrm. This utility requires one argument, the name of the package you want to remove. Example:

 $ pkgrm bash

Warning

Querying the Package Database

Querying the package database is done using pkginfo. This utility has a few options to answer different queries.

Examples:

 $ pkginfo -i
 acl 2.3.2-1
 alsa-lib 1.2.13-1
 alsa-utils 1.2.13-1
 ...>
 zip 3.0-2
 zlib 1.3.1-1
 zstd 1.5.6-1

 $ pkginfo -l bash
 bin/
 bin/bash
 bin/sh
 etc/
 etc/profile
 usr/
 usr/man/
 usr/man/man1/
 usr/man/man1/bash.1.gz
 usr/man/man1/sh.1.gz

 $ pkginfo -l grep#3.11-1.pkg.tar.gz
 usr/
 usr/bin/
 usr/bin/egrep
 usr/bin/fgrep
 usr/bin/grep
 usr/man/
 usr/man/man1/
 usr/man/man1/egrep.1.gz
 usr/man/man1/fgrep.1.gz
 usr/man/man1/grep.1.gz

 $ pkginfo -o bin/ls
 e2fsprogs  usr/bin/lsattr
 fileutils  bin/ls
 modutils   sbin/lsmod

Creating Packages

Creating a package is done using pkgmk. This utility uses a file called Pkgfile, which contains information about the package (such as name, version, etc) and the commands that should be executed in order to compile the package in question.

To be more specific, the Pkgfile file is actually a bash(1) script, which defines a number of variables (name, version, release and source) and a function (build). Below is an example of what a Pkgfile file might look like. The example shows how to package the grep(1) utility. Some comments are inserted for explanation.

# Specify the name of the package.
name=grep

# Specify the version of the package.
version=3.11

# Specify the package release.
release=1

# The source(s) used to build this package.
source=(ftp://ftp.ibiblio.org/pub/gnu/$name/$name-$version.tar.gz)

# The build() function below will be called by pkgmk when
# the listed source files have been unpacked.
build() {
   # The first thing we do is to cd into the source directory.
   cd $name-$version

   # Run the configure script with desired arguments.
   # In this case we want to put grep under /usr/bin and
   # disable national language support.
   ./configure --prefix=/usr --disable-nls

   # Compile.
   make

   # Install the files, BUT do not install it under /usr, instead
   # we redirect all the files to $PKG/usr by setting the DESTDIR
   # variable. The $PKG variable points to a temporary directory
   # which will later be made into a tar.gz-file. Note that the
   # DESTDIR variable is not used by all Makefiles, some use prefix
   # and others use ROOT, etc. You have to inspect the Makefile in
   # question to find out. Some Makefiles do not support redirection
   # at all. In those cases you will have to create a patch for it.
   make DESTDIR=$PKG install

   # Remove unwanted files, in this case the info-pages.
   rm -rf $PKG/usr/info
}

In reality you do not include all those comments, thus the real Pkgfile for grep(1) looks like this:

# Description: GNU grep, egrep and fgrep
# URL:         http://www.gnu.org/software/grep/grep.html
# Maintainer:  Per Lid�n, per at fukt dot bth dot se

name=grep
version=3.11
release=1
source=(ftp://ftp.ibiblio.org/pub/gnu/$name/$name-$version.tar.gz)

build() {
  cd $name-$version
  ./configure --prefix=/usr --disable-nls
  make
  make DESTDIR=$PKG install
  rm -rf $PKG/usr/info
}

Note

When the build() function has been executed, the $PKG directory will be made into a package named <name>#<version>-<release>.pkg.tar.gz. Before the package creation is completed, pkgmk will check the contents of the package against the .footprint file. If this file does not exist, it will be created and the test will be skipped. The .footprint file will contain a list of all files that should be in the package if the build was successful or a list of all the files that were installed in $PKG (if the .footprint did not already exist). If there is a mismatch the test will fail and an error message will be printed. You should not write the .footprint file by hand. Instead, when a package has been upgraded and you need to update the contents of the .footprint file you simply do pkgmk -uf. This test ensures that a rebuild of the package turned out as expected.

If the package built without errors it's time to install it by using pkgadd and try it out. I highly recommend looking at the Pkgfile in another package(s), since looking at examples is a great way to learn.

Note

Adjusting/Configuring the Package Build Process

Many settings pertaining to the package build process can be adjusted by editing the pkgmk(8) configuration file /etc/pkgmk.conf. Some of these configurable settings include:

• CFLAGS, CXXFLAGS - these settings control optimization and architecture options for package compiles. It is best NOT to change these unless you absolutely know what you're doing!
• PKGMK_SOURCE_MIRRORS - this setting defines locations from which pkgmk will attempt to fetch source archives. If this array only contains mirrors administered by the CRUX team, building a port from your personal collection might result in 404 Not Found errors until all the mirrors are exhausted, and then the source defined in the Pkgfile will be tried.
• PKGMK_SOURCE_DIR - this setting defines where pkgmk will store (if downloading) and use source archives when building
• PKGMK_PACKAGE_DIR - this setting defines where pkgmk will create package files once the build process is complete
• PKGMK_WORK_DIR - this setting defines a work area pkgmk will use to build the package

Here are some examples:

PKGMK_SOURCE_MIRRORS=(http://fileserver.intranet/crux/sources/)

This setting instructs pkgmk to attempt to fetch all source archives from http://fileserver.intranet/crux/sources/ before falling back to the source URL specified in the Pkgfile. Multiple URLS can be separated by spaces.

PKGMK_SOURCE_DIR="/usr/ports/srcfiles"

This example instructs pkgmk to store and find source archives in /usr/ports/srcfiles. An example benefit of this setup would be the ability to store /usr/ports/srcfiles on an NFS server on your local network for use by multiple crux installations. (PKGMK_PACKAGE_DIR can be set and used the same way. But if NFS is too challenging to set up on your local network and you find it easier to configure an http server instead, pkg-get from the opt collection gives you an alternative mechanism for sharing built packages.)

PKGMK_WORK_DIR="/usr/ports/work/$name"

This example instructs pkgmk to use /usr/ports/work/$name as a work area for building the specified package. Building the grep package would result in the work area being /usr/ports/work/grep. An alternative would be to use a tmpfs as your work directory.

There are a few more settings which can be found in the pkgmk.conf man page.

Package Guidelines

General

• The name of a package should always be lowercase (e.g. name=eterm and not name=Eterm). In case the package is added to the CRUX ports system the exact same name should be use for the name of the directory in the ports structure, i.e. /usr/ports/???/eterm.
• Do not combine several separately distributed programs/libraries into one package. Make several packages instead.
• The build function should never touch anything outside the work directory, and ideally should not rely on having an internet connection (say, to download sources not listed in the source array). While pkgmk does not currently recognize git urls in the source array, efforts are underway to add this feature, thereby removing the temptation to write a build function that reaches out to the internet for the latest git source tree.
  Apart from ensuring a consistent mechanism to verify the integrity of sources (signify), the policy of separating 'download' from 'build' allows a user with intermittent internet access to run pkgmk -do in the directory of every outdated package, and then go offline to finish the sysup operation. Language-specific toolchains, like those embraced by rust and python (cargo and pip, respectively), are making this policy more difficult to enforce. You are free to forego CRUX pkgutils and let the ecosystems of the N different languages manage their respective software in separate subdirectories (like python's ~/.local/share/virtualenv or rust's ~/.cargo), at the expense of having to learn N+1 administration suites rather than just the 1 suite of CRUX pkgutils. Every language-specific project that appears in the official repositories represents a hard-won effort by the CRUX team to sustain the historic method of software deployment, in the face of software development trends all heading away from this model.

Directories

• In general packages should install files in these directories. Exceptions are of course allowed if there is a good reason. But try to follow the following directory structure as close as possible.

• /opt directory is reserved for manually compiled/installed applications. Packages should never place anything there.
• /usr/libexec/ is not used in CRUX, thus packages should never install anything there. Use /usr/lib/<prog>/ instead.

Remove Junk Files

• Packages should not contain “junk files”. This includes info pages and other online documentation, man pages excluded (e.g. usr/doc/*, README, *.info, *.html, etc).
• Files related to NLS (national language support). Ports built using GNU autoconf can be told not to install these files by passing the option --disable-nls. Other compiler toolchains often provide no such option, forcing you to clean up the package footprint by hand.
• Useless or obsolete binaries (e.g. /usr/games/banner and /sbin/mkfs.minix).

Pkgfile Variables

• Do not add new variables to the Pkgfile. Only in very few cases does this actually improve the readability or the quality of the package. Further, the only variables that are guranteed to work with future versions of pkgmk are name, version, release, and source. Other names could be in conflict with internal variables in pkgmk.
• Use the $name and $version variables to make the package easier to update/maintain. For example, source=(http://xyz.org/$name-$version.tar.gz) is better than source=(http://xyz.org/myprog-1.0.3.tar.gz) since the URL will automatically updated when you modify the $version variable.
• Remember that source is an array, i.e. always do source=(...) and not source=...
• Another array that pkgmk will make use of, if defined, is renames (introduced in CRUX 3.7). This array lets you save the downloaded source tarballs with a more descriptive filename, to avoid filename collisions when using a shared directory for downloads. See the Pkgfile man page for more details.

Pkgfile Header

The commented lines at the beginning of a Pkgfile do not affect the operation of pkgmk, but prt-get and other CRUX tools will make use of the information provided there. Four headers of utmost importance are:

Depends on can be omitted if there are no dependencies. To keep this line short, runtime dependencies in the 'core' collection are omitted, unless they provide a library that is dynamically linked to the built program. See the Pkgfile man page for more details.

Two other headers you will often encounter when inspecting Pkgfiles are Optional and Nice to have. None of the official CRUX tools will be affected by the contents of these lines, but maintainers find it helpful to store in Optional the list of “soft” dependencies (libraries that will be linked to the built program, if present during compilation). This header then serves as a reminder of tests that might need to be included in the build() function in order to set the appropriate configure flags. As an end user of official ports, you might find in these headers some clues about:

• the additional functionality that can be unlocked, when dependencies beyond the bare minimum are installed before the desired port.
• the reason that pkgmk reports a footprint mismatch.
• the breakage that might happen if you try to install on your CRUX system a package built on someone else's CRUX system.

Example header

# Description: CUPS - Common UNIX Printing System
# URL: https://openprinting.github.io/cups/
# Maintainer: Juergen Daubert, jue at crux dot nu
# Depends on: acl libusb zlib linux-pam
# Optional: dbus gnutls

Building ports as unprivileged user

Building packages requires root privileges in order to create files with the correct owner and group. This is a security concern because a malicious or badly designed port can run arbitrary commands when its Pkgfile is sourced by the shell. The fakeroot command provides a way to build ports as normal user. Particularly when you build packages from user contributed repositories you are advised to use fakeroot:

$ fakeroot pkgmk -d

Ports Collections

Borrowing terminology from the BSD world, CRUX refers to the directory containing Pkgfile, .footprint, and .signature as a port. The parent directory of a port is known as a collection or repository. Distributing ports from one user to another, or from the CRUX team to users, requires a server listening on an HTTP, git, or rsync port, and a client program capable of downloading ports from that repository. The administrator of a CRUX system runs one of the scripts under /etc/ports/drivers to copy the remote files to a local directory (usually under /usr/ports/). Currently there are scripts for the protocols rsync(1), httpup(1), git(1). Although you can run the driver script by hand, the usual approach is to automate this task with a wrapper script (historically, ports -u). The wrapper script will choose the appropriate driver based on the extension of the file defining the remote repository (e.g., /etc/ports/core.rsync will be processed by the rsync driver). Feel free to define your own driver if you need support for a different protocol (e.g., git through SSH). But avoid naming your driver inactive, because that extension is reserved for disabling a repository.

There are three different levels of port collections:

The official collections 'core', 'opt', 'xorg' and 'compat-32'

core, opt, xorg and compat-32 are the four primary collections of CRUX. They're maintained by the CRUX development team which ensures that they're consistent and working well together. The first three are enabled by default. The compat-32 collection is disabled by default and contains 32-bit compatibility ports.

The user contributed collection 'contrib'

The contrib collection is a collection which is provided by experienced port maintainers: some are part of the CRUX development team, while others are regular users. Its goal is to reduce the number of duplicate ports provided in the individual collections. If you're a seasoned port maintainer, you might even want to join the contrib team.

As those ports are not provided officially by the CRUX development team, this collection is disabled by default. You can enable it by renaming the repository sync file:

$ cd /etc/ports
$ mv contrib.rsync.inactive contrib.rsync

The individual collections from CRUX users

Using HttpUp or git, every user can publish his or her own ports easily; the only requirement for that is some webspace to upload the ports. Maintaining an HttpUp repository of ports, which you've tested and gotten successfully running, is a simple way to contribute back to the CRUX community.

Package Management Frontend: prt-get

In its current form pkgutils does not have a concept of dependency handling. To address this a frontend utility called prt-get was created. It supports dependency handling (with the caveat mentioned below) as well as some overlap with pkgutils features and has been an official part of CRUX for some time.

Functionality

Some examples of prt-get's functionality and use are as follows:

Listing installed ports:

$ prt-get listinst
acl
alsa-lib
alsa-utils
[...]

$ prt-get listinst -v
acl 2.3.1-1
alsa-lib 1.2.7.2-1
alsa-utils 1.2.7-1
[...]

Querying information about a port:

$ prt-get info acl
Name:         acl
Path:         /usr/ports/core
Version:      2.3.1
Release:      1
Description:  Access Control Lists library
URL:          http://savannah.nongnu.org/projects/acl
Maintainer:   CRUX System Team, core-ports at crux dot nu
Dependencies: attr

Searching for ports by name:

$ prt-get search glibc
glibc
glibc-32

$ prt-get search --regex '(ba|z)sh$'
bash
zsh

Searching for ports by installed file:

$ prt-get fsearch gconv
Found in /usr/ports/core/glibc:
  /usr/lib/gconv/

Found in /usr/ports/core/glibc-32:
  /usr/lib32/gconv/

Searching for ports by words in their descriptions:

$ prt-get dsearch shell
dash
dialog
dsh
[...]
zsh

Viewing dependency lists:

$ prt-get depends bash
-- dependencies ([i] = installed)
[i] ncurses
[i] readline
[i] bash

$ prt-get quickdep bash
ncurses readline bash

$ prt-get deptree bash
-- dependencies ([i] = installed, '-->' = seen before)
[i] bash
[i]   readline
[i]     ncurses

Installing ports:

$ prt-get install xterm

Note

$ prt-get depinst xterm

Viewing and updating outdated ports (generally after 'ports -u'):

Listing installed ports which are out of date:

$prt-get diff
Differences between installed packages and ports tree:

Port                Installed           Available in the ports tree

imagemagick         7.1.1-41            7.1.1-44

Updating an individual port:

$ prt-get update imagemagick

Updating all installed ports:

$ prt-get sysup

Note

$ prt-get isinst $(prt-get quickdep $(prt-get quickdiff)) | awk '/not installed/ {print $2}'

Configuring prt-get

/etc/prt-get.conf

prt-get's main configuration file, '/etc/prt-get.conf', contains options that can be used to change prt-get's behavior. Notably in this file the following options can be configured:

prtdir - This option can occur multiple times and specifies a collection that prt-get should search for ports. By default the 'core', 'opt', and 'xorg' collections are enabled. The 'compat-32' and 'contrib' collections are disabled by default, but by removing the '.inactive' suffix you can enable them.

logfile - This option configures a file for prt-get to log its operation if desired.

runscripts - This option configures prt-get to run pre-/post-install scripts if they exist in ports being installed or updated. It is recommended that this be enabled as in many cases if a pre- or post-install script exists in a port, it is required to be run for proper operation.

makecommand - Setting this option as fakeroot pkgmk would allow a non-root user running prt-get to create packages with no footprint mismatches (wrong owner/group).

addcommand - Setting this option as doas /usr/bin/pkgadd would allow a non-root user to escalate privileges for the purpose of installing the built package. You could also point to a pkgadd under a different path, for example a version built with CXXFLAGS += -DNDEBUG if you want verbose debug output.

You might also consider writing a sudo or doas configuration that grants permission to run prt-get itself, and then as a non-root user you would always run sudo prt-get rather than prt-get directly. Read the man pages for prt-get.conf and sudo or doas to learn how these pieces fit together, and choose whichever option you like best. To give extra scrutiny to ports you find outside the official repositories, it helps to restrict yourself to the low-level commands pkgmk and pkgadd rather than the wrapper program prt-get. This practice forces you to be in the directory of the unofficial port, where the Pkgfile source array and build function are more likely to receive your careful attention.

/etc/prt-get.aliases

prt-get has a concept of aliases which can be used in a fashion similar to the concept of 'provides' in some other Linux distributions. This file is /etc/prt-get.aliases and contains lines in the following format:

postfix: sendmail
exim: sendmail
qmail: sendmail
masqmail: sendmail

The above set of aliases indicates that postfix, exim, qmail, and masqmail are all considered sufficient to satisfy a dependency on 'sendmail' in a port.

Sometimes the port maintainer will list among the required dependencies a port that bundles together a library and some demo utilities. If you only need the library when compiling the dependent port, you can use prt-get.aliases to avoid an attempted installation of the port with demo utilities. For example, on a system where aom does not build, but a similar port libaom with -D EXAMPLES=OFF does build, you would want prt-get depinst firefox to be satisfied with libaom and not try building aom. This can be accomplished with the following line in prt-get.aliases:

libaom: aom

Note

This is NOT an exhaustive list of all of prt-get's commands, features, and configuration options, merely a starting point. More information can be found in the man-page prt-get(8) and the quick start documentation.