CRUX : Home

Home :: Documentation :: Download :: Development :: Community :: Wiki :: Ports :: Bugs :: Links :: About

Back to wiki start page

Categories: Development

Attributes support in pkgutils

Please note that right this is a personal draft to spread the idea of attributes. This is not even a proposal, just a couple of notes.

1. Changes

This introduces meta-data; no big thing, however we want to try make it as easy as possible to manipulate packages with tar, so all we do is install meta data to a magic directory.

1.1. pkgmk

pkgmk stores meta data in a magic subdirectory, e.g. /CRUX_META within the tarball. This includes the attributes and *-install/*-remove scripts. A possible layout could look like this:

/CRUX_META/
    attributes
    pre-install
    post-install
    pre-remove
    post-remove

The information contained in attributes is potentially more than the average user needs, but unwanted meta data will be dropped before installing the package.

Alternatives could involve something like a nested tar/tar or ar/tar

/
    package.tar.gz
    attributes
    pre-install
    post-install
    pre-remove
    post-remove

where package.tar.gz is the package itself. This would niceley separate meta-data and package, or rather treat both meta-data and package as equal first class objects.

1.2. pkgadd

pkgadd checks for the existence of a /CRUX_META directory, and extracts the required meta data from it (name, version, release). It also stores additional meta data as configured, allowing a system designer [a person maintaining a particular flavour of CRUX, i.e. uCRUX, regular CRUX, server CRUX etc] to choose a set of attributes to be stored there. Obviously, /CRUX_META is not installed into the file system.

Two things should be consided:

  1. should it fail if there's no /CRUX_META package, or derive the most important information from the package name
  2. those packages would give a conflict starting from the second package when used with a pre-attribute version of pkgutils. Not really an issue AFAICS, but worth remembering. Using /CRUX_META/<pkg> would solve that, but seems overkill.

In addition, users can inject attributes at pkgadd time, allowing to store additional information per package, like installation date, file md5sums/permissions etc- (might be interesting for servers).

attributes, install/remove scripts and rejects are then saved to a per port directory in the package db directory.

1.3. pkgrm

Since all information of a particular port is in a subdirectory, reject can now be cleanly removed by removing that directory.

2. Package database

2.1. Filesystem structure

This has a couple of consequences:

2.2. Example attributes files

This file is supposed to be easy to parse by tools and scripts. It has a simple key: value syntax, and 1:n relations are represented with multiple entries.

User injected attributes are marked with an @-sign to avoid overwritting attributes provided by the attributes file.

name: kdelibs
version: 3.5.2
release: 3
description: Base set of libraries needed by KDE programs
maintainer: sten, nick dot steeves at shaw dot ca
packager: sten, nick dot steeves at shaw dot ca
url: http://www.kde.org
depends: sudo
depends: perl
depends: db
depends: libidn
depends: imlib
depends: hicolor-icon-theme
depends: freeglut
depends: libart_lgpl
depends: qt3
depends: audiofile
depends: libxml2
depends: libxslt
depends: libpcre
depends: openssl
depends: aspell
depends: gnupg
depends: gpgme
depends: gamin
depends: cups
depends: openldap
depends: arts
build-date: Wed Apr 19 13:25:13 CEST 2006
build-user: jw
build-path: /usr/ports/opt/kdelibs

file: usr/
file: usr/bin/
file: usr/bin/artsmessage
file: usr/bin/checkXML
file: usr/bin/cupsdconf


# optional: pkgadd injected, marked with @
@install-data: Wed Apr 19 13:25:13 CEST 2006
@filemode: drwxr-xr-x usr/
...
@fileowner: root/root user/
...

2.3. Example pkgfile

To make it easier to pass attributes from the shell script to the attributes file, we change from the "meta data in comment" approach to "everything is a shell variable". Slightly more strict when it comes to whitespaces, but that's not a bad thing per se.

Note that this allows users to execute shell commands within those variables, which is probably a bad idea. We should at least define some rules, and maybe variables the user is support to use there ($KERNEL_VERSION etc.)

name=kdelibs
version=3.5.1
release=2

description="Base set of libraries needed by KDE programs"
url="http://www.kde.org"
maintainer="sten, nick dot steeves at shaw dot ca"
packager="sten, nick dot steeves at shaw dot ca"
dependencies=(sudo perl db libidn imlib hicolor-icon-theme freeglut \
              libart_lgpl qt3 audiofile libxml2 libxslt libpcre openssl \
              aspell gnupg gpgme gamin cups openldap arts)


source=(...)

2.4. Further possibilities:

2.4.1. Mass tagging

If such an API is available, tools like 'prt-tag' would be might simple, allowing to add tags to all ports, like defining a snapshot of installed packages, and rolling back after i.e. a depinst gnome

3. Prototype aka "the above was just the theory"

I've started to write some prototyping code to display the ideas here; it's available from http://crux.nu/~jw/git/pkgutils-attr.git/ via git. More information on this soon :-).

3.1. Testing it

Important:

  1. I suggest to run all this as non-root user
  2. pkginfo, pkgadd, pkgrm and pkgattr look for configurations files in the current working directory
  3. it will install into a special root directory called 'pkg_root' inside the current working directory, so it won't have any influence on your current system.

3.1.1. Preparation

First, obtaining it

$ git clone http://crux.nu/~jw/git/pkgutils-attr.git/ pkgutils-attr
...
$ cd pkgutils-attr

Then, copy over some ports to play with; I've only checked some of them, so some might fail (autoconf, automake, filesystem and prt-get work :-)).

$ cp -r /usr/ports/core .
$ cd core
$ for d in *; do ../conv_pkgfile $d/Pkgfile; done
$ cd ..

This will create a Pkgfile.attr for port

3.1.2. Building, installing and basic info queries

Now, you can build a couple of ports (pkgmk will download automatically; there's no pkgmk.conf)

$ cd core/prt-get
$ ../../pkgmk
$ cd -

... and install them

$ ./pkgadd core/prt-get/prt-get#5.12-1.pkg.tar.gz

Currently, you can't update a package, so if you want to reinstall it, ./pkgrm it first. You can also use ./pkginfo (-o|-i|-l) as you're used to; ./pkginfo -i will require a -v argument if you want to see the version.

3.1.3. Attributes, lots of attributes

Now for the fun part:

Here's the pkgadd.conf used ($ROOT sed hack to go away)

install-date() {
    LC_ALL=C date
}

file-md5sum() {
    for f in $files; do
	if [ -f $ROOT/$f ]; then
        	md5sum $ROOT/$f|sed -e "s|$ROOT||g"
        fi
    done
}

PKGADD_ATTR_INJECT="install-date file-md5sum"

The last line tells pkgadd to inject those two attributes when installing the package

After a package is installed, you can use the ./pkgattr tools to obtain information; calling it without additional arguments besides the package name will print a list of available attributes:

$ ./pkgattr prt-get 
name
version
release
file
@install-date
@file-md5sum

You can then specify one or more arguments to obtain those attributes:

$ ./pkgattr prt-get name version
name:prt-get
version:5.12

This works the same for 1:n relationships

$ ./pkgattr prt-get file
file:etc/
file:etc/prt-get.conf
file:usr/
file:usr/bin/
file:usr/bin/prt-get
file:usr/bin/prt-cache
file:usr/man/
file:usr/man/man5/
file:usr/man/man5/prt-get.conf.5
file:usr/man/man8/
file:usr/man/man8/prt-get.8
file:usr/man/man8/prt-cache.8
file:var/
file:var/lib/
file:var/lib/pkg/
file:var/lib/pkg/prt-get.aliases

Of course, we can also query the attributes we configured to be injected:

$ ./pkgattr prt-get @install-date @file-md5sum
@install-date:Mon Apr 24 12:04:54 CEST 2006
@file-md5sum:cac4a5f0545033bee15432956112cdfd  /etc/prt-get.conf
@file-md5sum:ff4e3988ac028d759e3c15488b274d8b  /usr/bin/prt-get
@file-md5sum:ff4e3988ac028d759e3c15488b274d8b  /usr/bin/prt-cache
@file-md5sum:1baebfbcda349230178a8e7a43c57788  /usr/man/man5/prt-get.conf.5
@file-md5sum:e4388be780f5ed08e02efd5538ecdb78  /usr/man/man8/prt-get.8
@file-md5sum:9e72309c58920d3132425d54498658b8  /usr/man/man8/prt-cache.8
@file-md5sum:9fefdc42a8a106f7d2c2036ba3b2f9fb  /var/lib/pkg/prt-get.aliases

3.1.4. Configuration of attributes (i.e. which attributes go into the db)

To add more meta data to the package database, there's a configuration option in pkgadd

PKGADD_DB_ATTRIBUTES=(description maintainer url)

name, version, release and files are automatically added. This option is in pkgadd since it's not meant to be adjusted on a running system since otherwise not all packages will have all the information. Rather a system designed (i.e. for server crux, embedded crux etc.) would do that, and build a respective pkgutils variant.

Going back to the previous call:

$ ./pkgattr prt-get 
name
version
release
file
@install-date
@file-md5sum

Edit pkgadd and enable the PKGADD_DB_ATTRIBUTES=(description maintainer url) line (while commenting the PKGADD_DB_ATTRIBUTES=() line of course), then remove and reinstall the package (since there's no pkgadd update yet...).

$ ./pkgrm prt-get
...
$ ./pkgadd core/prt-get/prt-get#5.12-1.pkg.tar.gz

Then, run the above command again:

$ ./pkgattr prt-get 
name
version
release
description
maintainer
url
file
@install-date
@file-md5sum
$ ./pkgattr prt-get description
description:A utility to simplify ports searching/installing

As you can see, the wanted information is now available in the package database