Categories: Ports

How to set up an httpup repository

Author

Description

This article describes how to publish ports using httpup

Instructions

Prerequisites

In order to publish ports, you need some webspace; typically, one port is a less than 100kB, so space requirements are really modest and it shouldn't be a problem to find a free provider for it.

Once you have that, make sure you have httpup installed; with that, you're ready to go.

Preparing the repository on the local disk

The structure on your local disk can be chosen at will, therefore the following is just to give you an idea how it could be done.

First, you need to choose a location to keep your ports; I'm using ~/build here (note that I'm doing this with a non-root user). Copy the ports you want to upload to that directory.

In a next step, we create a REPO file which httpup will use to syncronize the ports later on. Do this with the httpup-repgen command:

$ cd ~/build
$ httpup-repgen

Now, check the REPO file using your favourite text edit; chances are it includes tar.gz files which shouldn't be there. In order to exclude them, create a file called .httpup-repgen-ignore in your home directory; mine looks like this:

\.svn$
\.tar\.gz$
\.tgz$
\.partial$
\.tar\.bz2$
\.zip$
(^|/)CVS(/|$)
.*~$
sync.sh
\.gif$
\.gem$
index.html

As you see, it filters a couple of other things beyond tar.gz's.

After you created that file, rerun httpup-repgen; repeat that until the REPO file really just contains the files you want in there. Note that Pkgfile, .footprint and .md5sum must be there for every port.

If you want to provide a nice index listing, have a look at the portspage tool from the prt-utils package. I added an exclusion for index.html to the ignore expressions already.

Uploading the collection

The uploading step depends a lot on how your webspace can be accessed; probably, that's either ftp, or if you're lucky, ssh/sftp. If it's ftp, have a look at the weex tool. If you have ssh access, I'd recommend you to use rsync. Again, make sure you only upload the files that are needed.

Creating an httpup file

Next, to allow the ports utility to easily sync your ports, create a httpup collection definition; it looks like this:

#
# /etc/ports/<shortname>.httpup: <Your name's>'s port collection
#

ROOT_DIR=/usr/ports/<shortname>
URL=http://<url>

# End of file

replace <shortname> with a nickname for your collection, and <Your name> with your full name. My collection definition looks like this:

#
# /etc/ports/jw.httpup: Johannes Winkelmann's port collection
#

ROOT_DIR=/usr/ports/jw
URL=http://jw.tks6.net/files/crux/ports/

# End of file

Verify its functionality

It's recommended to upload this file as well, to make it easy for others to subscribe to your ports collection. First however, test its functionality on your machine: as root, copy the file to /etc/ports, and run ports -u:

# cp <shortname>.httpup /etc/ports
# ports -u <shortname>
...
# cd /usr/ports/<shortname>
# ls
...

Closely watch the output of ports -u to spot potential errors. If all that worked fine and you're happy with the quality of your ports, go to http://crux.nu/portdb and register your collection.

A sync and upload helper script for ssh-enabled webhosts

Here's what I use to sync with my webserver; I'm in the lucky position to have an ssh account there, so I can use rsync through ssh.

#!/bin/bash
f=`find . -mindepth 3`
d=`find . -mindepth 2 -type d`

if [ -n "$d" ] || [ -n "$f" ]; then
    echo "invalid files:"
    echo "$f"
    echo "dirs:"
    echo "$d"
    exit -1
fi

httpup-repgen . 
portspage --title="Johannes Winkelmann's CRUX Ports" . > index.html

export RSYNC_RSH=ssh
rsync -avz --delete --delete-excluded \
      --exclude "*tar.gz"   \
      --exclude "*tar.bz2"  \
      --exclude "*.gem"  \
      --exclude "*tgz"  \
      --exclude "*.zip"     \
      --exclude "sync.sh"   \
      --exclude "*CVS/"     \
        . <user>@<host>:<directory>

This script includes an additional protection to cancel when there are unwanted files, like subdirectories within the ports; the common case when this happened was when I tried to upload the repository which building a port in a different shell.

I place this within my collection directory, and run it from there like this ./sync.sh

Note: ftp script wanted :-)

#!/bin/sh -x
#
# need ncftpput ("prt-get install ncftp")

REPOLOCAL="/usr/ports/<NAME>"

USERNAME="username"
PASSWORD="password"
FTPURL="ftp://"
REMOTEDIR="/crux/ports"

cd "$REPOLOCAL"

httpup-repgen .
portspage --title="My Crux Ports" . > index.html

ncftpput -R -u $USERNAME -p $PASSWORD $FTPURL $REMOTEDIR $PWD/*

CRUX : Home