mirroring-ports man page on OpenBSD

Man page or keyword search:  
man Server   11362 pages
apropos Keyword Search (all sections)
Output format
OpenBSD logo
[printable version]

MIRRORING-PORTS(7)	   OpenBSD Reference Manual	    MIRRORING-PORTS(7)

     mirroring-ports - how to build a mirror for ports distfiles

     The OpenBSD Ports Collection offers some powerful tools to mirror
     software sources.	The ports infrastructure provides a mirror-maker
     target that can be used to build a Makefile to facilitate mirroring
     distfiles.	 This target builds ${DISTDIR}/Makefile, which can be used on
     almost any Unix-like machine to mirror OpenBSD distfiles.	Optionally,
     port dependencies may also be recorded in the Makefile, which greatly
     increases the execution time of mirror-maker.  The variable
     RECURSIVE_FETCH_LIST can be set to `Yes' to request that this is done.

     A sample Makefile entry is formatted like this:

     all:: audio/tracker/tracker-5.3
     .PHONY: audio/tracker/tracker-5.3
     audio/tracker/tracker-5.3:	 tracker-5.3.tgz

     tracker-5.3.tgz: $F
	     @MAINTAINER="espie@openbsd.org" \
	      SITES="ftp://ftp.uni-trier.de/pub/unix/audio/tracker/ " \
	      CIPHER="sha1" CKSUM="b0973d6a9c363caebd3a71547412f42b0681f323" \
	      exec ${FETCH} "$@"

     This Makefile is usually invoked by the user from the directory where
     they wish to mirror distfiles, with variables FETCH and F set on the
     command line, e.g.,

	   cd mirror && make -k -j 5 -f path_to_makefile FETCH=fetch_script

     Targets are set up so that each port is referenced by its full name, and
     retrieves all its distfiles.  The all target can be used to retrieve all
     distfiles.	 Targets ftp and cdrom can be used to retrieve all distfiles
     necessary to build ftp and CD-ROM packages, respectively.	Default is ftp
     plus cdrom.  Actual fetching is usually invoked with a parallel-make
     option, so that several retrievals through ftp can proceed

     The F variable can be set to a dummy name, or a recent filename, to force
     re-fetching of anything which is older than the filename.	Its intended
     use is to force re-fetching existing files, or to checksum all files.

     The ${FETCH} script should be supplied by the user, and will download and
     verify the archive file.  It must obey the following variables:

     MAINTAINER	  Port maintainer, used to report errors,

     ERROR	  Some ports problems can be detected while building the
		  Makefile, in which case this variable will be set to a
		  proper error message.

     DIST_SUBDIR  See ports(7) for more details.  The ${FETCH} script is
		  responsible for creating this subdirectory and cd'ing to it
		  before performing the actual fetch.

     SITES	  A list of sites to try for fetching the distribution file.

     CIPHER	  The checksumming utility to use for verifying the
		  distribution file.  It will normally be set to sha1(1)
		  unless you tinker with PREFERRED_CIPHER while building the
		  mirroring Makefile.

     CKSUM	  The corresponding checksum.  If neither CIPHER nor CKSUM nor
		  ERROR are set, the distribution file needs not be checked.

     A standard fetch strategy is to try all sites in order: whenever the
     distribution file is found, download it; verify the checksum; erase the
     file and try the next site if it doesn't match.

     Mirroring sites should update their master Makefile fairly often.
     Activities a proper mirror should offer (in order of decreasing

     Mirror new files
		Use a proper fetch script to download missing files,

     Run ${PORTSDIR}/infrastructure/fetch/link-checksums
		This script creates permanent hardlinks that preserve
		distfiles against checksum changes.

     Verify all checksums
		All checksums should be verified from time to time, and
		maintainers notified of persistent discrepancies,

     Check mastersites liveliness
		Use a tool such as `mirror' to check that the master sites
		haven't fallen off the Earth.  Even though the first site in
		the site list is the most important site, good mirrors will
		scan all sites and report all problems,

     Remove old files
		To gain room this, the mirror should maintain a list of
		`active' files (easy enough, just provide a fetch script that
		just lists the file names), and remove files that are no
		longer active.	Since OpenBSD releases happen every six
		months, this delay should be longer than that.
     Sample scripts are provided in the ${PORTSDIR}/infrastructure/fetch

		Main mirroring Makefile
		Sample script usable to retrieve distfiles.
		Sample script to check all distfiles checksums.
		Populating checksums subdirectories with links, to guard
		against shifting checksums.


     This infrastructure was introduced for OpenBSD 2.7 by Marc Espie, with
     feedback from Bob Beck, Todd Fries, Camiel Dobbelaar, and a few other

     Changing checksums is a recurring problem that is outside the direct
     control of the OpenBSD Project.  Some software distributors change
     distribution files without warning, without changing the file name
     proper.  Once the problem has been identified, the port maintainer should
     usually contact the software author to fix the problem, or, if the
     software author is unresponsive, the maintainer should use DIST_SUBDIR to
     provide some state to guard against shifting checksums.

     However, a more robust approach is also needed, so that ports users can
     depend on distfiles mirrors to carry what they need irrespective of those
     synchronization issues.  The link-checksums script creates another access
     to the distfiles, indexed through the actual checksums that the files
     should match.  Provided mirroring is run sufficiently often, together
     with link-checksums, two versions of the same distfile with respective
     checksums cksum1 and cksum2 will be available under the names
     ${DISTFILES}/sha1/cksum1/distfile and ${DISTFILES}/sha1/cksum2/distfile.

     Starting revision 1.281, if REFETCH is set to true, bsd.port.mk will try
     to retrieve files under that naming scheme as a last resort.

OpenBSD 4.9			March 26, 2010			   OpenBSD 4.9

List of man pages available for OpenBSD

Copyright (c) for man pages and the logo by the respective OS vendor.

For those who want to learn more, the polarhome community provides shell access and support.

[legal] [privacy] [GNU] [policy] [cookies] [netiquette] [sponsors] [FAQ]
Polarhome, production since 1999.
Member of Polarhome portal.
Based on Fawad Halim's script.
Vote for polarhome
Free Shell Accounts :: the biggest list on the net