Context Navigation

OVERVIEW @ 1393bb5

gcc484ntopperl-5.22

Last change on this file since 1393bb5 was c5c522c, checked in by Edwin Eefting <edwin@datux.nl>, 8 years ago
initial commit, transferred from cleaned syn3 svn tree
Property mode set to `100644`
File size: 26.7 KB

Line
1	##############################################################################
2	# Document: OVERVIEW
3	# Purpose : Provide an overview of the Slackware package system incorporating
4	# the use of 'slacktrack'
5	# Please note that this is not a guide to the use of slacktrack;
6	# the man page and SlackBuild scripts in the 'examples' directory
7	# aim to fulfill that requirement.
8	# Date....: 15-May-2009
9	# Author..: Stuart Winter <mozes@slackware.com>
10	# Homepage: http://www.slackware.com/~mozes
11	##############################################################################
12
13
14	CONTENTS:
15	---------
16	1.0 Preamble
17	2.0 Slackware packages
18	2.0.1 Format of official Slackware packages
19	2.0.1.1 Package names
20	2.0.1.2 Version numbers
21	2.0.1.3 Architectures
22	2.0.1.4 Build numbers
23	2.1 An overview of the creation of official Slackware packages
24	2.1.1 'Clean' and 'dirty' builds
25	2.1.1.1 Clean builds
26	2.1.1.2 Dirty builds
27	2.1.1 SlackBuild and .build scripts
28	2.1.1.2 SlackBuild scripts
29	2.1.1.3 .build scripts
30	2.2 slacktrack in the scheme of things
31	2.2.1 Using slacktrack with official .build scripts
32	2.2.2 Writing your own .build scripts for slacktrack
33	2.2.2.1 Making use of slacktrack's environment variables
34	3.0 slacktrack in operation
35	3.1 How slacktrack finds which files have been installed
36	3.1.1 installwatch's author
37	4.0 Example scripts
38	4.0.1 non-slackware example build scripts
39	4.0.2 slackware example wrapper build scripts
40	5.0 Known problems/features of slacktrack
41	6.0 New features
42	7.0 Licence
43
44
45
46	1.0 Preamble
47	---------
48
49	I used to work for a company that provided web hosting and shell accounts.
50	When I started there were a number of shell servers all running various
51	releases of RedHat Linux, in various states of repair. I managed to convince
52	the management to let me try Slackware on there instead because I have a much
53	stronger understanding of how to maintain and build server using the
54	Slackware distribution. This trial went well and within a few months all
55	servers were converted to Slackware version 8.1.
56
57	In order to ease the speed of installation (and to prevent against
58	forgetting to install or configure something critical), I wrote a
59	set of post installation scripts that upgraded Slackware packages,
60	configured numerous config files and copied pre-compiled software
61	(such as courier IMAP) onto the new system.
62	For other software that I could not simply copy, I'd compile it on the
63	new server.
64
65	However, it soon became clear that due to security updates and so on,
66	it became incredibly time consuming (not to mention seriously boring)
67	to compile PHP, Apache and so on on every server.
68
69	At this point, I began to investigate how to create packages for Slackware.
70
71
72	2.0 Slackware packages
73	------------------
74
75	The Slackware Linux distribution consists of a variety of 'packages'
76	that make up the distribution.
77
78	Traditionally, all packages are in '.tgz' format (a gzipped tar archive).
79	Starting with Slackware v13, new compression formats are supported which
80	are represented by three additional file extensions:
81
82	.tgz - Gzip
83	.tbz - Bzip2
84	.tlz - LZMA
85	.txz - XZ
86
87	In this document, we use the file extension '.t?z' to represent
88	one of the above file formats.
89
90	Once you have initially installed Slackware from the installer screen,
91	you have the facilities to install, remove or upgrade new or existing
92	packages using the package management tools:
93
94	installpkg <package.t?z> - install 'package.t?z'
95
96	upgradepkg <package.t?z> - upgrade existing 'package' with the
97	new version specified at the command line.
98
99	removepkg <package> - remove specified package
100
101
102	Whilst the Slackware package system is not especially feature rich,
103	Slackware's user base (including me) like it because it is simple.
104	If we want to write our own package utilities then we can easily do so
105	by examining the existing package tools and querying and amending the
106	package database (text files).
107
108	2.0.1 Format of official Slackware packages
109	-------------------------------------
110
111	In Slackware 8.1 and up, each package file is named as follows:
112
113	packagename-version-architecture-buildnumber.t?z
114
115	2.0.1.1 Package names
116	-------------
117
118	The package name is either the name of the single program
119	or the name of a collection of utilities that fall under
120	a single umbrella.
121	For example:
122	autoconf-2.54-noarch-1.tgz
123
124	That is the name of the autoconf package I have on my
125	Slackware 8.1 box.
126	'autoconf' is the name of the the entire collection of
127	binaries and associated documents that are extracted from
128	the autoconf source distribution archive.
129
130	However, if we consider another example:
131	tcpip-0.17-i386-15.tgz
132
133	There is no single piece of software called 'tcpip'.
134	This package contains a number of associated utilities
135	written by different authors but bundled into one single
136	'package'.
137
138	2.0.1.2 Version numbers
139	---------------
140
141	If the package is the name of a particular piece of software
142	such as 'autoconf' from the example above, then the version
143	number represents the version number that its authors distribute.
144
145	If the package is a 'bundle' such as 'tcpip' then the version
146	number increases as and when you add a new piece of software to
147	the package, or upgrade a particular piece of major software
148	contained within the package.
149	For example, with 'tcpip' above, the 0.17 refers to the version of
150	Linux Netkit. However, there are other programs included within
151	the Slackware tcpip package that are not part of 'Linux netkit'.
152
153	2.0.1.3 Architectures
154	-------------
155
156	The architecture is just that -- it tells you which architecture
157	the package is for.
158
159	The current values in use are:
160
161	----- [ Official Slackware architecures ]
162
163	noarch - Architecture independent files such as config files
164	i386 - packages for the x86 (Slackware v8.1 & 9)
165	i486 - packages for the x86 (Slackware 9.1+)
166	i586 - packages for the x86
167	s390 - packages for the IBM s/390 mainframe
168
169	Note: Whilst Slackware v10 is primarily built for i486, you may
170	find that there are some packages whose architecture versions
171	are higher than i486. This is for two reasons:
172
173	[a] There is no source for the package - it is a repackaged
174	binary distribution (SUN's j2sdk is an example).
175
176	[b] The package is not required or otherwise not suitable for
177	earlier revisions of the architecture (this is especially
178	the true with ARM and SPARC).
179
180	----- [ Unofficial ports of Slackware ]
181
182	arm - packages for the ARM architecture
183	sparc - packages for the SUN Sparc architecture
184	powerpc - packages for the PowerPC architecture
185
186	2.0.1.4 Build numbers
187	-------------
188
189	A build number suplements the version number and is changed
190	when the package maintainer makes a change to the package but
191	the original source code and version number remains the same.
192
193	For example, I build a package of 'foo' version 0.14 for the
194	sparc. I have never built this package before, thus it becomes
195	foo-0.14-sparc-1.tgz
196	However, I realise that I haven't configured
197	the 'bar' parameter correctly in /etc/foo.conf. I fix it
198	and re-build the package. The package is now named
199	foo-0.14-sparc-2.tgz
200
201
202	2.1 An overview of the creation of official Slackware packages
203	-----------------------------------------------------------
204
205	This section gives a brief introduction of the two methods of
206	used when building the official Slackware packages.
207
208
209	2.1.1 'Clean' and 'dirty' builds
210	--------------------------
211
212	I am assuming the reader has some experience with Makefiles
213	and has compiled and installed a number of software packages.
214
215	2.1.1.1 Clean builds
216	------------
217
218	I term a 'clean' package one where you can specify a variable
219	to the 'make install' which contains the directory you wish to install
220	the package in, rather than installing it properly over the root file system.
221	For example:
222	# ./configure --prefix=/usr --sysconfdir=/etc
223	# make
224	# make install DESTDIR=/tmp/package-foo
225
226	With a 'Makefile' that honours the DESTDIR variable, this will
227	install the whole package into /tmp/package-foo. This directory
228	effectively is the 'root' directory '/' as far as the Makefile
229	is concerned.
230
231	From here you can use the Slackware 'makepkg' program and build
232	a package.
233
234	This is by far the safest and most preferred method by all
235	users that make packages.
236
237	You will find that DESTDIR is called prefix, TOPDIR and other names;
238	you need to examine the Makefile scripts in order to determine whether
239	it contains this functionality and if it does, then discover what
240	the variable name is.
241
242	2.1.1.2 Dirty builds
243	------------
244
245	A 'dirty' build is the name I give to source distribution archives
246	whose Makefile scripts do not have any provisioning to install
247	in an alternate directory other than root.
248
249	For these type of builds, you will typically do:
250	# ./configure --prefix=/usr --sysconfdir=/etc
251	# make
252	# make install
253
254	The package will then be installed on the root filesystem.
255
256	So how do you know what files were installed where and
257	even if you did, how do you pull all these files together in order
258	to run makepkg over them ?
259	That's the purpose of slacktrack! :-)
260
261
262	2.1.1 SlackBuild and .build scripts
263	-----------------------------
264
265	Slackware has a number of packages by a great number of authors.
266	Some of the packages's source Makefiles honour a DESTDIR type
267	variable but others do not.
268
269	2.1.1.2 SlackBuild scripts
270	------------------
271
272	SlackBuild scripts can be 'interesting'. They are
273	scripts that install into a 'clean' environment (usually /tmp).
274
275	Some of the scripts follow the make install DESTDIR=
276	style, if the Makefile permits.
277
278	Others have a 'framework' or 'controller tarball' which is
279	a _packagename.t?z (note the prefixing underscore).
280
281	The SlackBuild script uses the Slackware 'explodepkg' script
282	to untar the contents of the _.t?z into the /tmp-package<name>
283	directory.
284	Slackbuild then runs 'make' to compile the binaries
285	and libraries, but then uses the 'cat' program such as:
286	# cat foobar > /tmp/package-foobar/usr/bin/foobar
287
288	By using 'cat', the new version of 'foobar' retains
289	the original permissions and ownerships that are in the
290	controller tar ball.
291
292	However, you may be wondering how, if the package does not
293	have a facility to install into somewhere other than root,
294	do we get the file names and permissions for the
295	controller _.t?z in the first place.
296	The answer is simple:
297	[a] find all files on the file system and dump to a file.
298	[b] compile and install the software
299	[c] find all files on the file system and compare the file
300	produced by the first 'find'. After a little pruning, you
301	have the list of files for the controller tar ball.
302
303
304	2.1.1.3 .build scripts
305	---------------
306
307	For those software distributions whose Makefile does not hounour
308	the DESTDIR type system, there are Slackware's .build scripts.
309
310	These scripts literally ./configure ; make ; make install
311	and copy docs and other goodies into the root file system.
312
313	One of the problems with these scripts is that they are
314	often incomplete -- they build and install the package but
315	do not gzip the man pages or strip the binaries and libraries;
316	this is done manually afterwards.
317
318	These are the scripts that slacktrack and altertrack were
319	written for.
320
321	* Note: Whilst some software's Makefiles may appear to honour
322	the DESTDIR variable, the Makefile may be broken which can
323	result in files missing or corrupt within your new package.
324	For example: I built Apache v2.0.48 and built a package using
325	make install DESTDIR. However, a problem exists in that some of
326	the Perl scripts it installs have temporary build paths
327	hard coded into the resulting scripts.
328	This is why you may find a .build instead of a SlackBuild
329	script within Slackware's source tree.
330
331	However, the primary reason is because the build script just
332	hasn't been updated to make use of DESTDIR. *
333
334
335	2.2 slacktrack in the scheme of things
336	----------------------------------
337
338	I follow Slackware-current quite closly. Often I want to
339	'back port' a -current package to an earlier release of Slackware .
340	I can't simply upgrade with -current's package because it was
341	compiled for a newer GLIBC than Slackware 8.1's, for example.
342	For packages that use a 'clean' 'SlackBuild' script, this is
343	an easy job -- I simply run 'SlackBuild' on an 8.1 box.
344
345	However, for .build scripts, I needed a way of building packages
346	using Slackware's .build scripts.
347
348	I found a great program called 'CheckInstall' which fulfilled most of my
349	requirements. However, as time went on and I started building
350	more of Slackware's packages and writing my own build scripts, I found
351	that checkinstall was lacking some features that I required.
352	At this time I was also considering porting Slackware to run on
353	the ARM architecture and helping with the Splack (Slackware on SPARC project),
354	and therefore wanted a robust program that can deal with every .build script
355	I threw at it, and if it can't handle it, I needed to be able to make modifications.
356	The easiest way of doing this was to write my own script; thus
357	'slacktrack' was born.
358
359	slacktrack is based on the idea behind 'CheckInstall', but uses
360	only my own code (and contributions from other users), and only contains
361	Slackware-specific facilities -- it can not build Debian or RedHat packages
362	and never will.
363
364	slacktrack does not have all of the facilities of CheckInstall either,
365	but then these are not required for building Slackware packages
366	from .build scripts.
367
368	Also, slacktrack only works with 'official' Slackware directory locations
369	and /usr/local.
370	For example, if your make install installs binaries in /opt/packagename/bin
371	and the man pages in anywhere other than /usr/man or /usr/local/man, then
372	slacktrack's relevant options (eg stripping libs, bins, gzman) will
373	not detect them.
374
375
376	2.2.1 Using slacktrack with official .build scripts
377	---------------------------------------------
378
379	Building a replicar Slackware package from a .build script is
380	typically fairly trivial with slacktrack.
381
382	If we wanted to build slackware-9.1's elvis, we could do
383	# cd slackware/slackware-9.1/source/a/elvis
384	# chmod 755 elvis.build
385	# slacktrack -jefkzp "elvis-2.1-i386-2.tgz" ./elvis.build
386
387	The resulting package (by default) be stored in
388	/tmp/built-slackwarepackages/
389
390	As already mentioned, some of the Slackware .build scripts
391	are incomplete with regard to gzipping man pages, stripping binaries
392	and so on -- fetchmail.build is one of them.
393	Therefore you can specify various options to slacktrack that
394	will take care of this for you.
395	The options in the example above :
396	j - compress libraries
397	e - chown root:bin /bin,/sbin,/usr/bin,/usr/sbin directories
398	f - chown root:bin files in the binary dirs listed above
399	k - strip binaries found in the binary dirs listed above
400	z - gzip man pages
401	p - the resulting Slackware package .t?z name
402
403	The way I re-create the packages is to build a 'trackbuild' script that
404	launches slacktrack with the required options and the name
405	of the Slackware .build script. You can find examples of such
406	scripts within the docs directory after installing slacktrack:
407
408	/usr/doc/slacktrack*/buildscript-examples/
409
410	You will also find that in Slackware versions 12.0 and upwards,
411	the .build scripts are accompanied by 'trackbuild' scripts because
412	slacktrack is used by the Slackware team to produce the packages.
413
414	2.2.2 Writing your own .build scripts for slacktrack
415	----------------------------------------------
416
417	There isn't any specific that you have to do to write a build
418	script for use with slacktrack -- the script name specified to
419	slacktrack doesn't even have to be a script - it can be a binary -
420	as long as it is executable, it is acceptable.
421
422	You can see some of my own build scripts that I have written
423	for use with slacktrack by looking in the documents directory
424	after installing slacktrack:
425
426	/usr/doc/slacktrack*/buildscript-examples/
427
428	2.2.2.1 Making use of slacktrack's environment variables
429	-------------------------------------------------
430
431	slacktrack exports two environment variables:
432	SLACKTRACKFAKEROOT and SLACKTRACKSCRATCHDIR
433
434	SLACKTRACKFAKEROOT:
435	````````````````````
436
437	The purpose of this to allow your .build script to access the
438	'root' directory of the resulting package.
439	There are two scenarios where you may want to use this
440	variable:
441
442	1. During the build:
443
444	The reason you may wish to do this is to pre-populate
445	the package with files that you may not wish to place directly
446	onto the root filesystem.
447	The package contents will only appear inside this directory
448	once your build script has finished, and slacktrack has determined
449	what to put into this directory.
450
451	In previous slacktrack versions which used a pseudo-root filesystem
452	(where the package contents were populated during the build), this
453	made sense, but in slacktrack v2.00, it is unlikely that you'd want
454	to use this vairable from the build script.
455
456	2. Post-build -- cleanups after the build:
457
458	The most likely use of this script is to perform package cleanup
459	tasks after the build. This is mainly to perform tasks that slacktrack
460	does not handle itself - such as deleting known files/directories that
461	creep into packages (due to a system daemon such as CUPS), or setting
462	some file and directory permissions.
463
464	An example post build script is below.
465	A post build script can be specified by slacktrack's -R option:
466
467	# Incase you had CUPS running:
468	rm -rf etc/cups etc/printcap
469	# crond:
470	rm -rf var/spool/cron
471	rmdir var/spool
472
473	# perllocal.pod files don't belong in packages.
474	# SGMLSPL creates this:
475	find . -name perllocal.pod -print0 \| xargs -0 rm -f
476
477	# Some doc dirs have attracted setuid.
478	# We don't need setuid for anything in this package:
479	chmod -R a-s .
480
481
482	SLACKTRACKSCRATCHDIR:
483	`````````````````````
484
485	The purpose of this variable is to provide some temporary
486	space to untar your source archives and so on. slacktrack
487	will manage the creation and deletion of this directory.
488
489	For example:
490	# cd ${SLACKTRACKSCRATCHDIR}
491	# tar zxvvf ${ORIGPATH}/source/foobar.tar.gz
492
493	You can see in some of the example 'non-slackware' scripts
494	how I have used these variables
495
496
497	3.0 slacktrack in operation
498	-----------------------
499
500	The basic event flow is as follows:
501
502	[1] Parse command line arguments
503	-> Check they are OK, if not bomb out
504	[2] Launch the supplied build script
505	[3] Run any specified functions (eg gzman, strip bins, chowns) over the
506	package 'root' directory and contents
507	[4] Run Slackware's 'makepkg' program over the package contents
508	[5] Move the .t?z to the specified build store path
509	[6] Scan for any hard links that may be in the package
510	-> If any found, alert the user on screen and also
511	log to a packagename.hardlinks.log file in the build store path
512
513	The slacktrack shell script is fairly simple and well commented; it should be
514	relatively easy for anybody who understands bash to quickly comprehend what
515	is happening and why.
516
517	3.1 How slacktrack finds which files have been installed
518	----------------------------------------------------
519
520	In order to track the files and directories that have been installed
521	or changed, slacktrack follows this ordered process:
522
523	[1] Scans a pre-set list of directories on the filesystem and
524	logs the contents.
525	[2] Launches build script which installs the package onto the
526	filesystem
527	[3] Scans the filesystem once again
528	[4] Compares the differences in the before and after snapshots
529	[5] Copies the differences into a 'package root directory' and
530	runs makepkg over them.
531
532	In slacktrack version 1, we used 'installwatch' which overloaded
533	some of glibc's filesystem calls, creating new files and directories
534	into a pseudo-root filesystem, avoiding the need to install onto
535	the host's filesystem (and also allowing to build as a non-root user).
536	However, this library is ill maintained and even in the early days
537	when it worked well, it still had problems and workarounds were required.
538
539	4.0 Example scripts
540	---------------
541
542	Included with the slacktrack binary distribution are a number of example
543	scripts that hopefully should provide you with a basis of how to use slacktrack
544	to build from 'dirty' source distributions.
545
546	The examples are bundled in the documentation directory available
547	after installing slacktrack:
548
549	/usr/doc/slacktrack*/buildscript-examples/
550
551
552	5.0 Known problems/features of slacktrack
553	-------------------------------------
554
555	Current problems:
556
557	[1] slacktrack doesn't have sufficient error checking.
558
559	I'm in two minds about where to put error checking, you see.
560	Do I go no further if the supplied build script exits with a non-zero
561	value?
562	No, that's a bad idea because what if you didn't write the build script?
563	it might be one of these qmail type binary installer programs that
564	you can't (easily without hacking the source) fix. The author may
565	have their own systems, and the program exits with a value that their
566	own controller script recognises as non-failure.
567
568	What should I do if makepkg has failed? You see it on screen
569	and in the logs anyway, plus makepkg is one of the last things
570	that slacktrack calls -- how can you recover?
571
572	That said, version 1.03 now has some error handling. There's still
573	not enough though, imo.
574
575	[2] No support for a default config file
576
577	I don't see the point. Each .build script is different
578	and although I typically use the same options to slacktrack
579	for most of my build scripts, I don't see why I'd need a
580	config file just to save 4 bytes or so in a trackbuild wrapper :-)
581
582
583	6.0 New features
584	-------------
585
586	See the TODO file in the docs directory.
587
588	If you have any specific features you would like to see
589	included, or have found any bugs, please
590	email me <mozes@slackware.com>
591
592	7.0 Licence
593	-------
594
595	slacktrack is distributed under BSD Licence.

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

Original Format