Building From Source: Difference between revisions
(Marked page as being outdated) |
|||
(16 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
<p class="wikiNote">'''NOTE''': This page is outdated and should no longer be used as a reference for building a NST development system. If you are interested in putting together a system for the purpose of building the NST distribution from source, please refer to the [[Developer Info]] section.</p> | |||
__TOC__ | |||
= Preparing The Build Machine = | = Preparing The Build Machine = | ||
You will need to install [http://fedoraproject.org Fedora | You will need to install [http://fedoraproject.org Fedora 7] (the ''i386'' variant) onto your development system. | ||
If you want to build a 64 bit variant (''x86_64'') of the '''NST''', you will need to download the ''x86_64'' variant of [http://fedoraproject.org Fedora | If you want to build a 64 bit variant (''x86_64'') of the '''NST''', you will need to download the ''x86_64'' variant of [http://fedoraproject.org Fedora 7]. While the ''i386'' variant will run on both 32 bit and 64 bit machines, the ''x86_64'' variant will only run on 64 bit machines. Most, if not all, 64 bit '''AMD''' processors and the ''Intel Core 2'' series should be compatible with the ''x86_64'' variant. | ||
If you don't want to use physical hardware, you can use a Virtual Machine (even running under Windows) to do your '''NST''' development under. We have found that either ''VMWare Server'' (our preference) or ''VMWare Player'' from [http://www.vmware.com/ VMware] can be used (even if you want to build a ''x86_64'' version of the '''NST'''). We have also found that [http://www.microsoft.com/windows/products/winfamily/virtualpc/default.mspx Virtual PC] can be used. | If you don't want to use physical hardware, you can use a Virtual Machine (even running under Windows) to do your '''NST''' development under. We have found that either ''VMWare Server'' (our preference) or ''VMWare Player'' from [http://www.vmware.com/ VMware] can be used (even if you want to build a ''x86_64'' version of the '''NST'''). We have also found that [http://www.microsoft.com/windows/products/winfamily/virtualpc/default.mspx Virtual PC] can be used. | ||
See the [[Fedora Installation Notes]] for additional information on installing the base '''OS'''. | |||
= Getting the NST Source Code = | = Getting the NST Source Code = | ||
Line 44: | Line 50: | ||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>CVSROOT=":pserver:anonymous@nst.cvs.sourceforge.net:/cvsroot/nst"</div> | <div class="userInput"><span class="prompt">[root@localhost nst]# </span>CVSROOT=":pserver:anonymous@nst.cvs.sourceforge.net:/cvsroot/nst"</div> | ||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>export CVSROOT</div> | <div class="userInput"><span class="prompt">[root@localhost nst]# </span>export CVSROOT</div> | ||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>cvs update - | <div class="userInput"><span class="prompt">[root@localhost nst]# </span>cvs update -dP</div> | ||
<pre class="computerOutput"> | <pre class="computerOutput"> | ||
Line 67: | Line 73: | ||
* Before configuring the system, it is recommend that one run: "'''yum update'''" to make sure the latest kernel and package updates are installed. | * Before configuring the system, it is recommend that one run: "'''yum update'''" to make sure the latest kernel and package updates are installed. | ||
* Sun's Java Run Time Environment (JRE) is used when building the '''NST'''. The '''configure''' script will attempt to download and install it automatically (if it isn't found on your development system). If the script is unable to automatically install it, you will need to download and install the '''JRE''' from: [http://java.com/] (get the appropriate '''RPM''' version). | * Sun's Java Run Time Environment (JRE) is used when building the '''NST'''. The '''configure''' script will attempt to download and install it automatically (if it isn't found on your development system). If the script is unable to automatically install it, you will need to download and install the '''JRE''' from: [http://java.com/ http://java.com/] (get the appropriate '''RPM''' version). | ||
* You will probably need to run the '''configure''' multiple times the first time you setup your development system (as you will most likely be missing some packages). | * You will probably need to run the '''configure''' multiple times the first time you setup your development system (as you will most likely be missing some packages). | ||
Line 110: | Line 116: | ||
... Lot's of yum output as it figures out package dependencies, | ... Lot's of yum output as it figures out package dependencies, | ||
then you will be prompted. Enter 'y' when you see: | then you will be prompted. Enter 'y' when you see: | ||
</pre> | </pre> | ||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>./configure</div> | <div class="userInput"><span class="prompt">[root@localhost nst]# </span>./configure</div> | ||
<pre class="computerOutput"> | <pre class="computerOutput"> | ||
Checking for DocBook XSL [ OK ] | |||
Verifying settings [ OK ] | |||
Creating: "./config/config.sh" [ OK ] | |||
... Omitting Output. | |||
This time configure found the system up to date and completed normally ... | |||
Use "make" to build src/nst-1. | Use "make" to build src/nst-1.7.0.iso.gz, or "make help" to see | ||
additional targets. | additional targets. | ||
Line 165: | Line 135: | ||
"src/bin" to aid in the package install/update process). | "src/bin" to aid in the package install/update process). | ||
</pre> | </pre> | ||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | <div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | ||
</div> | </div> | ||
= Installing Packages = | = Installing Packages = | ||
Line 197: | Line 160: | ||
and some will build from source) ... | and some will build from source) ... | ||
------------------------------------------------------------------------------- | |||
***ERROR*** Following package updates failed: | |||
snort unicornscan wireshark autohide colorful_tabs dictionarysearch | |||
image_zoom LinkChecker long_titles quickrestart reloadevery tidy webdeveloper | |||
Check the log files under: /root/nst/tmp/updates for details | |||
------------------------------------------------------------------------------- | |||
make: *** [tsv-update] Error 1 | |||
</pre> | </pre> | ||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | <div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | ||
</div> | </div> | ||
The above output shows that several of the "automatic" package installations failed. This typically occurs for one of the following reasons: | |||
* The package required that a '''X''' desktop be running. All of the '''firefox''' add-ons fall into this category (autohide, colorful_tabs, ...). | |||
* The package is no longer available. This often occurs when a new release of a package comes out and indicates that the file: "'''include/data/packages.tsv'''" needs to be updated. This is what happened with '''snort''' (they were at release '''2.7.0''' where as the "'''include/data/packages.tsv'''" file was configured for '''2.6.1.5'''). | |||
* A library was not found because the '''ldconfig''' command needs to be run (this is what caused the issue with '''wireshark''' shown in the above output). | |||
* The package has dissappeared, moved, or no longer builds cleanly (this is what occurred with '''unicornscan''' in the output shown above). | |||
When you encounter packages which fail to install cleanly, you should be able to review the entire log related to the installation by looking in the: "'''tmp/updates'''" directory (for example, the file: "'''tmp/updates/unicornscan.log'''" contained many error messages). | |||
While the "''make package-update''" will automate the installation of 98% of the packages needed to build a full '''NST''' system, there are a handful of packages which require user interaction to complete the installation. For each of these packages, you should find an appropriately named script under the: "'''src/bin'''" directory. | |||
To see what packages still need to be installed, the "''make package-check''" command can be used. The following demonstrates how to identify which packages still need to be installed, and then installs the '''ieee80211''' package: | |||
<div class="screen"> | <div class="screen"> | ||
<div class="screenTitle">'' | <div class="screenTitle">''Identifying Missing Packages''</div> | ||
<div class="userInput"><span class="prompt">[root@localhost ~]# </span>cd $HOME/nst</div> | <div class="userInput"><span class="prompt">[root@localhost ~]# </span>cd $HOME/nst</div> | ||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>make package-check</div> | <div class="userInput"><span class="prompt">[root@localhost nst]# </span>make package-check | grep -i fail</div> | ||
<pre class="computerOutput"> | |||
</pre> | |||
<pre class="computerOutput"> | |||
autohide (version 1.1.5) [failed] | |||
colorful_tabs (version 2.0.1) [failed] | |||
dictionarysearch (version 2.0.1) [failed] | |||
ieee80211 (version 1.2.16) [failed] | |||
image_zoom (version 0.2.7) [failed] | |||
ipw2200 (version 1.2.1) [failed] | |||
LinkChecker (version 0.6.1) [failed] | |||
long_titles (version 1.2.4) [failed] | |||
madwifi (version 0.9.3) [failed] | |||
metasploit (version 2.7) [failed] | |||
nessus (version 2.2.9) [failed] | |||
netw (version 5.34.0) [failed] | |||
ntop (version 3.3) [failed] | |||
quickrestart (version 1.1.0) [failed] | |||
reloadevery (version 2.0) [failed] | |||
snort (version 2.6.1.5) [failed] | |||
snort_inline (version 2.6.1.5) [failed] | |||
snort-mysql (version 2.6.1.5) [failed] | |||
snorter (version 2.1) [failed] | |||
tidy (version 0.8.3.9) [failed] | |||
vmware-tools (version 1.0.3) [failed] | |||
webdeveloper (version 1.1.4) [failed] | |||
</pre> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>src/bin/ieee80211_fetch_build</div> | |||
<pre class="computerOutput"> | |||
... Lots of output followed by a series of questions you must | |||
answer, followed by the build/install of the package ... | |||
------------------------------------------------------------------------------- | |||
SUCCESS: Successfully installed ieee80211 | |||
------------------------------------------------------------------------------- | |||
NOTE: You may need to re-build modules which depended upon | |||
this one. Look for failures when you run "make package-check". | |||
</pre> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>make package-check | grep ieee80211</div> | |||
<pre class="computerOutput"> | |||
ieee80211 (version 1.2.16) [ok] | |||
</pre> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | |||
</div> | |||
= Building a ISO Image = | |||
To build the '''NST ISO''' image, one runs the following command: | |||
<div class="screen"> | |||
<div class="screenTitle">''Building a ISO Image''</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>make</div> | |||
<pre class="computerOutput"> | |||
... You will see a lot of output and it will take awhile to build ... | |||
</pre> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>ls -l src/*.iso</div> | |||
<pre class="computerOutput"> | <pre class="computerOutput"> | ||
-rw-r--r-- 1 root root 509790208 2007-08-25 17:27 src/nst-1.7.0.iso | |||
</pre> | </pre> | ||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | <div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | ||
</div> | </div> | ||
= | <p class="wikiNote">You will always see a ''[warn]'' indicator when the '''java''' package is installed. At the time of this writing, you may also expect to see a ''[warn]'' indicator for the '''cups''' and '''clamav''' packages (this should be resolved prior to the final release). Other packages flagged by a ''[warn]'' and/or ''[failed]'' indicator, would indicate that the layout of the package has changed since its installation script was last updated (the '''NST''' developers will get to it before the next release).</p> | ||
Now that a '''ISO''' image has been produced, one can do any of the following: | |||
* Burn the image to a '''CD''' or '''DVD''' (we've had good luck with '''CDRW''' media during our development). You might be able to use the: "'''make cd'''", "'''make cdrw'''" and/or "'''make dvd+rw'''" (see the output of: "'''make help'''" for more information). NOTE: If you are developing inside a virtual machine, you probably won't be able to burn a disk immediately. Instead, you'll need to transfer the image to a different system which has a burner. | |||
* Transfer the '''ISO''' image to a different machine (we typically use '''scp''', if you are a Windows user, you may want to consider [http://winscp.sourceforge.net/ WinSCP]). | |||
* Boot directly from the '''ISO''' image using a '''VMware''' product or '''Virtual PC'''. NOTE: At the time of this writing, there is an issue with '''NST''' and the recommended virtual LSI SCSI controller in '''VMware Server''' (beta testers should opt for the '''BusLogic''' controller when creating new virtual machines for the time being). | |||
= Tweaking The Configure/Build Process = | = Tweaking The Configure/Build Process = | ||
During the configuration and build process, several files in the: "'''${HOME}/.nst'''" directory are examined. These files are optional (you don't have to create any of them). However, creating these files, allows one to customize the results of configuring and building the '''NST''' '''ISO''' image without having to specify a lot of command line options to the '''configure''' script. | |||
<p class="wikiNote">The top level: "'''README'''" file contains a lot of information about the various configuration options available.</p> | |||
== The Magic "${HOME}/.nst" Directory == | == The Magic "${HOME}/.nst" Directory == | ||
By default, the configuration and build process will look under the: "'''${HOME}/.nst'''" directory for build customization files. You can append the: "''--config-dir DIRECTORY''" option to the '''./configure''' command if you would like to use a different directory. | |||
== "${HOME}/.nst/configure.sh" == | == "${HOME}/.nst/configure.sh" == | ||
The first file you will likely want to create is the: "'''${HOME}/.nst/configure.sh'''" file. This file can be used to set MANY of the available options supported by the top level '''./configure''' script. | |||
<pre class="programListing"> | |||
# Set a custom password and disable prompt to set the initial password | |||
NSTPASSWD="MY_NST_PASSWORD"; | |||
NSTPASSWD_PROMPT="false"; | |||
# Uncomment if you want the extra development packages added (src/packages/devextra) | |||
#PKGCATS="${PKGCATS} devextra"; | |||
# Uncomment if you want the "extra" disk packages added (src/packages/diskextra) | |||
#PKGCATS="${PKGCATS} diskextra"; | |||
# Where source code is downloaded to and built under | |||
BUILDROOT="/usr/local/src"; | |||
# Default boot mode | |||
NSTBOOT="desktop"; | |||
# Set to true if you want to build the NST documents, false for ISO | |||
ON_LINE_DOCS="false"; | |||
# Compress files so ISO will likely be small enough for a CD? | |||
NSTCOMPRESSED_ISO="true"; | |||
# Take time to strip executables to produce a smaller ISO image? | |||
NSTSTRIPPED="true"; | |||
</pre> | |||
There are many other settings you can add to this file, but those shown above will cover 95% of what you'll want to tweak. After you run the '''configure''' script, you can check the resulting configuration by looking at the file: "'''config/config.sh'''" (the top portion of this file can also be used as a reference for other variables you might wish to set). | |||
<p class="wikiNote">'''NOTE''': You will need to re-run the '''./configure''' script after making changes to this file. Also, make sure you set the permissions of this file to: ''0600'' - especially if you decide to put a clear text password in it.</p> | |||
== "${HOME}/.nst/disable.txt" == | == "${HOME}/.nst/disable.txt" == | ||
As time has passed, many packages have been added and removed from the '''NST''' distribution. As packages are removed from the '''NST''' distribution, they are often added to the: "'''include/data/disable.txt'''" file to indicate that they should not be included in the build. | |||
During the build process, the file: "'''${HOME}/.nst/disable.txt'''" will be searched for before falling back to: "'''include/data/disable.txt'''". This allows one to customize what packages are omitted from the build. | |||
If you are interested in omitting packages (or categories), copy the file: "'''include/data/disable.txt'''" to: "'''${HOME}/.nst/disable.txt'''" and then edit it's contents. You should find comments at the top of the file indicating how one disables additional categories and/or packages. | |||
== "${HOME}/.nst/post_configure.sh" == | |||
This script is sourced (if it exists) after each invocation of the '''./configure''' command. It allows one to perform "post configuration" updates to the build environment. Very few people will have need to implement this script. But if you do, your script will have access to all of the variables found in: '''config/config.sh''' (useful for locating files making up the system). | |||
== "${HOME}/.nst/post_install.sh" == | == "${HOME}/.nst/post_install.sh" == | ||
This script is run (if it exists) after all of the packages have been installed during the build process (when running: '''make'''). This script can be VERY useful to customizing your resulting '''NST''' '''ISO'''. | |||
The following example, does the following: | |||
* Defines some function which will adjust files in the build area prior to forming the '''ISO''' image. | |||
* Invokes a subset of the defined functions for the features we want installed. | |||
<pre class="programListing"> | |||
# | |||
# Customized files PRIOR to building ISO | |||
# | |||
# install_custom_isolinux_cfg | |||
# | |||
# | |||
install_custom_isolinux_cfg() { | |||
if [ -d "${NSTHOMEDIR}/base/isolinux" -a "${CFGDIR}/isolinux.cfg" ]; then | |||
${CP} -f "${CFGDIR}/isolinux.cfg" "${NSTHOMEDIR}/base/isolinux"; | |||
fi | |||
} | |||
# add_setup_home | |||
# | |||
# Adds helper script "/root/bin/setup_home", to configure probe for home network | |||
add_setup_home() { | |||
create_dirs "${NSTUTILSDIR}/root/bin" | |||
cat >| "${NSTUTILSDIR}/root/bin/setup_home" <<EOF | |||
#!/bin/bash | |||
# | |||
# Detect/add swap space | |||
# | |||
auto_add_swap | |||
# | |||
# Enable NTP | |||
# | |||
/etc/rc.d/init.d/ntpd start >/dev/null 2>&1 & | |||
# | |||
# Enable home proxy | |||
# | |||
nstsetproxy --http-host 192.168.12.1 --http-port 3128 | |||
# | |||
# Mount NFS drives | |||
# | |||
service portmap start | |||
mount -t nfs 192.168.12.1:/home /home > /dev/null 2>&1 & | |||
if [ ! -d /lan ]; then | |||
mkdir /lan | |||
fi | |||
mount -t nfs 192.168.12.1:/opt /lan > /dev/null 2>&1 & | |||
EOF | |||
chmod +x "${NSTUTILSDIR}/root/bin/setup_home" | |||
} | |||
# config_ssh | |||
# | |||
# Add custom .ssh configuration (authorized_keys in particular) | |||
config_ssh() { | |||
if [ -d "${NSTINITRDDIR}" ]; then | |||
${RM} -fr "${NSTINITRDDIR}/root/.ssh" | |||
fi | |||
create_dirs "${NSTUTILSDIR}/root/.ssh" | |||
${CHMOD} 700 ${NSTUTILSDIR}/root/.ssh | |||
for f in authorized_keys config id_rsa id_dsa; do | |||
if [ -e "${HOME}/.ssh/${f}" ]; then | |||
${CP} -p "${HOME}/.ssh/${f}" "${NSTUTILSDIR}/root/.ssh" | |||
fi | |||
done | |||
} | |||
# | |||
# Enable CD eject on shutdown | |||
# | |||
enable_cd_eject() { | |||
create_dirs "${NSTUTILSDIR}/sbin" | |||
${LN} -s /usr/bin/eject ${NSTUTILSDIR}/sbin/halt.local | |||
} | |||
# | |||
# Call features we want | |||
# | |||
install_custom_isolinux_cfg; | |||
# enable_cd_eject; | |||
config_ssh; | |||
add_setup_home; | |||
</pre> | |||
You can use this to customize pretty much anything in the build (add other packages, configure your network setttings, install wireless keys, etc). | |||
As this file is sourced, you will have access to all of the variables defined in: "'''config/config.sh'''". Those that will be of most interest include: | |||
; '''${NSTLOCALDIR}''' : This is the root directory which will appear as: '''/usr/local''' once the system has booted off of the '''Live CD'''. All files placed under here will be '''READ ONLY''' (don't put a configuration file which is likely to be editted here). This is where most executables and libraries are placed. | |||
; '''${NSTUTILSDIR}''' : This directory contains a collection of files that will be copied to '''RAM''' once the '''Live CD''' boots and mounts the '''CD'''. Configuration files, small scripts, frequently used tools are often placed here. Be aware that any file placed here will consume '''RAM''' as the system runs. | |||
; '''${NSTINITRDDIR}''' : This is the root directory which contains the files and directories which will end up in the initial '''RAM''' disk when booting from the '''Live CD'''. It typically contains small scripts, configuration files and enough executables and modules to get the '''CDROM''' mounted. You should avoid adding large files to this area. | |||
= Adding Packages To The Build = | |||
During the build process (when you type: '''make'''), package installation scripts for the various package categories under: '''src/packages''' are run. These scripts transfer information from the development system to the directory heirarchy that is used to form the final '''ISO''' image. | |||
== Adding Files From a RPM Package == | |||
The easiest packages to add to a '''NST''' build are from packages which are installed on the development system from '''RPM''' files (typically via: '''yum'''). | |||
To add additional packages to the '''NST''', it is recommended to use the following steps: | |||
* Create a: "'''src/packages/custom'''" directory (for your custom additions to the '''NST'''). | |||
* Update your configuration file such that the ''custom'' category is added to the: '''${PKGCATS}''' category list. | |||
* Install the package onto your development system. | |||
* Add a script ending with the ''.sh'' extension for each package you want to add. For example, to add the ''bison'' package to the distribution, one would want to create a: "'''src/packages/custom/bison.sh'''" installation script. | |||
The following outlines the steps involved to add ''bison'' to the distribution under the ''custom'' category. | |||
=== Creating the Category Directory === | |||
This step is trivial: | |||
<div class="screen"> | |||
<div class="screenTitle">''Adding The "custom" Directory''</div> | |||
<div class="userInput"><span class="prompt">[root@localhost ~]# </span>cd $HOME/nst</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>mkdir src/packages/custom</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | |||
</div> | |||
NOTE: If you want to keep your work separate from the '''NST''' area, you may want to use a symbolic link for: "'''src/packages/custom'''" instead of creating the directory directly in the '''NST''' heirarchy. | |||
=== Adding The Custom Directory To Your Build === | |||
First add the following lines to the bottom of your: "'''${HOME}/.nst/configure.sh'''" configuration file: | |||
<pre class="programListing"> | |||
# Following will cause the NST build to process ALL scripts found under the | |||
# src/packages/custom directory when building the ISO | |||
PKGCATS="${PKGCATS} custom"; | |||
</pre> | |||
Then run the: "'''configure'''" script and verify that the category was added: | |||
<div class="screen"> | |||
<div class="screenTitle">''Configure and Verify "custom" Directory''</div> | |||
<div class="userInput"><span class="prompt">[root@localhost ~]# </span>cd $HOME/nst</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>./configure</div> | |||
<div class="computerOutput"> | |||
... Omitting the initial output from running configure ... | |||
PKGCATS: boot boot2 bootnet system time printing db x gnome server | |||
development php java applications networking wireless dockapps recovery sound custom | |||
SFUSER: pblankenbaker | |||
SNORTRSCACHENAME: snort_rs_cache | |||
SOURCE_FORGE_MIRROR: http://easynews.dl.sourceforge.net/sourceforge | |||
WEBHOST_DOMAIN: | |||
WEBHOST_USER: | |||
Use "make" to build src/nst-1.7.0.iso.gz, or "make help" to see | |||
additional targets. | |||
NOTE: You may want to run "make package-update" to update MANY of the | |||
packages needed for the NST. You can then run "make package-check" to | |||
see what is left to do by hand (there are many helper scripts under | |||
"src/bin" to aid in the package install/update process). | |||
</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | |||
</div> | |||
The important thing to notice is the presence of: "'''custom'''" in the list of categories next to the: ''PKGCATS:'' label above. | |||
=== Install The "bison" Package === | |||
For this example, we'll use '''yum''' to install the '''bison''' package and then get a listing of the files in the package. It should be noted that the use of '''yum''' is not required, this example only requires that the package is installed on the development system from a '''RPM''' file. | |||
<div class="screen"> | |||
<div class="screenTitle">Installing "bison"</div> | |||
<div class="userInput"><span class="prompt">[root@localhost ~]# </span>yum install bison</div> | |||
<div class="computerOutput"> | |||
... output from yum as bison is downloaded and installed on your development system ... | |||
</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>rpm -ql bison</div> | |||
<div class="computerOutput"> | |||
/usr/bin/bison | |||
/usr/share/aclocal/bison-i18n.m4 | |||
/usr/share/bison | |||
/usr/share/bison/README | |||
/usr/share/bison/c++.m4 | |||
/usr/share/bison/c.m4 | |||
/usr/share/bison/glr.c | |||
/usr/share/bison/glr.cc | |||
/usr/share/bison/lalr1.cc | |||
/usr/share/bison/location.cc | |||
/usr/share/bison/m4sugar | |||
/usr/share/bison/m4sugar/m4sugar.m4 | |||
/usr/share/bison/yacc.c | |||
/usr/share/doc/bison-2.3 | |||
/usr/share/doc/bison-2.3/AUTHORS | |||
/usr/share/doc/bison-2.3/ChangeLog | |||
/usr/share/doc/bison-2.3/NEWS | |||
/usr/share/doc/bison-2.3/OChangeLog | |||
/usr/share/doc/bison-2.3/README | |||
/usr/share/doc/bison-2.3/THANKS | |||
/usr/share/doc/bison-2.3/TODO | |||
/usr/share/info/bison.info.gz | |||
/usr/share/locale/da/LC_MESSAGES/bison.mo | |||
/usr/share/locale/de/LC_MESSAGES/bison.mo | |||
/usr/share/locale/es/LC_MESSAGES/bison.mo | |||
/usr/share/locale/et/LC_MESSAGES/bison.mo | |||
/usr/share/locale/fr/LC_MESSAGES/bison.mo | |||
/usr/share/locale/ga/LC_MESSAGES/bison.mo | |||
/usr/share/locale/hr/LC_MESSAGES/bison.mo | |||
/usr/share/locale/id/LC_MESSAGES/bison.mo | |||
/usr/share/locale/it/LC_MESSAGES/bison.mo | |||
/usr/share/locale/ja/LC_MESSAGES/bison.mo | |||
/usr/share/locale/ms/LC_MESSAGES/bison.mo | |||
/usr/share/locale/nb/LC_MESSAGES/bison.mo | |||
/usr/share/locale/nl/LC_MESSAGES/bison.mo | |||
/usr/share/locale/pl/LC_MESSAGES/bison.mo | |||
/usr/share/locale/pt_BR/LC_MESSAGES/bison.mo | |||
/usr/share/locale/ro/LC_MESSAGES/bison.mo | |||
/usr/share/locale/ru/LC_MESSAGES/bison.mo | |||
/usr/share/locale/rw/LC_MESSAGES/bison.mo | |||
/usr/share/locale/sv/LC_MESSAGES/bison.mo | |||
/usr/share/locale/tr/LC_MESSAGES/bison.mo | |||
/usr/share/locale/vi/LC_MESSAGES/bison.mo | |||
/usr/share/man/man1/bison.1.gz | |||
</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | |||
</div> | |||
=== Creating "src/packages/custom/bison.sh" === | |||
Having seen the list of files which belong to the '''bison''' package, we'll create the installation script: "'''src/packages/custom/bison.sh'''" as shown below: | |||
<pre class="programListing"> | |||
# $Id$ | |||
# Description: This script installs the bison package. | |||
# Vars: | |||
# ----- | |||
PKG="${PNAME}"; | |||
# Code: | |||
# ----- | |||
# Copy files installed from RPM to /usr/local area on ISO (we'll put | |||
# include most of the files from the package, but omit the locale | |||
# specific stuff and some of the /usr/share/doc stuff to reduce the | |||
# package size some). | |||
check_rpm ${PKG} "${NSTLOCALDIR}" "/usr" \ | |||
"^/usr/bin/${PNAME}" \ | |||
"^/usr/share/${PNAME}" \ | |||
"^/usr/share/doc/${PNAME}.*/AUTHORS" \ | |||
"^/usr/share/doc/${PNAME}.*/README" \ | |||
"^/usr/share/info/" \ | |||
"^/usr/share/man/"; | |||
# Add a symlink so "bison" can be found as /usr/bin/bison as well | |||
# as under "/usr/local/bin/bison" and "/usr/share/aclocal" is the | |||
# same as "/usr/local/share/aclocal" | |||
create_dirs "${NSTUTILSDIR}/bin" "${NSTUTILSDIR}/share"; | |||
${LN} -s "/usr/local/bin/bison" "${NSTUTILSDIR}/bin/bison"; | |||
${LN} -s "/usr/local/share/aclocal" "${NSTUTILSDIR}/share/aclocal"; | |||
</pre> | |||
The '''check_rpm''' function does most of the work in copying files from the development system to the proper directory for the build. The following section describes this function: | |||
=== check_rpm ''PKG_NAME'' ''DST_ROOT'' ''STRIP_PATH'' ''MATCH0'' [''MATCH1'' ...] === | |||
The '''check_rpm''' function works in the following manner: | |||
* It gets a list of possible source files belonging to ''PKG_NAME'' (A "'''rpm -ql bison'''" in this case) | |||
* For each matching file ("^/usr/bin/bison" matches a single file whereas "^/usr/share/bison" matches many files and directories), it forms a destination file or directory name by stripping off ''STRIP_PATH'' ("'''/usr'''" in our case, so "'''/usr/bin/bison'''" becomes "'''/bin/bison'''"), and then prepends ''DST_ROOT'' ("''/bin/bison''" becomes "''${NSTLOCALDIR}/bin/bison''"). | |||
The tricky part is learning which ''DST_ROOT'' variable to use, we have 3: | |||
; ''${NSTLOCALDIR}'' : This is the ''DST_ROOT'' one should prefer, files copied here will live on the '''CD''' '''ISO''' image (not be loaded into '''RAM'''), and will end up under the "'''/usr/local'''" directory once the system is booted. However, this ends up "relocating" a lot of executables (for example, "bison" is found under "'''/usr/bin/bison'''" on a Fedora system, but under "'''/usr/local/bin/bison'''" on a '''NST''' system). Sometimes, one needs to add symlinks to work around this. | |||
; ''${NSTUTILSDIR}'' : this directory is tarred up initially, but after boot, it is extracted to the '''RAM''' disk area. This directory is used for "'''/etc'''" files (any file which may need to be modified), or adding helper symbolic links. In our example above, we used this variable to add two symbolic links. | |||
; ''${NSTINITRDDIR}'' : this directory is for critical files that need to be available in the initial '''RAM''' disk. These files are used to boot the system and mount the '''CDROM''' drive. Typically you will want to avoid this area. | |||
=== Test The Installation === | |||
The following method can be used to test the installation of the '''bison''' package WITHOUT building the entire '''NST''' (the ''clean'' target will cause the command to execute slowly the first time it is invoked after building an entire '''ISO''' - it will run much more quickly on subsequent invocations). | |||
<div class="screen"> | |||
<div class="screenTitle">Testing The Install</div> | |||
<div class="userInput"><span class="prompt">[root@localhost ~]# </span>make -C src clean install PKGS=bison</div> | |||
<div class="computerOutput"> | |||
make: Entering directory `/root/nstdev/nst/src' | |||
NST: Package Name Category State | |||
----------------------------------------------------------------- | |||
Adding package: bison DEVELOPMENT [ok] | |||
Running lddcheck on /root/nst/tmp/iso/nst_utils... | |||
Running lddcheck on /root/nst/tmp/iso/custom... | |||
Creating the initrd libs "ld.so.cache" file... | |||
Installation complete, review /root/nst/src/make.log for details... | |||
*** Time: 0:02.82 *** To install packages | |||
make: Leaving directory `/root/nstdev/nst/src' | |||
</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>(cd tmp/iso/nst_utils; find)</div> | |||
<div class="computerOutput"> | |||
. | |||
./share | |||
./share/aclocal | |||
./bin | |||
./bin/bison | |||
./etc | |||
./etc/ld.so.conf.d | |||
./etc/ld.so.conf.d/nst.conf | |||
./etc/ld.so.conf | |||
</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span>(cd tmp/iso/custom; find local)</div> | |||
<div class="computerOutput"> | |||
local | |||
local/share | |||
local/share/bison | |||
local/share/bison/glr.cc | |||
local/share/bison/glr.c | |||
local/share/bison/m4sugar | |||
local/share/bison/m4sugar/m4sugar.m4 | |||
local/share/bison/README | |||
local/share/bison/location.cc | |||
local/share/bison/c.m4 | |||
local/share/bison/c++.m4 | |||
local/share/bison/yacc.c | |||
local/share/bison/lalr1.cc | |||
local/share/doc | |||
local/share/doc/bison-2.3 | |||
local/share/doc/bison-2.3/AUTHORS | |||
local/share/doc/bison-2.3/README | |||
local/share/info | |||
local/share/info/bison.info.gz | |||
local/share/man | |||
local/share/man/man1 | |||
local/share/man/man1/bison.1.gz | |||
local/bin | |||
local/bin/bison | |||
</div> | |||
<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div> | |||
</div> | |||
The two '''find''' commands issued above are not necessary, but give one a good idea of what files were installed by the new: "'''src/packages/custom/bison.sh'''" script. | |||
=== Some Trouble Shooting Tips === | |||
Don't be surprised if things don't work the first time. There are several things that may cause problems: | |||
* Not copying enough files from the package. | |||
* Not installing dependencies (running '''ldd''' on your binaries can help track down the missing ones). | |||
* Not creating the necessary symbolic links (not everythine will relocate nicely to the '''/usr/local''' directory - you may need to add some symbolic links to work around this). | |||
* Placing a configuration file (or some other file which needs to be writable) under '''/usr/local''' on the '''NST''' system (sometimes you need to relocate the file to the ${NSTUTILSDIR} and sometimes you might find that you need to relocate AND add symbolic links). |
Latest revision as of 09:56, 22 September 2010
NOTE: This page is outdated and should no longer be used as a reference for building a NST development system. If you are interested in putting together a system for the purpose of building the NST distribution from source, please refer to the Developer Info section.
Preparing The Build Machine
You will need to install Fedora 7 (the i386 variant) onto your development system.
If you want to build a 64 bit variant (x86_64) of the NST, you will need to download the x86_64 variant of Fedora 7. While the i386 variant will run on both 32 bit and 64 bit machines, the x86_64 variant will only run on 64 bit machines. Most, if not all, 64 bit AMD processors and the Intel Core 2 series should be compatible with the x86_64 variant.
If you don't want to use physical hardware, you can use a Virtual Machine (even running under Windows) to do your NST development under. We have found that either VMWare Server (our preference) or VMWare Player from VMware can be used (even if you want to build a x86_64 version of the NST). We have also found that Virtual PC can be used.
See the Fedora Installation Notes for additional information on installing the base OS.
Getting the NST Source Code
There are several ways to acquire the source code for NST developement (one can find an entire section related to this in the NST FAQ).
For our purposes we will choose the anonymous CVS approach. This permits one to build a current snapshot of the NST distribution as it is being developed.
The following demonstrates how one can create a: nst sub-directory on a Fedora Core 6 based system, and then populate it with the current NST source code (NOTE: Just press the Enter key when prompted for the password as there is no password required for anonymous access):
Logging in to :pserver:anonymous@nst.cvs.sourceforge.net:2401/cvsroot/nst CVS password:
... You should see file names streaming by in your console window - it may take several moments to download all of the source files ...
As time goes on, you may want to update your source files (the NST developers check in new code quite frequently). Use the following commands:
... You should see directory names streaming by in your console window as CVS searches for updated files ...
As setting the CVSROOT variable can be quite tedious, it is recommended that you include its definition in your: "~/.bashrc" file. Add the following lines to the end of: "~/.bashrc".
CVSROOT=":pserver:anonymous@nst.cvs.sourceforge.net:/cvsroot/nst"; export CVSROOT;
Configuring The System
After downloading the source code, you will need to run the: configure command from the top level directory. Before doing so, here are a couple of things to note:
- Before configuring the system, it is recommend that one run: "yum update" to make sure the latest kernel and package updates are installed.
- Sun's Java Run Time Environment (JRE) is used when building the NST. The configure script will attempt to download and install it automatically (if it isn't found on your development system). If the script is unable to automatically install it, you will need to download and install the JRE from: http://java.com/ (get the appropriate RPM version).
- You will probably need to run the configure multiple times the first time you setup your development system (as you will most likely be missing some packages).
- If configure determines that there are missing packages, it will indicate the yum command which you need to run in order to add the necessary packages to your system (or at least indicate which packages need to be added).
Here's an example of what one might go through when trying to initially configure a development system (NOTE: In this example, yum found that the current packages installed on the system were up to date):
Loading "installonlyn" plugin Setting up Update Process Setting up repositories core 100% |=========================| 1.1 kB 00:00 updates 100% |=========================| 1.2 kB 00:00 extras 100% |=========================| 1.1 kB 00:00 Reading repository metadata in from local files primary.xml.gz 100% |=========================| 1.6 MB 00:11 extras : ################################################## 5097/5097 No Packages marked for Update/Obsoletion
***ERROR*** unable to find executable program 'docbook2html' on system ***ERROR*** unable to find executable program 'docbook2pdf' on system ***ERROR*** unable to find executable program 'make' on system ***ERROR*** unable to find executable program 'mkzftree' on system ***ERROR*** unable to find executable program 'ncftpput' on system ***ERROR*** unable to find executable program 'pear' on system ***ERROR*** unable to find executable program 'rpmbuild' on system ***ERROR*** unable to find executable program 'svn' on system Try the following to add the missing packages: yum install docbook-utils docbook-utils-pdf make zisofs-tools ncftp php-pear rpm-build subversion
... Lot's of yum output as it figures out package dependencies, then you will be prompted. Enter 'y' when you see:
Checking for DocBook XSL [ OK ] Verifying settings [ OK ] Creating: "./config/config.sh" [ OK ] ... Omitting Output. This time configure found the system up to date and completed normally ... Use "make" to build src/nst-1.7.0.iso.gz, or "make help" to see additional targets. NOTE: You may want to run "make package-update" to update MANY of the packages needed for the NST. You can then run "make package-check" to see what is left to do by hand (there are many helper scripts under "src/bin" to aid in the package install/update process).
Installing Packages
At this point you might be tempted to run: "make" and produce a bootable ISO image. However, you would be missing MANY of the extra packages which are typically included in a NST distribution.
To get "most" of the extra packages installed onto your development system, you can use the: "make package-update" command as shown below (NOTE: This may take hours to complete):
... First a check will be made for the RPM packages which can be installed or updated via yum (you will need to answer yes if any packages are found ... ... After the yum installation of RPM packages completes, most of the packages defined in the include/data/packages.tsv (or include/data/packages.x86_64.tsv) file will be installed using custom scripts (some will be binary installs and some will build from source) ... ------------------------------------------------------------------------------- ***ERROR*** Following package updates failed: snort unicornscan wireshark autohide colorful_tabs dictionarysearch image_zoom LinkChecker long_titles quickrestart reloadevery tidy webdeveloper Check the log files under: /root/nst/tmp/updates for details ------------------------------------------------------------------------------- make: *** [tsv-update] Error 1
The above output shows that several of the "automatic" package installations failed. This typically occurs for one of the following reasons:
- The package required that a X desktop be running. All of the firefox add-ons fall into this category (autohide, colorful_tabs, ...).
- The package is no longer available. This often occurs when a new release of a package comes out and indicates that the file: "include/data/packages.tsv" needs to be updated. This is what happened with snort (they were at release 2.7.0 where as the "include/data/packages.tsv" file was configured for 2.6.1.5).
- A library was not found because the ldconfig command needs to be run (this is what caused the issue with wireshark shown in the above output).
- The package has dissappeared, moved, or no longer builds cleanly (this is what occurred with unicornscan in the output shown above).
When you encounter packages which fail to install cleanly, you should be able to review the entire log related to the installation by looking in the: "tmp/updates" directory (for example, the file: "tmp/updates/unicornscan.log" contained many error messages).
While the "make package-update" will automate the installation of 98% of the packages needed to build a full NST system, there are a handful of packages which require user interaction to complete the installation. For each of these packages, you should find an appropriately named script under the: "src/bin" directory.
To see what packages still need to be installed, the "make package-check" command can be used. The following demonstrates how to identify which packages still need to be installed, and then installs the ieee80211 package:
autohide (version 1.1.5) [failed] colorful_tabs (version 2.0.1) [failed] dictionarysearch (version 2.0.1) [failed] ieee80211 (version 1.2.16) [failed] image_zoom (version 0.2.7) [failed] ipw2200 (version 1.2.1) [failed] LinkChecker (version 0.6.1) [failed] long_titles (version 1.2.4) [failed] madwifi (version 0.9.3) [failed] metasploit (version 2.7) [failed] nessus (version 2.2.9) [failed] netw (version 5.34.0) [failed] ntop (version 3.3) [failed] quickrestart (version 1.1.0) [failed] reloadevery (version 2.0) [failed] snort (version 2.6.1.5) [failed] snort_inline (version 2.6.1.5) [failed] snort-mysql (version 2.6.1.5) [failed] snorter (version 2.1) [failed] tidy (version 0.8.3.9) [failed] vmware-tools (version 1.0.3) [failed] webdeveloper (version 1.1.4) [failed]
... Lots of output followed by a series of questions you must answer, followed by the build/install of the package ... ------------------------------------------------------------------------------- SUCCESS: Successfully installed ieee80211 ------------------------------------------------------------------------------- NOTE: You may need to re-build modules which depended upon this one. Look for failures when you run "make package-check".
ieee80211 (version 1.2.16) [ok]
Building a ISO Image
To build the NST ISO image, one runs the following command:
... You will see a lot of output and it will take awhile to build ...
-rw-r--r-- 1 root root 509790208 2007-08-25 17:27 src/nst-1.7.0.iso
You will always see a [warn] indicator when the java package is installed. At the time of this writing, you may also expect to see a [warn] indicator for the cups and clamav packages (this should be resolved prior to the final release). Other packages flagged by a [warn] and/or [failed] indicator, would indicate that the layout of the package has changed since its installation script was last updated (the NST developers will get to it before the next release).
Now that a ISO image has been produced, one can do any of the following:
- Burn the image to a CD or DVD (we've had good luck with CDRW media during our development). You might be able to use the: "make cd", "make cdrw" and/or "make dvd+rw" (see the output of: "make help" for more information). NOTE: If you are developing inside a virtual machine, you probably won't be able to burn a disk immediately. Instead, you'll need to transfer the image to a different system which has a burner.
- Transfer the ISO image to a different machine (we typically use scp, if you are a Windows user, you may want to consider WinSCP).
- Boot directly from the ISO image using a VMware product or Virtual PC. NOTE: At the time of this writing, there is an issue with NST and the recommended virtual LSI SCSI controller in VMware Server (beta testers should opt for the BusLogic controller when creating new virtual machines for the time being).
Tweaking The Configure/Build Process
During the configuration and build process, several files in the: "${HOME}/.nst" directory are examined. These files are optional (you don't have to create any of them). However, creating these files, allows one to customize the results of configuring and building the NST ISO image without having to specify a lot of command line options to the configure script.
The top level: "README" file contains a lot of information about the various configuration options available.
The Magic "${HOME}/.nst" Directory
By default, the configuration and build process will look under the: "${HOME}/.nst" directory for build customization files. You can append the: "--config-dir DIRECTORY" option to the ./configure command if you would like to use a different directory.
"${HOME}/.nst/configure.sh"
The first file you will likely want to create is the: "${HOME}/.nst/configure.sh" file. This file can be used to set MANY of the available options supported by the top level ./configure script.
# Set a custom password and disable prompt to set the initial password NSTPASSWD="MY_NST_PASSWORD"; NSTPASSWD_PROMPT="false"; # Uncomment if you want the extra development packages added (src/packages/devextra) #PKGCATS="${PKGCATS} devextra"; # Uncomment if you want the "extra" disk packages added (src/packages/diskextra) #PKGCATS="${PKGCATS} diskextra"; # Where source code is downloaded to and built under BUILDROOT="/usr/local/src"; # Default boot mode NSTBOOT="desktop"; # Set to true if you want to build the NST documents, false for ISO ON_LINE_DOCS="false"; # Compress files so ISO will likely be small enough for a CD? NSTCOMPRESSED_ISO="true"; # Take time to strip executables to produce a smaller ISO image? NSTSTRIPPED="true";
There are many other settings you can add to this file, but those shown above will cover 95% of what you'll want to tweak. After you run the configure script, you can check the resulting configuration by looking at the file: "config/config.sh" (the top portion of this file can also be used as a reference for other variables you might wish to set).
NOTE: You will need to re-run the ./configure script after making changes to this file. Also, make sure you set the permissions of this file to: 0600 - especially if you decide to put a clear text password in it.
"${HOME}/.nst/disable.txt"
As time has passed, many packages have been added and removed from the NST distribution. As packages are removed from the NST distribution, they are often added to the: "include/data/disable.txt" file to indicate that they should not be included in the build.
During the build process, the file: "${HOME}/.nst/disable.txt" will be searched for before falling back to: "include/data/disable.txt". This allows one to customize what packages are omitted from the build.
If you are interested in omitting packages (or categories), copy the file: "include/data/disable.txt" to: "${HOME}/.nst/disable.txt" and then edit it's contents. You should find comments at the top of the file indicating how one disables additional categories and/or packages.
"${HOME}/.nst/post_configure.sh"
This script is sourced (if it exists) after each invocation of the ./configure command. It allows one to perform "post configuration" updates to the build environment. Very few people will have need to implement this script. But if you do, your script will have access to all of the variables found in: config/config.sh (useful for locating files making up the system).
"${HOME}/.nst/post_install.sh"
This script is run (if it exists) after all of the packages have been installed during the build process (when running: make). This script can be VERY useful to customizing your resulting NST ISO.
The following example, does the following:
- Defines some function which will adjust files in the build area prior to forming the ISO image.
- Invokes a subset of the defined functions for the features we want installed.
# # Customized files PRIOR to building ISO # # install_custom_isolinux_cfg # # install_custom_isolinux_cfg() { if [ -d "${NSTHOMEDIR}/base/isolinux" -a "${CFGDIR}/isolinux.cfg" ]; then ${CP} -f "${CFGDIR}/isolinux.cfg" "${NSTHOMEDIR}/base/isolinux"; fi } # add_setup_home # # Adds helper script "/root/bin/setup_home", to configure probe for home network add_setup_home() { create_dirs "${NSTUTILSDIR}/root/bin" cat >| "${NSTUTILSDIR}/root/bin/setup_home" <<EOF #!/bin/bash # # Detect/add swap space # auto_add_swap # # Enable NTP # /etc/rc.d/init.d/ntpd start >/dev/null 2>&1 & # # Enable home proxy # nstsetproxy --http-host 192.168.12.1 --http-port 3128 # # Mount NFS drives # service portmap start mount -t nfs 192.168.12.1:/home /home > /dev/null 2>&1 & if [ ! -d /lan ]; then mkdir /lan fi mount -t nfs 192.168.12.1:/opt /lan > /dev/null 2>&1 & EOF chmod +x "${NSTUTILSDIR}/root/bin/setup_home" } # config_ssh # # Add custom .ssh configuration (authorized_keys in particular) config_ssh() { if [ -d "${NSTINITRDDIR}" ]; then ${RM} -fr "${NSTINITRDDIR}/root/.ssh" fi create_dirs "${NSTUTILSDIR}/root/.ssh" ${CHMOD} 700 ${NSTUTILSDIR}/root/.ssh for f in authorized_keys config id_rsa id_dsa; do if [ -e "${HOME}/.ssh/${f}" ]; then ${CP} -p "${HOME}/.ssh/${f}" "${NSTUTILSDIR}/root/.ssh" fi done } # # Enable CD eject on shutdown # enable_cd_eject() { create_dirs "${NSTUTILSDIR}/sbin" ${LN} -s /usr/bin/eject ${NSTUTILSDIR}/sbin/halt.local } # # Call features we want # install_custom_isolinux_cfg; # enable_cd_eject; config_ssh; add_setup_home;
You can use this to customize pretty much anything in the build (add other packages, configure your network setttings, install wireless keys, etc).
As this file is sourced, you will have access to all of the variables defined in: "config/config.sh". Those that will be of most interest include:
- ${NSTLOCALDIR}
- This is the root directory which will appear as: /usr/local once the system has booted off of the Live CD. All files placed under here will be READ ONLY (don't put a configuration file which is likely to be editted here). This is where most executables and libraries are placed.
- ${NSTUTILSDIR}
- This directory contains a collection of files that will be copied to RAM once the Live CD boots and mounts the CD. Configuration files, small scripts, frequently used tools are often placed here. Be aware that any file placed here will consume RAM as the system runs.
- ${NSTINITRDDIR}
- This is the root directory which contains the files and directories which will end up in the initial RAM disk when booting from the Live CD. It typically contains small scripts, configuration files and enough executables and modules to get the CDROM mounted. You should avoid adding large files to this area.
Adding Packages To The Build
During the build process (when you type: make), package installation scripts for the various package categories under: src/packages are run. These scripts transfer information from the development system to the directory heirarchy that is used to form the final ISO image.
Adding Files From a RPM Package
The easiest packages to add to a NST build are from packages which are installed on the development system from RPM files (typically via: yum).
To add additional packages to the NST, it is recommended to use the following steps:
- Create a: "src/packages/custom" directory (for your custom additions to the NST).
- Update your configuration file such that the custom category is added to the: ${PKGCATS} category list.
- Install the package onto your development system.
- Add a script ending with the .sh extension for each package you want to add. For example, to add the bison package to the distribution, one would want to create a: "src/packages/custom/bison.sh" installation script.
The following outlines the steps involved to add bison to the distribution under the custom category.
Creating the Category Directory
This step is trivial:
NOTE: If you want to keep your work separate from the NST area, you may want to use a symbolic link for: "src/packages/custom" instead of creating the directory directly in the NST heirarchy.
Adding The Custom Directory To Your Build
First add the following lines to the bottom of your: "${HOME}/.nst/configure.sh" configuration file:
# Following will cause the NST build to process ALL scripts found under the # src/packages/custom directory when building the ISO PKGCATS="${PKGCATS} custom";
Then run the: "configure" script and verify that the category was added:
... Omitting the initial output from running configure ...
PKGCATS: boot boot2 bootnet system time printing db x gnome server
development php java applications networking wireless dockapps recovery sound custom
SFUSER: pblankenbaker SNORTRSCACHENAME: snort_rs_cache SOURCE_FORGE_MIRROR: http://easynews.dl.sourceforge.net/sourceforge WEBHOST_DOMAIN: WEBHOST_USER:
Use "make" to build src/nst-1.7.0.iso.gz, or "make help" to see additional targets.
NOTE: You may want to run "make package-update" to update MANY of the packages needed for the NST. You can then run "make package-check" to see what is left to do by hand (there are many helper scripts under "src/bin" to aid in the package install/update process).
The important thing to notice is the presence of: "custom" in the list of categories next to the: PKGCATS: label above.
Install The "bison" Package
For this example, we'll use yum to install the bison package and then get a listing of the files in the package. It should be noted that the use of yum is not required, this example only requires that the package is installed on the development system from a RPM file.
... output from yum as bison is downloaded and installed on your development system ...
/usr/bin/bison /usr/share/aclocal/bison-i18n.m4 /usr/share/bison /usr/share/bison/README /usr/share/bison/c++.m4 /usr/share/bison/c.m4 /usr/share/bison/glr.c /usr/share/bison/glr.cc /usr/share/bison/lalr1.cc /usr/share/bison/location.cc /usr/share/bison/m4sugar /usr/share/bison/m4sugar/m4sugar.m4 /usr/share/bison/yacc.c /usr/share/doc/bison-2.3 /usr/share/doc/bison-2.3/AUTHORS /usr/share/doc/bison-2.3/ChangeLog /usr/share/doc/bison-2.3/NEWS /usr/share/doc/bison-2.3/OChangeLog /usr/share/doc/bison-2.3/README /usr/share/doc/bison-2.3/THANKS /usr/share/doc/bison-2.3/TODO /usr/share/info/bison.info.gz /usr/share/locale/da/LC_MESSAGES/bison.mo /usr/share/locale/de/LC_MESSAGES/bison.mo /usr/share/locale/es/LC_MESSAGES/bison.mo /usr/share/locale/et/LC_MESSAGES/bison.mo /usr/share/locale/fr/LC_MESSAGES/bison.mo /usr/share/locale/ga/LC_MESSAGES/bison.mo /usr/share/locale/hr/LC_MESSAGES/bison.mo /usr/share/locale/id/LC_MESSAGES/bison.mo /usr/share/locale/it/LC_MESSAGES/bison.mo /usr/share/locale/ja/LC_MESSAGES/bison.mo /usr/share/locale/ms/LC_MESSAGES/bison.mo /usr/share/locale/nb/LC_MESSAGES/bison.mo /usr/share/locale/nl/LC_MESSAGES/bison.mo /usr/share/locale/pl/LC_MESSAGES/bison.mo /usr/share/locale/pt_BR/LC_MESSAGES/bison.mo /usr/share/locale/ro/LC_MESSAGES/bison.mo /usr/share/locale/ru/LC_MESSAGES/bison.mo /usr/share/locale/rw/LC_MESSAGES/bison.mo /usr/share/locale/sv/LC_MESSAGES/bison.mo /usr/share/locale/tr/LC_MESSAGES/bison.mo /usr/share/locale/vi/LC_MESSAGES/bison.mo /usr/share/man/man1/bison.1.gz
Creating "src/packages/custom/bison.sh"
Having seen the list of files which belong to the bison package, we'll create the installation script: "src/packages/custom/bison.sh" as shown below:
# $Id$ # Description: This script installs the bison package. # Vars: # ----- PKG="${PNAME}"; # Code: # ----- # Copy files installed from RPM to /usr/local area on ISO (we'll put # include most of the files from the package, but omit the locale # specific stuff and some of the /usr/share/doc stuff to reduce the # package size some). check_rpm ${PKG} "${NSTLOCALDIR}" "/usr" \ "^/usr/bin/${PNAME}" \ "^/usr/share/${PNAME}" \ "^/usr/share/doc/${PNAME}.*/AUTHORS" \ "^/usr/share/doc/${PNAME}.*/README" \ "^/usr/share/info/" \ "^/usr/share/man/"; # Add a symlink so "bison" can be found as /usr/bin/bison as well # as under "/usr/local/bin/bison" and "/usr/share/aclocal" is the # same as "/usr/local/share/aclocal" create_dirs "${NSTUTILSDIR}/bin" "${NSTUTILSDIR}/share"; ${LN} -s "/usr/local/bin/bison" "${NSTUTILSDIR}/bin/bison"; ${LN} -s "/usr/local/share/aclocal" "${NSTUTILSDIR}/share/aclocal";
The check_rpm function does most of the work in copying files from the development system to the proper directory for the build. The following section describes this function:
check_rpm PKG_NAME DST_ROOT STRIP_PATH MATCH0 [MATCH1 ...]
The check_rpm function works in the following manner:
- It gets a list of possible source files belonging to PKG_NAME (A "rpm -ql bison" in this case)
- For each matching file ("^/usr/bin/bison" matches a single file whereas "^/usr/share/bison" matches many files and directories), it forms a destination file or directory name by stripping off STRIP_PATH ("/usr" in our case, so "/usr/bin/bison" becomes "/bin/bison"), and then prepends DST_ROOT ("/bin/bison" becomes "${NSTLOCALDIR}/bin/bison").
The tricky part is learning which DST_ROOT variable to use, we have 3:
- ${NSTLOCALDIR}
- This is the DST_ROOT one should prefer, files copied here will live on the CD ISO image (not be loaded into RAM), and will end up under the "/usr/local" directory once the system is booted. However, this ends up "relocating" a lot of executables (for example, "bison" is found under "/usr/bin/bison" on a Fedora system, but under "/usr/local/bin/bison" on a NST system). Sometimes, one needs to add symlinks to work around this.
- ${NSTUTILSDIR}
- this directory is tarred up initially, but after boot, it is extracted to the RAM disk area. This directory is used for "/etc" files (any file which may need to be modified), or adding helper symbolic links. In our example above, we used this variable to add two symbolic links.
- ${NSTINITRDDIR}
- this directory is for critical files that need to be available in the initial RAM disk. These files are used to boot the system and mount the CDROM drive. Typically you will want to avoid this area.
Test The Installation
The following method can be used to test the installation of the bison package WITHOUT building the entire NST (the clean target will cause the command to execute slowly the first time it is invoked after building an entire ISO - it will run much more quickly on subsequent invocations).
make: Entering directory `/root/nstdev/nst/src'
NST: Package Name Category State
Adding package: bison DEVELOPMENT [ok]
Running lddcheck on /root/nst/tmp/iso/nst_utils...
Running lddcheck on /root/nst/tmp/iso/custom... Creating the initrd libs "ld.so.cache" file... Installation complete, review /root/nst/src/make.log for details...
- Time: 0:02.82 *** To install packages
make: Leaving directory `/root/nstdev/nst/src'
. ./share ./share/aclocal ./bin ./bin/bison ./etc ./etc/ld.so.conf.d ./etc/ld.so.conf.d/nst.conf ./etc/ld.so.conf
local local/share local/share/bison local/share/bison/glr.cc local/share/bison/glr.c local/share/bison/m4sugar local/share/bison/m4sugar/m4sugar.m4 local/share/bison/README local/share/bison/location.cc local/share/bison/c.m4 local/share/bison/c++.m4 local/share/bison/yacc.c local/share/bison/lalr1.cc local/share/doc local/share/doc/bison-2.3 local/share/doc/bison-2.3/AUTHORS local/share/doc/bison-2.3/README local/share/info local/share/info/bison.info.gz local/share/man local/share/man/man1 local/share/man/man1/bison.1.gz local/bin local/bin/bison
The two find commands issued above are not necessary, but give one a good idea of what files were installed by the new: "src/packages/custom/bison.sh" script.
Some Trouble Shooting Tips
Don't be surprised if things don't work the first time. There are several things that may cause problems:
- Not copying enough files from the package.
- Not installing dependencies (running ldd on your binaries can help track down the missing ones).
- Not creating the necessary symbolic links (not everythine will relocate nicely to the /usr/local directory - you may need to add some symbolic links to work around this).
- Placing a configuration file (or some other file which needs to be writable) under /usr/local on the NST system (sometimes you need to relocate the file to the ${NSTUTILSDIR} and sometimes you might find that you need to relocate AND add symbolic links).