diff --git a/misc/jenkins/README.org b/misc/jenkins/README.org
index 4555109..e484893 100644
--- a/misc/jenkins/README.org
+++ b/misc/jenkins/README.org
@@ -1,208 +1,216 @@
* Notes
** Overview
|----------------------+--------+---------+-----------+-----+-----------|
| Configuration Matrix | native | in-tree | sanitizer | w32 | distcheck |
|----------------------+--------+---------+-----------+-----+-----------|
| arch | x | | | | |
| debian | x | | | | |
| macos | x | | | | |
| master | | x | x | x | x |
| openbsd60 | x | | | | |
|----------------------+--------+---------+-----------+-----+-----------|
There are two dimensions, build host ("label") and build profile
("XTARGET"). The build hosts are described below. The "debian" label
is the same configuration as "master".
bin/build.bash is the build script. It creates a suitable build
environment, builds, tests, and installs all our packages. The
different build profiles are implemented there.
*** Build profiles
**** native
A straight forward out-of-tree build.
**** in-tree
A straight forward in-tree build.
**** sanitizer
A build with -fsanitize=undefined -fsanitize=address. This catches
many memory errors by instrumenting the code and running the test
suites.
**** w32
Cross-compile the package for Windows, run the tests using a virtual
machine.
**** distcheck
Executes 'make distcheck'. Makes sure that we can always create
releases.
** Setting up a Jenkins build slave
- on soro, create an entry in /etc/hosts
- copy root@soro's ssh key to /root/.ssh/authorized_keys
- install a jre, make, autoconf, automake, libtool, gcc, git, bison,
fig2dev, ghostscript, gnutls, sqlite3, pkg-config, imagemagick,
rngd, python2/3, SWIG, Qt5 base, ccache
- setup rngd (test suites will consume quite a bit of entropy)
- create a user jenkins
- clone gnupg-doc
$ git clone git://git.gnupg.org/gnupg-doc.git
- link ~/bin
$ ln -s gnupg-doc/misc/jenkins/bin
- download slave.jar
$ wget https://jenkins.gnupg.org/jnlpJars/slave.jar -O bin/slave.jar
Note:
The jar should be updated from time to time, but the documentation
says that the protocol changes rarely.
- copy and adapt launcher
$ cp bin/jenkins-slave.dist bin/jenkins-slave
- make sure that jenkins@soro can ssh to the new node
- go to https://jenkins.gnupg.org/computer/new and copy an existing
configuration, adapting it as needed
- setup 'GPGME tests for GnuPG' as described below
- for each project, add the new nodes distinct label to the
configuration matrix, and force a rebuild. Start with libgpg-error
and walk your way up the dependency chain:
- libgpg-error, libnpth, libassuan, libksba, libgcrypt, ntbtls,
gnupg, gpgme
** GPGME tests for GnuPG
There is a reasonably up-to-date (but this is currently a manual
process) GPGME source at
$ mkdir $HOME/src
$ git clone git://git.gnupg.org/gpgme.git $HOME/src/gpgme-for-gnupgs-tests
$ cd $HOME/src/gpgme-for-gnupgs-tests
$ ./autogen.sh
and a build tree at
$ mkdir $HOME/src/gpgme-for-gnupgs-tests/obj
$ cd $HOME/src/gpgme-for-gnupgs-tests/obj
$ export PATH=$HOME/prefix/native/bin:$PATH
$ ../configure --enable-maintainer-mode
$ make
$ make check
and specialized build trees, e.g. for the sanitizer target at
$ mkdir $HOME/src/gpgme-for-gnupgs-tests/obj-sanitizer
$ cd $HOME/src/gpgme-for-gnupgs-tests/obj-sanitizer
$ export PATH=$HOME/prefix/sanitizer/bin:$PATH
$ ../configure --enable-maintainer-mode \
--enable-languages="cpp qt" \
CFLAGS="-fsanitize=undefined -fsanitize=address" \
CXXFLAGS="-fsanitize=undefined -fsanitize=address"
$ make
$ make check
and a w32 build tree at
$HOME/src/gpgme-for-gnupgs-tests/obj.w32
The tests from there are executed in GnuPG's test suite.
* Build hosts
** zygalski
Werner manages this box.
** Virtual machines
*** openbsd60
Packages installed:
# pkg_add zile zsh git autoconf-2.69p2 automake-1.15p0 gettext-tools \
gmake xfig bison readline libusb-compat ImageMagick makeinfo \
gcc-4.9.3p9 g++-4.9.3p9 qt5 python swig ccache
Add some compatibility links to PATH:
$ mkdir -p ~/compat/{bin,lib,include}
$ cd ~/compat/bin
$ ln -s /usr/local/bin/gmakeinfo makeinfo
$ [ add $HOME/compat/bin to ~/.profile ]
$ cd ~/compat/lib
$ for F in /usr/local/lib/libbz2* /usr/local/lib/libiconv* /usr/local/lib/libintl* ; do ln -s $F ; done
$ cd ~/compat/include
$ for F in /usr/local/include/*bz*h /usr/local/include/*iconv*h /usr/local/include/*intl*h ; do ln -s $F ; done
Tweak limits:
# echo 'jenkins:\
:maxproc-max=1024:\
:maxproc-cur=1024:\
:tc=pbuild:
' >> /etc/login.conf
# user mod -L jenkins jenkins
*** win8.1
Configuration: One user "gpg", disabled Windows update (cpu hog),
disable animations ("make computer easier to see again").
Installed applications: https://github.com/PowerShell/Win32-OpenSSH/releases/latest
**** How tests are executed
To run the test suite, the machine is reverted to the snapshot
'tests', and the tests are executed in-place from an ISO image.
See:
- bin/run-tests-w32.bash
- bin/run-tests.bat
- bin/make-windows-cd.sh
**** Win32-OpenSSH
The implementation seems a bit brittle at the moment. Often, the ssh
server will stop responding to requests, I do not know why.
To update the ssh server, follow
https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH
first uninstall the old one, then install the new one.
**** Updating & maintenance
Get a lock on bin/run-tests-w32.bash to avoid it stomping over your
changes:
jenkins@soro:~$ flock /var/lib/jenkins/bin/run-tests-w32.bash bash
Start the machine using
jenkins@soro:~$ virsh -c qemu:///system snapshot-revert --snapshotname tests --force --running win8.1
Connect to the machine from your desktop machine:
you@home $ virt-viewer -c qemu+ssh://jenkins@soro.g10code.com/system win8.1
-Do whatever maintenance work is necessary. Shutdown the machine.
-Create a new snapshot 'test-new':
+Do whatever maintenance work is necessary.
+
+Hints:
+ - Start a powershell:
+ Open the file explorer, enter 'powershell' into the location bar (ctrl-l).
+ - Right mouse button pastes in powershell.
+ - Elevate privileges in powershell:
+ > Start-Process powershell -verb runAs
+
+Shutdown the machine. Create a new snapshot 'test-new':
jenkins@soro:~$ virsh -c qemu:///system snapshot-create-as win8.1 --name "tests-new" --description "Updated OpenSSH to xxx"
Archive the current snapshot:
jenkins@soro:~$ virsh -c qemu:///system snapshot-edit win8.1 --snapshotname "tests" --rename
[... editor pops open, change "tests"
to "tests YYY-MM-DD", save, exit ...]
Note: The snapshots creation times can be found using:
jenkins@soro:~$ virsh -c qemu:///system snapshot-list win8.1 | grep tests
tests 2017-03-15 14:21:17 +0100 shutoff
tests 2017-01-31 2017-01-31 11:05:17 +0100 shutoff
Rename the new snapshot:
jenkins@soro:~$ virsh -c qemu:///system snapshot-edit win8.1 --snapshotname "tests-new" --rename
[... editor pops open, change "tests-new"
to "tests", save, exit ...]
Exit the shell to release the lock:
jenkins@soro:~$ exit
exit
Voila.
**** Ideas
- Build the installer, put it on the ISO image, and test that as well.
*** openindiana20161030
So I wanted the most alien UNIX I could get my hands on. I never
configured the build environment though, so this machine lies dormant.
**** Packages installed
pkg install pkg://openindiana.org/runtime/java/openjdk8 top git autoconf automake libtool bison readline
*** archlinux
**** Packages installed
pacman --sync zile bind-tools openssh zsh jre8-openjdk-headless git autoconf automake libtool make wget gcc bison fig2dev ghostscript gnutls sqlite3 pkg-config imagemagick librsvg rng-tools python swig qt5-base
**** Upgrading packages
Note: Some breakage can happen when upgrading the system. Arch users
deal with that by reading the website and following instructions
there.
pacman -Syu