Hudson

Locally, we use http://hudson-ci.org/ Hudson for the nightly builds.

Ptolemy and Kepler builds

The ptII nightly build is at http://sisyphus.eecs.berkeley.edu:8079/hudson/job/ptII/

The kepler nightly build is at http://sisyphus.eecs.berkeley.edu:8079/hudson/job/ptII/

Hudson has its own user database. Users can view the configuration for the builds (http://sisyphus:8079/hudson/job/ptII/configure).

To get an account, email Christopher

Updating Hudson

As root on sisyphus.

1. Probably a good idea to stop hudson first:

/etc/init.d/hudson stop

2. Run yum

yum --nogpgcheck update hudson

3. Restart

/etc/init.d/hudson stop

Install Hudson for RHEL 6.2

1. See http://wiki.hudson-ci.org/display/HUDSON/Installing+Hudson+using+RPM+repository

sudo wget -O /etc/yum.repos.d/hudson.repo http://hudson-ci.org/redhat/hudson.repo
sudo yum check-update
sudo yum install hudson

2. Log in to sisyphus with ssh -X sisyphus, run firefox and, on sisyphus, go to http://localhost:8080

3. Hudson Plugins to add

  • Maven 2
  • Maven 3
  • Cobertura
  • Email-ext
  • FindBugs

4. Update /etc/sysconfig/hudson:

HUDSON_PORT="8079"
HUDSON_HOME="/home/hudson"
HUDSON_ARGS="--prefix=/hudson"

Note that HUDSON_HOME has to be set to /home/hudson to find our preexisting ptII and kepler jobs.

5. /etc/init.d/hudson was not terminating. I'm not sure what was up here, I ended up adding a & to the line that invokes java. I think the problem is that --daemon is not supported in Hudson3.0.

daemon --user "$HUDSON_USER" --pidfile "$HUDSON_PID_FILE" $JAVA_CMD $PARAMS > /dev/null

to

daemon --user "$HUDSON_USER" --pidfile "$HUDSON_PID_FILE" $JAVA_CMD $PARAMS & > /dev/null

Getting Hudson to run on port 80 under RHEL

Typically, Hudson appears on port 8080, but since we are testing web applications, we moved it to :8079 and set up apache on sisyphus to redirect http://sisyphus.eecs.berkeley.edu/hudson to http://sisyphus.eecs.berkeley.edu:8079.

See http://www.zzorn.net/2009/11/setting-up-hudson-on-port-80-on-debian.html for Debian instructions. I'm running RHEL 5.8, so things were different.

The problem is that port 80 requires that applications run as root, which is a security issue for large Java applications. So I'm using an Apache server set up as a proxy.

Below are details about what we did:

1. Request that port 80 be opened up to sisyphus. We did this on https://iris.eecs.berkeley.edu/db/network/ and then "Update a Device"

2. Make sure that Apache is installed. We are running RHEL 5.8, so yum install httpd.x86_64

3. Edit /etc/httpd/conf/httpd.conf Important: Be sure to set allow only .berkeley.edu hosts, or else offsite visitors will use the machine as a proxy to access non-public library resources.

#
# Proxy Server directives. Uncomment the following lines to
# enable the proxy server:
#
<IfModule mod_proxy.c>
ProxyRequests On

<Proxy *>
Order deny,allow
Deny from all
Allow from .berkeley.edu
</Proxy>

#
# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
#
ProxyVia On

#
# To enable a cache of proxied content, uncomment the following lines.
# See http://httpd.apache.org/docs/2.2/mod/mod_cache.html for more details.
#
<IfModule mod_disk_cache.c>
CacheEnable disk /
CacheRoot "/var/cache/mod_proxy"
</IfModule>
#

</IfModule>
# End of proxy directives.

And

<VirtualHost *>
ServerAdmin cxh@eecs.berkeley.edu
DocumentRoot "/var/www/html"
ServerName sisyphus.eecs.berkeley.edu
ErrorLog logs/sisyphus-error_log
CustomLog logs/sisyphus-access_log common

ProxyPass /hudson http://sisyphus.eecs.berkeley.edu:8079/hudson
</VirtualHost>

3. Configure Hudson to use port 8079. Edit /etc/sysconfig/hudson and update HUDSON_PORT="8079" and HUDSON_ARGS="--prefix=/hudson". Then run /etc/init.d/hudson shutdown and then /etc/init.d/hudson start and check http://sisyphus.eecs.berkeley.edu:8079

4. Open up port 80, 443 and 8079 on sisyphus. As root, I ran system-config-firewall, which worked in a terminal and opened up the ports. /sbin/iptables -L should include something like:

root@sisyphus ~]# /sbin/iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:http
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:https
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:8079
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target prot opt source destination
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@sisyphus ~]#

5. Start apache: /etc/init.d/httpd start http://sisyphus.eecs.berkeley.edu/hudson should now work. If it does not, try http://sisyphus.eecs.berkeley.edu:8079/hudson.

Also, try creating a dummy page for the webserver, httpd.conf has a line like

DocumentRoot "/var/www/html"

Create a index.html file at that location:

<html>
<head>
<title>Sisyphus</title>
</head>
<body>
<h2>Sisyphus</h2>
</body>
</html>

and then visit http://sisyphus.eecs.berkeley.edu

If you are having firewall problems, try using wget on sisyphus to see if the page is up: wget http://sisyphus.eecs.berkeley.edu/ should download a index.html file. If it does, but that URL is not visible outside of the machine, then either the machine firewall or the dept. firewall is blocking traffic.

6. Enable httpd after a reboot:

[root@sisyphus tmp]# /sbin/chkconfig httpd on
[root@sisyphus tmp]# /sbin/chkconfig --list httpd
httpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
[root@sisyphus tmp]#

Set up users for hudson

http://wiki.hudson-ci.org/display/HUDSON/Standard+Security+Setup

Install modules

  • Cobertura Plugin
  • SetEnv

ptII job

Build Trigger: 5 0 * * *

Set environment variables:

PTII=${WORKSPACE}
LD_LIBRARY_PATH=${WORKSPACE}/lbnl/lib/util:${WORKSPACE}/lib:/usr/local/matlab-r2008a/bin/glnxa64:${WORKSPACE}/vendors/misc/dbxml-2.5.16/install/lib:/usr/local/lib:${LD_LIBRARY_PATH}
CLASSPATH=/usr/java/JMF-2.1.1e/lib/jmf.jar:/usr/java/jai-1_1_3/lib/jai_codec.jar:/usr/java/jai-1_1_3/lib/jai_core.jar:/usr/java/jai-1_1_3/lib/mlibwrapper_jai.jar:.
PATH=/usr/local/bin:${PATH}:${WORKSPACE}/vendors/misc/dbxml-2.5.16/install/bin:/usr/local/sbin:/usr/local/bin
TOSROOT=${WORKSPACE}/vendors/ptinyos/tinyos-1.x
TOSDIR=${WORKSPACE}/vendors/ptinyos/tinyos-1.x/tos
PTINYOS_MOMLROOT=${WORKSPACE}/vendors/ptinyos/moml

Other installations

Below are other packages that needed to be installed.

sudo yum install expat-devel.i686
sudo yum install expat-devel.x86_64

Running svn from the command line fails because Hudson uses Subversion 1.7

bash-4.1$ svn update
svn: The path '.' appears to be part of a Subversion 1.7 or greater
working copy. Please upgrade your Subversion client to use this
working copy.
bash-4.1$ which svn
/usr/bin/svn
bash-4.1$ svn --version
svn, version 1.6.11 (r934486)
compiled Apr 12 2012, 11:09:11

Copyright (C) 2000-2009 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
This product includes software developed by CollabNet (http://www.Collab.Net/).

The following repository access (RA) modules are available:

* ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
- handles 'http' scheme
- handles 'https' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
- with Cyrus SASL authentication
- handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
- handles 'file' scheme

bash-4.1$

The solution is to find a more recent version of Subversion for RHEL 6.2.

I ended up installing from Collabnet: http://www.collab.net/downloads/subversion

I had to create a link:

cd /usr/local/bin
ln -s /opt/CollabNet_Subversion/bin/* .

Email from Hudson is failing

On January 27, I added my email address to the hudson list

On January 30, hudson had this log message, but ptXX did not

Sending email for trigger: Still Unstable
Sending email to: ptXXX cxh@eecs.berkeley.edu

On January 31, I started seeing messages like:

ERROR: Could not send email as a part of the post-build publishers.
^[[8mha:AAAAWB+LCAAAAAAAAABb85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=^[[0mjavax.mail.SendFailedException: Invalid Addresses;
nested exception is:
com.sun.mail.smtp.SMTPAddressFailedException: 554 <<ealt@XXX.berkeley.edu>>... Relay operation rejected
;
nested exception is:
com.sun.mail.smtp.SMTPAddressFailedException: 554 <<cBRXXXX>>... Relay operation rejected