Latest News

The status can be seen here on http://vp-continuous.cirad.fr

Continuous integration for all of VirtualPlant's projects is handled by BuildBot. Buildbot is structured in a master and slaves way.

• Master : Decides which tasks (aka builder) is to be executed by whom (aka slave) and when (aka sheduler)
• Slave : Only in charge of registering itself to the master and then executing the tasks the master sends to it.

The slave can by installed on any system supported python. This way we can use it to run compilation/tests/… on many different systems.

The slave can be any machine that can see the master through a network connection, and any OS that supports Python, on a physical on a virtual machine.

On the machine hosting the slave:

buildbot create-slave basedir host:port slave passwd.

The command is the same as for the server except we might not have virtual python environments.

cd Devlp/slave
buildbot [re]start .

It is a good idea to automatically start the buildbot slave when the system starts up. The following sections show how to setup boot scripts for various systems. Be careful with the paths, fix them as needed!

Create the /etc/init/buildbot_slave.conf file:

sudo vi /etc/init/buildbot_slave.conf

Then type in:

# We want the slave to start after ''network-interface'' 
# is started (so that it can contact the master).
start on started network-interface
 
# Print errors to the console
console output
 
# start-stop-daemon will ''start'' the executable given by ''--exec''  
# as a background (''-b'') service with the identity of the 
# user given by ''--chuid''
script
	start-stop-daemon --start --chuid buildmaster -b --exec /usr/bin/buildbot -- restart ~/devlp/slave/
end script

See RunningBuildbotOnWindows

We have a build server running VirtualBoxed systems called Mango. This section will not present how to install an OS on a VB but how to make easier to use as a build system.

Like we did here, we will extend or create the /etc/init/vplants_vboxes.conf file.

# Start one the network in running
start on started network-interface
 
# print things to console
console output
 
# Start the virtual machines as headless (no gui) background services.
# Each VM will watch a different host port for RDP connections.
script
        start-stop-daemon --start --chuid buildmaster -b --exec /usr/bin/VBoxHeadless -- -startvm "FC15-i686" -v on -e "TCP/Ports"=3389
 
        start-stop-daemon --start --chuid buildmaster -b --exec /usr/bin/VBoxHeadless -- -startvm "Ubuntu-11.04x64" -v on -e "TCP/Ports"=3390
 
        start-stop-daemon --start --chuid buildmaster -b --exec /usr/bin/VBoxHeadless -- -startvm "win7_32" -v on -e "TCP/Ports"=3391
end script

Since the machines are started user VBoxHeadless, they are accessible using rdesktop on linux or Remote Desktop Connxion (Connexion Bureau à distance) on Windows.

In the following examples, PORT refers to the port numbers given to the machines in the previous section.

For Linux

rdesktop mango.cirad.fr:PORT

For Windows

Start>>Programs>>Accessories>>Remote Desktop Connection
# Now type mango.cirad.fr:PORT in the server (or computer) field.

You can also setup ssh forwarding from host to guest like on this blog post. If you set the forwarding from host port PORT to UbuntuXX guest and want an ssh shell to UbuntuXX:

ssh username@mango.cirad.fr:PORT''

Windows can support SSH using freesshd or CYGWIN (quick tuto).

svn co –username svnuser –password svnpass https://scm.gforge.inria.fr/svn/vplants/vplants/devel/continuous_integration/master basedir.

svnuser and svnpass are given is the common.py file in that same repository. You need to be a developer to see it. basedir is the working dir for the master, eg: ”/home/buildmaster/continuous_integration/master”.

The master is installed on vp-continuous.cirad.fr. Use the master login and password to log onto the machine via ssh.

The host OS is CentOs 5. The provided python is too old to make BuildBot run confortably so Python 2.6 is compiled (in ~/Devlp/Python-2.6) and the master is running in a virtualenv.

Missing Python packages can be installed using easy_install in the virtual environment.

cd Devlp/buildbot
source bin/activate 
easy_install <package>

As stated above the server configuration is stored on our VPlants subversion repository in vplants/vplants/devel/continuous_integration/master/common.py. You can use a cron task to update the server configuration from the svn at regular intervals (crontab -e).

You can force an update like this:

cd Devlp/buildbot
source bin/activate 
cd master
svn up
sudo buildbot restart .

A typical buildbot installation has a master.cfg file which describes the how schedulers/repositories/slaves/… are coordinated. It is actually a python file with a few special structures. For our usage we have created another python file named common.py which is imported by master.cfg and declares the exact same thing but in a more declarative way (This file is inline documented). If you need to modify the server behaviour, first see if you can't do the change in common.py. If you're sure you can't then modify master.cfg.

To know what's going on with the server and see if it complains about anything have a look at the Devlp/buildbot/master/twistd.log file. Use the tail -n 50 twistd.log command to what the 50 latest logs.