25 May

On the use and abuse of size_t

Just about everyone has used the size_t type – and everybody and their grandmothers know what the type means of course! The usual answers I get to ‘tell me what size_t is without Googling it’ is;

  1. It is the parameter type for malloc to allocate memory!
  2. It is a platform dependent 32/64 bit unsigned integer type!
  3. It is what size_type in all the C++ std template hell is! (ok I made this one up – but fuck me the size_type lark really annoys me in the std code.)

In general though the main assumption is that the size_t type is a platform agnostic way to represent any potential size of memory you could allocate or use.

The definition from ISO/IEC 9899:TC3, section 7.17 is;

size_t

which is the unsigned integer type of the result of the sizeof operator

And also relevant  is section 7.18.3;

limit of size_t

SIZE_MAX 65535

So the minimum maximum value that size_t must be able to hold is 65535, which is 16 bits of precision, and size_t is only defined to be an unknown unsigned integer type. Let that sink in – size_t can, by the standard, be an uint16_t type on a platform with 32 or 64 bit addressing.

Now before the reader decides to flip their table like this fine gentleman;tableflipI completely understand that on every platform the modern C/C++ developer uses and cares about size_t is either uint32_t on 32 bit platforms, or uint64_t on 64 bit platforms. What annoys me though is that I could take my perfectly portable (among standard MSVC/Clang/GCC compilers) C/C++ code and run it on some random compiler for a given platform, and it could fail in a non-obvious way! This in turn would probably lead me to say ‘this new platform must have shit tools!’ when in fact the platform is supporting the C standard to the letter in their approach.

This is why I don’t use size_t in my interfaces or code, it irks me that some random compiler somewhere could legitimately fail on my code.

So what is the alternative? Really the type we should be using is uintptr_t – EG. the unsigned integer type that matches the width of the pointer type. That way if I decide to allocate all of my memory I will always have a type that can encompass that size value always. But of course, there is a problem there too, from section 7.18.1.4;

The following type designates a signed integer type with the property that any valid pointer to void can be converted to this type, then converted back to pointer to void, and the result will compare equal to the original pointer:

intptr_t

The following type designates an unsigned integer type with the property that any valid pointer to void can be converted to this type, then converted back to pointer to void, and the result will compare equal to the original pointer:

uintptr_t

These types are optional.

And just for emphasis  These types are optional’. So anyone thinking of using these types too? Please don’t! An entirely conformant compiler and toolchain could give you a big;

fuckhugh

 

What I tend to do (and I understand the grievances in this) is either define my own size type in my interfaces (involves a ton of preprocessor checks for arm/x86/x86-64/ppc/mips/etc) or just use uint64_t for anything that could represent memory. So the next time you see a uint64_t representing an allocatable size in some of my code, you know why!

 

21 May

Connecting a build node to Jenkins on Raspberry Pi 2

So I previously installed Jenkins on my Raspberry Pi 2, and ran my first job through Jenkins. The final obvious step was to get another build node to connect to my Jenkins server so that workload can be offloaded from the Raspberry Pi 2 to another device.

The lovely folks over at Imagination Technologies sent me a MIPS Creator CI20 board a while back, and this seemed like the perfect node to add to my new Jenkins build farm!

So first step, you go to ‘Manage Jenkins’ -> ‘Manage Nodes’;

Capture

 

Next, you want to add a new Node (picking whatever name you choose), I went with ci20 for a rather obvious name. My only choice was a ‘Dumb Slave’, so I went with that.Capture1

 

Next up, we need to configure how our Jenkins server running on the Raspberry Pi 2 will connect to the ci20!

Capture2

I use the stock settings and debian 7 that comes pre-installed on the ci20, so I used launch via ssh, stuck in the username password I use to ssh into the ci20 myself, and lastly I had to set the ‘Remote root directory’ to the ci20 username’s home directory.

On the ci20 itself, I had to run;

sudo apt-get install openjdk-7-jdk cmake git

For the missing packages I needed on the device.

The last thing I did was edit my previous doboz job to only run on the ci20 label, forcing it to run on the new ci20 build node exclusively. And the results?Capture3

That concludes my foray into the Jenkins on Raspberry Pi 2 world – in conclusion it is definitely possible now to run Jenkins on the Raspberry Pi with the beefier new model.

19 May

Running your first job on the Raspberry Pi 2 / Jenkins

So in my last post Installing Jenkins on Raspberry Pi 2 I took us through the steps it took me to get a Jenkins server running on my Raspberry Pi 2 – but I didn’t actually run any jobs on the Raspberry Pi 2! Time to rectify.

So I decided to build doboz – a really awesome little compression/decompression library written by Attila Áfra. I first forked it from HG -> Git, and fired it up onto my GitHub here.

I did some minor cleaning up (removed the VC10 folder, moved COPYING.txt to LICENSE as is the GitHub norm) but all in all the remainder of the code was the same.

I then tried to add my first Jenkins job to pull this repo – but oh wait! Jenkins by default only has the CVS and SVN version control plugins installed, so I had to first at the Git SCM Plugin. After installing that, I created a simple job called doboz that pulled the repository, and huzzah! It worked!

Next up, to actually build the thing. Now I primarily use CMake for all my builds (it is awful in many ways, but it is cross platform and the LLVM project uses it so that is good enough for me) – so I added a CMakeLists.txt that would build a libdoboz.so for me. I then added a build step/execute shell that simply did;

mkdir -p build
cd build
cmake ..
make

And ran it… and it failed.

Screenshot from 2015-05-19 21:56:25

Basically CMake wasn’t installed by default on the Pi! So I ssh onto the Pi, run;

sudo apt-get install cmake

And then once it completes, I re-ran the Jenkins job, and it failed again.

Screenshot from 2015-05-19 22:47:39

Luckily I recognized this error from a previous customer project, basically before including stdint.h on some wacky platforms you need to;

#define __STDC_LIMIT_MACROS
#include <stdint.h>

To actually get UINT64_MAX and all the other *_MIN and *_MAX defines.

I then ran the job again, and it passed.

Screenshot from 2015-05-19 22:33:07

So there you have it! Our first job to run on our Jenkins / Raspberry Pi 2. In a follow-up post I’ll show you how to connect another build node to the Raspberry Pi 2, so that the job is actually being built on another machine entirely (which is entirely the point of Jenkins!).

 

 

 

15 May

Installing Jenkins on Raspberry Pi 2

So I’ve always wanted to roll my own home continuous integration (CI) server – I use Jenkins as my main CI at work (having used Atlassian’s Bamboo and the hideousness that is buildbot in the past), so I thought I may as well start there.

I previously tried to get Jenkins running on my Raspberry Pi B+ – but the single core and general slowness of the device just meant it wasn’t feasible. Then, I got a Raspberry Pi 2 (naturally) so I wondered if it was any more capable?

For reference – I’m running stock Raspbian on the device.

First up, lets install Jenkins! I run sudo apt-get install Jenkins, and voilà, Jenkins is installed! I navigate to the default Jenkins port 8080 in a web browser, and it is indeed the Jenkins I know and love have developed Stockholm Syndrome for. But wait – it is a rather old version! It turns out that the default package included in Debian is a much older Jenkins installation. Jenkins has a built in mechanism in the web interface to allow you to update Jenkins from the webpage, I tried this and it spectacularly failed. The crux of the matter is that since I installed Jenkins via sudo apt-get, it has placed the jenkins.war file in a sudo location, such that the web updating method can’t replace the original file.

After reading on the Jenkins website, I noticed they had an alternative solution for installing on ubuntu – since Ubuntu and Raspbian share the same .deb, dpkg and apt-get infrastructure at this point, I just followed the instructions;

wget -q -O - https://jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

This worked, kinda. Basically because Raspbian already has a method for acquiring the jenkins.war, it didn’t like me installing Jenkins via the new method (said there was a .deb file that conflicted). After some Googling, I found this article which had the command;

sudo dpkg -i --force-overwrite /var/cache/apt/archives/smplayer_0.6.9+svn3595-1ppa1~maverick1_i386.deb

Which I used but replaced the .deb file with the offending jenkins .deb, and it installed ok.

For some reason Jenkins wouldn’t start though, so I manually ran the init service that is installed to /etc/init.d/jenkins and ran it as follows;

sudo /etc/init.d/jenkins start

And it complained that I was using Java 6 instead of the minimum required Java 7. If you use the Linux command update-alternatives, you can do;

$> sudo update-alternatives --list java
/usr/lib/jvm/java-6-openjdk-armhf/jre/bin/java
/usr/lib/jvm/jdk-7-oracle-armhf/jre/bin/java
/usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt/jre/bin/java

You can then use;

$> sudo update-alternatives --config java

 

Which will ask you to select which java version to by default use – I selected the jdk 8 one.

I then in my browser went to the 8080 port for my Raspberry Pi 2 – and I got the shiny latest version of Jenkins, huzzah!

I’ve not had time to add any actual build jobs yet – but from first impressions, immediately on reboot loading each page takes a good 10 seconds. I ran top on the Pi at the same time and the Java process spikes to 130% of CPU. Once you have loaded the page once, it loads quickly the second time, and this is not being client-side cached, it seems that on the Pi the Jenkins process is doing some form of caching of content which is welcome!

Once I add some actual build jobs that generate output I’ll do a follow up post on how responsive it gets under load.