Blogs

mollekopf's picture

Reproducible testing with docker

Reproducible testing is hard, and doing it without automated tests, is even harder. With Kontact we’re unfortunately not yet in a position where we can cover all of the functionality by automated tests.

If manual testing is required, being able to bring the test system into a “clean” state after every test is key to reproducibility.

Fortunately we have a lightweight virtualization technology available with linux containers by now, and docker makes them fairly trivial to use.

Docker

Docker allows us to create, start and stop containers very easily based on images. Every image contains the current file system state, and each running container is essentially a chroot containing that image content, and a process running in it. Let that process be bash and you have pretty much a fully functional linux system.

The nice thing about this is that it is possible to run a Ubuntu 12.04 container on a Fedora 22 host (or whatever suits your fancy), and whatever I’m doing in the container, is not affected by what happens on the host system. So i.e. upgrading the host system does not affect the container.

Also, starting a container is a matter of a second.

Reproducible builds

There is a large variety of distributions out there, and every distribution has it’s own unique set of dependency versions, so if a colleague is facing a build issue, it is by no means guaranteed that I can reproduce the same problem on my system.

As an additional annoyance, any system upgrade can break my local build setup, meaning I have to be very careful with upgrading my system if I don’t have the time to rebuild it from scratch.

Moving the build system into a docker container therefore has a variety of advantages:
* Builds are reproducible across different machines
* Build dependencies can be centrally managed
* The build system is no longer affected by changes in the host system
* Building for different distributions is a matter of having a couple of docker containers

For building I chose to use kdesrc-build, so building all the necessary repositories is the least amount of effort.

Timotheus Pokorra's picture

Submitting patches to Kolab via Phabricator

Just before the Kolab Summit, at the end of April 2015, the Phabricator instance for Kolab went online! Thanks to Jeroen and the team from Kolab Systems who made that happen!

I have to admit it took me a while to get to understand Phabricator, and how to use it. I am still learning, but I now know enough to write an initial post about it.

Phabricator describes itself as an “open source, software engineering platform”. It aims to provide all the tools you need to engineer a software. In their words: “a collection of open source web applications that help software companies build better software”

To some degree, it replaces solutions like Github or Gitlab, but it has much more than Code Repository, Bug Tracking and Wiki functionality. It also has tools for Code Review, Notifications and Continuous builds, Project and Task Management, and much more. For a full list, see http://phabricator.org/applications/

In this post, I want to focus on how you work with the code, and how to submit patches. I am quite used to the idea of Pull Requests as Github does it. Things are a little bit different with Phabricator. But when you get used to it, they are probably more powerful.

Starting with browsing the code: there is the Diffusion application. You can see all the Kolab projects there.
It also shows the “git clone” command at the top for each project.
Admittedly, that is quite crowded, and if you still want the simple cgit interface, you get it here: https://cgit.kolab.org/

Now imagine you have fixed a bug or want to submit a change for the Kolab documentation (project docs). You clone the repository, locally edit the files, and commit them locally.

Aaron Seigo's picture

Roundcube Next crowdfunding success and community

Roundcube Next crowdfunding success and community

A couple days ago, the Roundcube Next crowdfunding campaign reached our initial funding goal. We even got a piece on Venture Beat, among other places. This was a fantastic result and a nice reward for quite a bit of effort on the entire team's part.

Reaching our funding goal was great, but for me personally the money is secondary to something even more important: community.

You see, Roundcube had been an Internet success for a decade now, but when I sat to talk with the developers about who their community was and who was participating from it, there wasn't as much to say as one might hope for such a significant project used by that many people.

Unlike the free software projects born in the 90s, many projects these days are not very community focused. They are often much more pragmatic, but also far less idealistic. This is not a bad thing, and I have to say that the focus many of them have on quality (of various sorts) is excellent. There is also a greater tendency to have a company founded around them, a greater tendency to be hosted on the mostly-proprietary Github system with little in the way of community connection other than push requests. Unlike the Free software projects I have spent most of my time with, these projects hardly try at all to engage with people outside their core team.

This lack of engagement is troubling. Community is one of the open source1 methodology's greatest assets. It is what allows for mutual interests to create a self-reinforcing cycle of creation and support. Without it, you might get a lot of software (though you just as well might not), but you are quite unlikely to get the buy-in, participation and thereby amplifiers and sustainability of the open source of the pre-Github era.

Aaron Seigo's picture

Riak KV, Basho and Kolab

Riak KV, Basho and Kolab

As I have mentioned in earlier blog entries, Kolab Enteprise has gained data loss prevention (DLP) functionality this year that goes above and beyond what one tends to find in other groupware products. Kolab's DLP is not just a back-up system that copies mails and other objects to disk for later restore, it actually creates a history of every groupware object in real-time that can later be examined and restored from. This will eventually lead to some very interesting business intelligent features.

The storage system for the Kolab DLP system is Basho's industry-leading distributed NoSQL database, Riak KV. (The "KV" stands for key/value.) We chose Riak KV because it scales naturally (it is designed to be run as a cluster of nodes by default), is robust by design (CAP Theorem ftw), and is dead simple to deploy on development and production machines alike. A further key factor for us is that Basho provides proven enterprise-grade support for its line of Riak products. This was a requirement for us as we need to provide enterprise-grade support for the entire Kolab Enterprise stack.

(It was a nice coincidence that both Riak and core parts of Kolab's DLP system are written in Erlang. ;)

I sat down with Manu Marchel, Managing Director for EMEA at Basho Technologies Inc., recently for a mutual interview. You can read my interview on the Basho blog (I'll update this entry with a link to their blog when it is published lived); here is a transcript of my conversation with Manu:

NoSQL is a quite a new technology in the Big Data space. Many people might have heard about things like Hadoop but how does NoSQL fit in? Could you give everyone the quick cheatsheet on what NoSQL databases are and specifically what is RiakKV your key value NoSQL database?

Aaron Seigo's picture

Riak KV, Basho and Kolab

Riak KV, Basho and Kolab

As I have mentioned in earlier blog entries, Kolab Enteprise has gained data loss prevention (DLP) functionality this year that goes above and beyond what one tends to find in other groupware products. Kolab's DLP is not just a back-up system that copies mails and other objects to disk for later restore, it actually creates a history of every groupware object in real-time that can later be examined and restored from. This will eventually lead to some very interesting business intelligent features.

The storage system for the Kolab DLP system is Basho's industry-leading distributed NoSQL database, Riak KV. (The "KV" stands for key/value.) We chose Riak KV because it scales naturally (it is designed to be run as a cluster of nodes by default), is robust by design (CAP Theorem ftw), and is dead simple to deploy on development and production machines alike. A further key factor for us is that Basho provides proven enterprise-grade support for its line of Riak products. This was a requirement for us as we need to provide enterprise-grade support for the entire Kolab Enterprise stack.

(It was a nice coincidence that both Riak and core parts of Kolab's DLP system are written in Erlang. ;)

I sat down with Manu Marchel, Managing Director for EMEA at Basho Technologies Inc., recently for a mutual interview. You can read my interview on the Basho blog (I'll update this entry with a link to their blog when it is published lived); here is a transcript of my conversation with Manu:

NoSQL is a quite a new technology in the Big Data space. Many people might have heard about things like Hadoop but how does NoSQL fit in? Could you give everyone the quick cheatsheet on what NoSQL databases are and specifically what is RiakKV your key value NoSQL database?

Timotheus Pokorra's picture

Installing Demo Version of Kolab 3.4 with Docker

This describes how to install a docker image of Kolab.

Please note: this is not meant to be for production use. The main purpose is to provide an easy way for demonstration of features and for product validation.

This installation has not been tested a lot, and could still use some fine tuning. This is just a demonstration of what could be done with Docker for Kolab.

Preparing for Docker
I am using a Jiffybox provided by DomainFactory for downloading a Docker container for Kolab 3.4 running on CentOS 7.

I have installed Fedora 21 on a Jiffybox.

Now install docker:

sudo yum install docker-io
systemctl start docker
systemctl enable docker

Install container
The image for the container is available here:
https://registry.hub.docker.com/u/tpokorra/kolab34_centos7/
If you want to know how this image was created, read my other blog post http://www.pokorra.de/2015/06/building-a-docker-container-for-kolab-3-4-on-jiffybox/.

To install this image, you need to type in this command:

docker pull tpokorra/kolab34_centos7

You can create a container from this image and run it:

Timotheus Pokorra's picture

Building a Docker container for Kolab 3.4 on Jiffybox

This article is an update of the previous post that built a Docker container for Kolab 3.3 from September 2014.

Preparation
I am using a Jiffybox provided by DomainFactory for building the Docker container.

I have installed Fedora 21 on a Jiffybox.

Now install docker:

sudo yum install docker-io
systemctl start docker
systemctl enable docker

Create a Docker image

To learn more about Dockerfiles, see the Dockerfile Reference

My Dockerfile is available on Github: https://github.com/TBits/KolabScripts/blob/Kolab3.4/kolab/Dockerfile. You should store it with filename Dockerfile in your current directory.

This command will build a container with the instructions from the Dockerfile in the current directory. When the instructions have been successful, an image with the name tpokorra/kolab34_centos7 will be created, and the container will be deleted:

sudo docker build -t tpokorra/kolab34_centos7 .

You can see all your local images with this command:

sudo docker images

To finish the container, we need to run setup-kolab, this time we define a hostname as a parameter:

Timotheus Pokorra's picture

Kolab as a Server Role on Fedora with rolekit

This post originates in the idea from Stephen Gallagher, who is working on rolekit: “rolekit is a daemon for Linux systems providing a stable D-BUS interface to manage the deployment of [Fedora] Server Roles”.
The code of Rolekit is available here: https://github.com/sgallagher/rolekit

On his blog, Stephen stated in this post:

A few that I’d love to see (but don’t have time to start on yet):

  • A fileserver role that manages Samba and NFS file-shares (maybe [s]ftp as well).
  • A mail and/or groupware server role built atop something like Kolab
  • A backup server

This made me wonder, how would that be, if Kolab became a Server Role for Fedora, and could be installed from the Fedora repositories? Through my work on OpenPetra and Mono I got involved with Fedora, and noticed that the Fedora community tries out new technology, proves if it works, and then the technology will eventually end up in other distributions as well.

First steps

On IRC, we agreed that the first step would be to create a Copr repo, that contains the Kolab packages, and to write this blog post describing how to install and configure Kolab.

Creating the Copr Repo

So, here is the Copr repository for Fedora 22: https://copr.fedoraproject.org/coprs/tpokorra/kolab/

I created it by getting the src rpm packages from the Kolab repository, from 3.4 and 3.4 updates, in this order:

Timotheus Pokorra's picture

Kolab 3.4 on Debian Jessie

Some weeks ago, I did significant work on getting Kolab 3.4 running on Debian Jessie.

I did this work in my own time, because at TBits.net we focus on CentOS7.
Still the work was benefitial to CentOS as well, because I had to do some fixes for PHP 5.6, which will eventually be part of CentOS sometime in the future.

For your interest, here are the bugs I have worked on:

For several weeks, my nightly tests succeed now for Debian Jessie as well, on LBS: see https://lbs.solidcharity.com/package/tbits.net/kolab-test/kolab-test#Kolab3.4_debian/jessie/amd64

I just updated the installation instructions: https://docs.kolab.org/installation-guide/debian.html

I am not using Debian Jessie in production, and that means two things:

  • I cannot say if it actually works in production. I only can say that it passes my nightly tests.
  • In the future, I think I might need to focus on CentOS more, and cannot invest so much of my own free time into the Debian packaging. I am open for suggestions or sponsorship :) (perhaps https://www.bountysource.com/?)
Timotheus Pokorra's picture

Kolab 3.4 Community Updates: Roundcube update and some other fixes

I realized it would be good to blog here about updates for the Kolab 3.4 Community Edition.

Although it is a community release, and therefore does not come with any guarantuee (that is up to the Enterprise version), some people are using the community edition in production, and we as the community are contributing fixes and maintaining the release.

Thanks to Daniel Hoffend, we now have the latest Roundcube 1.1.2 in Kolab 3.4 Updates. Just run yum update (CentOS) or apt-get update && apt-get upgrade (Debian)…

Daniel also backported a week ago a fix for the installation of Kolab, the Roundcube configuration for the Addressbook was not correct.
More details can be seen in the Change Reqest on OBS.
You might want to manually update your existing installation in the same way…

And another fix from 10 days ago: Daniel backported a fix for the Roundcube Context menu.
See details in the Change Request on OBS.

In the future, I will aim to post here as soon as we accept updates into Kolab 3.4 Updates.

If you want to contribute to make the community edition of Kolab more stable and secure, please make suggestions on the mailing list about fixes that you know of, and if you enjoy creating a change request on OBS yourself, then go for it, you would be very welcome!