Nov 16, 2013

SUSECon '13 - Final day

It's Friday - the final day of SUSECon. The closing keynote was by Lew Tucker from Cisco who's also the OpenStack Vice Chairperson.

He gave a great overview of cloud computing and OpenStack - and how Cisco plays a role in it. He also cited some impressive numbers on network traffic world wide.
It was interesting to see that Cisco UCS has an API that you can use to manage the Cisco hardware.
Also a demo was shown that showed the integration of Cisco UCS together with SUSE Cloud - and SUSE Manager and SUSE Studio for a complete lifecyle management.

At the end of the keynote the video "What's the Chameleon Say" was shown. SUSE's Russ Dastrup wrote a great script and filmed the  initial version  with his kids and some friends - and then during SUSECon filmed the version below with participants.

Btw. additional videos are available from the SUSEvideo channel on YouTube.

I attended then the session called "wicked trip into wicket network management". Olaf Kirch and Matthias Eckermann explained the complexities of networking on Linux and how the ifcfg framework will be replaced for SUSE Linux Enterprise 12 by a new concept called "wicked". Wicked uses a wickedd daemon and a wicked command line utility to manage complex network setup.  It handles renames of interfaces and can be configured by both config files and via dbus. The package wicked is available for installation in openSUSE 13.1 from the Open Build Service.

Thus ends my report from SUSECon '13. If you like to see more photos, check my updated gallery.

Nov 15, 2013

SUSECon '13 - third day

Thursday morning started for me with a presentation titled "Intel and OpenStack: Contributions and challenges". Krish Raghuram explained not only how Intel helps OpenStack to work great on Intel hardware but also how Intel uses OpenStack internally. For example, they have their own OpenStack private cloud where they run 1,500,000 (hope I got all the zeros right;-) VMs across data centers. Interesting was the concept of geolocation in the cloud: With trusted computing, an image can be encrypted and then will only run in certain geos, e.g. countries. If only the diagram wouldn't look that complex...

The next presentation I attended was by Udo Seidel about "Petabyte scale out Rocks - Ceph as Replacement for Openstack's Swift and Co". He first introduced ceph and explained how it's scalable, has a flexible configuration and not a single point of failure. Ceph can also be used in OpenStack as an alternative to OpenStack Object Storage (Swift), as backend for OpenStack Block Storage (cinder) and
also as storage for OpenStack Image Service (glance).  A single backend for these kinds of storage is a quite clever idea.

I listened again into our SLE 12 systems management outlook session and was happy that I had the evening off - and concluded the evening with a good time at the pool looking at the moon, swimming, sliding and talking.

Photos from the day have been added again to my gallery.

Nov 14, 2013

SUSECon '13 - second day

Today's program at SUSECon '13 had one major topic: SUSE Cloud and thus OpenStack: First I listened to two presentations about Apache Stratos, an PaaS solution that can be used on top of OpenStack. I was impressed by the level of process integration they have, how they can support devops and how they integrated java. Then, I listened to a presentation about the Cisco and OpenStack. A SUSE Cloud installation is nicely integrated with Cisco UCS and then there's the option to scale OpenStack networking with Cisco Nexus routers. Next was the exception to the rule: A presentation about SUSE's changes for system management in SUSE Linux Enterprise 12. Afterwards it was back to OpenStack - this time the VMware story where networking with NXS (formerly Nicira) and VMware vSphere integration was presented - and the SUSE Cloud integration using VMware as hypervisor was demoed.

The day closed with a visit to Disney's Epcot park where we even did a test drive of a "self designed" car.
Photos from the day have been added to my gallery.

Nov 13, 2013

SUSECon '13

Yesterday I flew to Orlando, Florida for SUSEcon '13 and today the conference started.

The opening keynote had two fun videos: "What Does the Chameleon Say?" and "Chameleon Dance". At the end of the second video most people turned their heads to the entrance of the hall to expect the chameleon to come in - but that did not happen. But the chameleon showed up at the evening pirate party.

In the exhibition hall I showed and discussed SUSE Cloud 2.0 - our OpenStack based cloud product - and SUSE Studio - the great image/appliance building tool with customers.

I attended two presentations: Jos Poortvliet and Robert Schweikert gave a view "behind the scenes: How SUSE and openSUSE collaborate". They explained the interaction between openSUSE community and its corporate sponsors and the sometimes conflicting interests.

Later I listed to Alan Clark to give some "OpenStack insights: the disruptive nature of open source cloud". He introduced OpenStack from the community and foundation site - not going into technical details of the implementation but more into the principles that are important for OpenStack and showed the great momentum that it has. From the discussions at the end, it looked that he might have recruited a new OpenStack contributor ;)

I also shot quite some photos and uploaded them to my gallery.

Oct 9, 2013

Easily read locally built OpenStack manuals

Writing and reviewing the OpenStack documentation, I often navigated through directory after directory to finally open a guide in my browser and see it. I considered bookmarking some links but then wrote a local "index.html" file that I can open in my browser and easily click on any guide.

This index file - called now  "local-files.html" is now part of the OpenStack documentation repository openstack-manuals in the doc directory. I've added also a few links to online OpenStack Documentation resources.
I hope this will make reviewing contents easier.

Oct 8, 2013

Improving the OpenStack Documentation Build Tools

During the last weeks, the tool for building and checking the OpenStack Documentation manuals has been improved.

Now we have one new tool with a few different options for fine-grained control. It is run as part of the gates and can be run locally as well. If you used tools/ to test your changes locally, you should now use tools/
Instead, of a single gate job, there are now four checking gates, you'll see now something like the following from Jenkins:
Build succeeded.

    gate-openstack-manuals-validate-niceness SUCCESS in 4s (non-voting)
    gate-openstack-manuals-validate-syntax SUCCESS in 2s
    gate-openstack-manuals-validate-deletions SUCCESS in 4s
    gate-openstack-manuals-validate-build SUCCESS in 8m 47s 

The advantage of all of these changes are that a writer submitting a patch gets quickly an overview about what fails. This is due to the separate jobs. These jobs only check what has changed and thus do not build all books but only those impacted by the change under review.

Description of Checks

Note that all checks parse the git output to record the modified files in a review and only look at these files. If you want to run checks locally on all files, use the --force option.

Niceness check

This tests for extra whitespace. It is non-voting for now and thus not run as part of the final gate.

Syntax check

This check validates the docbook XML code for each file against DocBook 5.1 (actually candidate release 1 - 5.1CR1).

Deletions check

This check tests whether any files have been removed and if those are still referenced anywhere. It handles images and include files.

Build Check

This check will build all books that might be modified by the commit. It checks which files are modified and which books include these files and runs a check of all books where one file was changed. The books are build in parallel to speed up building.

For the Install Guide, it builds all three variants of the Guide - the Fedora/RHEL/Centos Install Guide, the Ubuntu Install Guide and the openSUSE Install Guide. It now also builds the High Availability Guide (which is not using DocBook but asciidoc) if one of its files has changed.Thus, everything is build the same way as we do for publishing the manuals to the OpenStack docs site.

Since we only build the modified books and build them in parallel, you might get now a comment from Jenkins in a minute or two while you had to work 30 minutes in August for this. Jenkins currently needs around 10 minutes to build all books.


The tool is tools/ and has the following options:
tools/ --help
usage: [-h] [--force] [--check-build] [--check-syntax]
               [--check-deletions] [--check-niceness] [--check-all]
               [--non-voting] [--verbose]

Validate XML files against the DocBook 5 RELAX NG schema

positional arguments:
  path               Root directory that contains DocBook files, defaults to
                     `git rev-parse --show-toplevel`/doc

optional arguments:
  -h, --help         show this help message and exit
  --force            Force the validation of all files and build all books
  --check-build      Try to build books using modified files
  --check-syntax     Check the syntax of modified files
  --check-deletions  Check that deleted files are not used.
  --check-niceness   Check the niceness of files, for example whitespace.
  --check-all        Run all checks (default if no arguments are given)
  --non-voting       Do not exit on failures
  --verbose          Verbose execution

I'd like to thank Christian Berendt for starting this with the creation of (as a modified copy of and all the reviewers that helped moving this forward.

Jul 12, 2013

NIS and Password Length Fun

I have seen a request to "extend NIS/YP maximum password length" for SUSE Linux and openSUSE and digging into this further, found that there's really nothing to do but a lot of confusion.

NIS itself stores passwords and does not have limitations on the password implementation and on the used ciphers. Password authentification happens at login time on a client, at the imap server when connecting (if the imap server uses the NIS database) etc - and on the NIS server when changing the password (more below).

All systems that do authentication, do the password check locally and thus all systems, need to implement the same ciphers for encryption. The common lowest denominator is DES which only supports 8 characters. But if you know that all systems support e.g. SHA-512, you can use that one.

To change the cipher used for encryption on an openSUSE or SUSE Linux Enterprise 11 system, there are two different ways, depending on whether you use pam_unix2 or pam_unix in /etc/pam.d/common-password. pam_unix will be the default in new openSUSE 13.1 installations, pam_unix2 has been setup so far by default on openSUSE and SUSE Linux Enterprise.  If you use pam_unix2, edit /etc/default/passwd and change CRYPT_YP to the desired cipher. If you use pam_unix, edit /etc/login.defs and edit ENCRYPT_METHOD - and then run "passwd".

Regarding compatibility of systems, here's some uncomplete information on ciphers available besides DES that I quickly gathered:

  • SLES 10: blowfish
  • SLES 11: md5, sha-256, sha-512, blowfish
  • openSUSE 13.1: md5, sha-256, sha-512, blowfish
  • Solaris 11: sha-256, md5
  • RHEL 6: sha-256, sha-512, md5

Testing this, I stumpled upon one problem: When changing the NIS password, the old password is verified locally and then the old password is send unencrypted together with the encrypted new password to the NIS server. The NIS server then tests that the old password is correct (it cannot trust your client) and saves the encrypted new password in its database. When you now specify a cipher for encryption that is not known to the server, it will happily save the new encyrpted password - but once you change the next time, it will fail since it does not understand the new cipher and thus cannot check that the old password is correct. So, time to ask your admins to reset your NIS password...

Thanks for my colleague - and Linux NIS developer - Thorsten Kukuk for shedding some lights on these questions.

Apr 12, 2013

Chef and Crowbar for setting up OpenStack clouds - with openSUSE and SUSE Cloud

To easily setup an OpenStack-based cloud on server hardware, we at SUSE use the open source Crowbar project. Crowbar eases the setup of all the different OpenStack components and thus reduces the time spent deploying and managing an OpenStack-based cloud.

Crowbar is a platform for server provisioning and deployment from bare metal. It provides server discovery, firmware upgrades, and operating system installation using PXE Boot, and it deploys applications on top of functioning operating systems using Chef.

Currently development is happening on Crowbar 2 and I'd like to give a short overview of a few things that are done here to make this work useable for openSUSE and SUSE Cloud - SUSE's OpenStack based cloud offering.

Moving to a common Chef cookbase base

So far, Crowbar comes with its own Chef cookbooks to setup OpenStack, a forked and heavily adapted version.

We'd like to have one common set of Chef cookbooks that are shared between Crowbar and administrators using Chef directly. As Rob Hirschfeld said: "The real victory comes when multiple deployments use the same trunk instead of forking".
The additional benefit here is that these cookbooks will then also work without Crowbar on e.g. openSUSE and SUSE Linux Enterprise.

Right now it looks like the awesome cookbooks done by AT&T and Rackspace (e.g. are the best option for going forward. But these cookbooks have two limitations for our usage: they do not handle openSUSE and SUSE Linux Enterprise and are not ported for Grizzly yet. So, this gives us the next tasks:

Enhance existing Chef cookbooks to know about openSUSE and SUSE - and for Grizzly

We're enhancing these cookbooks now so that they support OpenStack Grizzly and we're also adding support for servers running openSUSE and SUSE Linux Enterprise.

Once the cookbooks are usuable on their own, they can be integrated into Crowbar.

Integrate the Chef cookbooks into Crowbar

To use the cookbooks from Crowbar, Crowbar needs to pass parameters to
them. This concept is called "Attribute Injection" and Judd Maltin has explained this in detail in a great blog post.

Crowbar 2 and Chef / Puppet

Of interest to some might be that in the coming Crowbar 2, the Chef usage has been hidden behind an extra layer called "Jig". Jig was introduced to overcome some design limitations of Crowbar, the extra abstraction also gives a more flexible framework. Now other configuration management options can be added besides Chef. Puppet is one framework that the Crowbar team would like to support as backend and this work is currently postponed after the initial Crowbar 2 release.

Apr 9, 2013

OpenStack Grizzly Packages for openSUSE and SUSE Linux Enterprise

Last week the OpenStack project released its 7th release named Grizzly and the packages are now available for openSUSE and SUSE Linux Enterprise via the openSUSE Build Service. Packages can be downloaded from the download server.

The repositories contain all the core OpenStack packages plus incubated projects that will be part of core in the Havana release (Heat for VM orchestration and Ceilometer for metering). Additional package dependencies are included as well.

For discussion about these packages, use the opensuse-cloud at mailing list and the freenode channel #opensuse-cloud. To learn more about OpenStack on openSUSE, see also our wiki page.

Mar 11, 2013

Moving to open development: OpenStack / Crowbar / Chef on SUSE

This blog post was written by Adam Spiers and Andreas Jaeger.
Here at SUSE we have based our business on Free and Open Source Software for over 20 years, so it is nothing new to say that we strongly believe in open development and collaboration as a way of producing software of the highest quality.
Therefore we are pleased to announce that we have opened up our work on development and packaging of OpenStack, Crowbar, and Chef, for both openSUSE and SUSE Linux Enterprise. This work is happening publically in the Open Build Service, and we warmly invite everyone to use the results and also join in with development, testing, documentation and packaging.
Open development is a process that does not end with code on public servers, but involves ongoing open communication as well. Please speak up on the opensuse-cloud mailing list if you see any areas where we could improve further!


OpenStack is our preferred cloud management platform, due to its open nature and backing from an already huge and rapidly growing community. We have packaged both OpenStack's Folsom release and the beta packages for the upcoming Grizzly release and have made them available in the Open Build Service.


OpenStack's incredible flexibility can also make it difficult to deploy. Therefore to ease the set-up of all the different OpenStack components out of the box, we are supporting the open source Crowbar project.  Our partners and customers find that using Crowbar dramatically reduces the time spent deploying and managing an OpenStack-based cloud.
Crowbar is a platform for server provisioning and deployment from bare metal. It provides server discovery, firmware upgrades, and operating system installation using PXE Boot, and it deploys applications on top of functioning operating systems using Chef.

Opening up Crowbar development

Similarly to how OpenStack was born at RackSpace and later grew into an independent entity in the form of the OpenStack Foundation, Crowbar was originally developed by Dell engineers, and is now a fully-fledged Open Source project involving close collaboration with SUSE and others. There are weekly public meetings, a public mailing list, a #crowbar IRC channel, public Trello boards and so on. As an indication of the project's independent nature, the authoritative location for the git repositories has changed from to, and a new homepage is currently under construction.

Crowbar 2.0

Work on the upcoming 2.0 release of Crowbar is proceeding at a furious pace, and packages for openSUSE and SUSE Linux Enterprise are available from the OBS. You are very welcome to join in and find out more!

Chef packages

Chef is an Open Source configuration management tool that allows remote administration of systems at scale. It has a thriving community which has a large overlap with the OpenStack community. We have packaged both Chef 10 and Chef 11 for openSUSE and SUSE Linux Enterprise.

Tying it all together

Since each major release of these projects is packaged separately in the Open Build Service, you are free to "mix'n'match" which of these components you want to use together:
  • OpenStack Essex
  • OpenStack Folsom
  • OpenStack Grizzly
  • Crowbar 2.0
  • Chef 10
  • Chef 11
Of course, each combination will work differently. To try any of them out, simply navigate to one of the project's Repository tab to obtain the link to the relevant download repository. Then you can add that to your host's list of software repositories in the normal way via YaST2 or zypper, and start installing packages from it. Alternatively, you can install directly via one-click install.

Automation and Continuous Integration

We use Jenkins to automatically package changes from upstream git repositories into rpms within an openSUSE environment, and if they successfully build and install, Jenkins commits those changes to the Open Build Service (OBS) which then automatically builds and publishes packages and images. Jenkins also does unit tests as well as full stack tests. By automating the packaging and testing process, we can spend more time on development while maintaining high quality, and gain more clarity around where problems exist.
You can also read in more detail about our development model using the Open Build Service.

Finding out more

If you have questions, ask them on the opensuse-cloud mailing list or on the recently launched #opensuse-cloud freenode IRC channel.
You can also find more details via the openSUSE wiki: This is work in progress, so feel free to help improve anything!

Feb 25, 2013

Coordinating efforts to create OpenStack packages for openSUSE and SUSE Linux Enterprise Server

Engineers from B1 Systems and SUSE have now joined forces to package OpenStack in the Open Build Service for openSUSE and SUSE Linux Enterprise. The B1 Systems engineers bring in their OpenStack consulting and packaging expertise, the SUSE team their packaging and hardening expertise from building a commercial OpenStack based release with SUSE Cloud.

The work is done in the Open Build Service which allows building of packages for various distributions and versions of it. There are stable packages available from the latest OpenStack release, Folsom, and beta packages of the upcoming Grizzly release. These OpenStack packages are build now for openSUSE 12.2, openSUSE 12.3 and SUSE Linux Enterprise 11 SP2.
The joint development project is contained in the Cloud:OpenStack and its subprojects.

Additionally, OpenStack Folsom is part of the upcoming openSUSE 12.3 release. Thus, you can build, administrate and use an OpenStack cloud within openSUSE.

If you like to use OpenStack packages or join the team, please join us on the mailing list (subscribing information is here).

I'll talk in future posts on the development progress of these packages in the Open Build Service. In the meantime, to get more information about OpenStack on openSUSE, see the openSUSE OpenStack Portal.

Some background: B1 Systems is a consulting partner of SUSE and offers support for OpenStack on openSUSE and SUSE Linux Enterprise.  Some of their engineers have been packaging OpenStack for over two years now (starting with Bexar) and made those packages available through the Open Build Service for both openSUSE and SUSE Linux Enterprise. SUSE has delivered last year its SUSE Cloud 1.0 release that is based on OpenStack Essex.