Many Linux distributions provide build services under various names: openSUSE Build Service (OBS), Fedora Copr, and so on. These resources are indispensable for upstream developers, and also for their users. I will demonstrate this through some examples from the syslog-ng project.
Note: this blog is loosely based on a talk idea I had for the FOSDEM Distributions Devroom. There is no deep technical information about syslog-ng in this blog. This is more like a history of syslog-ng packaging, and how the fantastic tools by openSUSE and Fedora made it a lot easier and made me an active part of these communities.
Definitions
The title needs a few clarifications. It uses some plain English words, but I suspect that only people creating packages for various Linux distributions may understand it.
A build service is a service that allows users to build software packages from source code for various Linux distributions and hardware architectures. Both openSUSE and Fedora maintain publicly available build services, where project members can build and publish any open-source software. You can also maintain private instances of various build services, where you are not restricted to work only on open-source software.
From a Linux distribution point of view, any software included in the distribution (and not developed in-house) comes from an upstream project. Thus, the syslog-ng project is the upstream for the syslog-ng package in various Linux distributions.
A bit of history
When the openSUSE Build Service became available, my original use of OBS was to build alpha/beta test packages of syslog-ng before a new release. This ensured that syslog-ng could be compiled on all (open)SUSE versions, both on i386 and on AMD64 (no other platforms yet at that time). These packages were far from perfect, as I was just learning the basics of RPM packaging. Luckily, they had worked, and could also be used by “official” distro packagers as a reference when they upgraded syslog-ng to the latest version.
Soon my OBS syslog-ng repos became popular, as the latest syslog-ng became available on all supported distribution variants. Even if I only had access to a single version of openSUSE on an AMD64 laptop, I could build syslog-ng for a wide variety of openSUSE and SLES versions on multiple architectures. Once the Fedora Copr also became available, Fedora and RHEL users also got access to the latest syslog-ng testing and final releases in a similar manner.
Upstream developers often get problem reports about distribution packages. Even if the syslog-ng project quickly publishes a fix, most users do not know how to build software from source code, or how to build a package to test those fixes. Often, just pointing users to the very latest syslog-ng version resolves their problems. If the latest version does not resolve their problems, build services make it easy to build and publish a fix for any supported distro, even if upstream developers do not have access to the given Linux distribution or architecture. Later some of these fixes can be backported to official distro packages.
Soon I was invited to become a co-maintainer of the official distro packages. A few years later I became the main maintainer with some occasional help from other maintainers.
Why build services are important?
The way I see it, build services are the perfect gateway drug for upstream developers to become a distribution package maintainer while also serving their original communities.
Build services are a playground, where users can easily build packages for various Linux distributions without the responsibilities of an official package maintainer (dealing with issue tickets, over complicated release processes and long waits). They are very useful resources for developers. Even with just openSUSE on my laptop and Fedora in a VM, all AMD64, I could build syslog-ng packages for all supported openSUSE, SLES, Fedora and RHEL releases. Furthermore, I could build for i386 too, and later also for POWER, ARM, and now even for the RISC-V architecture.
Working on the various build services gives several useful experiences. First of all, you can learn how the given build service works. OpenSUSE uses the exact same OBS instance to build the whole Linux distribution. All distros have different packaging guidelines, which are often enforced by the build services. Being aware of those is useful, even if your software project is not part of the given Linux distribution: you receive a lot fewer problem reports later…
Building packages on a build service is also a great opportunity to get familiar not just with the technology but also with the community, both the Linux distribution’s users and developers. If your software is already part of the distro, you can show the official package maintainer the very latest features and how to enable them. So, next time the package receives not just a version bump, but also becomes more feature complete. Eventually I became the primary package maintainer of syslog-ng in both RPM distributions.
My unofficial syslog-ng repos are still built for various reasons. I have now a repository built regularly from syslog-ng git snapshots, so build problems surface early. There is a new syslog-ng release every other month, while Linux distributions are not released this frequently. Also, some features are missing from the official distro packages due to missing or old dependencies in the distro, which can be easily worked around in 3rd party repos. Which means that syslog-ng users are served both within the Linux distros, and also from external repositories if they need the latest and greatest version.
What is next?
It is just a guesstimate, but if we do not count syslog-ng running on the Amazon Kindle, then well over 80% of syslog-ng users run a package built by me. The bus factor is pretty high, and I do not feel too comfortable with being a single point of failure. This year I plan to change it and distribute the responsibility of building syslog-ng packages. I am not a git aficionado, but there is a possibility to automate and share packaging responsibilities both in Copr and OBS using git.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.