Brian Behlendorf

There was once a time when we marveled at the global nature of the open source user and contributor community, when it was a thrill to get a question or patch from an address ending in .nz or .jp or .cl., or to hear about your software running at the Vatican or the International Space Station. These days, it’s a given that the more popular an open source project is, the more likely its user and contributor community span continents and cultures. 

Well-run open source projects recognize that fact, and often take a series of steps to build their global user and contributor base. Those steps can include basic ones like ensuring Unicode and 8-bit-clean text handling in their UI, providing localization via resource bundles, or translating documentation. Others realize there’s often a human and cultural gap to cross, so they invest in growing local user communities and support forums, or flying core maintainers to present at conferences they might not otherwise reach. The hardest, but potentially the most important thing, is for projects to look at the pathways for users to become participants and contributors, everything from the forums and mailing lists to regular conference calls all can inadvertently create barriers for people in other time zones and who use other communication methods.

We are just beginning this journey at OpenSSF. To date the vast majority of active contributors are based in the US, with a smidge in Europe and Australia. However, the people we’d like to reach with the OpenSSF’s different guides, specifications, services and software are global, and we know there are potential contributors to our efforts everywhere too. To support that, you’ll see us start to prioritize things like moving meetings to more globally convenient times; investing in translations of our work products; and putting together more virtual (and eventually face-to-face) meetings focused on the global audience.

On Thursday, March 24th, at 11am Hong Kong time (also 8:30am IST, 11am SGT, 12pm Japan, Korea and 2pm Australia) we’ll be hosting a virtual event introducing OpenSSF to the Asia-Pacific audience. David Wheeler, Julian Gordon and I will be joined by VM Brasseur from Wipro to give an overview of what we’re doing and how people from the region can get involved. Please come if you’re at all interested!

If there’s other things you think we can do to be more globally accessible, don’t hesitate to jump on the OpenSSF Slack, we’d love to hear from you.

Thanks!

– Brian

As someone who has spent their entire career in open source software (OSS), the Log4Shell scramble (an industry-wide four-alarm-fire to address a serious vulnerability in the Apache Log4j package) is a humbling reminder of just how far we still have to go. OSS is now central to the functioning of modern society, as critical as highway bridges, bank payment platforms, and cell phone networks, and it’s time OSS foundations started to act like it.

Organizations like the Apache Software Foundation, the Linux Foundation, the Python Foundation, and many more, provide legal, infrastructural, marketing and other services for their communities of OSS developers. In many cases the security efforts at these organizations are under-resourced and hamstrung in their ability to set standards and requirements that would mitigate the chances of major vulnerabilities, for fear of scaring off new contributors. Too many organizations have failed to apply raised funds or set process standards to improve their security practices, and have unwisely tilted in favor of quantity over quality of code.

What would “acting like it” look like? Here are a few things that OSS foundations can do to mitigate security risks:

1. Set up an organization-wide security team to receive and triage vulnerability reports, as well as coordinate responses and disclosures to other affected projects and organizations.
2. Perform frequent security scans, through CI tooling, for detecting unknown vulnerabilities in the software and recognizing known vulnerabilities in dependencies.
3. Perform occasional outside security audits of critical code, particularly before new major releases.
4. Require projects to use test frameworks, and ensure high code coverage, so that features without tests are discouraged and underused features are weeded out proactively.
5. Require projects to remove deprecated or vulnerable dependencies. (Some Apache projects are not vulnerable to the Log4j v2 CVE, because they are still shipping with Log4j v1, which has known weaknesses and has not received an update since 2015!)
6. Encourage, and then eventually require, the use of SBOM formats like SPDX to help everyone track dependencies more easily and quickly, so that vulnerabilities are easier to find and fix.
7. Encourage, and then eventually require, maintainers to demonstrate familiarity with the basics of secure software development practices.

Many of these are incorporated into the CII Best Practices badge, one of the first attempts to codify these into an objective comparable metric, and an effort that has now moved to OpenSSF. The OpenSSF has also published a free course for developers on how to develop secure software, and SPDX has recently been published as an ISO standard.

None of the above practices is about paying developers more, or channeling funds directly from users of software to developers. Don’t get me wrong, open source developers and the people who support them should be paid more and appreciated more in general. However, it would be an insult to most maintainers to suggest that if you’d just slipped more money into their pockets they would have written more secure code. At the same time, it’s fair to say a tragedy-of-the-commons hits when every downstream user assumes that these practices are in place, being done and paid for by someone else.

Applying these security practices and providing the resources required to address them is what foundations are increasingly expected to do for their community. Foundations should begin to establish security-related requirements for their hosted and mature projects. They should fundraise from stakeholders the resources required for regular paid audits for their most critical projects, scanning tools and CI for all their projects, and have at least a few paid staff members on a cross-project security team so that time-critical responses aren’t left to individual volunteers. In the long term, foundations should consider providing resources to move critical projects or segments of code to memory-safe languages, or fund bounties for more tests.

The Apache Software Foundation seems to have much of this right, let’s be clear. Despite being notified just before the Thanksgiving holiday, their volunteer security team worked with the Log4j maintainers and responded quickly. Log4j also has almost 8000 passing tests in its CI pipeline, but even all that testing didn’t catch the way this vulnerability could be exploited. And in general, Apache projects are not required to have test coverage at all, let alone run the kind of SAST security scans or host third party audits that might have caught this.

Many other foundations, including those hosted at the Linux Foundation, also struggle to do all this – this is not easy to push through the laissez-faire philosophy that many foundations have regarding code quality, and third-party code audits and tests don’t come cheap. But for the sake of sustainability, reducing the impact on the broader community, and being more resilient, we have got to do better. And we’ve got to do this together, as a crisis of confidence in OSS affects us all.

This is where OpenSSF comes in, and what pulled me to the project in the first place. In the new year you’ll see us announce a set of new initiatives that build on the work we’ve been doing to “raise the floor” for security in the open source community. The only way we do this effectively is to develop tools, guidance, and standards that make adoption by the open source community encouraged and practical rather than burdensome or bureaucratic. We will be working with and making grants to other open source projects and foundations to help them improve their security game. If you want to stay close to what we’re doing, follow us on Twitter or get involved in other ways. For a taste of where we’ve been to date, read our segment in the Linux Foundation Annual Report, or watch our most recent Town Hall.

Hoping for a 2022 with fewer four alarm fires,

Brian

Brian Behlendorf is General Manager of the Linux Foundation’s Open Source Security Foundation (OpenSSF). He was a founding member of the Apache Group, which later became the Apache Software Foundation, and served as president of the foundation for three years.