Fans of free and open source software (FOSS) may recall a report from Coverity last year that found open source code typically has fewer defects per thousand lines of code than proprietary software code does.
Fast forward to this year, and the news is even more striking.
Following the analysis of more than 450 million lines of software code through the Coverity Scan service, Coverity’s 2012 Coverity Scan Open Source Report, which was released Tuesday, concludes that “Linux remains the benchmark for quality.”
‘Differing dynamics’
Coverity’s service, which was initiated in 2006 by Coverity and the U.S. Department of Homeland Security, has now become a widely accepted standard for measuring the state of open source software quality.
According to this year’s results, open source projects with between 500,000 and 1,000,000 lines of code had an average ‘defect density’ of just .44, whereas proprietary code scored at .98 for such projects. Defect density refers to the number of defects per 1000 lines of software code.
For projects with more than one million lines of code, on the other hand, defect density decreased to .66 in proprietary code but increased all the way to .75 for open source projects.
“This discrepancy can be attributed to differing dynamics within open source and proprietary development teams, as well as the point at which these teams implement formalized development testing processes,” Coverity explained.
For two years, both proprietary and open source users of Coverity’s Scan Service have demonstrated better quality than the accepted industry standard defect density of 1.0.
Defect densities below 0.7
Particularly interesting for fans of Linux, however, is that the free and open source operating system “remains a benchmark for quality,” in Coverity’s estimation.
“Since the original Coverity Scan report in 2008, scanned versions of Linux have consistently achieved a defect density of less than 1.0, and versions scanned in 2011 and 2012 demonstrated a defect density below .7,” the company explained.
Whereas Coverity scanned more than 6.8 million lines of Linux code in 2011 and found a defect density of .62, the 2012 report included a scan of more than 7.4 million lines of Linux code and found a defect density of .66.
Most recently, Coverity scanned 7.6 million lines of code in Linux 3.8 and found a defect density of just .59.
A copy of Coverity’s full report is available as a free download.