click here to enlarge image
In our component database, where we collect meta and clearing information on components, we added the Open Source Security Foundation (OpenSSF) Scorecard to help exploring the security status of open source projects. The score, introduced by the OpenSSF project of the Linux Foundation in 2020 and currently evaluated on regular basis for about 1 million open source projects on Github, is an aggregated value reflecting the security measures taken by the open source project. It can be used as an indication on how much you may trust the security efforts of a particular project without having evaluated it further.
What does the Scorecard tell?
The scorecard value or score is the result of sixteen checks reflecting secure software development best practises. They comprise the domains of development, testing, maintenance and vulnerabilities but also code and build management. Based on the a comprehensive set of best practises the tests scan the code repository for evidence, that the practises are actively supported by the project.
Currently 18 tests are available, 16 of which are available through the API. The detailed documentation can be found here. Each test will receive a score between 0 and 10, with 10 being the best possible score. The tests come with a result and and a risk as weight. The sum of all tests together with their weight derives the total score.
Some tests may sometimes not be applicable due to project design decisions, e.g. if the project does not supply packages through Github, the packaging test will not apply, since the current implementation does not yet provide a mechanism to verify the different package managers.
However, given you want to make a decision whether or not to use a particular component, running a scorecard test – or looking at the component in our database – will help you getting an impression on what effort you might need to invest in securing the component. The higher the score the more you may trust on the component.
What does the Scorecard NOT tell?
Please do not understand a high score as a guarantee for a secure component! Also a low score does not immediately relate to a weak or flawy component! There is no logic in assuming that a low score is an indication for a vulnerable component!
The score indicates which steps the project takes to ensure the code it provides follows best practises and therefor has a high likeliness of being free from errors and vulnerabilities. But it is no guarantee! If all is done fine, all tests boost to 10, there still might be the chance that a vulnerability occurs in an upstream component which is not simple or possible to fix for the project itself.
Use the score as an indicator but make the decision of whether to use a component or not based on its functionality not only on the score. You will – especially in these early days when the score is not yet widely adopted –
What comes next?
However, we highly recommend using scorecards because they give an indication of how strongly you may rely on your upstream components.
Since TrustSource knows all the components you apply inside your solution, it will now be possible to make more out of the single scores. A simple average will not make sense. Due to the amount of components an average score will have to be expected somewhere at a meaningless 5. But we are currently experimenting with quantiles or top 10 and low 10 averages as well as the relation of not scored components compared to scored ones.
In addition we will provide a service, that will allow you to check your own components by just providing a URL and transferring the scorecard to non github projects. Given we achieve some success, we will contribute our developments back to OpenSSF.