Multiple software products often exist on the same server and therefore vulnerability in one product might compromise the entire system. It is imperative to perform a security risk assessment during the selection of the candidate software products that become part of a larger system. Having a quantitative security risk assessment model provides an objective criterion for such assessment and comparison between candidate software systems. In this paper, we present a software product evaluation method using such a quantitative security risk assessment model. This method utilizes prior research in quantitative security risk assessment, which is based on empirical data from the National Vulnerability Database (NVD), and compares the security risk levels of the products evaluated. We introduced topic modeling to build a security risk assessment model. The risk model is created using Latent Dirichlet Allocation (LDA) to classify the vulnerabilities into topics, which are then used as the measurement instruments to evaluate the candidate software product. Such a procedure could supplement the existing selection process, to assist the decision makers to evaluate open-source software (OSS) systems, to ensure that it is safe and secure enough to be put into their environment. Finally, the procedure is demonstrated using an experimental case study.

1. J. L. Bayuk, "Systems Security Engineering," Security & Privacy, IEEE, vol. 9, pp. 72-74, 2011.
2. INCOSE, ""INCOSE Systems Engineering Handbook," vol. 3. 2.
3. D. Childs, "Information technology security system engineering methodology," in Aerospace Conference, 2003. Proceedings. 2003 IEEE, 2003, pp. 3393-3401.
4. N. F. Schneidewind, "Methods for assessing COTS reliability, maintainability, and availability," in Software Maintenance, 1998. Proceedings. International Conference on, 1998, pp. 224-225.
5. R. Khan, "Secure software development: a prescriptive framework," Computer Fraud & Security, vol. 2011, pp. 12-20, 2011.
6. I. Mkpong-Ruffin, D. Umphress, J. Hamilton, and J. Gilbert, "Quantitative software security risk assessment model," presented at the Proceedings of the 2007 ACM workshop on Quality of protection, Alexandria, Virginia, USA, 2007.
7. J. A. Wang, M. Guo, H. Wang, M. Xia, and L. Zhou, "Ontology-based security assessment for software products," presented at the Proceedings of the 5th Annual Workshop on Cyber Security and Information Intelligence Research: Cyber Security and Information Intelligence Challenges and Strategies, Oak Ridge, Tennessee, 2009.
8. S. H. Houmb, V. N. L. Franqueira, and E. A. Engum, "Quantifying security risk level from CVSS estimates of frequency and impact," Journal of Systems and Software, vol. 83, pp. 1622-1634, 2010.
9. W. Zhihu and W. Xin, "Research on technologies in quantitative risk assessment and forcast of network security," in Advanced Computer Theory and Engineering (ICACTE), 2010 3rd International Conference on, 2010, pp. V6-524-V6-528.
10. H. Joh, "Quantitative analyses of software vulnerabilities," Ph. D. 3489881, Colorado State University, United States -- Colorado, 2011.
11. Y. L. Chenmeng Sui, Yun Liu, "A Software Security Assessment System Based On Analysis of Vulnerabilities," Journal of Convergence Information Technology, vol. 7, p. 211 ~ 219, 2012.
12. S. -W. Woo, H. Joh, O. H. Alhazmi, and Y. K. Malaiya, "Modeling vulnerability discovery process in Apache and IIS HTTP servers," Computers & Security, vol. 30, pp. 50-62, 2011.
13. A. Austin and L. Williams, "One Technique is Not Enough: A Comparison of Vulnerability Discovery Techniques," in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, 2011, pp. 97-106.
14. W. J. Sung, J. H. Kim, and S. Y. Rhew, "A Quality Model for Open Source Software Selection," in Advanced Language Processing and Web Information Technology, 2007. ALPIT 2007. Sixth International Conference on, 2007, pp. 515-519.
15. MITRE. (6/24/2012). CWE -Common Weakness Enumeration. Available: http://cwe. mitre. org/compatible/category. html
16. "Probabilistic Topic Models," Communications of the ACM, vol. 55, pp. 77-84, 2012.
17. G. Anthes, "Topic Models Vs. Unstructured Data," Communications of the ACM, vol. 53, pp. 16-18, 2010.
18. H. U. Asuncion, A. U. Asuncion, and R. N. Taylor, "Software traceability with topic modeling," in Software Engineering, 2010 ACM/IEEE 32nd International Conference on, 2010, pp. 95-104.
19. E. Linstead, P. Rigor, S. Bajracharya, C. Lopes, and P. Baldi, "Mining concepts from code with probabilistic topic models," presented at the Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, Atlanta, Georgia, USA, 2007.
20. C. Kuan-Yu, C. Hsuan-Sheng, and B. Chen, "Latent topic modeling of word vicinity information for speech recognition," in Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, 2010, pp. 5394-5397.
21. R. A. Negoescu and D. Gatica-Perez, "Modeling Flickr Communities Through Probabilistic Topic-Based Analysis," Multimedia, IEEE Transactions on, vol. 12, pp. 399-416, 2010.
22. S. W. Thomas, B. Adams, A. E. Hassan, and D. Blostein, "Validating the Use of Topic Models for Software Evolution," in Source Code Analysis and Manipulation (SCAM), 2010 10th IEEE Working Conference on, 2010, pp. 55-64.
23. C. Xin, H. Xiaohua, S. Xiajiong, and G. Rosen, "Probabilistic topic modeling for genomic data interpretation," in Bioinformatics and Biomedicine (BIBM), 2010 IEEE International Conference on, 2010, pp. 149-152.
24. D. Magatti, S. Calegari, D. Ciucci, and F. Stella, "Automatic Labeling of Topics," in Intelligent Systems Design and Applications, 2009. ISDA '09. Ninth International Conference on, 2009, pp. 1227-1232.
25. A. Hindle, N. A. Ernst, M. W. Godfrey, and J. Mylopoulos, "Automated topic naming to support cross-project analysis of software maintenance activities," presented at the Proceedings of the 8th Working Conference on Mining Software Repositories, Waikiki, Honolulu, HI, USA, 2011.
26. A. R. Hevner, S. T. March, J. Park, and S. Ram, "Design science in information systems research," MIS Q. , vol. 28, pp. 75-105, 2004.
27. NVD. National vulnerability database. Available: http://nvd. nist. gov
28. A. Ozment, "Improving vulnerability discovery models," presented at the Proceedings of the 2007 ACM workshop on Quality of protection, Alexandria, Virginia, USA, 2007.
29. G. Schryen and R. Kadura, "Open source vs. closed source software: towards measuring security," presented at the Proceedings of the 2009 ACM symposium on Applied Computing, Honolulu, Hawaii, 2009.
30. C. Fruhwirth and T. Mannisto, "Improving CVSS-based vulnerability prioritization and response with context information," presented at the Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement, 2009.
31. Y. J. Chung, I. Kim, N. Lee, T. Lee, and H. P. In, "Security risk vector for quantitative asset assessment," presented at the Proceedings of the 2005 international conference on Computational Science and Its Applications - Volume Part II, Singapore, 2005.
32. W. Ju An, Z. Linfeng, G. Minzhe, W. Hao, and J. Camargo, "Measuring Similarity for Security Vulnerabilities," in System Sciences (HICSS), 2010 43rd Hawaii International Conference on, 2010, pp. 1-10.
33. S. Neuhaus and T. Zimmermann, "Security Trend Analysis with CVE Topic Models," in Software Reliability Engineering (ISSRE), 2010 IEEE 21st International Symposium on, 2010, pp. 111-120.
34. (06-01-2012). CVE -Common Vulnerabilities and Exposures (CVE). Available: http://cve. mitre. org/
35. J. A. Wang, H. Wang, M. Guo, and M. Xia, "Security metrics for software systems," presented at the Proceedings of the 47th Annual Southeast Regional Conference, Clemson, South Carolina, 2009.
36. N. Mendes, J. Duraes, and H. Madeira, "Benchmarking the Security of Web Serving Systems Based on Known Vulnerabilities," in Dependable Computing (LADC), 2011 5th Latin-American Symposium on, 2011, pp. 55-64.
37. K. Peffers, T. Tuunanen, M. Rothenberger, and S. Chatterjee, "A Design Science Research Methodology for Information Systems Research," Journal of Management Information Systems, vol. 24, pp. 45-77, 2008.
38. J. V. Juhani Iivari "Action Research and Design Science Research – Seemingly similar but decisively dissimilar," 17th European Conference on Information Systems, 2009.
39. K. E. N. Peffers, T. Tuunanen, M. A. Rothenberger, and S. Chatterjee, "A Design Science Research Methodology for Information Systems Research," Journal of Management Information Systems, vol. 24, pp. 45-77, Winter2007/2008 2007.
40. Stanford Topic Modeling Toolbox. Available: http://nlp. stanford. edu/software/tmt/tmt-0. 4/
41. (05/05/2012). Google Code. Available: http://code. google. com/p/skipfish/
42. (05/05/2012). Software [In]security: Comparing Apples, Oranges, and Aardvarks (or, All Static Analysis Tools Are Not Created Equal) | As the Market Grows | InformIT. Available: http://www. informit. com/articles/article. aspx?p=1680863
43. A. Doupe, M. Cova, and G. Vigna, "Why Johnny can't pentest: an analysis of black-box web vulnerability scanners," presented at the Proceedings of the 7th international conference on Detection of intrusions and malware, and vulnerability assessment, Bonn, Germany, 2010.
44. O. Hauge, T. Osterlie, C. -F. Sorensen, and M. Gerea, "An empirical study on selection of Open Source Software - Preliminary results," presented at the Proceedings of the 2009 ICSE Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development, 2009.
45. L. Suto, "Title," unpublished|.
46. A. G. Gary Stoneburner, and Alexis Feringa. (2002, 5/5/2012). Risk Management Guide for Information Technology Systems. NIST-SP 800:30.
47. T. Scholte, D. Balzarotti, and E. Kirda, "Have things changed now? An empirical study on input validation vulnerabilities in web applications," Computers & Security, vol. 31, pp. 344-356, 2012.
48. B. Jason, "State of the Art: Automated Black-Box Web Application Vulnerability Testing," 2010, pp. 332-345.

Software Security Quantitative Risk Assessment Software Evaluation Topic Modeling Lda