Inspiration - Correlation between Security Experience and Requirements Quality

As we analyze our survey responses, we see a pattern emerging. A sort of a correlation between security experience and the quality of requirements they put in. We have not analyzed the data yet in terms of this aspect, but is one definitely worth studying. Increasing the scope of the paper? Maybe yes. It'll be an interesting observation.

One more thing we observe is a correlation between security experience and awareness of the CWE database. That would be an interesting study too. Looking forward to the results.

Follow Up Interviews - SE Faculty

As a follow up to our surveys, we did a couple of interviews with SE faculty. Both of them seemed to concur with a majority of the requirements we obtained through the survey. However, two requirements stood out and showed how Security Experience could influence the quality of requirements we got. Could that be added as a result to our paper? We think yes. Any Opinions? 
To people who are curious to know what stood out:
1. The ability to add and customize the rules we have to analyze security.
2. Domain specific rules 

They could form a significant part of the requirement space. Security Experience does influence the quality of requirements we get. Surprised? Maybe not. But could the others have thought about this? Definitely not.

Requirements Trends Till Date

It has been interesting to observe the responses to the survey till date. The following are the trends we observed till now:

1. Developers do not want a static analyzer, as they are already spoilt for choices.

2. 80% of the respondents wanted the application to be able to run as a plugin to their IDE's or other development environments

3. 87% (intermediate) felt that this would be useful throughout the implementation iterations.

Research Questions - Only Certainty of Grad Life?

Some of the questions we have come up with during our literature survey, lectures and other ideas that have come up during our discussion:
1. What does it take to find out potential vulnerabilities?
2. Can attacks be generalized into regular expressions for parsing and detection?
3. Can solutions for countering attacks be generalized? Again, regex?
4. How can static analyzers lead us into vulnerability modelling? 
5. What is the applicability of Static Analyzers in vulnerability analysis?

Anyone with inputs, please do post your comments

Security Metrics - A perspective with respect to the research

Software Metrics are a very useful way of measuring software artifacts. Software security metrics especially enable software engineers to have a good understanding of the potential threats and the mitigation strategies that they can use to counter the threats. The ISO 15408 standard specifies a set of security requirements that a software should have with resepect to security. While the tool that we propose might not incorporate this feature, as this is with respect to the requirements phase, a paper that we read, and related papers [REF 1-10] gave us inputs on what metrics we could measure on code with respect to security. The tool that we propose parses the codebase and points out potential areas of vulnerability. 

This could be extended to point out the number of security errors, which in turn could be integrated with any bug tracking system that the team might use and gather the number of bugs. The ratio of the number of security issues to the number of issues in total is a good indicator of how secure the software is as per the references. This could be a potential requirement out of the tool, which can point out this metric to give a lead-in to the developers on concentrating their fortifications efforts.

Opinions, Readers?

References to this Idea: (Might be of use to what you're doing)

[1] D. P. Gilliam, T. L.Wolfe, J. S. Sherif, Software Security Checklist for the Software Life Cycle, In Proc. of the 12th International Workshops on Enabling Technologies, 2003, 243-248.

[2] J. A. Chaula, L. Yngström, and S. Kowalski; Security Metrics and Evaluation of Information Systems Security, In Proceedings of the 4th Annual Conference on Information Security for South Africa, 2004.

[3] R. Scandariato, B. D. Win, and W. Joosen, Towards a Measuring Framework for Security Properties of Software, In Proc. of the 2nd workshop on Quality of Protection, 2006, 27-30.

[4] A. Sachitano, R. O. Chapman, and J. A. Hamilton, “Security in software architecture: a case study,” in Proceedings from the Fifth Annual IEEE SMC Information Assurance Workshop, 2004, pp. 370–376.

[5] M. Swanson, N. Bartol, J. Sabato, J. Hash, and L. Graffo, Security Metrics Guide for Information Technology Systems, NIST Special Publication 800- 55, National Institute of Standards and Technology, 2003.

[6] Sultan, K.; En-Nouaary, A.; Hamou-Lhadj, A.; , "Catalog of Metrics for Assessing Security Risks of Software throughout the Software Development Life Cycle," Information Security and Assurance, 2008. ISA 2008. International Conference on , vol., no., pp.461-465, 24-26 April 2008
doi: 10.1109/ISA.2008.104
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4511611&isnumber=4511515

[7] V. R. Basili, G. Caldiera and H. D. Rombach, Goal Question Metric Paradigm, In J. J. Marciniak (ed.), Encyclopedia of Software Engineering 1, New York: John Wiley & Sons, 1994, 528-532.

[8] ISO/IEC15408 Standards: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=40612

[9] R. Scandariato, B. D. Win, and W. Joosen, Towards a Measuring Framework for Security Properties of Software, In Proc. of the 2nd workshop on Quality of Protection, 2006, 27-30.

[10] Meneely, A.,Williams, L.,Proceedings CCS '09 Proceedings of the 16th ACM conference on Computer and communications security ACM New York, NY, USA ©2009  ISBN: 978-1-60558-894-0 doi>10.1145/1653662.1653717

Research Methodolgy

The survey that we have planned will consist of two sets of questions, which will be as follows:

• The first set of questions will be on developer awareness with respect to software security.
• The second set of questions will center on the expectations/requirements of a tool that will parse through the code-base of an application and indicate to the developers the fragments where there is a potential for attack based on the CWE top 25 techniques.

Once the responses have been obtained, we plan to statistically analyze them to substantiate our hypothesis that developers are in general unaware of the CWE identifiers and instead would be better off using a tool like the one we have proposed to build to minimize security vulnerabilities in their software.

We plan to analyze the second set of responses which are user expectations and prioritize them based on responses that had the highest consensus amongst the participants.

The prioritized requirements will form the basic vision for the tool we plan to develop and act as guidance for meeting the user expectations/requirements during the course of development.

Motivations for the Paper

• A study we conducted shows that 70% of the vulnerabilities involve fixes to them through new code, that is, adding a new functionality that will render the attack ineffective
• 45% of the vulnerabilities that formed a part of the study were from the CWE top 25 vulnerabilities
• Since most vulnerability fixes involved new code, static analyzers could be rendered ineffective
• Developers might not have time to learn about all the vulnerabilities and would be benefitted by a tool that will point the potential places where the code is vulnerable

Research Objective and Contributions - Scope

The objective of this research is to:

1. Gauge security awareness amongst developers.
2. Gather requirements for a tool that will parse through the code-base of an application and identify areas in the code with a potential vulnerability with respect to the CWE top 25 errors.

In order to perform this research, we made use of the Grounded Theory Analysis [REF] to gauge the awareness of security principles amongst developers in the industry and to analyze a set of common requirements which the developers and analysts need and expect out of the tool mentioned in item 2.

Security-aware Software Development Life Cycle (SaSDLC) - Processes and Tools)

This article talks about a Security-aware Software Development Life Cycle(SaSDLC). It’s main focus is on Next Generation Internet applications but of course the content is helpful, I would say, to any type of application being developed. 
The process is described as follows and a tool was developed to help with the process. Functional requirements are gathered using UML tools, assets are identified, evaluated, and categorized, misuse cases are identified taking STRIDE (Spoofing Identity, Tampering with Data, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege) and C15A (Confidentiality, Integrity, Availability, Authentication, Authorization, Accounting, and Anonymity) into account, an attack tree is developed with each misuse case as a node, threats are then rated using DREAD, threats become categorized as high, moderate, or low, and are then compared to asset values, high values have to be secure and if the threat is too expensive to do in-vivo (within the application), it doesn’t need to be done in-vitro, the threats then become functional requirements and then just iterate through the process. It is also talked about using security design patterns when developing, and the different types of testing that should be done.

RUP Extension for the Development of Secure Systems

This paper proposes an extension to RUP that deals with security. It was noted that RUP doesn’t deal with security directly but its documented through supplementary requirements. The idea then was to have security directly involved with the process. Instead of using UML (Unified Modeling Language) to model the process, this paper uses UMA (Unified Modeling Architecture), which combines concepts from various sources. Security becomes directly involved by being a discipline that follows activities, tasks, roles, and work products. 

This was chosen; extending RUP, instead of customizing it because with customizing it could get difficult since there are workflows that interact with each other. There are 6 activities proposed; those being Security Plan Development, Misuse Cases Definition, Threat Identification and Analysis, Security Requirements Definition, Security Policy, Architecture Refinement for Security, and Security Test Elaboration.  

There are five roles proposed: Security Manager, Security Engineer, Security Software Architect, Security Manager, and Security Tester. There are quite a few workproducts that can be read in the paper. An example is given of how a system was built with the proposed extension.

Review of Improving Software Security by Eliminating the CWE Top 25 Vulnerabilities

In the article Improving Software Security by Eliminating the CWE Top 25 Vulnerabilities, Michael Howard describes the Top 25 CWE vulnerabilities. He explains how attackers use these vulnerabilities to do their "work," and how to best combat them. This is very helpful in understanding what exactly what CWE is and how it can/should be used.

Each CVE is described and possible mitigation strategies are mentioned. Its useful theoretically, but what needs to be analyzed is how to translate these theories into practice. It is one thing to say CVE's can be mitigated by using so and so strategies. But actually translating them into developer aids, thats what we're planning to do. 

There goes your introduction. We're a team of 3 students at Rochester Institute of Technology, New York trying to gather requirements for a Security Analyzer tool, tentatively titled "Code-Sentinel". Your inputs in the comments section would be very valuable to us, and in turn aid the developer community to produce secure software. Yes, our goals seem lofty, but we're equally determined to achieve them.