Inspiration - Correlation between Security Experience and Requirements Quality

As we analyze our survey responses, we see a pattern emerging. A sort of a correlation between security experience and the quality of requirements they put in. We have not analyzed the data yet in terms of this aspect, but is one definitely worth studying. Increasing the scope of the paper? Maybe yes. It'll be an interesting observation.

One more thing we observe is a correlation between security experience and awareness of the CWE database. That would be an interesting study too. Looking forward to the results.

Follow Up Interviews - SE Faculty

As a follow up to our surveys, we did a couple of interviews with SE faculty. Both of them seemed to concur with a majority of the requirements we obtained through the survey. However, two requirements stood out and showed how Security Experience could influence the quality of requirements we got. Could that be added as a result to our paper? We think yes. Any Opinions? 
To people who are curious to know what stood out:
1. The ability to add and customize the rules we have to analyze security.
2. Domain specific rules 

They could form a significant part of the requirement space. Security Experience does influence the quality of requirements we get. Surprised? Maybe not. But could the others have thought about this? Definitely not.

Requirements Trends Till Date

It has been interesting to observe the responses to the survey till date. The following are the trends we observed till now:

1. Developers do not want a static analyzer, as they are already spoilt for choices.

2. 80% of the respondents wanted the application to be able to run as a plugin to their IDE's or other development environments

3. 87% (intermediate) felt that this would be useful throughout the implementation iterations.

Research Questions - Only Certainty of Grad Life?

Some of the questions we have come up with during our literature survey, lectures and other ideas that have come up during our discussion:
1. What does it take to find out potential vulnerabilities?
2. Can attacks be generalized into regular expressions for parsing and detection?
3. Can solutions for countering attacks be generalized? Again, regex?
4. How can static analyzers lead us into vulnerability modelling? 
5. What is the applicability of Static Analyzers in vulnerability analysis?

Anyone with inputs, please do post your comments

Security Metrics - A perspective with respect to the research

Software Metrics are a very useful way of measuring software artifacts. Software security metrics especially enable software engineers to have a good understanding of the potential threats and the mitigation strategies that they can use to counter the threats. The ISO 15408 standard specifies a set of security requirements that a software should have with resepect to security. While the tool that we propose might not incorporate this feature, as this is with respect to the requirements phase, a paper that we read, and related papers [REF 1-10] gave us inputs on what metrics we could measure on code with respect to security. The tool that we propose parses the codebase and points out potential areas of vulnerability. 

This could be extended to point out the number of security errors, which in turn could be integrated with any bug tracking system that the team might use and gather the number of bugs. The ratio of the number of security issues to the number of issues in total is a good indicator of how secure the software is as per the references. This could be a potential requirement out of the tool, which can point out this metric to give a lead-in to the developers on concentrating their fortifications efforts.

Opinions, Readers?

References to this Idea: (Might be of use to what you're doing)

[1] D. P. Gilliam, T. L.Wolfe, J. S. Sherif, Software Security Checklist for the Software Life Cycle, In Proc. of the 12th International Workshops on Enabling Technologies, 2003, 243-248.

[2] J. A. Chaula, L. Yngström, and S. Kowalski; Security Metrics and Evaluation of Information Systems Security, In Proceedings of the 4th Annual Conference on Information Security for South Africa, 2004.

[3] R. Scandariato, B. D. Win, and W. Joosen, Towards a Measuring Framework for Security Properties of Software, In Proc. of the 2nd workshop on Quality of Protection, 2006, 27-30.

[4] A. Sachitano, R. O. Chapman, and J. A. Hamilton, “Security in software architecture: a case study,” in Proceedings from the Fifth Annual IEEE SMC Information Assurance Workshop, 2004, pp. 370–376.

[5] M. Swanson, N. Bartol, J. Sabato, J. Hash, and L. Graffo, Security Metrics Guide for Information Technology Systems, NIST Special Publication 800- 55, National Institute of Standards and Technology, 2003.

[6] Sultan, K.; En-Nouaary, A.; Hamou-Lhadj, A.; , "Catalog of Metrics for Assessing Security Risks of Software throughout the Software Development Life Cycle," Information Security and Assurance, 2008. ISA 2008. International Conference on , vol., no., pp.461-465, 24-26 April 2008
doi: 10.1109/ISA.2008.104
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4511611&isnumber=4511515

[7] V. R. Basili, G. Caldiera and H. D. Rombach, Goal Question Metric Paradigm, In J. J. Marciniak (ed.), Encyclopedia of Software Engineering 1, New York: John Wiley & Sons, 1994, 528-532.

[8] ISO/IEC15408 Standards: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=40612

[9] R. Scandariato, B. D. Win, and W. Joosen, Towards a Measuring Framework for Security Properties of Software, In Proc. of the 2nd workshop on Quality of Protection, 2006, 27-30.

[10] Meneely, A.,Williams, L.,Proceedings CCS '09 Proceedings of the 16th ACM conference on Computer and communications security ACM New York, NY, USA ©2009  ISBN: 978-1-60558-894-0 doi>10.1145/1653662.1653717

Research Methodolgy

The survey that we have planned will consist of two sets of questions, which will be as follows:

• The first set of questions will be on developer awareness with respect to software security.
• The second set of questions will center on the expectations/requirements of a tool that will parse through the code-base of an application and indicate to the developers the fragments where there is a potential for attack based on the CWE top 25 techniques.

Once the responses have been obtained, we plan to statistically analyze them to substantiate our hypothesis that developers are in general unaware of the CWE identifiers and instead would be better off using a tool like the one we have proposed to build to minimize security vulnerabilities in their software.

We plan to analyze the second set of responses which are user expectations and prioritize them based on responses that had the highest consensus amongst the participants.

The prioritized requirements will form the basic vision for the tool we plan to develop and act as guidance for meeting the user expectations/requirements during the course of development.

Motivations for the Paper

• A study we conducted shows that 70% of the vulnerabilities involve fixes to them through new code, that is, adding a new functionality that will render the attack ineffective
• 45% of the vulnerabilities that formed a part of the study were from the CWE top 25 vulnerabilities
• Since most vulnerability fixes involved new code, static analyzers could be rendered ineffective
• Developers might not have time to learn about all the vulnerabilities and would be benefitted by a tool that will point the potential places where the code is vulnerable