Software Engineering Institute Copyright (c) 2013 Carnegie Mellon University All rights reserved. http://repository.cmu.edu/sei Recent documents in Software Engineering Institute en-us Sun, 31 Mar 2013 01:31:40 PDT 3600 Quantifying Uncertainty in Expert Judgment: Initial Results http://repository.cmu.edu/sei/733 http://repository.cmu.edu/sei/733 Fri, 29 Mar 2013 07:17:50 PDT The work described in this report, part of a larger SEI research effort on Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE), aims to develop and validate methods for calibrating expert judgment. Reliable expert judgment is crucial across the program acquisition lifecycle for cost estimation, and perhaps most critically for tasks related to risk analysis and program management. This research is based on three field studies that compare and validate training techniques aimed at improving the participants’ skills to enable more realistic judgments commensurate with their knowledge.

Most of the study participants completed three batteries of software engineering domain-specific test questions. Some participants completed four batteries of questions about a variety of general knowledge topics for purposes of comparison. Results from both sets of questions showed improvement in the participants' recognition of their true uncertainty. The domain-specific training was accompanied by notable improvements in the relative accuracy of the participants' answers when more contextual information to the questions was given along with “reference points” about similar software systems. Moreover, the additional contextual information in the domain-specific training helped the participants improve the accuracy of their judgments while also reducing their uncertainty in making those judgments.

]]> Dennis Goldenson et al. Justification of a Pattern for Detecting Intellectual Property Theft by Departing Insiders http://repository.cmu.edu/sei/732 http://repository.cmu.edu/sei/732 Fri, 29 Mar 2013 07:17:47 PDT This paper describes an analysis that justifies applying the pattern “Increased Review for Intellectual Property (IP) Theft by Departing Insiders.” The pattern helps organizations plan, prepare, and implement a strategy to mitigate the risk of insider theft of IP. The analysis shows that organizations can reduce their risk of insider theft of IP through increased review of departing insiders’ actions during a relatively small window of time prior to their departure. Preliminary research results show that approximately 70% of insider IP thieves can be caught by following the pattern’s recommendation of reviewing insiders’ actions for theft events during only the last two months of their employment. These results provide practical guidance for practitioners wishing to fine tune the application of the pattern for their organizations. “Increased Review for IP Theft by Departing Insiders” is part of the CERT(R) Insider Threat Center’s evolving library of enterprise architectural patterns for mitigating the insider threat, based on the Center’s collected data. The Center’s larger goal is to foster greater organizational resilience to insider threat, using repeated application of patterns from the library.

]]> Andrew P. Moore et al. A Workshop on Architecture Competence http://repository.cmu.edu/sei/731 http://repository.cmu.edu/sei/731 Fri, 29 Mar 2013 07:17:45 PDT This report summarizes a workshop on architecture competence that was held at the Carnegie Mellon ® Software Engineering Institute (SEI) in June of 2008. The SEI invited accomplished practitioners from government, academia, and industry to discuss key issues in assessing the competence of organizations that use architecture to produce software-reliant systems. After several opening talks by individuals who recounted their experience in competence improvement efforts, workshop participants divided into working groups. Each group was tasked with working on a specific set of issues and was asked to produce a set of questions that could appear in a competence assessment instrument.

]]> Len Bass et al. The MAL: A Malware Analysis Lexicon http://repository.cmu.edu/sei/730 http://repository.cmu.edu/sei/730 Wed, 13 Mar 2013 10:15:27 PDT The lack of a controlled vocabulary for malware analysis is a symptom of the field's immaturity and an impediment to its growth. Malware analysis is a splintered discipline, with many small teams that for cultural reasons do not, or cannot, readily communicate among themselves; this condition encourages the growth of many local dialects. This report presents the results of the Malware Analysis Lexicon (MAL) initiative, a small project to develop the discipline's first common vocabulary.

]]> David McIntire et al. Detecting and Preventing Data Exfiltration Through Encrypted Web Sessions via Traffic Inspection http://repository.cmu.edu/sei/729 http://repository.cmu.edu/sei/729 Wed, 13 Mar 2013 10:15:24 PDT Web-based services, such as email, are useful for communicating with others either within or outside of an organization; however, they are a common threat vector through which data exfiltration can occur. Despite this risk, many organizations permit the use of web-based services on their systems. Implementing a method to detect and prevent data exfiltration through these channels is essential to protect an organization's sensitive documents. This report presents methods that can be used to detect and prevent data exfiltration using a Linux-based proxy server in a Microsoft Windows environment. Tools such as Squid Proxy, Clam Antivirus, and C-ICAP are explored as means by which information technology (IT) professionals can centrally log and monitor web-based services on Microsoft Windows hosts within an organization. Also introduced is a Tagger tool developed by the CERT Insider Threat Center that enables information security personnel to quickly insert tags into documents. These tags can then be used to create signatures for use on the proxy server to prevent documents from leaving the organization. In addition, the use of audit logs is also explored as an aid in determining whether sensitive data may have been uploaded to an internet service by a malicious insider.

]]> George J. Silowash et al. Insider Threat Control: Using Universal Serial Bus (USB) Device Auditing to Detect Possible Data Exfiltration by Malicious Insiders http://repository.cmu.edu/sei/728 http://repository.cmu.edu/sei/728 Tue, 22 Jan 2013 12:01:53 PST Universal serial bus (USB) storage devices are useful for transferring information within an organization; however, they are a common threat vector through which data exfiltration can occur. Despite this, many organizations permit the use of USB devices on their systems. Implementing controls to track the use of these devices is necessary if organizations wish to retain situational awareness and auditing capabilities during a data theft incident. This report presents methods to audit USB device use within a Microsoft Windows environment. Using various tools-the Windows Task Scheduler, batch scripts, Trend Micro's OSSEC host-based intrusion-detection system (HIDS), and the Splunk log analysis engine-we explore means by which information technology (IT) professionals can centrally log and monitor USB device use on Microsoft Windows hosts within an organization. In addition, we discuss how the central collection of audit logs can aid in determining whether sensitive data may have been copied from a system by a malicious insider.

]]> George J. Silowash et al. A Decision Framework for Selecting Licensing Rights for Noncommercial Computer Software in the DoD Environment http://repository.cmu.edu/sei/727 http://repository.cmu.edu/sei/727 Thu, 17 Jan 2013 07:48:35 PST A major acquisition challenge for a program where computer software is a critical element of the system is the upfront determination of an appropriate licensing rights strategy. This report describes standard noncommercial software licensing alternatives as defined by U.S. government and Department of Defense (DoD) regulations. It also suggests an approach for objectively identifying agency needs for license rights and the appropriate license type for systems with noncommercial computer software or as standalone software in the DoD environment. There are three standard license types for noncommercial computer software: Unlimited, Government Purpose, and Restricted. Each of these license types for noncommercial computer software conveys different rights to the agency. This report presents distinguishing characteristics of the three standard license types, a method to develop the supporting rationale or traceability for DoD agency needs, a high-level description of circumstances that fall outside of standard license types, and a discussion of the importance of deliverables as necessary components for implementing license rights.

]]> Charlene Gross A Preliminary Model of Insider Theft of Intellectual Property http://repository.cmu.edu/sei/726 http://repository.cmu.edu/sei/726 Thu, 17 Jan 2013 07:48:33 PST A study conducted by the CERT Program at Carnegie Mellon University's Software Engineering Institute analyzed hundreds of insider cyber crimes across U.S. critical infrastructure sectors. Follow-up work involved detailed group modeling and analysis of 48 cases of insider theft of intellectual property. In the context of this paper, insider theft of intellectual property includes incidents in which the insider's primary goal is stealing confidential or proprietary information from the organization. This paper describes general observations about and a preliminary system dynamics model of this class of insider crime based on our empirical data. This work generates empirically-based hypotheses for validation and a basis for identifying mititgative measures in future work.

]]> Andrew P. Moore et al. Issues and Opportunities for Improving the Quality and Use of Data in the Department of Defense http://repository.cmu.edu/sei/725 http://repository.cmu.edu/sei/725 Thu, 17 Jan 2013 07:48:30 PST The Department of Defense (DoD) is becoming increasingly aware of the importance of data quality to its operations, leading to an interest in methods and techniques that can be used to determine and improve the quality of its data. The Office of the Secretary of Defense for Acquisition, Technology, and Logistics (OSD [AT&L]), Director, Defense Research & Engineering (DDR&E) sponsored a workshop to bring together leading researchers and practitioners to identify opportunities for research focused on data quality, data analysis, and data use. Seventeen papers were accepted for presentation during the workshop. During workshop discussion participants were asked to identify challenging areas that would address technology gaps and to discuss research ideas that would support future DoD policies and practices. The Software Engineering Institute formed three primary recommendations for areas of further research from the information produced at the workshop. These areas were integrating data from disparate sources, employing provenance analytics, and developing models, methods, and tools that support data quality by design.

]]> Mark Kasunic et al. Trusted Computing in Embedded Systems Workshop http://repository.cmu.edu/sei/724 http://repository.cmu.edu/sei/724 Thu, 17 Jan 2013 07:48:27 PST This report describes the November 2010 Trusted Computing in Embedded Systems Workshop held at Carnegie Mellon University. This workshop brought together various groups concerned with advancing research into improving the trustworthiness in embedded systems. The workshop format provided the opportunity to focus on embedded systems while examining the application of related trust technologies in order to foster collaborative approaches and information exchange in this area. Presentations and discussion addressed the capabilities and limitations of effectively employing trusted hardware-enabled components in embedded systems. This included, but was not restricted to, the following areas: new research and development in enabling trust in embedded systems, methods and techniques for establishing trust in embedded systems, lessons learned from research and development projects on embedded systems security, and gaps in current research. The workshop resulted in identification of gaps in current research and recommendations for potential research directions.

]]> Archie D. Andrews Jr. et al. Best Practices for National Cyber Security: Building a National Computer Security Incident Management Capability, Version 2.0 http://repository.cmu.edu/sei/723 http://repository.cmu.edu/sei/723 Thu, 17 Jan 2013 07:48:24 PST As nations recognize that their critical infrastructures have integrated sophisticated information and communications technologies (ICT) to provide greater efficiency and reliability, they quickly realize the need to effectively manage risk arising from the use of these technologies. Establishing a national computer security incident management capability can be an important step in managing that risk. In this document, this capability is referred to as a National CSIRT, although the specific organizational form may vary among nations. Nations face various challenges when working to strengthen incident management, such as the lack of information providing guidance for establishing a national capability, determining how this capability can support national cyber security, and managing the national incident management capability. This document, first in the Best Practices for National Cyber Security series, provides information that interested organizations and governments can use to develop a national incident management capability. The document explains the need for national incident management and provides strategic goals, enabling goals, and additional resources pertaining to the establishment of National CSIRTs and organizations like them.

]]> John Haller et al. Appraisal Requirements for CMMI® Version 1.3 (ARC, V1.3) http://repository.cmu.edu/sei/722 http://repository.cmu.edu/sei/722 Thu, 17 Jan 2013 07:48:21 PST This report, the Appraisal Requirements for CMMI, Version 1.3 (ARC, V1.3), defines the requirements for appraisal methods intended for use with Capability Maturity Model Integration (CMMI) and with the People CMM. The ARC may also be useful when defining appraisals with other reference models. The ARC defines three appraisal classes distinguished by the degree of rigor associated with the application of the method. These classes are intended primarily for people who develop appraisal methods to use with reference models such as those in the CMMI product suite.

]]> SCAMPI Upgrade Team Insider Threat Control: Using Centralized Logging to Detect Data Exfiltration Near Insider Termination http://repository.cmu.edu/sei/721 http://repository.cmu.edu/sei/721 Thu, 17 Jan 2013 07:31:25 PST Since 2001, the CERT Insider Threat Center has built an extensive library and comprehensive database containing more than 600 cases of crimes committed against organizations by insiders. A significant class of insider crimes, insider theft of intellectual property, involves highly damaging attacks against organizations that result in significant tangible losses in the form of stolen business plans, customer lists, and other proprietary information. The Insider Threat Center's behavioral modeling of insiders who steal intellectual property shows that many insiders who stole their organization's intellectual property stole at least some of it within 30 days of their termination. This technical note presents an example of an insider threat pattern based on this insight. It then presents an example implementation of this pattern on an enterprise-class system using the centralized log storage and indexing engine Splunk to detect malicious insider behavior on a network.

]]> Michael Hanley et al. Smart Grid Maturity Model, Version 1.2: Model Definition http://repository.cmu.edu/sei/720 http://repository.cmu.edu/sei/720 Thu, 17 Jan 2013 07:31:22 PST The Smart Grid Maturity Model (SGMM) is a business tool stewarded by the Software Engineering Institute at Carnegie Mellon University. It was originally developed by electric power utilities for use by electric power utilities. The model provides a framework for understanding the current extent of smart grid deployment and capability within an electric utility, a context for establishing strategic objectives and implementation plans in support of grid modernization, and a means to evaluate progress over time toward those objectives. The SGMM is composed of eight domains and six maturity levels as detailed in this document, which contains the full definition and description of the model. Introductory material to aid in understanding the purpose and use of the SGMM is also provided. The primary audiences for the SGMM, and for this document, are electric power utilities that are seeking guidance related to the modernization of their operations and practices for delivering electricity. The audience also includes any related stakeholders for such utilities. Currently, the model is better suited for utilities with transmission and distribution operations than for pure generation utilities.

]]> The SGMM Team An Acquisition Perspective on Product Evaluation http://repository.cmu.edu/sei/719 http://repository.cmu.edu/sei/719 Thu, 17 Jan 2013 07:31:18 PST This technical note focuses on software acquisition and development practices related to the evaluation of products before, during, and after implementation. From engagements with numerous DoD acquisition programs, it has been observed that a number of recurring issues reduce the effectiveness of how software-reliant products are evaluated. An acquisition effort consists of identifying the customer's needs, selecting or developing a product that is responsive to those needs, and then evaluating the product to determine if it properly addresses the identified needs. This technical note describes the Product Evaluation (verification, validation, and certification) process including test, reviews, and formal methods. It also makes the argument that Product Evaluation should not be deferred until after a product has been built, but should begin as soon as the customer's needs have been identified and should continue throughout the acquisition effort.

]]> Grady Campbell et al. Software Assurance Curriculum Project Volume IV: Community College Education http://repository.cmu.edu/sei/718 http://repository.cmu.edu/sei/718 Thu, 17 Jan 2013 07:31:14 PST The fourth volume in the Software Assurance Curriculum Project led by a team at the Software Engineering Institute, this report focuses on community college courses for software assurance. The report includes a review of related curricula, outcomes and body of knowledge, expected background of target audiences, and outlines of six courses. The courses are intended to provide students with fundamental skills for continuing with graduate-level education or to provide supplementary education for students with prior undergraduate technical degrees who wish to become more specialized in software assurance. Previous volumes of this project are Volume I: Master of Software Assurance Reference Curriculum, Volume II: Undergraduate Course Outlines, and Volume III: Software Assurance Course Syllabi.

]]> Nancy R. Mead et al. Proceedings of the Fourth International Workshop on a Research Agenda for Maintenance and Evolution of Service-Oriented Systems (MESOA 2010) http://repository.cmu.edu/sei/717 http://repository.cmu.edu/sei/717 Thu, 17 Jan 2013 07:31:11 PST The Fourth International Workshop on Maintenance and Evolution of Service-Oriented Systems (MESOA 2010), organized by members of the Carnegie Mellon Software Engineering Institute's technical staff, was held at the 26th International Conference on Software Maintenance (ICSM 2010) in Timisoara, Romania, on September 17, 2010. The goal for MESOA 2010 was to share current research efforts and discuss emerging technologies in the maintenance and evolution of service-oriented systems. A second goal of the workshop was to identify areas of future work needed to address existing gaps and problems in the taxonomy of research topics in service-oriented architecture (SOA). This report summarizes the workshop and includes the accepted papers that were the basis for the presentations given during the workshop. Topics include using simulation models to evolve business processes, a requirements-driven framework for root cause analysis in SOA environments, SOA integration as an alternative to source migration, proactive adaptation as illustrated by the S-Cube service life cycle, a dynamic framework for quality web-service discovery, a characterization of policies that govern SOAs, and context-driven adaptive monitoring for supporting SOA governance. The report concludes with highlights from the discussions among workshop attendees.

]]> Grace A. Lewis et al. Architecting Service-Oriented Systems http://repository.cmu.edu/sei/716 http://repository.cmu.edu/sei/716 Thu, 17 Jan 2013 07:31:07 PST Service orientation is an approach to software systems development that has become a popular way to implement distributed, loosely coupled systems, because it offers such features as standardization, platform independence, well-defined interfaces, and tool support that enables legacy system integration. From a quality attribute point of view, the primary drivers for service orientation adoption are interoperability and modifiability. However, a common misconception is that an architecture that uses a service-oriented approach can achieve these qualities by simply putting together a set of vendor products that provide an infrastructure and then using this infrastructure to expose a set of reusable services to build systems. In reality, there are many architectural decisions that need to be made. An architectural decision that promotes interoperability or modifiability can negatively impact other qualities, such as availability, reliability, security and performance. The goal of this report is to present general guidelines for architecting service-oriented systems, how common service-oriented system components support these principles, and the effect that these principles and their implementation have on system quality attributes.

]]> Philip Bianco et al. Measures for Managing Operational Resilience http://repository.cmu.edu/sei/715 http://repository.cmu.edu/sei/715 Thu, 17 Jan 2013 07:31:04 PST How resilient is my organization? Have our processes made us more resilient? Members of the CERT Resilient Enterprise Management (REM) team are conducting research to address these and other related questions. The team's first report, Measuring Operational Resilience Using the CERT Resilience Management Model, defined high-level objectives for managing an operational resilience management (ORM) system, demonstrated how to derive meaningful measures from those objectives, and presented a template for defining resilience measures, along with example measures. In this report, REM team members suggest a set of top ten strategic measures for managing operational resilience. These measures derive from high-level objectives of the ORM system defined in the CERT Resilience Management Model, Version 1.1 (CERT-RMM). The report also provides measures for each of the 26 process areas of CERT-RMM, as well as a set of global measures that apply to all process areas. This report thus serves as an addendum to CERT-RMM Version 1.1. Since CERT-RMM practices map to bodies of knowledge and codes of practice such as ITIL, COBIT, ISO2700x, BS25999, and PCI DSS, the measures may be useful for measuring security, business continuity, and IT operations management processes, either as part of adoption of CERT-RMM or independent of it.

]]> Julia H. Allen et al. Standards-Based Automated Remediation: A Remediation Manager Reference Implementation http://repository.cmu.edu/sei/714 http://repository.cmu.edu/sei/714 Thu, 17 Jan 2013 07:31:00 PST This report describes the Software Engineering Institute's work in calendar year 2010 for the National Security Agency Computer Network Defense Research and Technology Program Management Office to develop standards for remediation of vulnerabilities and compliance issues on Department of Defense (DoD) networked systems. The overall goals are to assist in the development of remediation standards, demonstrate the functionality DoD would like in a remediation manager, and increase efficiency and effectiveness of remediation by automating the remediation process. The 2010 Remediation Manager reference implementation demonstrates the following potential applications of remediation and other security automation standards: (1) Ingest scan findings in Security Content Automation Protocol (SCAP) format, extracting host compliance issues (in Common Configuration Enumeration [CCE] format) and vulnerabilities (in Common Vulnerability Enumerations [CVE] format). (2) Map CCE and CVE to remediation actions (in Common Remediation Enumeration [CRE] format). (3) Build remediation tasks in Remediation Tasking Language (RTL), based on CRE. (4) Transmit remediation tasks to a Remediation Tool on a host system. (5) Receive remediation task execution status, in RTL Results Format, from the Remediation Tool. This report identifies capabilities considered for future versions of the reference implementation and the operational system as well as challenges for future work.

]]> Sagar Chaki et al.