Testing cots systems evaluation testing in parallel. The use of commercial offtheshelf cots software is increasingly becoming a necessity for many european organisations. Cots testing is best way of evaluating the cots products. Criteriabased assessment mike jackson, steve crouch and rob baxter criteriabased assessment is a quantitative assessment of the software in terms of sustainability, maintainability, and usability.
This approach forms the basis of our online sustainability evaluation, a webbased assessment you can use straight out of the box. It considers the issues and risks in using cots software over the life cycle and how to control them. Any package implementation operation will involve a degree of configuration and code development. While this paper is specifically focused on software, the discussion here pertains equally to software services instantiated as hardware productsappliances for functions like search or firewall. The limitations of current decisionmaking techniques in. Kung, chitkwan lin, youngjune gwon school of engineering and applied sciences harvard university abstractwe are interested in estimating the angle of arrival of an rf signal by using commercialofftheshelf cots software. Security considerations in managing cots software cisa. Boehm confirms that the usage of cots products has increased significantly in building software systems during the last decade.
They report on a recent experience in selecting a complex commercial offtheshelf software system to be compliant with over customer requirements, and. Rfi rfp responses may be evaluated using points scoring criteria and scorecards. The limitations of current decisionmaking techniques in the procurement of cots software components. Sampson nasa goddard space flight center greenbelt, md 20771 phone. Results can then be compared within a software evaluation matrix.
One of these is the need to evaluate candidates cots systems early in the development life cycle. Evaluation takes time, study, understanding and a holistic view of. Resources for the evaluation of commercial offtheshelf cots software. A management guide to software maintenance in cots. So there is a need of evaluation of cots products before procuring them for software reuse. During the requirements analysis phase, the agency will conduct any procurement needed for the project. In particular, the use of commercial offtheshelf cots products as elements of larger systems is becoming increasingly commonplace, due to shrinking budgets. An evaluation of the parts and assurance methodologies jay a.
While this may be the case, the actual benefits have not been confirmed. It describes changes in the software maintenance process that are needed to manage a cotsbased system. See rfp evaluation template for a quick and easytouse template. It arises because of the gap between stateofart which represented by many methods that published in literature for evaluating and selecting cots software and stateofpractice which. A management guide to software maintenance in cotsbased systems. The following documents were referred to by epa officials as they decided on the requirements for an erm product to test in a pilot project. Cots commercial off the shelf lifecyle model methodology. Human resources hr, retail point of sale, case management, finance etc. Thus, this study was conducted to investigate the current practices, mechanisms and techniques for evaluating and selection cots software in real industries.
This research paper aims to discuss the empirical study and analysis on current practices and benefits of cotsbased system cbs from jordan organizations perspectives. Cots project management strategy from a state government. Cots software evaluation techniques semantic scholar. Five commandments for successful cots package testing. Casm version control product selection unpublished project at the. This article is one of a series covering cots commercials off the shelf lifecycle model methodology, also know as package software. A sister of cots, the saas or softwareasaservice where the commercial solutions are made available over the web, with no hosting or installation required, has made these solutions even easier to implement technically. Evaluation theory, development of a new decision making technique by. In a later paper we will provide a detailed examination of the intersection of the. This coordinate system can be used descriptively for example to relate different approaches to cots software evaluation, or prescriptively as a tool for improving practice by focusing on key evaluation issues, and the techniques that address these issues. Pdf assessment and evaluation of cots software products has become a.
Dec 14, 2006 cots products are commercial offtheshelf software and hardware. Vendor evaluation matrix template vendorsub project managers technical architects development leads processfunctional leads. The techniques discussed are software wrapping, compatibility and regression testing, contract based. Cots project management strategy from a state government pmo. Descriptive statistics analysis was used to rank the identification techniques according to their importance.
When the cots product is primarily implemented for backoffice systems oracle peoplesoft, sterling commerce, etc. Proceedings of the nato information systems technology. Dynamically reflect results of cotsbased tradeoffs in the requirements description. These components range from standalone elements, such as a web server or database system, to platform software or an operating system, to embedded functional components, such as a calendar manager or an inventory. Survey and data collection was conducted in jordon. The business unit could buy a software product from among a number of vendors called commercial offtheshelf software or cots, eliminating most software development except for what is necessary to configure and deploy it in your environment. Request pdf cots software evaluation techniques employing commercial offtheshelf cots software products as components in largescale longlived systems has been proposed as a way to reduce. Cots software selection process sandia national laboratories. Analyze software requirements software requirements analysis is a critical part of the software development process, although too often this activity is overlooked or glossed over in the rush to start building. Iso 25051, isoiec 25051 software engineering software product quality requirements and evaluation square requirements for quality of commercial offtheshelf cots software product and instructions for testing, first ed international standards organization, geneva, 2006. A formal process for evaluating cots software products computer. Commercialofftheshelf cots based software development has been considered as an alternative strategy to inhouse development. Cots based integration institute for software research. In this paper, a survey of various techniques used for testing cots is presented.
From system requirements to cots evaluation criteria. I first started working on cots software products and their integration with other systems back in the 90s when working for a large systems integrator. As the commercial market provides increased diversification of software products each with uncertain pedigree, there is a critical need for cots software evaluation techniques to analyze and compare the fit or misfit between. What requirements to specify for cots and saas projects. Software evaluation guide software sustainability institute. This approach has advantages in terms of cost and software components for systems development, to describe time because it results in a more directed evaluation of.
The use of commercial offtheshelf cots software products as components of. Electronic records management guidance on methodology for. Use these four evaluation criteria to help you choose the perfect business solution. Most implementations do not identify testing as an independent function required during the implementation of the cots product. This can inform highlevel decisions on specific areas for software improvement. The standard technical evaluation process step developed in g024 outlines a rigorous process for technology evaluations of one or more cots products. These are products that are readymade and sold as packages. Employing commercial offtheshelf cots software selection. As adapted from timing the testing of cots software products, the streams of evaluation testing would focus on the following. Rfps are best evaluated as a team effort, following an agreed process and. Employing commercial offtheshelf cots software products as components in largescale longlived systems has been proposed as a way to reduce both implementation and operating cost for the user communities. Our research is concentrated in the area of physical evaluation of candidate products, that is, actual testing of the products themselves. From which, the optimum software vendor can be shortlisted or selected. Our criteriabased approach is a quantitative assessment of the software in terms of sustainability, maintainability, and usability.
Citeseerx document details isaac councill, lee giles, pradeep teregowda. A formal process for evaluating cots software products. However, using cots packages introduces various burdens, including cots evaluation, assessment, familiarization, and vendor interaction. A basis for evaluation of commercial software sciencedirect. Brusse qss group, incorporated nasa goddard space flight center greenbelt, md 20771 phone.
Dec 17, 2014 what is cots commercial off the shelf cots or commercial off the shelf software, also known as package software refers to a software application that is specifically designed to meet the needs of particular business type e. We all work in environments in which government funding is stretched, program resources are syphoned, disparate applications are waiting on future. This is especially true of techniques for evaluating cots products. Vendor evaluation matrix template vendorsub project managers technical architects development leads processfunctional leads change management leads bpr leads importance definitions definition detailed description high. But this necessity introduces new problems and changes for all levels of the organisations both from the business point of view. Architecture and design activities generate alternative architectures and designs and select candidate cots products and technologies using cots evaluation results. It applies to a variety of areas of technology and provides substantial benefits for evaluation teams and their government sponsors. The complete concolic execution of cots oses would never terminate in a reasonable amount of time due to the wellknown limitation of the symbolic execution, state or path explosion, where the number of feasible program. Commercial offtheshelf software includes such products as the microsoft office suite, which includes word for word processing, excel for spreadsheets, access for building databases, and other applications. Abstract definition and specification that sets the criteria for cots. Social technical approach to cots software evaluation.
Practical concolic testing techniques for cots operating systems su yong kim. Performance techniques for cots systems cots components can provide much of the functionality of distributed information systems. Building new systems is highprofile, difficult work that receives appropriate attention, but it operations of an organization rely most. A process for cots software product evaluation carnegie mellon. Cots project management strategy from a state government pmo perspective. Determining rf angle of arrival using cots antenna arrays. Overviewinformation technology tends to focus on new systemsthe processes for designing, developing, testing them, and making them secure has been the subject of thousands of books and the focus of hundreds of processes. The purpose of this paper is to present a discussion of proposed evaluation techniques used to select cots software components for systems development, to describe appropriate testing techniques for cots candidates, and to propose an evaluation system which will provide support to ensure timely selection of suitable cots products.
Association for information and image management aiim. This 2004 report focuses on cots product evaluations conducted for the purpose of selecting products to meet a known need in a system. Paper presented at pmi global congress 2011north america, dallas, tx. Use these four evaluation criteria to help you choose the. The purpose of the requirements analysis phase is to transform the needs and highlevel requirements specified in earlier phases into unambiguous measurable and testable, traceable, complete, consistent, and stakeholderapproved requirements. Periodically reexamine cots products for opportunities to optimize user processes. Testing is not easily done, for instance, and a large part of understanding of the component depends on the vendors claims for the product. Cots, mots, gots, and nots are abbreviations that describe prepackaged software or less commonly hardware purchase alternatives. Performing organization name and address united technologies research center 411 silver lane east hartford, ct 06108 10. The background fundamentals for that evaluation process, as well as steps and techniques to follow, are described in this report. Other types of cots software are for organizational level systems rather than office or personal use.
A process for cots software product evaluation july 2004 technical report santiago comelladorda, john dean, grace lewis, edwin j. Not only does it require clear objectives and goals, but also a dedicated group of key stakeholders with substantial business process insight. Course objectives the objective in this course is to identify the unique challenges of using cots products in software intensive systems and to cover a range of topics and techniques to enable you to be successful when using cots products in your system or project. In this regard, the formulation of evaluation criteria and the corresponding assessment methods developed for commercialofftheshelf cots components may be. Keywords commercial offtheshelf, cots, software, evaluation 1 introduction in modern cotsbased systems development we need to. A formal process for evaluating cots software products a s government agencies and businesses become more dependent on commercial offtheshelf cots software products to automate tasks, the ability to determine product quality and suitability has become increasingly important. Forecasts had that figure rising in 1998 to around 28. Selecting a commercial off the shelf cots software product is usually a demanding process. A field evaluation hsiehchung chen, tsunghan lin, h. Some techniques for soliciting information from vendors. It describes changes in the software maintenance process that are needed to manage a cots based system. The number commercialofftheshelf cots software options have increased exponentially in recent years. Commercial offtheshelf cots avionics software study.
The central problem is how to select a desirable cots software. The limitations of current decisionmaking techniques in the. Standardized technology evaluation process step users. Specifics about the quantification and application of these factors can be found in 6. Defense technical information center compilation part notice. Adopting commercial offtheshelf cots products or packages like erp, crm, and hr management systems to fulfil a range of enterprise functions is a crucial decision involving huge investment. A software system called descots description, evaluation and selection of cots component developed by grau includes various tools to support the cots component selection process. Evaluation takes time, study, understanding and a holistic view of current and future needs. A successful implementation of a commercial offtheshelfintensive software system can save programs money if you have the right solution and understand the potential risks involved. Pdf from system requirements to cots evaluation criteria. Analyze software requirements software requirements analysis is a critical part of the software development process, although too often this activity is overlooked or glossed over in the rush to start building software.
1272 930 1117 765 764 19 1053 1300 658 121 1352 1133 1508 1232 1040 938 1617 1433 443 1420 1209 1114 1298 437 1443 377 40 744 867 308