COTS Series 2: Problem Definition & Risk Assessment Part

We talked previously about how Commercial Off The Shelf Solutions (COTS) use offers many benefits:

·         Reduce development costs due to the mass production of reusable modules

·         Reduced development time and effort as the coding has been done

·         Improved quality as the COTS has been used and tested by other users

But COTS come with a level of uncertainty:

·         What features exist and what features will be added/removed in the future?

·         Are there latent (hidden) defects within the COTS solution?

·         What is the capability of the COTS solution to integrate with other solutions?

·         What is the likelihood of customisation to meet all of the organisation needs?

·         What are the supplier’s stability, capability, support, quality and trustworthiness?

Furthermore, it is difficult to assess the strengths and weaknesses of COTS solutions with certainty as:

·         The black box nature of COTS solutions forces the consumer to rely on the vendors claims.

·         The description of the COTS solution can be incomplete.

·         The evaluation of COTS solutions is often rushed; poorly thought through and incomplete; with limited resources and under false assumptions that are not relevant to the organisation’s environment.

COTS is really is a risky business.

A poor implementation strategy and a poor understanding of the requirements leads to cost overruns, schedule delays, poor validated state and even system abandonment. Yet we get the question “if the coding is already done, surely buying a COTS system is quick and easy?” Capers Jones (Applied software measurement, 3^rd ed, 2008) has found that the percentage of software development life-cycle effort involved for coding ranges from: 16% for military systems; 20% for enterprise systems and 35% for software written by users who are not employed in the software industry. EmpowermentQE’s own metrics show that coding accounts for between 15% and 21% of the software development life-cycle effort for complex, engineered enterprise systems. Our point being that there is still a lot of effort required for a successful COTS deployment.

A 6 point COTS selection approach that although is geared towards enterprise based systems and can be tailored to any size of system – the scope of approach is based on risk.

TIP: Based upon the above start with the hypothesis that all COTS solutions and suppliers are not complete for the regulatory industry rather than the having mind-set that the COTS will be relatively quick and easy to deploy.  Starting with a pessimistic approach will increase thoroughness, capture and tie down more risks, and help increase success.

Today we’re going to introduce briefly look at the initial phase o four approach.

The approach is applicable to any solution and caters for the regulatory industry needs:

·         Provide a high level of confidence that the COTS solution will meet the business, technical and regulatory needs.

·         Leverage the knowledge, experience and documentation of the supplier.

·         Judge the quality and reliability of the COTS suppliers.

·         Gain documented evidence that COTS will always perform as intended – differentiate the need between supplier audit and supplier assessment.

·         Gain documented evidence of the data integrity of the COTS solution.

The above lists some of the steps to consider when researching and planning a COTS implementation.

This phase presents the opportunity to fully examine the business/manufacturing/operations/quality domain broadly and identify the goals, how the work is performed, who performs the work and why the work is performed. Examine the risks, stakeholders and so forth for setting the foundations for the requirements definition.  Review of key indicators or project drivers leading to the initiation of this project. Compare those project drivers to the overall business objectives held by the organisation to ensure they are aligned. Next develop a project plan and business case, which allow for executive support and approval for the selection process. All are necessary to ensure the project is headed down the right path at the project’s onset.

This stage will help identify:

·         How the business unit fits within the organisation.

·         The Business needs to consider (IT, regulatory, training, HR,…).

·         The business units strengths and weaknesses – can improvements be introduced?

·         Barriers to a COTS solution.

·         Regulatory and legal requirements.

This helps to ascertain the project context – how the project will be structured, success factors, staff, technology, constraints and initial risks.

Document the activities performed above to provide trace-ability of the initial research. For example, capture:

·         A list of all sources used: resource name and contact/access details

·         Copy of documentation sources

·         Details of any interviews, meetings or other consultative form

·         Details of all relevant stakeholders

·         Project Initiation Documentation

·         Project Plan   … and so forth

A top down and bottom up approach will help form a better understanding of the organisation needs from a micro and macro level, such as:  establish current problems that a COTS solution will solve; establish current assumptions and constraints; timelines; security needs; and  ability to meet regulatory compliance requirements

The breadth and depth of our activities is dependent of risk. A risk based approach to COTS is fundamental to ensure its success. Consideration is given to the existing systems performance.  After which a strategic systems and project plan can be designed and approved.  As per any software implementation approach, risk must be managed (identified, managed and evaluated) throughout the approach and tailored where required.

The above steps are applicable to major and complex solutions and the depth and breadth of their use is wholly dependent on the risk, complexity and regulatory needs of a proposed COTS solution.

Next time we will explore requirements definition.