Software development

Software development are the set of activities that results in software products. Software development may include new development, modification, reuse, re-engineering, maintenance, or any other activities that result in software products.^[1]

Overview

There are several different approaches to software development, much like the various views of political parties toward governing a country. Some take a more structured, engineering-based approach to developing business solutions, whereas others may take a more incremental approach, where software evolves as it is developed piece-by-piece. Most methodologies share some combination of the following stages of software development:

Gathering requirements for the proposed business solution
Analyzing the problem
Devising a plan or design for the software-based solution
Implementation (coding) of the software
Testing the software
Deployment
Maintenance and bug fixing

These stages are often referred to collectively as the software development lifecycle, or SDLC. Different approaches to software development may carry out these stages in different orders, or devote more or less time to different stages. The level of detail of the documentation produced at each stage of software development may also vary. These stages may also be carried out in turn (a “waterfall” based approach), or they may be repeated over various cycles or iterations (a more “Xtreme” approach). The more extreme approach usually involves less time spent on planning and documentation, and more time spent on coding and development of automated tests. More “extreme” approaches also promote continuous testing throughout the development lifecycle, as well as having a working (or bug-free) product at all times. More structured or “waterfall” based approaches attempt to assess the majority of risks and develop a detailed plan for the software before implementation (coding) begins, and avoid significant design changes and re-coding in later stages of the software development lifecycle.

There are significant advantages and disadvantages to the various methodologies, and the best approach to solving a problem using software will often depend on the type of problem. If the problem is well understood and a solution can be effectively planned out ahead of time, the more "waterfall" based approach may work the best. If, on the other hand, the problem is unique (at least to the development team) and the structure of the software solution cannot be easily envisioned, then a more "extreme" incremental approach may work best. A software development process is a structure imposed on the development of a software product. Synonyms include software life cycle and software process. There are several models for such processes, each describing approaches to a variety of tasks or activities that take place during the process.

Software development activities

Requirements analysis

The most important task in creating a software product is extracting the requirements or requirements analysis. Customers typically have an abstract idea of what they want as an end result, but not what software should do. Incomplete, ambiguous, or even contradictory requirements are recognized by skilled and experienced software engineers at this point. Frequently demonstrating live code may help reduce the risk that the requirements are incorrect. One specific method here is Software Elements Analysis.

Once the general requirements are gleaned from the client, an analysis of the scope of the development should be determined and clearly stated. This is often called a scope document. Certain functionality may be out of scope of the project as a function of cost or as a result of unclear requirements at the start of development. If the development is done externally, this document can be considered a legal document so that if there are ever disputes, any ambiguity of what was promised to the client can be clarified.

Domain Analysis is often the first step in attempting to design a new piece of software, whether it be an addition to an existing software, a new application, a new subsystem or a whole new system, is, what is generally referred to as "Domain Analysis". Assuming that the developers (including the analysts) are not sufficiently knowledgeable in the subject area of the new software, the first task is to investigate the so-called "domain" of the software. The more knowledgeable they are about the domain already, the less work required. Another objective of this work is to make the analysts, who will later try to elicit and gather the requirements from the area experts, speak with them in the domain's own terminology, facilitating a better understanding of what is being said by these experts. If the analyst does not use the proper terminology it is likely that they will not be taken seriously, thus this phase is an important prelude to extracting and gathering the requirements. If an analyst hasn't done the appropriate work confusion may ensue: "I know you believe you understood what you think I said, but I am not sure you realize what you heard is not what I meant."^[2]

Specification

Specification is the task of precisely describing the software to be written, possibly in a rigorous way. In practice, most successful specifications are written to understand and fine-tune applications that were already well-developed, although safety-critical software systems are often carefully specified prior to application development. Specifications are most important for external interfaces that must remain stable. A good way to determine whether the specifications are sufficiently precise is to have a third party review the documents making sure that the requirements and Use Cases are logically sound.

Architecture

The architecture of a software system or software architecture refers to an abstract representation of that system. Architecture is concerned with making sure the software system will meet the requirements of the product, as well as ensuring that future requirements can be addressed. The architecture step also addresses interfaces between the software system and other software products, as well as the underlying hardware or the host operating system.

Design, implementation and testing

Implementation is the part of the process where software engineers actually program the code for the project.

Software testing is an integral and important part of the software development process. This part of the process ensures that bugs are recognized as early as possible.

Documenting the internal design of software for the purpose of future maintenance and enhancement is done throughout development. This may also include the authoring of an API, be it external or internal.

Deployment and maintenance

Deployment starts after the code is appropriately tested, is approved for release and sold or otherwise distributed into a production environment.

Software Training and Support is important because a large percentage of software projects fail because the developers fail to realize that it doesn't matter how much time and planning a development team puts into creating software if nobody in an organization ends up using it. People are often resistant to change and avoid venturing into an unfamiliar area, so as a part of the deployment phase, it is very important to have training classes for new clients of your software.

Maintenance and enhancing software to cope with newly discovered problems or new requirements can take far more time than the initial development of the software. It may be necessary to add code that does not fit the original design to correct an unforeseen problem or it may be that a customer is requesting more functionality and code can be added to accommodate their requests. It is during this phase that customer calls come in and you see whether your testing was extensive enough to uncover the problems before customers do.

Models

Agile software development

Agile software development processes are built on the foundation of iterative development. To that foundation they add a lighter, more people-centric viewpoint than traditional approaches. Agile processes use feedback, rather than planning, as their primary control mechanism. The feedback is driven by regular tests and releases of the evolving software.

Interestingly, surveys have shown the potential for significant efficiency gains over the waterfall method. For example, a survey, published in August 2006 by VersionOne and Agile Alliance and based on polling more than 700 companies claims the following benefits for an Agile approach.^[3] The survey was repeated in August 2007 with about 1,700 respondents.^[4]

Iterative processes

Iterative development^[5] prescribes the construction of initially small but even larger portions of a software project to help all those involved to uncover important issues early before problems or faulty assumptions can lead to disaster. Iterative processes are preferred by commercial developers because it allows a potential of reaching the design goals of a customer who does not know how to define what they want.

XP: Extreme Programming

Extreme Programming (XP) is the best-known iterative process. In XP, the phases are carried out in extremely small (or "continuous") steps compared to the older, "batch" processes. The (intentionally incomplete) first pass through the steps might take a day or a week, rather than the months or years of each complete step in the Waterfall model. First, one writes automated tests, to provide concrete goals for development. Next is coding (by a pair of programmers), which is complete when all the tests pass, and the programmers can't think of any more tests that are needed. Design and architecture emerge out of refactoring, and come after coding. Design is done by the same people who do the coding. (Only the last feature - merging design and code - is common to all the other agile processes.) The incomplete but functional system is deployed or demonstrated for (some subset of) the users (at least one of which is on the development team). At this point, the practitioners start again on writing tests for the next most important part of the system.

Waterfall processes

The waterfall model shows a process, where developers are to follow these steps in order:

Requirements specification (AKA Verification)
Design
Construction (AKA implementation or coding)
Integration
Testing and debugging (AKA validation)
Installation (AKA deployment)
Maintenance

After each step is finished, the process proceeds to the next step, just as builders don't revise the foundation of a house after the framing has been erected.

There is a misconception that the process has no provision for correcting errors in early steps (for example, in the requirements). In fact this is where the domain of requirements management comes in which includes change control.

This approach is used in high risk projects, particularly large defense contracts. The problems in waterfall do not arise from "immature engineering practices, particularly in requirements analysis and requirements management." Studies of the failure rate of the DOD-STD-2167 specification, which enforced waterfall, have shown that the more closely a project follows its process, specifically in up-front requirements gathering, the more likely the project is to release features that are not used in their current form ^{[citation needed]}.

Often the supposed stages are part of review between customer and supplier, the supplier can, in fact, develop at risk and evolve the design but must sell off the design at a key milestone called Critical Design Review (CDR). This shifts engineering burdens from engineers to customers who may have other skills

Other models

Capability Maturity Model: The Capability Maturity Model (CMM) is one of the leading models. Independent assessments grade organizations on how well they follow their defined processes, not on the quality of those processes or the software produced. CMM is now replaced by CMMI. ISO 9000 describes standards for formally organizing processes with documentation.

ISO 15504: ISO 15504, also known as Software Process Improvement Capability Determination (SPICE), is a "framework for the assessment of software processes". This standard is aimed at setting out a clear model for process comparison. SPICE is used much like CMMI. It models processes to manage, control, guide and monitor software development. This model is then used to measure what a development organization or project team actually does during software development. This information is analyzed to identify weaknesses and drive improvement. It also identifies strengths that can be continued or integrated into common practice for that organization or team.

Six sigma: Six Sigma is a methodology to manage process variations that uses data and statistical analysis to measure and improve a company's operational performance. It works by identifying and eliminating defects in manufacturing and service-related processes. The maximum permissible defects is 3.4 per one million opportunities. However, Six Sigma is manufacturing-oriented and needs further research on its relevance to software development.

Test Driven Development: Test Driven Development (TDD) is a useful output of the Agile camp but some suggest that it raises a conundrum. TDD requires that a unit test be written for a class before the class is written. It might be thought, then, that the class firstly has to be "discovered" and secondly defined in sufficient detail to allow the write-test-once-and-code-until-class-passes model that TDD actually uses. This would be actually counter to Agile approaches, particularly (so-called) Agile Modeling, where developers are still encouraged to code early, with light design. However, to get the claimed benefits of TDD a full design down to class and responsibilities (captured using, for example, Design By Contract) is not necessary. This would count towards iterative development, with a design locked down, but not iterative design - as heavy refactoring and re-engineering might negate the usefulness of TDD.

Formal methods

Formal methods are mathematical approaches to solving software (and hardware) problems at the requirements, specification and design levels. Examples of formal methods include the B-Method, Petri nets, RAISE and VDM. Various formal specification notations are available, such as the Z notation. More generally, automata theory can be used to build up and validate application behavior by designing a system of finite state machines.

Finite state machine (FSM) based methodologies allow executable software specification and by-passing of conventional coding (see virtual finite state machine or event driven finite state machine).

Formal methods are most likely to be applied in avionics software, particularly where the software is safety critical. Software safety assurance standards, such as DO178B demand formal methods at the highest level of categorization (Level A).

Formalization of software development is creeping in, in other places, with the application of OCL (and specializations such as JML) and especially with MDA allowing execution of designs, if not specifications.

Another emerging trend in software development is to write a specification in some form of logic (usually a variation of FOL), and then to directly execute the logic as though it were a program. The OWL language, based on Description Logic, is an example. There is also work on mapping some version of English (or another natural language) automatically to and from logic, and executing the logic directly. Examples are Attempto Controlled English, and Internet Business Logic, which does not seek to control the vocabulary or syntax. A feature of systems that support bidirectional English-logic mapping and direct execution of the logic is that they can be made to explain their results, in English, at the business or scientific level.

The Government Accountability Office, in a 2003 report on one of the Federal Aviation Administration’s air traffic control modernization programs^[6], recommends following the agency’s guidance for managing major acquisition systems by

establishing, maintaining, and controlling an accurate, valid, and current performance measurement baseline, which would include negotiating all authorized, unpriced work within 3 months;
conducting an integrated baseline review of any major contract modifications within 6 months; and
preparing a rigorous life-cycle cost estimate, including a risk assessment, in accordance with the Acquisition System Toolset’s guidance and identifying the level of uncertainty inherent in the estimate.

The study of software development

Software engineering

Software engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software.^[7] It encompasses techniques and procedures, often regulated by a software development process, with the purpose of improving the reliability and maintainability of software systems.^[8] The effort is necessitated by the potential complexity of those systems, which may contain millions of lines of code.^[9]

The term software engineering was coined by F.L. Bauer and popularized by the 1968 NATO Software Engineering Conference, which was organized by Brian Randell.^[10]

Software development and marketing

Software development is sometimes understood to encompass the processes of software engineering combined with the research and goals of software marketing to develop computer software products.^[11] This is in contrast to marketing software, which may or may not involve new product development.

It is often difficult to isolate whether engineering or marketing is more responsible for the success or failure of a software product to satisfy customer expectations. This is why it is important to understand both processes and facilitate collaboration between both engineering and marketing in the total software development process. Engineering and marketing concerns are often balanced in the role of a project manager that may or may not use that title.

Because software development may involve compromising or going beyond what is required by the client, a software development project may stray into processes not usually associated with engineering such as market research, human resources, risk management, intellectual property, budgeting, crisis management, etc. These processes may also cause the role of business development to overlap with software development.

In the book "Great Software Debates", Alan M. Davis states in the chapter "Requirements", subchapter "The Missing Piece of Software Development":

Students of engineering learn engineering and are rarely exposed to finance or marketing. Students of marketing learn marketing and are rarely exposed to finance or engineering. Most of us become specialists in just one area. To complicate matters, few of us meet interdisciplinary people in the workforce, so there are few roles to mimic. Yet, software product planning is critical to the development success and absolutely requires knowledge of multiple disciplines.^[12]

Software development organizations

A largely growing body of software development organizations implement process methodologies. Many of them are in the defense industry, which in the U.S. requires a rating based on 'process models' to obtain contracts.

The international standard for describing the method of selecting, implementing and monitoring the life cycle for software is ISO 12207.

A decades-long goal has been to find repeatable, predictable processes that improve productivity and quality. Some try to systematize or formalize the seemingly unruly task of writing software. Others apply project management techniques to writing software. Without project management, software projects can easily be delivered late or over budget. With large numbers of software projects not meeting their expectations in terms of functionality, cost, or delivery schedule, effective project management appears to be lacking.

References

^ DRM Associates (2002). "New Product Development Glossary". Retrieved 2006-10-29.
^ Appears in Roger S. Pressman, Software Engineering (A practitioner's approach), 5th edition, 2000, Mc Graw-Hill Education, ISBN 978-0071181822; however the quote is attributed to many sources, including Richard Nixon, Robert McCloskey, and Alan Greenspan. It may have originated several decades earlier in an anonymous academic joke.
^ Agile Journal - Agile Survey Results: Solid Experience And Real Results
^ Agile Journal - Agile Survey Results: Widespread Adoption, Emphasis on Productivity and Quality
^ [1]
^ Government Accountability Report (January 2003). Report GAO-03-343, National Airspace System: Better Cost Data Could Improve FAA’s Management of the Standard Terminal Automation Replacement System. Retrieved from http://www.gao.gov/cgi-bin/getrpt?GAO-03-343
^ “IEEE Standard Glossary of Software Engineering Terminology,” IEEE std 610.12-1990, 1990, quoted at the beginning of Chapter 1: Introduction to the guide "Guide to the Software Engineering Body of Knowledge". February 6, 2004. Retrieved 2008-02-21.
^ Pecht, Michael (1995). Product Reliability, Maintainability, and Supportability Handbook. CRC Press. ISBN 0-8493-9457-0. {{cite book}}: Cite has empty unknown parameter: |coauthors= (help)
^ Pehrson, Ronald J. (1996). "Software Development for the Boeing 777". CrossTalk: The Journal of Defense Software Engineering. {{cite journal}}: Unknown parameter |month= ignored (help), "The 2.5 million lines of newly developed software were approximately six times more than any previous Boeing commercial airplane development program. Including commercial-off-the-shelf (COTS) and optional software, the total size is more than 4 million lines of code."
^ Randell, Brian (10 Aug 2001). "The 1968/69 NATO Software Engineering Reports". Brian Randell's University Homepage. The School of Computer Sciences, Newcastle University. Retrieved 2008-10-11. The idea for the first NATO Software Engineering Conference, and in particular that of adopting the then practically unknown term "software engineering" as its (deliberately provocative) title, I believe came originally from Professor Fritz Bauer.
^ Jim McCarthy. "Dynamics of Software Development" (August 1, 1995), pp:10-30
^ Alan M. Davis. Great Software Debates (October 8, 2004), pp:125-128 Wiley-IEEE Computer Society Press

External links

No Silver Bullet: Essence and Accidents of Software Engineering", 1986
Gerhard Fischer, "The Software Technology of the 21st Century: From Software Reuse to Collaborative Software Design", 2001
Lydia Ash: The Web Testing Companion: The Insider's Guide to Efficient and Effective Tests, Wiley, May 2, 2003. ISBN 0471430218
SaaSSDLC.com - Software as a Service Systems Development Life Cycle Project
Software development life cycle (SDLC) [visual image], software development life cycle
Iterative Development and The Leaning Tower of Pisa - From The Trench
Processes and Templates

Template:Computer Science

[1] DRM Associates (2002). "New Product Development Glossary". Retrieved 2006-10-29.

[2] Appears in Roger S. Pressman, Software Engineering (A practitioner's approach), 5th edition, 2000, Mc Graw-Hill Education, ISBN 978-0071181822; however the quote is attributed to many sources, including Richard Nixon, Robert McCloskey, and Alan Greenspan. It may have originated several decades earlier in an anonymous academic joke.

[3] Agile Journal - Agile Survey Results: Solid Experience And Real Results

[4] Agile Journal - Agile Survey Results: Widespread Adoption, Emphasis on Productivity and Quality

[5] [1]

[6] Government Accountability Report (January 2003). Report GAO-03-343, National Airspace System: Better Cost Data Could Improve FAA’s Management of the Standard Terminal Automation Replacement System. Retrieved from http://www.gao.gov/cgi-bin/getrpt?GAO-03-343

[7] “IEEE Standard Glossary of Software Engineering Terminology,” IEEE std 610.12-1990, 1990, quoted at the beginning of Chapter 1: Introduction to the guide "Guide to the Software Engineering Body of Knowledge". February 6, 2004. Retrieved 2008-02-21.

[8] Pecht, Michael (1995). Product Reliability, Maintainability, and Supportability Handbook. CRC Press. ISBN 0-8493-9457-0. {{cite book}}: Cite has empty unknown parameter: |coauthors= (help)

[9] Pehrson, Ronald J. (1996). "Software Development for the Boeing 777". CrossTalk: The Journal of Defense Software Engineering. {{cite journal}}: Unknown parameter |month= ignored (help), "The 2.5 million lines of newly developed software were approximately six times more than any previous Boeing commercial airplane development program. Including commercial-off-the-shelf (COTS) and optional software, the total size is more than 4 million lines of code."

[10] Randell, Brian (10 Aug 2001). "The 1968/69 NATO Software Engineering Reports". Brian Randell's University Homepage. The School of Computer Sciences, Newcastle University. Retrieved 2008-10-11. The idea for the first NATO Software Engineering Conference, and in particular that of adopting the then practically unknown term "software engineering" as its (deliberately provocative) title, I believe came originally from Professor Fritz Bauer.

[11] Jim McCarthy. "Dynamics of Software Development" (August 1, 1995), pp:10-30

[12] Alan M. Davis. Great Software Debates (October 8, 2004), pp:125-128 Wiley-IEEE Computer Society Press

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]