Software re-engineering of legacy software

There are legacy applications that have been there decades, and much younger applications but a few years.  Both can be subjects of software reengineering for a variety of reasons:

  1. technology no longer supported
  2. to inflexible and/or unable to harness emerging technologies
  3. newer technology can give a better solution
  4. it was just simply not written as best as it could have been
  5. the code has much duplicity, not scalable, and maintenance costs high
  6. badly architected
  7. after much maintenance the code has become disorganized
  8. skills to maintain are becoming rare
  9. compliance with updated or new regulation

What is software reengineering?

Software reengineering is the examination and modification of existing code to continue its use in a more refined state or to migrate it to a new modernized system.

Why not just start again and re-write?

This is the area that generates most debate with agile followers that say “we can just start again and throw away the legacy system when we are finished.  All we need are dedicated users to work along side us”.  This is re-inventing the wheel.

Software often has complex business-critical processes within.  Simply re-writing and you may lose valuable information hidden within the code.  Migration requires more upfront analysis, design, planning, and maybe contract negotiations as we move and probably in many steps to the desired outcome.  In terms of the agile manifesto we have a lot of value on the items on the right which is less agile and more waterfall.  Nothing in the agile manifesto says don’t do waterfall by the way – but it does say be as agile as you can be.

Not everything is obvious via the UI, documented, in someone’s memory, or can be accessed in anyway.  We need to access that hidden information buried inside the code.  Legacy software is also a completed document in its own right – it contains many lessons learned and algorithms that can be modified to fit the new technology.  Starting again and throwing away the legacy system when finished could be a very bad decision.

Many avoid the subject of reengineering but how many of today’s applications start off as a new idea compared to something already existing?  Software reengineering is a very valuable skill for a software engineer to have.

Reverse Engineer

Reverse engineering is the analysis of the existing system and development of high-level plans and designs – the existing system is a very detailed:

  1. functional specification
  2. requirements document
  3. working software

The primary purpose is:

  1. capturing requirements
  2. finding any lost or hidden information, and logic
  3. high level solutions

It is also a feasibility study should/can we

  1. modernize existing code?
  2. migrate to a new system?

If we decide to migrate to a new system then what is the best approach?

  1. The big bang – write a completely new replacement system that we begin using all at once.
  2. Migrate incrementally by modularization – identify modules (if any). We modify legacy code so it is organized into modules where each module represents a function or business area – combine related code parts in modules. We create integration/interface points between these modules so when we introduce a module that is part of the new system, we can redirect those integration points to it. If we are really clever, we use “feature switches” so we can switch between old and new without downtime.

We need to review and document what we will:

  1. Keep
  2. Remove
  3. Improve

We need to look at impact and risk and identify

  1. integration points
  2. interdependencies
  3. interfaces

Do we need to change or normalize the data?

  1. Determine if the interface and relationship with the data permit migration incrementally by modularization or forces a “big bang” approach.  We may need complementary strategies such as migration one region at a time to enable migration incrementally.
  2. Can we use the same database for legacy and reengineered concurrently?  If yes, we may need to make modifications to both legacy and reengineered applications to synchronize with the database as we go.  If no we need a data migration strategy and plan.
  3. If we are migrating incrementally by modularization and require a new database then how do we synchronize data between legacy and reengineered databases?

Forward Engineer

From the high-level plans and designs created in “Reverse Engineering” create low level plans and designs – especially if migrating incrementally by modularization

  1. Database(s)Database designs, migration, synchronization, concurrent support strategies.  Unlike a new product this is live and we have real data that we must not lose.  We have an opportunity to create this right first time – as we have lessons learned from the legacy application. There is little room to evolve a design.
  2. ApplicationObject Orientated Design – design patterns, and application frameworks.  This is not something new, we have an existing application and the majority of requirements.  There is less requirement to evolve an architecture because we know most of the requirements.If we have planned to migrate incrementally by modularization then we know what each of our modules are. We also need to plan for feature switches between them – to turn off modules in the legacy application and on in the reengineered application.

From the low-level plans and designs begin the physical implementation.

If we have planned to migrate incrementally by modularization the first step is to modify the legacy code into modules first or in synchronization with the reengineered application.  If we are creating a new database this would normally be created before the UI.

What is the definition of Success in the SDLC?

There are many software development projects that are classed as successful because they delivered on time and/or on budget and delivered the functionality requested – but survived a short while in the production environment due to their lack of scalability, and high maintenance costs due to lack of planning and foresight during the software’s initial production. These projects often require a complete rewrite, that is not successful. Every opportunity must be taken to make sure that software is written with maintenance, readability, skills availability, and scalability in mind so the software lasts for many years – future proof.

Keeping the maintenance cost down is paramount to a software project being successful; it is no use being a one hit wonder. A project must also be able to implement fixes and/or enhancements and release these speedily and in isolation to on-going development, this means having at least two separate streams of development. Firstly, there is the on-going development stream where we can create new features without impacting the ability to produce fixes. Secondly, there is a maintenance stream where we can fix any problems that arise in the production environment and release these without also releasing any work included in on-going development. Thirdly, we must make sure that fixes and enhancements made in each stream can be included in the other so one release does not overwrite fixes in the previous release.

With a good Software Development Manager and Systems Development Lifecycle (SDLC) the following will be achieved.

  • Prevent Failure
  • Have knowledge of the true state of any project
  • Enables far more accurate planning and costing
  • Better Management of expectation
  • Improved Communication
  • Less disputes
  • Better vision of what is expected
  • Better efficiency, less duplicity
  • More control – top down
  • Interoperability between projects and the business
  • Prevention of Process Creep – where one process takes authority from another

Common Problems for Leaders in the SDLC

  • Lack of support for multiple streams i.e. production and new development – versioning. Lack of knowledge how versioning works
  • Lack of integration with related projects
  • Development processes overlap into production processes and are not compatible.
  • Lack of clear input and output at each step.
  • Lack of clear ownership
  • Constantly changing release dates – forward and backward
  • Inability to deal with late requests
  • Crunch testing – development overruns into time allocated for testing, testing reduced. Can happen with many processes.
  • Management without technical expertise who override decisions made by technologists, and pressurising technologists to make change to fast

Common Problems for Software Engineers in the SDLC

Before getting into defining a Systems Development Lifecycle (SDLC) lets look at some of the main problems experienced by Software Engineers when it is done badly.

  • No time allocated for good design and architecture
    • Code becomes unreadable and unmaintainable
    • Code is disorganised limiting amount of software engineers and/or changes to be active at any time.
    • Makes full and patch releases difficult or impossible without severe downtime or refactoring of code.
    • Simple configuration or system administration requires a code change not variable/parameter change.
    • Makes change extremely difficult – not scalable or object orientated.
      Duplication of solutions i.e. not sharing such as application and hardware servers
    • Makes releases extremely difficult i.e. No thought how to release the project – to many manual processes in a release. Releases take several hours and even days. Build an object to large to get through the door.
  • Lack of necessary documentation or to much unnecessary documentation
  • No or very little enterprise development, testing, or security standards
  • Don’t manage what you don’t understand. Non technical staff getting involved in making technical decisions causes the two most common features in struggling and/or failing technology projects:
    • Pressure from business to implement change fast. Management scheduling without understanding impact and risk
    • Inability of IT to explain and the business to understand the intricate nature of technology projects.