20 problems experienced by Software Development Managers and Engineers with a poor SDLC

If you are or thinking of getting involved in Software Development Management and the Systems Development Life cycle (SDLC) take a look at the top 20 problems experienced by Software Development Managers and Software Engineers when the SDLC is managed poorly and without training. Untrained staff getting involved in making technical decisions causes the most common problems in struggling and/or failing software development and software engineering projects.

  1. No time allocated for good design and architecture
  2. Code becomes unreadable and unmaintainable
  3. Code is disorganized limiting amount of software engineers and/or changes to be active at any time.
  4. Makes full and patch releases difficult or impossible without severe downtime or refactoring of code.
  5. Lack of support for multiple streams i.e. production and new development – versioning. Lack of knowledge how versioning works. Development processes overlap into production processes.
  6. Simple configuration or system administration requires a code change not variable/parameter change.
  7. Makes change extremely difficult – not scalable or object orientated.
  8. Duplication of solutions i.e. not sharing such as application and hardware servers
  9. Lack of integration with related projects
  10. Makes releases extremely difficult i.e. No thought how to release the project – to many manual processes in a release. Releases take several hours and even days. Build an object to large to get through the door.
  11. Lack of necessary documentation or to much unnecessary documentation
  12. No or very little enterprise development, testing, or security standards
  13. Crunch testing – development overruns into time allocated for testing, testing reduced.
  14. Scope creep
  15. Lack of clear input and output at each step.
  16. Lack of clear ownership and accountability
  17. Constantly changing release dates – forward and backward
  18. Inability to deal with late requests
  19. Pressure from business to implement change fast. Management scheduling without understanding impact and risk
  20. Inability of IT to explain and the business to understand the intricate nature of technology projects.

All the above points can be solved by a trained Software Development Manager. Don’t manage what you don’t understand and make the same mistakes – get trained. A Software Development Manager is not just a people manager. They are also the one in the middle who resolves conflicts and maintains relationships and integrations between project managers, product owners, scrum-masters involvement with software development – both technical and people management.

That is why I coach, train, and mentor new and upcoming Software Development Managers. Don’t make the same mistakes that have been made by countless others since the inception of software development and engineering.

Internationally experienced software engineers and managers add value

They gain exposure to projects they could not have in their home countries. 

When I was in the UK, we had many Australians, New Zealanders, South Africans, and many others work on such projects.  With small projects or smaller populations such as Australia and New Zealand, it is important to have jack-of-all-trades, that is people who are multi-skilled.  That doesn’t mean they were not also highly specialised.  Likewise, larger projects and larger populations such as the UK, Germany, France, US, and China can afford more specialised people.  We gave exposure to large projects they didn’t often have at home – and they were inspiring us to be more multi-skilled.  As a result, the international market opened up with my new multi-skilled ability. 

There are large and complex projects, and small and simple.  There are a variety of project and product types.  It is rare if anyone can find such experience in one location. Working with or being an international software engineer has great benefits for knowledge transfer.

They also have a wider exposure to a variety of tools, processes, and technologies and at different stages of maturity.

Software engineers without international experience often focus on knowing software engineering trends for their country very well but less on what is happening globally – they may re-invent the wheel or not be knowledgeable about the latest worldwide trends.  International software engineers can bring new ideas to the table with knowledge & capability to compare & contrast from experience in variety of countries. It makes them more efficient and ultimately more strategic with their time as they pick from the best tools, process, and technology. 

Each country is at a different stage of its growth and cycle.  Legacy software in developed countries is often greenfield projects in developing countries.  Some countries are moving forwards with large legislation changes such as in financial trading, import and export, or taxation. Eventually each country will at some point go through the same process and need these internationally experienced software engineers to save re-inventing the wheel.

They have learnt more about team building and leadership working with a variety and mix of cultures. 

Team structure and reporting line can be very different between countries.  Across UK and Europe, the structure is often very flat

  1. Board
  2. IT Director
  3. Software Engineering Manager
  4. Software Engineer. 

In Canada it can be

  1. Board
  2. VP/CIO
  3. AVP Software Engineering
  4. Director of Software Engineering
  5. Assistant Director
  6. Software Engineering Manager
  7. Software Engineer. 

Even the well-known methodologies of agile and waterfall are modified to match local culture.  Some cultures prefer micro-managing and others more trusting – the international software engineer quickly adapts and knows when local culture trumps any favouritism to waterfall or agile.  

Places such as UK, Europe, Asia, and the US have no problems with a foreigner in a leadership position allowing the teams and leaders to flourish without borders.  A few do not allow that and teams stay stuck in their old ways – especially in a culture that micromanages more than they trust.  I have worked as a software engineer or software engineering manager in over 30 countries and it is the countries and their cultures that have inclusion and diversity that permits foreigners in leadership that flourish the most.  There is one “developed” country that will not even permit foreigners to call themselves a software engineer or teacher of software engineering, unless they were educated in that country, nor will they permit them beyond role of developer or computer programmer. It is no wonder that country had many examples of old tools, processes, and technology and struggled to fill them. This reluctance to be inclusive led to sales and administration staff being “drafted” into IT roles

Conclusion

Overcoming challenges when team members have different ideas, opinion, and methodology on a solution is one of the most important skills for a software engineer or manager.  Internationally experienced can do so when cultural differences exist too.  They know the right questions to ask, and when to listen.  They too have learnt to quickly adapt to other cultures and working styles.  Software engineers and managers with international experience have proven they are courageous enough to step outside their comfort zone and quick to learn on their feet.  Including them in homegrown teams is a must to transfer that knowledge.  Teams that don’t struggle when they have to work with overseas teams – or continue in ways that the rest of the world no longer uses.

Photo by Porapak Apichodilok from Pexels

Let the software engineering team build and recruit the team

It was once quite straightforward to recruit software engineers.  As resume’s arrived HR sent them to the hiring Software Engineering Manager who forwarded them to senior software engineers.  They scanned them for obvious yeas or nays. Yeas were called by HR for interview ASAP with the Software Engineering Team.  If yeas were good, they got an offer and didn’t wait for a closing date before starting to interview.  It wasn’t unusual from interview to verbal offer to be within 24 hours or the time between application and offer to be within one week.  We rarely lost a great software engineer due to process. When somebody was right we closed the deal.

The world is covered by water. A fisherman’s job is to choose the best parts to cast that net.  We can learn a lot from fisherman.  The way we build and recruit our team is all determined by our chosen fishing grounds and the bait we use.

A complaint from HR today is there are too many applicants – and they have automated tracking systems (ATS) to make filtering easier.  Applicants now work extremely hard to customize resumes for the ATS search algorithm.  Before there were just as many software engineers looking for work and they had little customization to do to their resumes.  They were read by software experts who understood them.  We never had such a deluge of resumes even during hard times. Why is that?  What did we do that was different?

Software engineering managers wrote well targeted job descriptions (JD’s) for a specific job title, function, and experience level which included a salary range – they used recognized descriptive terms.  They selected who to invite for interview.  Only software engineers with the majority of requirements and happy with the salary applied.  It let software engineers know they were a good fit, and software engineering teams know only those who were likely good fits apply. That limited the resumes from 100’s to 10’s. We got to the ideal software engineers in one or two steps.  Equally important software engineers could target the employers that wanted them and not waste time on employers with unclear JD’s.

Today’s JD’s are written by HR have fanciful non-standard job titles such as ninja and superhero.  They are not what software engineers search on.  The requirements cover every function and experience level in a software engineering department – and more.  They also have no salary range.  Now anyone with just a fraction of what is being asked applies – because they don’t really know what is wanted but it may be them.  We have a deluge of mainly unsuitable resumes.  HR use ATS to filter software engineers on non-software-engineering criteria to get resumes down to a manageable number.  In that filtering we lose great software engineers.  Often those who get through the ATS are unsuitable or if not decline when they discover the real JD and salary.  The best drop through the net and ghosted – because they didn’t use the right customization to get through the ATS.  It now takes many steps and much longer to find software engineers.

A carefully constructed JD targeting the specific software engineer we are after will encourage the correct person and no more to apply – and less steps in the interview process.  Broad JD’s get a broad response and difficult to filter.  The ATS could be used to much greater fruition if we only fed it suitable applicants.

I normally get JD’s between 20 and 30 lines – I have seen them 100’s of lines long.  Here is what I use in a JD to keep it short and get who I need:

  • Employer Name – links to employers’ site for more information, don’t put all the organization stuff in the JD as it hides the job.
  • Location – Be specific. Using neighboring cities to attract applicants not wanting to travel wastes time.
  • Job Description – use recognized descriptive terms and avoid internal, domain specific terms so you don’t have to describe them.
    • Department (or job function)
    • Job type (full time, part time etc.)
    • Job Title
    • Seniority level
    • Seniority level reporting to:
    • Salary range
    • Qualifications and Skills – List only the hard and soft skills, languages, and technologies used and level of importance. Don’t add qualifications or skills that are a standard part of that job title and seniority level.
    • Summary – project types, team size, culture, methodologies, main appeals of the job and what makes it unique. List any responsibilities and duties that are in addition to recognized job title and seniority level.
    • Benefits – vacation, retirement savings etc.

Photo by Quang Nguyen Vinh from Pexels

 

Software re-engineering of legacy software

There are legacy applications that have been there decades, and much younger applications but a few years.  Both can be subjects of software reengineering for a variety of reasons:

  1. technology no longer supported
  2. to inflexible and/or unable to harness emerging technologies
  3. newer technology can give a better solution
  4. it was just simply not written as best as it could have been
  5. the code has much duplicity, not scalable, and maintenance costs high
  6. badly architected
  7. after much maintenance the code has become disorganized
  8. skills to maintain are becoming rare
  9. compliance with updated or new regulation

What is software reengineering?

Software reengineering is the examination and modification of existing code to continue its use in a more refined state or to migrate it to a new modernized system.

Why not just start again and re-write?

This is the area that generates most debate with agile followers that say “we can just start again and throw away the legacy system when we are finished.  All we need are dedicated users to work along side us”.  This is re-inventing the wheel.

Software often has complex business-critical processes within.  Simply re-writing and you may lose valuable information hidden within the code.  Migration requires more upfront analysis, design, planning, and maybe contract negotiations as we move and probably in many steps to the desired outcome.  In terms of the agile manifesto we have a lot of value on the items on the right which is less agile and more waterfall.  Nothing in the agile manifesto says don’t do waterfall by the way – but it does say be as agile as you can be.

Not everything is obvious via the UI, documented, in someone’s memory, or can be accessed in anyway.  We need to access that hidden information buried inside the code.  Legacy software is also a completed document in its own right – it contains many lessons learned and algorithms that can be modified to fit the new technology.  Starting again and throwing away the legacy system when finished could be a very bad decision.

Many avoid the subject of reengineering but how many of today’s applications start off as a new idea compared to something already existing?  Software reengineering is a very valuable skill for a software engineer to have.

Reverse Engineer

Reverse engineering is the analysis of the existing system and development of high-level plans and designs – the existing system is a very detailed:

  1. functional specification
  2. requirements document
  3. working software

The primary purpose is:

  1. capturing requirements
  2. finding any lost or hidden information, and logic
  3. high level solutions

It is also a feasibility study should/can we

  1. modernize existing code?
  2. migrate to a new system?

If we decide to migrate to a new system then what is the best approach?

  1. The big bang – write a completely new replacement system that we begin using all at once.
  2. Migrate incrementally by modularization – identify modules (if any). We modify legacy code so it is organized into modules where each module represents a function or business area – combine related code parts in modules. We create integration/interface points between these modules so when we introduce a module that is part of the new system, we can redirect those integration points to it. If we are really clever, we use “feature switches” so we can switch between old and new without downtime.

We need to review and document what we will:

  1. Keep
  2. Remove
  3. Improve

We need to look at impact and risk and identify

  1. integration points
  2. interdependencies
  3. interfaces

Do we need to change or normalize the data?

  1. Determine if the interface and relationship with the data permit migration incrementally by modularization or forces a “big bang” approach.  We may need complementary strategies such as migration one region at a time to enable migration incrementally.
  2. Can we use the same database for legacy and reengineered concurrently?  If yes, we may need to make modifications to both legacy and reengineered applications to synchronize with the database as we go.  If no we need a data migration strategy and plan.
  3. If we are migrating incrementally by modularization and require a new database then how do we synchronize data between legacy and reengineered databases?

Forward Engineer

From the high-level plans and designs created in “Reverse Engineering” create low level plans and designs – especially if migrating incrementally by modularization

  1. Database(s)Database designs, migration, synchronization, concurrent support strategies.  Unlike a new product this is live and we have real data that we must not lose.  We have an opportunity to create this right first time – as we have lessons learned from the legacy application. There is little room to evolve a design.
  2. ApplicationObject Orientated Design – design patterns, and application frameworks.  This is not something new, we have an existing application and the majority of requirements.  There is less requirement to evolve an architecture because we know most of the requirements.If we have planned to migrate incrementally by modularization then we know what each of our modules are. We also need to plan for feature switches between them – to turn off modules in the legacy application and on in the reengineered application.

From the low-level plans and designs begin the physical implementation.

If we have planned to migrate incrementally by modularization the first step is to modify the legacy code into modules first or in synchronization with the reengineered application.  If we are creating a new database this would normally be created before the UI.

Deciding between agile and waterfall

A simple IF statement I use to determine which of agile or waterfall is best suited for a particular project.

IF the end picture is not clear & you don’t know how to achieve it:

  • without continuous feedback
  • most variables unknown
  • few if any requirements set
  • has complexity and novelty
  • defined deadline, scope, and budget not critical
  • team is high-performing, self-organized, allowed  great deal of flexibility, and overlapping skillset.

THEN

Agile

ELSE

Waterfall – there is nothing to be agile about i.e. no flexibility and almost everything already defined

END IF

It is a similar method to the agile manifesto where we value items on the left more.  Many may not like me saying it but the further to the right the closer we are to waterfall.  It could even be waterfall itself.

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

The IF statement shows how far we are to the right before we have started.  There could be many reasons for being nearer to waterfall than agile.

  • A customer already has a comprehensive idea of what they need
  • It is mission critical or scientific making any process fixed, and must practice being risk averse
  • There is simply to much information for the users to provide the software engineers daily that it must be documented
  • It is a re-engineering project and the existing software is in itself comprehensive documentation.

Likewise, the nearer we are to agile there can be many reasons

  • A customer only has a summarized idea of what they want
  • They can afford to be risk tolerant
  • Information can be given on-going and in drips
  • It is a new product, nothing exists yet.

Don’t close your mind

There is nothing in the agile manifesto or principles that says do not do waterfall – and nothing in waterfall that says don’t do agile.  The agile manifesto indicates if you are all the way to the right you are waterfall anyway.  If both are kept as simple a process as they need to be then both can produce great results.

I often think that if we didn’t formally name if we were doing waterfall or agile there would not be so much fuss over it.  Some Software Development Managers & Engineers are believers of one but skeptical of the other…

  • and believe only Agile can work for everything whilst…
  • others believe only Waterfall can work for everything

…but what I have found is not many have used the other – or didn’t use it well.

Conclusion

Great Software Development Managers and Engineers know both waterfall and agile and will select the most suitable.  If they make the wrong selection of tool, process, or technology they will adapt and get on with it.  Great Software Development Managers and Engineers know when to press the reset button and admit when wrong.

Photo by Leah Kelley from Pexels

Mythology versus methodology

This is not one of those waterfall v agile posts.  This is about debunking the mythology given to each.  I’ve used both extensively and find more similarities than differences, and many things said are just not what I have experienced.

Waterfall

  • Myths
    • Only have one sequence throughout the entire project
    • Customer cannot give feedback until the end of the project
    • Changes cannot be included until the end
  • Facts
    • You can have several sequences overlapping at the same time
    • Customer can provide feedback anytime
    • Changes can be included anytime

I never once worked on a waterfall project with one sequence.  I have worked on a waterfall project 30 months long with sequences 2 weeks long.  Sequences overlapped so when sequence 1 was in implementation stage, sequence 2 was already being designed. The customer gave feedback at the end of each sequence and this was fed back into the project.  Each sequence was iterative until complete.

Waterfall has a history in manufacturing/construction when things were be built in sequential order – basement, floors, then roof.  This is probably where the myth comes from.  When software came along a strict sequential order was no longer required.  We divided into more than one component and joined digital products later.  Manufacturing/construction also followed where many components are prefabricated long distance from where they are joined to together.  Today a roof may be built in a factory before the foundations even laid.

Agile

  • Myths
    • Must use one of the recognized processes such as Scrum and Kanban
    • Iterations must be of constant length
    • You cannot change the contents of an iteration once it is in action
  • Facts
    • Adherence to the agile manifesto and principles and you can add your own simple process to be agile.
    • Use a guide such as 2 weeks for iterations
    • one of the promoted benefits of agile is that you encourage change at any time

The agile manifesto states individuals and interactions over processes and tools – but many of these processes are commercialized versions of agile putting their processes and tools over individuals and interactions – that is how they make money.  I have worked on many agile products where we kept the process simple just using the manifesto and principles.  You only need as simple process as you need. We should own the process not the process own us.

For consistency many argue that all iterations are of the same length but I only ever had a guide for iteration length.  I would add as many user stories to that iteration without exceeding its length.  If that makes the iteration useful then fine, else add more user-stories until it is useful and make that the length of the iteration.   If iterations are constant then this requires the overhead of writing mocks and stubs, and feature switches to spread features over iterations.  The alternative is to change the contents of an iteration.  I always had the teams constantly monitoring progress so they could bring forward highest priority user stories into the current iteration, or push back the lowest priority user stories depending whether time was slack or tight.

Conclusion

Agile and waterfall are two different cultures that clash in software development by those looking for differences. When you focus on similarities not differences they are much closer to each other than you think.

Photo by Vera Arsic from Pexels

Team Building and Recruitment Strategy

Team building and recruitment for software engineers is part of the Software Development Managers job.  These are some of the “non-techie” and “soft skills” I look for when building a team.  Most cannot be identified by an ATS.

A carefully constructed and updated job description written by a Software Development Manager or Software Engineer for a specific job can target specific candidates – unlike some of todays “capture whole department” JD’s written by non-techies.   We never had a problem before reducing applicant amount using accurate JD’s, and filtering them by eye.  I would say we had a better outcome as great candidates always get to the front of the line.  It is imperative that the Software Development Manager and their team(s) are involved at all stages of recruitment.

Project Size and Complexity

For small projects it is important to have jack-of-all-trades, that is people who are multi-skilled but for larger projects specialized people in each area become more and more important.

We also need to consider the size and complexity of the project, and the time it will most likely take to complete.  There will be peaks and troughs in effort required and dependencies between tasks.  To fill these peaks and troughs it would be foolish and expensive to recruit more permanent staff – unless they can be re-allocated from elsewhere in the organization. If the resources are from the existing resource pool then there are no real risks but should you employ someone specifically for the project you need to consider local employment rights legislation.

Qualified or unqualified, Academic or Experienced?

Qualification is used by HR and ATS’s and filters out experienced candidates.  A Software Development Manager can see real experience with or without qualification in a resume or interview.  I also hold the view that qualification alone is not useful unless we have real experiences and there are people with experience but no qualification.

Academics can pass tests but have little if any practical experience to solve real life problems, whereas, experienced people have real-life practical experience and may find tests suited to academics harder – but given a real-world test will flourish.

We need experience and ability, but also responsibility to give newcomers a chance, therefore, we must balance the team, and we may require two interview strategies to do so.

Domain Specific or Global?

Domain specific software engineers focus on knowing software engineering for one industry very well.  Global Software Engineers bring new ideas to the table.  If we stuck with just domain specific software engineers, we are likely to get stuck in our ways – and create our own skills crisis.  Again, we can balance the team with both.  If we recruit from one domain we lose lots of opportunity to grow.

Pragmatic, Evangelist, or Zealot

With doubt pragmatic’s as they are guided by practical considerations than by ideals of any tool, process, or technology.  They will know the time when to implement a quick or elegant and robust solution, and most likely to see a balance between practicality and perfection.  Evangelist can be an excellent choice for a change program.

Skeptics or Optimists

Optimists assume their code will never fail and reduce testing to match, whereas, Skeptics will test far more thoroughly.

Asks questions?

Great Software Engineers question and constructively criticize a plan if it doesn’t make sense. Others do word for word whatever you say without question.  We need those who come forward – but we should accept responsibility as a manager to ensure we also give opportunity for them to come forward and not reject them.

Can take the rough with the smooth

Many software engineers love to write new code – but few loves to debug.  Fewer will want to debug someone else’s code.  Great software engineers even if not enjoying the task will dive in and resolve the bug – even if it wasn’t theirs.

After much time solving a problem with a particular tool, process, or technology evidence arises showing others are or even were better.  A great software engineer knows when to press the rest button and admit when being wrong

Many are fans of waterfall or agile but not both but a time may arise when a short project using the other came along.   Great Software Engineers will get on with it

Many like to work on the latest technology but there are times when we have to maintain legacy software.  Again, great Software Engineers will just do it.

Conclusion

It is very easy to exclude and create a skills crisis where none existed by being to specific with qualifications and domain requirements. We can also create problems in the team spirit if we recruit non-team players such as zealots or those that cannot take the rough with the smooth.  We need also to look at where a candidate and employer can collaborate with the candidate bringing new ideas to the table, and the employer offering the candidate a chance of new experiences.

There is a lot more to team building and recruitment that an ATS can cope with. Only a Software Development Manager or Software Engineer has ability to extract this information with discussion.

 

Empowering Software Engineers

One of the most difficult things for leaders to do is trust its software engineers to do what they do best – often because they don’t understand software engineering and always think the wool is being pulled over their eyes. They get a little knowledge and micro-manage.

We need to let the software engineers do what they do best. Software engineering as an art and a science. Have you ever noticed how you slow down and make mistakes when someone is looking over your shoulder? Artists make mistakes, they erase things, and begin again. Scientists make and break things, they use and refine calculations until they get perfection. That is also how software engineers work.

My course on Software Development Management will give you the Knowledge & experience to promote Teams that are:

  • self-managing
  • self-organizing

I can provide you the knowledge to be a Leader of high performing organized Software Development Teams using traditional or agile methods to:

  • trust to make decisions
  • value their opinion and expertise
  • listen more and talk less to value their expertise
  • give them breathing space to make mistakes – that is how people learn.
  • be efficient without compromising quality.
  • reduce costs through automation, reusability, & efficiency.
  • be subject matter experts in the evolutionary control of software using tools, processes, & methodology.

I teach you to understand so the wool cannot be pulled over your eyes – and your team will know that. They will value you, and you will value them. You will know how to recruit, retain, and develop the best team possible.

Hiring Software Engineers Checklist

When hiring I focus on software engineering & mindset which is 90% of the job & cannot be captured by ATS or tests. Discussion with experts will. Language & qualification are secondary. I created this guide to write my job descriptions, scan resumes, & interview to capture what the ATS cannot.

  1. Developer codes single objects. Engineers systems of multiple reusable objects.
  2. Smaller projects a jack-of-all-trades. Larger a specialist in each area.
  3. Qualification alone is no use. Real experience is. Academics can solve tests but not necessarily real problems. Experienced people solve problems but not necessarily tests.
  4. Domain specific know one industry. Others bring new ideas to the table from a variety of domains.
  5. With tools, processes, and technology zealots are fanatical & uncompromising, pragmatists are practical, evangelists seek to convert others to their ideals.
  6. Optimists assume code never fails so reduce tests. Skeptics assume code will fail and increase tests.
  7. Great engineers’ question & constructively criticize. Others do whatever is said without question.
  8. Many love to write new code – but few to debug. Fewer will debug someone else’s.

Academic v Experienced Software Engineers

A while ago, I did a root cause analysis for a Toronto client. They had problems with a software engineering team having the most trivial of tasks always missing deadlines. We found the root cause in recruitment not Software Engineering.

Job descriptions listed academic certifications as must-have. ATS found academic certification but could not convert to equivalent experience. Candidates then subjected to academic tests by HR which were assessed using a third-party. The Software Development Manager was out the loop until HR and ATS had filtered out the experienced. The team was 100% academic and needlessly over complicating solutions.

  • Academics can pass tests but have little if any experience to solve real life problems. Theoretical knowledge often is irrelevant in the workplace.
  • Experienced people have real-life experience and may find tests suited to academics harder – but in the real-world flourish.
  • Academics focus on theoretical perfection even if it doesn’t make a difference.
  • Experienced people focus on what makes a difference to the business.
  • Academics over complicate to prove a theory whereas the experienced simplify to make it economical and efficient.

The solution was the Software Development Manager scanning resume’s after the ATS and HR had their duties reduced to within their capabilities.