9. September 2022 By Stefan Mönk
#5 Agile software development – progress requires empirical knowledge
In previous blog posts in this series, we found out about agility in software development. When we take an agile approach to software development, we typically propose a hypothesis. This generally includes assumptions regarding what the customer needs and a concept for how we can meet this need.
When we now go about implementing the hypothesis, we subsequently review our findings with the customer. We also examine how we came to the results and discuss what to keep in the process and what can be improved. We scrutinise the steps we have taken and change the way we work in order to be even more successful in the next iteration.
What does empirical knowledge mean?
I do not want to go too far back into the roots of the word ‘empirical’, but I would like to explain what the term means since not everyone will be familiar with it. Empirical derives from the Greek and more or less translates as experience or experience-based knowledge. The Cambridge dictionary defines empirical knowledge as ‘knowledge based on what is experienced or seen rather than on theory’.
Most of us are not scientists, so here is a simply definition. Empirical knowledge relates primarily to the systematic collection of data in order to check a theory or hypothesis that has been proposed or to validate or refute it. That means we do not simply rely on theoretical science to obtain empirical knowledge – we gain it through personal experience.
But that is not enough, nor is gathering information on its own. We strive to be efficient in what we do. How do we gain empirical knowledge in practice?
Empirical process control
Empirical knowledge is built on three pillars, all of which we can find in the principles of the Agile Manifesto as well as in different agile frameworks. Empirical process control lets us apply empirical knowledge and make full use of it.
The three pillars are: transparency, inspection and adaptation.
The more transparently we work, either individually or as a team, the more data and information we will have about what we do. But how do we create transparency? First and foremost through communication. If we regularly discuss our goals, work and challenges, we are already doing a lot right. Looking beyond one’s own department, being open about giving and receiving feedback and having the courage to question findings all help promote transparency.
Communication is generally fostered and improved in agile frameworks, which in turn can have a positive impact on transparency. Artefacts, events or even predefined processes make it easier for us to work transparently. Under the Scrum approach, the daily scrum meeting helps developers visualise challenges that have an impact on the sprint goal. The sprint retrospective fosters discussion on which work processes we should keep from the previous sprint, and which ones we should change.
Generally speaking, Scrum is prime example of an empirical approach. The framework promotes transparency, while each of the events incorporates the concept of inspection and adaptation.
Inspection refers to the assessment of a process, output or working process. We analyse if we are working effectively or if there might be room for improvement. As part of the inspection, we describe what we have observed, for example, we regularly take too long to complete a specific meeting, we have not reached the final sprint goals or certain people always argue with each other during the discussions. However, it could just as easily be the case that there are far fewer changes being made to developer tickets since we began holding the workshops on site with the customer. Or we could as well discover that we meet the sprint goal more often and there are fewer open tickets at the end of a sprint because developers actively negotiate the sprint goal with the product owner.
Just as we cannot scrutinise our every action or everything that happens around us, we also cannot change everything all at once. We have limited time to make changes. That means we focus on what is most important. Within our environment, we need to deal primarily with the things that either promise to deliver the greatest success or the things that need to be resolved most urgently.
During the adaptation phase, we give it great thought how we want to spend our time. Time spent making changes to processes is not available for project-related business.
Once we decide on something, we define the goal and try to limit the amount of time and work that will be needed to achieve it. How important is it to us that we solve the problem? If we do not have enough time for everything, we go back and look at what we have done, check to see what we still need to do and prioritise.
It is good to know each of the problems that need to be addressed, but that alone will not make anything better. This means we need people in charge who focus on implementing the ideas that have the highest priority. That could be a single member of the team, someone from outside the team or the whole team.
An approach based on empirical knowledge delivers results, even if I only use it once. But is that enough? This approach is not so much a process that you have to begin in order to get where you want to go – it is more a way of working and thinking. This means that until people absorb and actively apply the empirical approach, it will not be able to unleash its true potential. Values and culture can support us here.
We do not brush our teeth every few days or so. And if you want to get a tan, you do not lay down in the sun once. It is not enough to talk about being stuck and needing help in one of many situations. Who does it help if we do not say anything the next time around and simply go on with our work?
If we notice that our development cycle is too long and reduce it from four to two weeks, never bringing up the matter again simply does not do. What if we find out that the results are even worse after six weeks? How much damage would be done if we do not openly discuss the matter in order to finally make changes and learn from this?
Taking an approach based on empirical knowledge requires that we have an open mind. It requires that we give and receive feedback. It can be really fun to regularly meet to discuss what went right, what went wrong, what we want to keep and what we want to change, and then plan how to make the changes that are needed.
We analyse the actions we have taken and make changes, if necessary. We continue to evolve and get better all the time. It is hard work, but once you have enjoyed success, it is something you will want to experience again.
Do you scrutinise your actions, think about what you could have done better and carry over this knowledge for the future? How transparently do you work?
You will find more exciting topics from the adesso world in our latest blog posts.