8 reasons why you shouldn’t rely on source lines of code as a software metric

The estimate of the value of production of software based on the number of lines of code (LOC or KLOC or SLOC) is as popular as it is controversial. The main criticism is that there are too many factors influencing the final measurement value. Robert E. Park (1992, page 140), software metrics specialist & staunch defender of the method, responded to critics with the following:

“When we hear criticism of SLOC as a software measure, we are reminded of a perhaps apocryphal story about a ditch digger who, when asked one day how he was doing, replied, “Dug seventeen feet of ditch today.” He didn’t bother to say how wide or how deep, how rocky or impenetrable the soil, how obstructed it was with roots, or even how limited he was by the tools he was using. Yet his answer conveyed information to his questioner. It conveyed even more information, we suspect, to his boss, for it gave him a firm measure to use as a basis for estimating time and cost and time to completion.”

Originally, this technique could probably be used in the conditions mentioned by the Park. Later models such as COCOMO (Boehm 1981) also allowed developers take into account a number of parameters whose variability was probably reasonable at the time. But since then, the number of factors affecting the number of lines of code has become so important that it is very unwise to take this action seriously both in the evaluation of the software and the productivity of the design team.  I will try to illustrate the problem using eight arguments.

1. Different languages and different frameworks

Today hundreds of different languages exist (Wikipedia 2013).  For each of these languages there are several frameworks.  For the same functionality, there may be a very different number of lines of code produced depending on which technology is chosen.  In addition, modern architectures use different technologies, which further complicates calculations. Correction factors exist but they hardly seem defensible given the wide variety of types of applications that are being developed today.

2. Experience and competence of the developers

We must also take into account the experience of the developers involved as this may affect the calculation in many ways. A very competent developer often writes fewer lines of code than other less experienced developers because they will use design methods created for the sole purpose of reducing the number of lines and increasing readability and maintainability. In addition, they are more competent with the functionalities offered by tools (technology stack). Indeed, through ignorance of these, many programmers rewrite existing code, greatly increasing the number of lines of code.  In this regard, many experts in the area do not hesitate to speak of “lines of code spent” as opposed to “lines of code produced” (Dijkstra 1983).

3. The practice of refactoring

The fact that the same piece of code can change over time with the refactoring (reworking of code) can skew the results. This practice reworks the source code without adding functionality (Wikipedia 2013) and it is becoming more common because it can increase code quality and reduce technical debt. This can cause unexpected situations: if many developers practice this technique while the lines of code are being measured, the result could give the appearance of a reduction in output (fewer lines of code than in the previous measurement), while it is clear that the opposite occurs.

4. The practice of reuse and / or code generation

The reuse of existing code is very common and highly recommended in DRY (Do not Repeat Yourself). So many parts of the code can be retrieved from a previous project or copied from an open source project, library or another blog post. In addition, modern development tools can automatically generate code for the developer who works with various high level design tools.

5.  Tasks outside development

Activity in the development of software is not limited to writing code on a keyboard. In fact, many other tasks are needed to produce quality code. Here, a high variability can emerge according to the different methods used, the composition of the team or the documentation requirements.

6. The reliability of the measurement tool

A wide variety of measurement tools are available on the market. Given the lack of consensus on the method of counting the amount of lines of code in a source file, the outcome may be materially different depending on the tool used.  In addition, certain technical problems can arise when it comes to identifying what should actually be counted or not. For example, some software has difficulty differentiating comments from instructions when they are mixed (Danial 2013). The efficiency and quality of those source line counter is also very variable.

7. The (potential) manipulations

When a measure may have an impact on one or more person, we need to consider the possibility that some of them try to manipulate it to their advantage. Thus, if the productivity of a developer is measured based on the number of lines of code (or functions), they could very easily manipulate the source code to inflate the results. This problem is very common in companies that use KPIs to conduct assessments of their employees. One can also easily imagine a company trying to maximise the numbers if they know they will be evaluated based on this metric.

8. Time

Almost all the above elements are time sensitive. For example, the competence of a developer does change with practice (this includes the famous learning curve). More features of languages ​​and frameworks are also evolving to increase the productivity of the developers. The longer a project takes, the longer the measurement will be sensitive to this bias

Conclusion

In conclusion we can say that estimating the production effort or value of a program using this software metric is very risky. However, this technique is widely used. Some estimate experts such as Steve McConnell (2006) are very aware of the ineffectiveness of the method but still use it in the absence of anything better. Other methods based on “function point” (business functionality) have attempted to resolve some of the issues addressed above, but the values ​​remain highly correlated with the number of lines of code (Albrecht 1983).  For me, the information obtained by these metrics, and anything based on them, should never be considered as reliable and should be used with great caution in your decision making process.

Note: Some of the information in this text come from the fruit of the research I have done for LIEU (Liaison Entreprises-Universités) a network of valorisation units of Universities and colleges of the Wallonia-Brussels federation.

References

Albrecht, A. (1983). Software Function, Source Lines of Code, and Development Effort Estimation. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=1703110&searchWithin%3Dp_Authors%3A.QT.Albrecht%2C+%2FA%2F.J..QT.%26searchWithin%3Dp_Author_Ids%3A37850740200

Boehm, B. W. (1981). Software Engineering Economics.  Englewood Cliffs, NJ. http://userfs.cec.wustl.edu/~cse528/Boehm-SE-Economics.pdf

Danial, A. (2013). CLOC Limitations. Retrieved the 2 august 2013 from http://cloc.sourceforge.net/#Limitations

Dijkstra, E. W. (1983). The fruit of misunderstanding. Retrieved the 2 august 2013 from http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD854.html

List of programming languages. (2013, July 30). In Wikipedia, The Free Encyclopedia. Retrieved 12:48, August 2, 2013, from http://en.wikipedia.org/w/index.php?title=List_of_programming_languages&oldid=566431816

McConnell, S. (2006). Software Estimation : Demystifying the Black Art.Microsoft Press. http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351/

Park, R. E. (1992). Software Size Measurement : A Framework for Counting Source Statements. http://www.sei.cmu.edu/reports/92tr020.pdf

Réusinage de code. (2013, juillet 5). Wikipédia, l’encyclopédie libre. Retrieved the 12:04, august 2, 2013 from  http://fr.wikipedia.org/w/index.php?title=R%C3%A9usinage_de_code&oldid=94719037

From a group of workers to a powerful software development team

All software development team project managers hope for a dream team of successful developers working to assist the company to achieve its objectives. Unfortunately, this type of team is very rare in all professions, including in the area of ​​software development. I am thinking particularly of enterprise applications or software companies where teams of developers are working simultaneously on the same code base of the same product.

The reason probably lies in the way we generally go about increasing developer productivity. It is probably too focused on improving the hard skills of the individuals (e.g. sending them on technical training and evaluating them individually on these points) and not their soft skills which without doubt positively and significantly impact on the effectiveness of the whole team. We should also consider “groupal” improvements: organizing the team so that it can naturally grow and reach a much desired level of maturity.  Raising awareness of the groupal phenomena is an example that I will specifically address in a future article.

I propose here a brief introduction to the concept of team dynamics that I will discuss more thoroughly and specifically in future posts. To begin with, I offer a definition from “high performance teams” research in group dynamics and organizational psychology. This approach is put forward by Katzenbach & Smith (1993) in their book “The Wisdom of Teams” and seems to me to be a particularly interesting way to begin addressing this issue. Both authors propose, as a first step, to clarify the difference between a single group of individuals and a team:

“A team is a small number of people with complementary skills who are committed to a common purpose, performance goals and approach for they hold themselves mutually accountable.”

We will see how this breaks down operationally. According to Katzenbach & Smith, the characteristics that differentiate between a normal team and a successful team are:

1. A limited number of members

As the number increases, it is less easy to maintain high efficiency. As suggested by the authors in their work, it is clear that problems of communication grow along with the size of the team.

2. They are composed of people with complementary skills

We immediately think of the different technical skills, but it goes further than that. The authors propose three categories of skills:

  • Technical & functional expertise
  • Problem-solving and decision-making skills
  • Interpersonal skills

In software development, we tend to wrongly focus on the first category (hard skills) while neglecting others. One of the best ways to increase the efficiency of an entire team is to invest in developing their soft skills. This should also allow them to see things from a “groupal” perspective as opposed to the “individual” view which is overwhelmingly the case. In addition it should be noted that the multi-complementary members of a team involves active support – and that even within the same skills category. Thus, the developer of the backend will fully rely on the expertise of the DBA when needed and vice versa. All promotion of individualism, for example by individual evaluations (explicit or implicit), would make a good team ineffective.

3. Committed to a common purpose and performance goals

This is a common objective defined in agreement with the team. The goals should never be imposed by any individual (eg the team leader). The performance of the team to achieve the goals must be measurable. The measurement method is paramount and should in no case enter into conflict with the above principles, for example by measuring the team members individually and not collectively.

4. Committed to a common approach

This approach should also be clearly defined. It seems to be that the first requirement is the establishment of coding guidelines that must be enforced by a continuous integration server and / or frequent code reviews. The latter, to be effective, must not be made by the same people but by the team in rotation. This allows different members of the team to share learning and leads to a mutually enriched team. The criteria should be re-discussed and re-evaluated in a retrospective sprint (if you work in an iterative model as Scrum). The team should also have overall responsibility for development tools it uses. If they have a strategic interest in the company, its main users should be involved in one way or another in the discussions. Which brings us to the last point.

5. Mutual accountability

This element seems to me particularly important to ensure good cohesion in the team. If you want to get to an optimum efficiency level, you must delegate some control over the project to the team (eg the choice of tools as mentioned above). The worst situation would be to make the team fully responsible for the result but taking away any means of control over their performance. This would create very severe stress that would be counterproductive for the entire organization.

As you may have noticed, there is plenty of room for discussion. And this is just based on the findings from a single study. There are hundreds of other studies that will allow me to return to these questions of team dynamics (aka group dynamics) very frequently in the future.  I think that this is a fundamental element in the field of software development. The specifics of this sector will allow us to approach things differently. I am thinking especially of the psychological profile of the developer in general (myth or reality), but also the technical means available to create an optimal working environment to meet the criteria for achieving optimum performance. For example the use of the continuous integration server but also the various meetings organized within the framework of a method of iterative development.

If you are interested in the subject, do not hesitate to share your experiences in a comment or by contacting me directly.