Evaluating IT projects is one of the most sensitive topics for programmers because clients usually come unexpected and they always have a burning task. A client usually does not know whether the task is large or small, assuming that it is small and could be done quickly.
During the very first meeting or phone call, a client asks for an accurate estimate of how much it will cost to create the application and wants to stick to that price. And of course, if that price is agreed, would like the programming process to start immediately. This customer approach puts a lot of stress on salespeople, project managers, business analysts, as well as the programmers themselves. Customers want to have a product, you want to have a client, for income, new experience, and challenges in developing new and quality products.
Thus, the problem is that the assessment needs to be fast and accurate. The simplest solution that everyone makes is guesstimate. This is an assessment by guess. Its accuracy can be improved by working with the same particular technology for a very long time, and the accuracy depends on experience in implementing specific standard components which are in every software system. If from the first conversation with the client you can already imagine what components could be in the system, you know how to connect them and how many of those components you already have from other projects, and from experience, you know how long it takes to connect those components, then guesstimate could be quite accurate. However, it is likely that the time range between an optimistic and a pessimistic estimation still be very large, for example, pessimistic estimation could be up to 10 times bigger than optimistic. Customers with a fixed budget will usually be dissatisfied with such ranges. In case the client starts the project with an optimistic guesstimate in mind, the price usually increases massively during the project. In such a situation, both, the client and the IT company become extremely dissatisfied. IT company cannot implement the project so cheaply, the cost to the client goes up and he can no longer easily terminate the contract. The quality of the project will also be badly affected by the low budget.
So project evaluation is a very important issue. Is it still possible to do it quickly and accurately?
Yes, accurate enough. This will not be an ideal accuracy, such as is achieved during project implementation, when the speed of features implementation is visible after each sprint, but still, it can be done with sufficient precision.
Who should be involved in that assessment? One of the biggest contributions, of course, is from programmers. A modern programmer spends a significant portion of his day interacting with clients. We know our programming languages, frameworks, technologies. If we don’t know something, at least we have a sense of how long it will take us to learn and integrate these new technologies.
Programmers must be involved in the initial conversations with the client. In a situation, when it is still unclear whether the competition with other companies for the project will be won, an evaluation is still needed, but it is not necessary to involve specialists from the whole organization. Based on the methodology I am describing below, it is sufficient for several programmers to directly communicate with the client and produce a sufficiently accurate estimate without incurring massive evaluation costs.
- Preparation of a requirements summary
If you have direct contact with a client, it is professional to say no to guesstimate. Ask the customer to prepare a very approximate specification for his desired software system. This means that the customer could write a few pages or just describe in more detail what kind of system they want.
Typical questions customer should be asked are these:
- the purpose of the system
- where it will be used
- on what hardware it should run
- for which user groups it will be
- what are the functions of these user groups
After having the client’s requirements, we ask only for a day or a few to prepare an assessment.
Based on the client’s story, the system description, and a summary of the system requirements are prepared. In system requirements summary we list functions of each user group, for example, grouping them by application screens. Let’s say that there will be a login screen, several other screens, and the logout screen. For every screen, we briefly describe what functions will be performed there.By doing this we get a short, more technical document. Now it will be clear which parts of the project are difficult to implement and which have already been done many times before and can be completed quickly. The proportion of difficult and easy tasks in the project will be visible. It will also highlight the most important, critical functions that should be done first. We should start with the easiest and most important functions, and do not implement the most difficult and least worthy ones.
- Translation of requirements summary into hours, monetary expenses and story points
I use the Project evaluation and review technique (PERT) to translate the requirements summary into the hours of how long it will take to complete the project. PERT was developed primarily to simplify the planning and scheduling of large and complex projects. It was developed for the U.S. Navy Special Projects Office in 1957 to support the U.S. Navy’s Polaris nuclear submarine project. It found applications all over industry. An early example was it was used for the 1968 Winter Olympics in Grenoble which applied PERT from 1965 until the opening of the 1968 Games. This project model was the first of its kind, a revival for scientific management, founded by Frederick Taylor (Taylorism) and later refined by Henry Ford (Fordism). DuPont’s critical path method was invented at roughly the same time as PERT.
It is also highly recommended by Robert Cecil Martin, colloquially called “Uncle Bob,” an American software engineer, instructor, and best-selling author. He is now recognized for developing numerous software design principles and for being a founder of the influential Agile Manifesto.
For our assessment, we will apply only a part of the PERT, only to the extent we need it to prepare a preliminary evaluation, still to perform that task as accurately and quickly as possible.
Basics of PERT methodology
Several formulas will be used for evaluation:
- standard deviation
- a sum of means
- a sum of standard deviations
O – optimistic implementation time
N – nominal (average) implementation time
P – pessimistic implementation time
A weight for N, average implementation time, is 4.
The standard deviation (mean deviation from the mean) is the difference between the pessimistic and optimistic implementation time divided by the sum of all weights, which is equal to 6. In calculating the final project implementation time, the usual practice is to add two standard deviations to the mean.
The mean and standard deviation are calculated for each feature, then the sums of the mean and standard deviation are calculated. These are project implementation time estimates. Knowing the hourly rate, we can easily find out the cost of the project.
PERT implementation using Excel spreadsheet
Eating an elephant can only be done one way: one bite at a time. The best way to accomplish something big is to approach it in smaller pieces.
As we can see, at the top of the table are the weights: 1 to optimistic, 4 to nominal, and 1 to pessimistic estimates. Below we can see what information should be entered into the spreadsheet. It’s the name of the feature, three estimates in hours, and story points for evaluation of task complexity. The spreadsheet calculates the mean, standard deviation, and its square (used to calculate the standard deviation of the project) itself. We can see that each feature is broken down into at least 4 parts. The finer the decomposition, the more accurately each part can be estimated. Also do not forget to include the preparatory work and the publication of the application (in the case of Android, the publication in the Google Play Store).
After entering information for every feature, the spreadsheet automatically calculates the whole project time and complexity estimates.
In this part of the spreadsheet, we can see the project completion time estimate and also it’s pessimistic figure (sum of feature averages + 2 standard deviations). Estimates are presented in hours and months (6 hours of coding daily). We can easily turn hours into monetary expenses multiplying by the hourly rate. Also, we can see the number of hours per story point and how many story points a programmer could perform in a 2-week sprint.
If the project was evaluated by more than one programmer, we can calculate the average of these estimates, thus further increasing the accuracy.
Not all customers want an application to be covered by tests (the usual wrong argument is that writing tests will take longer to complete the project), but even then we can easily provide an evaluation of adding TDD and gaining long-term benefits (making refactoring and expansion of the application much easier).
So, now we have all the information the customer needs. It may not make sense to provide the customer with a table of such detail, but leaving the main parts and perhaps not showing an optimistic option (which is often misleading) can be the result a client wants. Most often the client needs an average and pessimistic project completion time estimates also translated in monetary expenses. This result also gives a client an understanding of project complexity.
The evaluation provided in this way allows the client to decide whether to start the implementation of the project and maybe decide not to implement certain expensive but not very important functions.
As you can see, such evaluation is not very expensive in time and money terms. As a result, a client would know you better as a professional, and you will be more prepared for such situations in the future. That means in future you could make such kinds of evaluations faster and better.
That’s it for this time. I hope the method presented will greatly facilitate your daily routine and give you more enjoyment in your work.