AGTEK R & D Wiki

This is an old revision of the document!

With SmartDirt 1.0 completed (December 2011), we're over due to analyze the measured results of the SmartDirt team. This page will summarizes the actual SmartDirt effort and durations, and computes the teams velocity. Velocities can be used to calculate estimates and predictions of future projects.

First, the raw data:

The following charts show the velocity as measured for the development duration of the project. The first chart shows the estimates of stories (in ideal days) and the measured duration (in actual days). If the team is “hitting it's mark” the two numbers will be identical for each story (variation between stories is due to the actual technical content of each story).

Another measure of “hitting the mark” is EV values very close to 1.0 (calendar velocities are lower because of weekends, 5/7 to be accurate).

The second chart shows the velocities of each story, as measured in actual task work days, and as measured against the calendar. The first velocity is called the “Effort Velocity” (EV) while the second is called “Calendar Velocity” (CV). In general velocities are applied to story estimates to predict actual end dates. An Effort Velocity allows you to calculate the actual task days, while the Calendar Velocity allows you to calculate the story completion date.

If all work was homogeneous throughout the project the velocity numbers shouldn't vary much from story to story. This is, of course, not how the world operates and the SmartDirt project was no exception. There were several interruptions on the project which took one or both developers away from core coding for extended (months) periods of time:

• New License System
• ADF Converter work
• ADF Implementation and rework
• Vacations and Holidays

The Effort Velocity does not take this into account because we didn't record specific tasks for “developer doing something else”. On the other hand, Calendar Velocity does take this into account because it only looks at start and end dates compared with estimated work days. Calendar Velocity also takes into account weekends. Because of this Calendar Velocity is ideal for applying to stories or projects to get a more reasonable expected end date.

Summary Velocities It's not reasonable to use any specific velocity from any one story, but it's better to compute aggregate velocities over the entire 1.0 project. The aggregate values will lump many real world distractions in and do a better job of modeling future performance. Average V incorporate all distractions encountered on during SmartDirt 1.0, while median V only take into account moderate distractions (multiple month distractions are thrown out).

Given that we have EVa, EVm, CVa, CVm a final value is to be calculated, which is Calendar Velocity for the total project (CVt). The estimated duration was 257 days while the actual duration was 410 days. This provides the final value:

NOTE: Story zero was thrown out of the data calculations because it was so short and wasn't technically similar to the other stories. The artificially high velocities skews the average and median values to higher than actual.

Because of the small number of stories, median velocity values are significantly higher than measured project velocity or average. I (mja) believe a more reasonable velocity value is somewhere between the average and median values.

How to apply. For future projects planning can use these numbers to calculate more likely end dates and durations on a story by story basis or the entire project. Use average velocity values to apply to individual stories, while the total velocity (CVt) should be applied to the entire project.

Example 1: Suppose a story is estimating 37 ideal days, using the average EV, we can compute probable duration = 37 / EVa = 42. This tells us the engineers are probably going to “run over” by 5 days on the original estimate for the story. Using the average CV we see an end date of 37 / 0.82 = 45.1 or probably 46 days in the future.

Example 2: SmartDirt stories total to 257 ideal days, slightly more than one work year. With two engineers we'd expect it to be done in about 6 months, which didn't happen. A more likely date would be 257 / CVa = 313 days. In fact the actual end of SmartDirt ended up being 410 days, almost 3 months shy of this prediction. This is where we apply CVt. 257 / 0.63 = 407.

Using story level velocities is usually a good measure of where the current story will end up. Engineers are usually not taken in the beginning of a story, usually toward the last half. Using the total velocity (CVt) is useful for determining the end date of the entire project.

Caveats The following conditions, if encountered, invalidate the use of these velocities:

• Major changes to team - changing engineers. The team isn't big so any change is significant.
• Major new technology introduced - e.g. NDK on x86, 'porting to Windows Mobile'.
• Borrowing (poaching) engineers for other effort - Will invalidate Median V numbers.
• The small number of stories mean that the median values aren't in any sense typical.
• Individual velocities have not been measured.

Recommendations

• The current velocity numbers should be used to compute more likely dates for communication to management or internally to decide if specific work can be done within a specified time frame.
• Future projects should continue to measure velocity and in particular add to the data so median calculations are more statistically reliable.
  • Engineers should not apply velocity to their initial estimates and continue to estimate in “ideal days”.
• Engineers need to do a better job of marking accurate actual dates.
• Engineers need to do a better job of closing tasks (several tasks in story 6 are still “open”).