Towards Reliable Agile Iterative Planning via Predicting Documentation Changes of Work Items

Authors: Jirat Pasuksmit, Patanamon Thongtanunam, Shanika Karunasekera

Venue: The International Conference on Mining Software Repositories (MSR)

Abstract:

In agile iterative development, an agile team needs to analyze documented information for effort estimation and sprint planning. While documentation can be changed, the documentation changes after sprint planning may invalidate the estimated effort and sprint plan. Hence, to help the team be aware of the potential documentation changes, we developed DocWarn to estimate the probability that a work item will have documentation changes. We developed three variations of DocWarn, which are based on the characteristics extracted from the work items (DocWarn-C), the natural language text (DocWarn-T), and both inputs (DocWarn-H). Based on nine open-source projects that work in sprints and actively maintain documentation, DocWarn can predict the documentation changes with an average AUC of 0.75 and an average F1-Score of 0.36, which are significantly higher than the baseline. We also found that the most influential characteristics of a work item for determining the future documentation changes are the past tendency of developers and the length of description text. Based on the qualitative assessment, we found that 40%-68% of the correctly predicted documentation changes were related to scope modification, and such changes could impact the accuracy of estimated effort and the sprint plan. With the prediction of DocWarn, the team will be better aware of the potential documentation changes during sprint planning, allowing the team to manage the uncertainty and reduce the risk of unreliable effort estimation and sprint planning.

Preprint: PDF