Amassing and indexing a large sample of version control systems: towards the census of public source code history

TitleAmassing and indexing a large sample of version control systems: towards the census of public source code history
Publication TypeConference Paper
Year of Publication2009
AuthorsMockus, A
Conference Name6th IEEE Working Conference on Mining Software Repositories
Date PublishedMay 16–17
Keywordsbazaar, cvs, flossmole, git, mercurial, source code, sourceforge, subversion, version control
Abstract

The source code and its history represent the output and process of software development activities and are an in- valuable resource for study and improvement of software development practice. While individual projects and groups of projects have been extensively analyzed, some fundamental questions, such as the spread of innovation or genealogy of the source code, can be answered only by considering the entire universe of publicly available source code and its history. We describe methods we developed over the last six years to gather, index, and update an approximation of such a universal repository for publicly accessible version control systems and for the source code inside a large corporation. While challenging, the task is achievable with limited resources. The bottlenecks in network bandwidth, processing, and disk access can be dealt with using inherent parallelism of the tasks and suitable tradeoffs between the amount of storage and computations, but a completely automated discovery of public version control systems may require enticing participation of the sampled projects. Such universal repository would allow studies of global properties and origins of the source code that are not possible through other means.

AttachmentSize
PDF icon 11amassing.pdf108.04 KB