Code Metrics Statistics with TeamCity

Code metrics can be a very useful tool for monitoring some aspects of code quality. To get the most out of it you need to calculate the metrics on a regular basis in order to find trends. Code quality tend to go down during intense phases of a development project and also when a product is in low-intensity development, a.k.a the maintenance phase. The obvious thing to do for a modern developer is to integrate calculation of code metrics into the Continuous Integration process.

In this post I will demonstrate how you can calculate code metrics and display graphs of the evolution over time as part of your Continuous Integration process. There are a few tools available for calculating code metrics in .NET, the most capable is without doubt NDepend. Here I will use another tool, SourceMonitor, which is free of charge and very lightweight. We use Team City at work, so that is the CI server that I will use here also, but you could probably implement this idea in the CI server you are using.

One of the goals I had when I started experimenting with code metrics statistics was that I wanted it to be easy to add statistics to any project, without changing anything in the project itself. All changes should be in the Team City configuration.

Step 1: Create Metrics project in VCS

The first step is to create a project in your version control system. This project should contain the following artifacts:

  • The SourceMonitor executable
  • A SourceMonitor command file, SourceMonitorCommands.xml
  • A MSBuild script file, SourceMonitor.proj
  • MSBuild community tasks

I put all the files in a directory called SourceMonitor.

Step 2: Create a SourceMonitor command file

I will not go into the details of working with SourceMonitor. If you are interested, you can read the documentation that is included in the download. Here is the command file that I used:

<?xml version="1.0" encoding="utf-8"?>
<sourcemonitor_commands>
  <write_log>true</write_log>
  <command>
    <project_file>MyProject.smp</project_file>
    <checkpoint_name>Baseline</checkpoint_name>
    <project_language>C#</project_language>
    <source_directory>..</source_directory>
    <source_subdirectory_list>
      <exclude_subdirectories>true</exclude_subdirectories>
      <source_subtree>bin\</source_subtree>
      <source_subdirectory>obj\</source_subdirectory>
    </source_subdirectory_list>
    <parse_utf8_files>True</parse_utf8_files>
    <ignore_headers_footers>True</ignore_headers_footers>
    <export>
      <export_file>SourceMonitor-details.xml</export_file>
      <export_type>2</export_type>
    </export>
  </command>
  <command>
      <project_file>MyProject.smp</project_file>
    <checkpoint_name>Baseline</checkpoint_name>
    <export>
    <export_file>SourceMonitor-summary.xml</export_file>
    <export_type>1</export_type>
  </export>
  </command>
</sourcemonitor_commands>

This command file will create two xml-files: SourceMonitor-details.xml and SourceMonitor-summary.xml. It is from the latter that we will extract the values to publish to Team City.

Step 3: Create a build script

Here I have used MSBuild, but you can of course use NAnt, Rake or whatever. The build script will do the following:

  • Run SourceMonitor on your source files
  • Extract the interesting values from the resulting xml-file
  • Publish these values to Team City

The MSBuild community task XmlRead is used to extract the values from the xml-file. The Team City task TeamCityReportStatsValue is used to publish the values. The community tasks has to be explicitly imported, but the Team City tasks are imported automatically when the script is run by Team City. Here is my MSBuild script:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="3.5" DefaultTargets="Analyze"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
  <MSBuildCommunityTasksPath>.</MSBuildCommunityTasksPath>
</PropertyGroup>
  <Import Project="MSBuild.Community.Tasks.Targets"/>

  <Target Name="Analyze">
  <Exec Command="SourceMonitor.exe /C SourceMonitorCommands.xml"/>
    <XmlRead XPath="//*/metric[@id='M0']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="NumberOfLines" />
    </XmlRead>
    <TeamCityReportStatsValue Key="NumberOfLines" Value="$(NumberOfLines)" />

    <XmlRead XPath="//*/metric[@id='M5']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="MethodsPerClass" />
    </XmlRead>
    <TeamCityReportStatsValue Key="MethodsPerClass" Value="$(MethodsPerClass)" />

    <XmlRead XPath="//*/metric[@id='M7']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="StatementsPerMethod" />
    </XmlRead>
    <TeamCityReportStatsValue Key="StatementsPerMethod" Value="$(StatementsPerMethod)" />

    <XmlRead XPath="//*/metric[@id='M10']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="MaxComplexity" />
    </XmlRead>
    <TeamCityReportStatsValue Key="MaxComplexity" Value="$(MaxComplexity)" />

    <XmlRead XPath="//*/metric[@id='M14']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="AvgComplexity" />
    </XmlRead>
    <TeamCityReportStatsValue Key="AvgComplexity" Value="$(AvgComplexity)" />
    </Target>
</Project>

The build script above will extract the following metrics and publish them:

  • Number of lines of code
  • Average number of methods per class
  • Average number of statements per method
  • Maximum cyclomatic complexity
  • Average cyclomatic complexity

Step 4: Create a build configuration in Team City

The next step is to create a new build configuration for the project that you want to analyze and display statistics for.

Attach your VCS project root and the SourceMonitor root as well. Your version control settings will look like the example below.

Version Control settings for Metrics

Version Control Settings in Team City

Configure TeamCity to use the MSBuild runner for your build and specify the path to the build script that will run SourceMonitor. Also specify the target, in my case “Analyze”.

Build runner configuration in Team City

Setup build triggering the way you like it, either as a dependent build or scheduled at some interval, for example every night.

Step 5: Configure Team City to display statistics

TeamCity has built in capablities to display statistics in graphs. You can easily add your own graphs to a project by adding configuration to the file plugin-settings.xml which is located in the folder: (TeamCity path)\.BuildServer\config\(project name). Note that you have to change the buildTypeId value to the Id of your Metrics build configuration. You can find the buildTypeId in the URL as a query parameter. Below is a sample plugin-settings file:

<?xml version="1.0" encoding="UTF-8"?>
<settings>
 <custom-graphs>
    <graph title="Number of Lines" defaultFilters="" hideFilters="showFailed">
      <valueType key="NumberOfLines" title="Number Of Lines" buildTypeId="bt61" />
    </graph>
    <graph title="Methods per Class" defaultFilters="" hideFilters="showFailed">
      <valueType key="MethodsPerClass" title="Methods per Class" buildTypeId="bt61" />
    </graph>
    <graph title="Statements per Method" defaultFilters="" hideFilters="showFailed">
      <valueType key="StatementsPerMethod" title="Statements per Method" buildTypeId="bt61" />
    </graph>
    <graph title="Complexity" defaultFilters="" hideFilters="showFailed">
      <valueType key="MaxComplexity" title="Max Complexity" buildTypeId="bt61" />
      <valueType key="AvgComplexity" title="Average Complexity" buildTypeId="bt61" />
    </graph>
  </custom-graphs>
  </settings>

Whenever the Metrics build is run, the graphs will be updated with the latest values. You will find them on the Statistics tab on the project overview page. It will look something like this after the first successful run:

Sample Metrics graphs

Final remarks

SourceMonitor only supports a few metrics, so if you want some more advanced metrics, like code cohesion, then go for NDepend.

In this post I haven’t explained how to display detailed reports from SourceMonitor in TeamCity. Maybe that will come in a later post.

Good luck with your metrics!

About these ads
Leave a comment

6 Comments

  1. Perry

     /  February 3, 2011

    Have you managed to get this working in TeamCity 6.0? I have gotten everything working except the custom statistic graphs. Thanks for the write up.

    Reply
  2. pwigle

     /  February 3, 2011

    We haven’t upgraded to 6.0 at work yet so I can’t tell. Let me know if you found a solution.

    Reply
  3. Perry

     /  February 4, 2011

    FYI you have to edit (TeamCity path)\.BuildServer\config\main-config in 6.0 instead of (TeamCity path)\.BuildServer\config\(project name)\plugin-settings.xml. The plugin-settings.xml file is still present in 6.0 but doesn’t appear to do anything anymore. After that all works as expected. Now I just have to edit my SourceMonitorCommands.xml so I get the results I am looking for. Thanks for the write up.

    Reply
  4. Perry

     /  February 8, 2011

    I also had to add

    to the MSBuild file because it was reporting the same values over and over again.

    Reply
  5. acarum

     /  April 8, 2011

    @Perry…what did you have to add to the MSBuild?

    Reply
  6. @Perry if I understood correctly, the graphics in teamcity will always show the same values, because sourcemonitor’s command file is ponting to only one checkpoint: “baseline”.
    This checkpoint is the history file of code metrics in a especific date.

    @Pwingle Is there any way to automate the process of updating the checkpoint name in the SourceMonitor command file?

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: