Code Metrics Statistics with TeamCity

Code metrics can be a very useful tool for monitoring some aspects of code quality. To get the most out of it you need to calculate the metrics on a regular basis in order to find trends. Code quality tend to go down during intense phases of a development project and also when a product is in low-intensity development, a.k.a the maintenance phase. The obvious thing to do for a modern developer is to integrate calculation of code metrics into the Continuous Integration process.

In this post I will demonstrate how you can calculate code metrics and display graphs of the evolution over time as part of your Continuous Integration process. There are a few tools available for calculating code metrics in .NET, the most capable is without doubt NDepend. Here I will use another tool, SourceMonitor, which is free of charge and very lightweight. We use Team City at work, so that is the CI server that I will use here also, but you could probably implement this idea in the CI server you are using.

One of the goals I had when I started experimenting with code metrics statistics was that I wanted it to be easy to add statistics to any project, without changing anything in the project itself. All changes should be in the Team City configuration.

Step 1: Create Metrics project in VCS

The first step is to create a project in your version control system. This project should contain the following artifacts:

  • The SourceMonitor executable
  • A SourceMonitor command file, SourceMonitorCommands.xml
  • A MSBuild script file, SourceMonitor.proj
  • MSBuild community tasks

I put all the files in a directory called SourceMonitor.

Step 2: Create a SourceMonitor command file

I will not go into the details of working with SourceMonitor. If you are interested, you can read the documentation that is included in the download. Here is the command file that I used:

<?xml version="1.0" encoding="utf-8"?>
<sourcemonitor_commands>
  <write_log>true</write_log>
  <command>
    <project_file>MyProject.smp</project_file>
    <checkpoint_name>Baseline</checkpoint_name>
    <project_language>C#</project_language>
    <source_directory>..</source_directory>
    <source_subdirectory_list>
      <exclude_subdirectories>true</exclude_subdirectories>
      <source_subtree>bin\</source_subtree>
      <source_subdirectory>obj\</source_subdirectory>
    </source_subdirectory_list>
    <parse_utf8_files>True</parse_utf8_files>
    <ignore_headers_footers>True</ignore_headers_footers>
    <export>
      <export_file>SourceMonitor-details.xml</export_file>
      <export_type>2</export_type>
    </export>
  </command>
  <command>
      <project_file>MyProject.smp</project_file>
    <checkpoint_name>Baseline</checkpoint_name>
    <export>
    <export_file>SourceMonitor-summary.xml</export_file>
    <export_type>1</export_type>
  </export>
  </command>
</sourcemonitor_commands>

This command file will create two xml-files: SourceMonitor-details.xml and SourceMonitor-summary.xml. It is from the latter that we will extract the values to publish to Team City.

Step 3: Create a build script

Here I have used MSBuild, but you can of course use NAnt, Rake or whatever. The build script will do the following:

  • Run SourceMonitor on your source files
  • Extract the interesting values from the resulting xml-file
  • Publish these values to Team City

The MSBuild community task XmlRead is used to extract the values from the xml-file. The Team City task TeamCityReportStatsValue is used to publish the values. The community tasks has to be explicitly imported, but the Team City tasks are imported automatically when the script is run by Team City. Here is my MSBuild script:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="3.5" DefaultTargets="Analyze"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
  <MSBuildCommunityTasksPath>.</MSBuildCommunityTasksPath>
</PropertyGroup>
  <Import Project="MSBuild.Community.Tasks.Targets"/>

  <Target Name="Analyze">
  <Exec Command="SourceMonitor.exe /C SourceMonitorCommands.xml"/>
    <XmlRead XPath="//*/metric[@id='M0']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="NumberOfLines" />
    </XmlRead>
    <TeamCityReportStatsValue Key="NumberOfLines" Value="$(NumberOfLines)" />

    <XmlRead XPath="//*/metric[@id='M5']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="MethodsPerClass" />
    </XmlRead>
    <TeamCityReportStatsValue Key="MethodsPerClass" Value="$(MethodsPerClass)" />

    <XmlRead XPath="//*/metric[@id='M7']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="StatementsPerMethod" />
    </XmlRead>
    <TeamCityReportStatsValue Key="StatementsPerMethod" Value="$(StatementsPerMethod)" />

    <XmlRead XPath="//*/metric[@id='M10']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="MaxComplexity" />
    </XmlRead>
    <TeamCityReportStatsValue Key="MaxComplexity" Value="$(MaxComplexity)" />

    <XmlRead XPath="//*/metric[@id='M14']" XmlFileName="SourceMonitor-summary.xml">
      <Output TaskParameter="Value" PropertyName="AvgComplexity" />
    </XmlRead>
    <TeamCityReportStatsValue Key="AvgComplexity" Value="$(AvgComplexity)" />
    </Target>
</Project>

The build script above will extract the following metrics and publish them:

  • Number of lines of code
  • Average number of methods per class
  • Average number of statements per method
  • Maximum cyclomatic complexity
  • Average cyclomatic complexity

Step 4: Create a build configuration in Team City

The next step is to create a new build configuration for the project that you want to analyze and display statistics for.

Attach your VCS project root and the SourceMonitor root as well. Your version control settings will look like the example below.

Version Control settings for Metrics

Version Control Settings in Team City

Configure TeamCity to use the MSBuild runner for your build and specify the path to the build script that will run SourceMonitor. Also specify the target, in my case “Analyze”.

Build runner configuration in Team City

Setup build triggering the way you like it, either as a dependent build or scheduled at some interval, for example every night.

Step 5: Configure Team City to display statistics

TeamCity has built in capablities to display statistics in graphs. You can easily add your own graphs to a project by adding configuration to the file plugin-settings.xml which is located in the folder: (TeamCity path)\.BuildServer\config\(project name). Note that you have to change the buildTypeId value to the Id of your Metrics build configuration. You can find the buildTypeId in the URL as a query parameter. Below is a sample plugin-settings file:

<?xml version="1.0" encoding="UTF-8"?>
<settings>
 <custom-graphs>
    <graph title="Number of Lines" defaultFilters="" hideFilters="showFailed">
      <valueType key="NumberOfLines" title="Number Of Lines" buildTypeId="bt61" />
    </graph>
    <graph title="Methods per Class" defaultFilters="" hideFilters="showFailed">
      <valueType key="MethodsPerClass" title="Methods per Class" buildTypeId="bt61" />
    </graph>
    <graph title="Statements per Method" defaultFilters="" hideFilters="showFailed">
      <valueType key="StatementsPerMethod" title="Statements per Method" buildTypeId="bt61" />
    </graph>
    <graph title="Complexity" defaultFilters="" hideFilters="showFailed">
      <valueType key="MaxComplexity" title="Max Complexity" buildTypeId="bt61" />
      <valueType key="AvgComplexity" title="Average Complexity" buildTypeId="bt61" />
    </graph>
  </custom-graphs>
  </settings>

Whenever the Metrics build is run, the graphs will be updated with the latest values. You will find them on the Statistics tab on the project overview page. It will look something like this after the first successful run:

Sample Metrics graphs

Final remarks

SourceMonitor only supports a few metrics, so if you want some more advanced metrics, like code cohesion, then go for NDepend.

In this post I haven’t explained how to display detailed reports from SourceMonitor in TeamCity. Maybe that will come in a later post.

Good luck with your metrics!

Deploying ASP.NET Web Applications using NAnt

We have started implementing Contiuous Integration at our company. We began with Cruise Control.NET but have decided to try with TeamCity instead. One thing that we would like to do is to deploy ASP.NET web applications as part of our Continuous Integration process. We use the Web Application Project template in Visual Studio. One problem that we faced was: How can we copy all the neccesary files to the web server? We didn’t want to specify the files explicitly nor did we want to copy all files with certain file name extensions. Those approaches seemed very error-prone.

The first approach we used with Cruise Control.NET was to use the MSBuild target _CopyWebApplication which is defined in Microsoft.WebApplication.targets. I was not very pleased with this solution, although it worked. My main concern that we didn’t have any control over the process. When we started with TeamCity it turned out that this approach was not working anymore. The _CopyWebApplication target copies all files needed for the web application to a directory called _PublishedWebSites\<web application project name>. For some reason (probably a good one), TeamCity mangles the name of the project file, with the effect that the name of the directory where the web application files were copied to had a different name than expected.

So I decided to try another approach. Since I like NAnt much more than MSBuild, I started to explore if it was possible to use NAnt for this task. The main problem was that the valuable information on which content files were part of the application was buried in the project file. To utilize this information I used an XSL transformation to transform parts of the project file (which is an MSBuild script) into a NAnt script.

We would like to select all the files where the Build Action property is set to Content. Those files are listed in the project file as a Content element, like in this example:

  <ItemGroup>
    <Content Include="Default.aspx" />

Hence they can be selected with the XPath expression: /Project/ItemGroup/Content.

What I wanted my XSLT script to do was to create a NAnt script with the following content:

  1. A fileset with all the content files included
  2. A target which copies the files in the fileset to some directory defined by a property.

This is what I came up with:

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet
    version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:msxsl="urn:schemas-microsoft-com:xslt"
    exclude-result-prefixes="msxsl"
    xmlns:s="http://schemas.microsoft.com/developer/
        msbuild/2003">
    <xsl:output method="xml" indent="yes"/>
    <xsl:template match="/">
        <project
            xmlns="http://nant.sf.net/release/0.85/nant.xsd"
            name="GeneratedFromMsbuild">
            <fileset id="content.files"
                basedir="${{project.root}}">
                <xsl:for-each select=
                    "/s:Project/s:ItemGroup/s:Content">
                    <include name="{@Include}"/>
                </xsl:for-each>
            </fileset>
            <target name="copy.content">
                <copy todir="${{destination.dir}}">
                    <fileset refid="content.files"/>
                </copy>
            </target>
        </project>
    </xsl:template>
</xsl:stylesheet>

The following excerpt from our main NAnt script shows how the XLS transformation is invoked and how the resulting script is used to copy the content files:

<target name="deploy.web" depends="generatenant">
    <nant buildfile="${generated.file}" target="copy.content">
        <properties>
            <property name="project.root"
                value="${web.projectrootdir}"/>
            <property name="destination.dir"
                value="${web.rootdir}"/>
        </properties>
    </nant>
   <copy
       todir="${web.rootdir}\bin">
       <fileset basedir="${web.projectrootdir}\bin">
            <include name ="*.*"/>
       </fileset>
   </copy>
</target>

<target name="generatenant">
    <delete file="${generated.file}" failonerror="false"/>
    <style style="GenerateNantFromMSBuild.xslt"
        in="${web.projectrootdir}/${web.projectfilename}"
        out="${generated.file}"/>
</target>

Closing remarks

Of course there are several other ways of accomplishing this, like using a Web Deployment Project, but I personally like having full control of what is going on and that can be achieved with NAnt.

Follow

Get every new post delivered to your Inbox.