Parsing VS 2005 Logs with Powershell
Published 1.25.2007 by ~mattg
I’ve been testing this week, but I’m testing our conversion, which is a lot of “setup and wait” type testing. In the “wait” time I’ve been trying to automate builds on my development machine using Powershell. I have a small batch file that does that already, but I wanted to use Powershell to provide some parsing and automatic notification of build errors and events. Plus, remembering the command line seems to “sharpen the mind,” so to speak.
Visual Studio 2005 provides a way to log output when running from the command line (just use /out when calling devenv.exe). However, if you allow building of more than one project at a time, the build log lines are preceded with a “#>” marker, where # is the project “number” being built. If you build only one project at a time, those markers are not present. Since the build numbers aren’t linear, it makes parsing errors fun and exciting, because you have to keep track of what project goes with what number, and keep different counts for each projects warnings and errors.
With the aid of a script for constructing generic objects in powershell, I was able to write a script that will parse a Visual Studio 2005 build log into just errors and warnings. I’ve posted the code below for your reference and use. Don’t forget you’ll have to download the New-GenericObject script for my script to work.
## Usage:
##
## Invoke-ParseLog "C:\Path\To\VisualStudio.log" "C:\Path\To\Output.txt"
##
##
## Parses Visual Studio Build Logs (with C++ and C# mixed projects) into an Error/Warning only log
##
param(
[string]$strBuildLog, [string]$strOutputFile
)
begin
{
[void][System.Reflection.Assembly]::LoadWithPartialName(“System.Collections.Generic”)
$strContent = Get-Content $strBuildLog
$nTotalErrors = 0
$nTotalWarnings = 0
$strSingleProject = “”
$nProjectsBuilt = 0
$nProjectsFailed = 0
$dictCurrentProject = New-GenericObject System.Collections.Generic.Dictionary System.String, System.Int32
$dictErrorMap = New-GenericObject System.Collections.Generic.Dictionary System.Int32, System.Int32
$dictWarningMap = New-GenericObject System.Collections.Generic.Dictionary System.Int32, System.Int32
$queueCSProjects = New-GenericObject System.Collections.Generic.Queue System.String
$listErrProjects = New-GenericObject System.Collections.Generic.List System.String
$listWarnProjects = New-GenericObject System.Collections.Generic.List System.String
$bMult = ($strContent[0] -match “^\d+>”)
if ($bMult -ne $true)
{
$dictWarningMap.Add(0, 0)
$dictErrorMap.Add(0, 0)
}
ac $strOutputFile “Build Log Summary”
foreach ($line in $strContent)
{
if (($line -match “(?<bnum>\d+)>——.+\sProject:\s(?<project>[^,]+),\sConfiguration:\s(?<config>[^\-]*)——”) -or
($line -match “——.+\sProject:\s(?<project>[^,]+),\sConfiguration:\s(?<config>[^\-]*)——”))
{
$strCurrentProject = $matches[“project”]
if ($bMult)
{
$strConfig = $matches[“config”]
if (($strConfig -match “Debug Any CPU”) -or ($strConfig -match “Release Any CPU”))
{
$queueCSProjects.Enqueue($strCurrentProject)
}
$nBuildNumber = $matches[“bNum”]
$dictCurrentProject.Add($strCurrentProject, $nBuildNumber)
$dictWarningMap.Item($nBuildNumber) = 0
$dictErrorMap.Item($nBuildNumber) = 0
}
else
{
$strSingleProject = $strCurrentProject
$dictWarningMap.Item(0) = 0
$dictErrorMap.Item(0) = 0
}
}
# this is the build summary for C++ builds
if (($line -match “(?<bnum>\d+)>(?<project>\w+)\s*-\s*(?<errors>\d+)\serror\(s\),\s(?<warnings>\d+)\swarning\(s\)”) -or
($line -match “(?<project>\w+)\s*-\s*(?<errors>\d+)\serror\(s\),\s(?<warnings>\d+)\swarning\(s\)”))
{
$strPrjName = $matches[“project”]
$nPrjWarnings = $matches[“warnings”]
$nPrjErrors = $matches[“errors”]
if ($nPrjErrors -gt 0)
{
$listErrProjects.Add($strPrjName)
ac $strOutputFile “$strPrjName : Build Failed with $nPrjErrors errors and $nPrjWarnings warnings.`r`n“
}
elseif ($nPrjWarnings -gt 0)
{
$listWarnProjects.Add($strPrjName)
ac $strOutputFile “$strPrjName : Build succeeded with $nPrjWarnings warnings`r`n“
}
}
#This is the failed summary for C# projects
if (($line -match “(?<bnum>\d+)>Done\sbuilding\sproject\s`”(?<project>[^`"]+)`”\s–\sFAILED.”) -or
($line -match “Done\sbuilding\sproject\s`”(?</project><project>[^`"]+)`”\s–\sFAILED.”))
{
if ($bMult)
{
$strCurrentProject = $matches[“project”]
$strCurrentProject = $strCurrentProject.Substring(0, $strCurrentProject.LastIndexOf(“.”))
$nBuild = $matches[“bNum”]
}
else
{
$strCurrentProject = $strSingleProject
$nBuild = 0
}
$nPrjErrors = 0
$nPrjWarnings = 0
if ($dictErrorMap.ContainsKey($nBuild))
{
$nPrjErrors = $dictErrorMap.Item($nBuild)
}
if ($dictWarningMap.ContainsKey($nBuild))
{
$nPrjWarnings = $dictWarningMap.Item($nBuild)
}
if ($nPrjErrors -gt 0)
{
$listErrProjects.Add($strCurrentProject)
ac $strOutputFile “$strCurrentProject : Build Failed with $nPrjErrors errors and $nPrjWarnings warnings.`r`n“
}
elseif ($nPrjWarnings -gt 0)
{
$listWarnProjects.Add($strCurrentProject)
ac $strOutputFile “$strCurrentProject : Build succeeded with $nPrjWarnings warnings`r`n“
}
}
#This is the successful summary for C# projects
if ($line -match “Compile\scomplete\s–\s0\serrors,\s(?<warnings>\d+)\swarnings”)
{
if ($bMult)
{
$strCurrentProject = $queueCSProjects.Dequeue()
$nBuild = $dictCurrentProject[$strCurrentProject]
}
else
{
$strCurrentProject = $strSingleProject
$nBuild = 0
}
$nPrjErrors = $dictErrorMap[$nBuild]
$nPrjWarnings = $dictWarningMap[$nBuild]
if ($nPrjWarnings -gt 0)
{
$listWarnProjects.Add($strCurrentProject)
ac $strOutputFile “$nCurrentProjectNum`> $strCurrentProject : Build succeeded with $nCurrentWarnings warnings`r`n“
}
}
if (($line -match “.+ error .+”) -or ($line -match “error .+”))
{
$nTotalErrors++
$printLine = $line
if ($bMult)
{
if ($line -match “^(?<bnum>\d+)>.+”)
{
$nBuild = $matches[“bNum”]
if ($dictErrorMap.ContainsKey($nBuild))
{
$dictErrorMap.Item($nBuild)++
}
else
{
$dictErrorMap.Add($nBuild, 1)
}
$printLine = $printLine.Substring($nBuild.Length + 1)
}
}
else
{
$dictErrorMap[0]++
}
ac $strOutputFile “Error $nTotalErrors> $printLine”
}
if (($line -match “.+: warning .+”) -or ($line -match “warning .+”))
{
$nTotalWarnings++
if ($bMult)
{
if ($line -match “(^?</bnum><bnum>\d+)>.+”)
{
$nBuild = $matches[“bNum”]
if ($dictWarningMap.ContainsKey($nBuild))
{
$dictWarningMap.Item($nBuild)++
}
else
{
$dictWarningMap.Add($nBuild, 1)
}
}
}
else
{
$dictWarningMap[0]++
}
ac $strOutputFile “Warning $nTotalWarnings> $line”
}
if (($line -match “==========\sRebuild\sAll:\s(?<success>\d+)\ssucceeded,\s(?<fail>\d+)\sfailed,\s(?<skip>\d+)\sskipped.+==========”) -or
($line -match “==========\sBuild:\s(?<success>\d+)\ssucceeded,\s(?<fail>\d+)\sfailed,\s(?<utd>\d+)\sup-to-date,\s(?<skip>\d+)\sskipped.+==========”) -or
($line -match “==========\sBuild:\s(?<success>\d+)\ssucceeded\sor\sup-to-date,\s(?<fail>\d+)\sfailed,\s(?<skip>\d+)\sskipped.+==========”))
{
$nProjectsBuilt = [int]$matches[“success”] + [int]$matches[“skip”]
if ($matches[“utd”])
{
$nProjectsBuilt = $nProjectsBuilt + [int]$matches[“utd”]
}
$nProjectsFailed = $matches[“fail”]
}
}
ac $strOutputFile “$nTotalErrors errors, $nTotalWarnings warnings”
ac $strOutputFile “$nProjectsBuilt projects built/skipped/up-to/date, $nProjectsFailed failed.”
if ($nTotalErrors -gt 0)
{
ac $strOutputFile “Projects with errors:”
foreach($prj in $listErrProjects)
{
ac $strOutputFile “`t$prj”
}
}
if ($nTotalWarnings -gt 0)
{
ac $strOutputFile “Projects with warnings:”
foreach($prj in $listWarnProjects)
{
ac $strOutputFile $prj
}
}
}
Filed under Windows
Did you know The Switch statement as also a regex switch ?
I this can help clearing it a bit up by removing some if stements
PoSH> switch -regex (”FooBar”) {
>> “foo.*” {’foo found’}
>> “.*Bar” {’bar found’}
>> “Foobar” {’foobar found break’;break}
>> “f.*” {’f found (no foobar)’}
>> }
>>
foo found
bar found
foobar found break
Greetings /\/\o\/\/
I did not know that, but how do I access the match groups (which are typically stored in $matches) if I use the switch? It doesn’t seem to populate those groups in the case of using switch -regex.
Also in the Switch statement $matches should get filled
This works for me :
PS Function:\> switch -regex (”FooBar”) {
>> “F(oo)Bar” {$matches[0];$matches[1]}
>> }
>>
FooBar
oo
Greetings /\/\o\/\/
I’ll have to mess with it a bit more. I also have multiple matches that require the same code, and would need some kind of fall-through. For example, in C#:
{
case 1:
case 2:
DoSomething1and2();
break;
case 3:
DoSomething3();
break;
default:
DefaultTasks();
break;
}
i tried wrapping the switch conditions in {}, and it ran, but I didn’t not get the $matches variable populated. In other words, I tried this:
{“f(.*)” -or “foo(.*)”}
{
$matches
}
}
In that case, $matches is empty.
Hey, quick question. Im totally new.
If i download the New-GenericObject.ps1 and it in the same directory as MyScript.ps1. and then call a New-GenericObject from Myscript.ps1, i get a “The term ‘New-GenericObject’ is not recognized as a cmdlet, function, operable program, or script file. Verify the term and try again.”.
How do i import it?
N00b,
Well, there are several reasons why New-GenericObject won’t be found, but the most likely scenario is that the path it’s in isn’t included in your execution path (the PATH environmental variable)
Also, make sure that, if you ExecutionPolicy is set to Signed, you sign the New-GenericObject.ps1 file.