Thursday, March 5, 2015

Windows -> Unix convert line endings with Git

Recently we met a problem, that files we fetch from TFVC to Linux build agents fail the build.

Problem was good old line endings - all our files had CRLF, since they were checked in from Windows.
Problem never appeared before, previously people were using Perforce clients, which can fix line ending for the "host" operating system. With Team Explorer everywhere that was not a case anymore.
So long story short, I ended up with need to fix line ending for the set of files, located in the same directory. Massive convertion of file endings... I need a tool... Git! So I created a script which convert line endings for all files in a specific directory (and subdirectories) and checks in to TFVC.

Configure Git line endings to unix style :
git config --global core.autocrlf input

Go into folder which need to be converted:
cd MYFOLDER

Make Folder a git repo, checkin all files:
git init
git add *
git commit -m "make it a unix, baby..."

At this moment in repository files are with Unix line endings, but locally on disk - still unchanged. Let's fix that.

Delete all files and then undo change so it will fetch back from repository:
git rm *
git reset
git checkout *

Now all files have correct line endings, we need to checkin to TFVC. So, delete .git in  your folder:
rd .git /S /Q

Go up one level and checkout your folder and all under:
cd ..
tf checkout MYFOLDER /recursive

Checkin to TFS forcibely (it does not see the difference if only line endings changed, so we need to force):
tf checkin MYFOLDER /recursive /force /noprompt


Voila :)

What is extra nice,  Visual Studio is clever enougth to deal with Linux style line endings. So I still can build my solution with devenv.exe. 
And even msbuild had nothing against unix style line endings.

PS: some more reading on subject

PPS: Of course, if you want to checkin changes or new files to TFVC, you need to watch line endings. It is possibel to configure your TFVC client to be aware. See more information here:
or here:


Wednesday, February 18, 2015

TeamCity + TFS Git repository: unable to find valid certification path to requested target

Accessing TFS Git repository from TeamCity, for an on-premises TFS installation gives you the following error?

Failed for the root '"XXX Git" {instance id=225, parent internal id=48, parent id=XXX_YYY_Git, description: "https://tfsinstance/tfs/DefaultCollection/TeamProject/_git/GitRepository#refs/heads/master"}: List remote refs failed: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target 

Your corporate certificate is not known by TeamCity. This is how you fix the problem.

Save your corporate root certificate into CompanyRoot.cer file

For server side checkout: 
  1. Login to the TeamCity server machine
  2. Add this root certificate to keystore of your TeamCity 
%TeamCityInstallDir%\jre\bin\keytool.exe  -importcert -trustcacerts -file CompanyRoot.cer -alias ca_alias -keystore "%TeamCityInstallDir%\jre\lib\security\cacerts"

(replace %TeamCityInstallDir% with actual path to your TeamCity)

If you haven't ever changed it, password for the keystore is changeit :)

For agent side checkout:
  1. Install Git on agent machine to lets say C:\Programs\Git\
  2. Run on agent machine git config --global http.sslCAInfo D:\Programs\Git\bin\curl-ca-bundle.crt
  3. Edit curl-ca-bundle.crt manually, add your Company root certificate to the end of the file
Mor information can be found here: 


Monday, February 16, 2015

TeamCity and multiple VCS root - be aware

Recently we hit the problem, that in some cases when using multiple VCS roots in TeamCity build configuration, files from some roots were missing.

Well, of course we did "something" - we changed server-side checkout to agent-side checkout. And that dramatically changed how the checkout directory looks like after fetching files.

The catch is:

  • On server-side checkout, all VCS roots are processed separately by the server and then files are sent to the agent. 
  • On agent-side checkout VCS roots are processed one after one in the SAME agent checkout directory. 


Disaster scenario:
One of VCS roots (lets say RootX) is set up to map files into root og agent checkout folder.
Checkout changes from server-side to agent-side.
TeamCity sees that as clean checkout for all VCS roots.
Agent fetches root1 into Folder1, root2 into Folder2, RootX ... into root of agent checkout directory. And prior to fetch it does cleanup files, which in this case means - delete Folder1 and Folder2.

Result: 
Files from some roots are missing.

Solution: 
Do not map files to root of checkout directory, if you have more then one VCS roots.
Or use server-side checkout.

Tuesday, November 4, 2014

Scientific bugs definitions

Bohrbug is a "good, solid bug".

Mandelbug is a bug whose causes are so complex it defies repair, or makes its behavior appear chaotic or even non-deterministic.

Schrödinbug is a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

Heisenbug is a software bug that seems to disappear or alter its behavior when one attempts to study it.

Hindenbug is a bug with catastrophic behavior.

PS: originated from http://en.wikipedia.org/wiki/Heisenbug

Thursday, October 23, 2014

Howto: version your nuget pack

There are several ways to set a version on your nuget pack:

  1. using project file information
  2. hardcoded version number in your .nuspec file
  3. sending -version parameter to "nuget pack" command
  4. mixture of above

At a very simple scenario we do nuget pack MyProject.csproj and it all works - nuget pack gets the same version as an assembly produced by building the project. There is no need in .nuspec file whatsoever.
In a real life nuget pack consists of more then one dll, sometimes some context files, sometimes some dependencies. So - we need a .nuspec file. And we make one by running nuget spec MyProject.csproj

The default generated .nuspec file already contains $version$ parameter. Now we can modify that nuspec file as we want, add description, extra dlls. Here we can hardcode version number and just call nuget pack MyProject.nuspec or send version in as a parameter like this: nuget pack MyProject.nuspec -Version 1.0.0.0
Mmmkey, but I want it to pick version from the project...

In that case we just call nuget pack MyProject.csproj again! Having MyProject.nuspec file next to project file nuget will be able to locate it and use it. The $version$ parameter gets populated based on the assembly information of the MyProject. And the rest gets picked up from .nuspec.

Now let's say we need an X.X.X-beta nuget pack - and assembly version does not allow any "beta" syntax. So - we use AssemblyInformationalVersion attribute in your AssemblyInfo.cs file.

[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyInformationalVersion("1.0.0.0-beta")]
[assembly: AssemblyFileVersion("1.0.0.0")]

The assembly version will still be 1.0.0.0, but the nuget pack gets version 1.0.0.0-beta. Perfect!

Here is a very good link explaining what is the actual difference between those attributes.

PS: some auto-magic happens when we do nuget pack for project file, - like auto-generating dependencies, auto-add content files, etc. I will tell about it in my next post.

Happy nuget-ing!

Wednesday, October 22, 2014

Which nuget.config am I using?

Those are my nuget usage scenarios:

  • from command line
  • from Visual Studio package manager
  • as a post-build event in Visual Studio
  • with automatic package restore in Visual Studio
  • on TeamCity build server (this I consider to be a command line as well)

At some point it was vital to know - which nuget.config am I actually using?
The default location for nuget.config is C:\Users\{UserName}\AppData\Roaming\NuGet\NuGet.config

Nuget chains configuration files, if there are several. It loads default nuget.config, then searches down from drive's root to {CurrentFolder}, then adds config file specified with -ConfigFile switch. {CurrentFolder} is either the solution root directory or an invocation directory. Later located config files are "stronger" and can override settings from previously loaded.

Calling nuget from command line - loads default config and then loads all nuget.config files searching from the  drive's root and down to invocation directory. Remember - than later found than stronger. So than closer to invocation directory than better.

When working in Visual Studio package manager - placing nuget.config in the solution root directory is good enough to know that's the one being used. Basically, working in package manager is same as working from command line, having solution root directory as an invocation directory.

For using nuget.exe in a post-build event my solution is to specify location of nuget.config in a post-build event explicitely, using -ConfigFile switch. I can not know for sure from where my project will be built.
Example: nuget push $(ProjectDir)$(OutputPath)*.nupkg -ConfigFile $(SolutionDir).nuget\nuget.config

A bit different though when Visual Studio does automatic package restore. We definitely want to avoid adding packages to Source Control, and there is a nice config setting for that: 
<add key="disableSourceControlIntegration" value="true" />
But - that one will work only if you have your nuget.config file in .nuget subfolder at the Solution Root directory. 

Happy nuget-ing :)

PS: For testing I used VS 2013 Update3 and nuget 2.8
PPS: a detailed, yet not very transparent documentation can be found here.

Wednesday, October 8, 2014

Howto: Set up TeamCity to use TFS Proxy

TeamCity usage of TFS as a source provider is not exactly perfect, but setting up fetching data using TFS Proxy worked surprisingly well!

So, we have some TFS server and a configured TFS proxy with address, let's say http://tfsProxy:8081
and now we need to explain TeamCity that it should fetch files using it.

Configure VCS root pointing to TFS server as ususal, do not think about proxy yet.

And now - all you need to do is to set up an environment variable TFSProxy with the value of TFS Proxy address.
Then restart TeamCity agent - if you use agent side checkout. Or restart TeamCity server - if you use server side checkout.

NB! Sending TFSProxy as an environment variable to build will not work. It MUST be an environment variable on a machine which is fetching code.

Testing scenario:
Set up a build to fetch some data from TFS.

1. Clean up TFS Proxy cache by deleting all under: %TFS_Proxy_install_directory%\Version Control Proxy\_tfs_data\Proxy

2. Clean up TeamCity cache:



  • Server side checkout - delete all under %TeamCity Data Directory%\system\caches\sources
  • Agent side checkout - delete all under %localappdata%\Microsoft\Team Foundation\5.0\Cache
3. Clean up build directory from source files, if not empty.
4. Start build and see Cache folder getting populated on TFS Proxy machine :)

Hope that helps!