Windows -> Unix convert line endings with Git

Recently we met a problem, that files we fetch from TFVC to Linux build agents fail the build.

Problem was good old line endings - all our files had CRLF, since they were checked in from Windows.
Problem never appeared before, previously people were using Perforce clients, which can fix line ending for the "host" operating system. With Team Explorer everywhere that was not a case anymore.
So long story short, I ended up with need to fix line ending for the set of files, located in the same directory. Massive convertion of file endings... I need a tool... Git! So I created a script which convert line endings for all files in a specific directory (and subdirectories) and checks in to TFVC.

Configure Git line endings to unix style :

git config --global core.autocrlf input

Go into folder which need to be converted:

cd MYFOLDER

Make Folder a git repo, checkin all files:

git init

git add *

git commit -m "make it a unix, baby..."

At this moment in repository files are with Unix line endings, but locally on disk - still unchanged. Let's fix that.

Delete all files and then undo change so it will fetch back from repository:

git rm *

git reset

git checkout *

Now all files have correct line endings, we need to checkin to TFVC. So, delete .git in  your folder:

rd .git /S /Q

Go up one level and checkout your folder and all under:

cd ..

tf checkout MYFOLDER /recursive

Checkin to TFS forcibely (it does not see the difference if only line endings changed, so we need to force):

tf checkin MYFOLDER /recursive /force /noprompt

Voila :)

What is extra nice,  Visual Studio is clever enougth to deal with Linux style line endings. So I still can build my solution with devenv.exe. 

And even msbuild had nothing against unix style line endings.

PS: some more reading on subject

http://blog.codinghorror.com/the-great-newline-schism/

PPS: Of course, if you want to checkin changes or new files to TFVC, you need to watch line endings. It is possibel to configure your TFVC client to be aware. See more information here:

https://msdn.microsoft.com/en-us/library/gg413272(v=vs.100).aspx

or here:

http://blogs.msdn.com/b/tfsxp/archive/2010/04/26/extended-attributes-in-team-explorer-everywhere.aspx

TeamCity + TFS Git repository: unable to find valid certification path to requested target

Accessing TFS Git repository from TeamCity, for an on-premises TFS installation gives you the following error?

Failed for the root '"XXX Git" {instance id=225, parent internal id=48, parent id=XXX_YYY_Git, description: "https://tfsinstance/tfs/DefaultCollection/TeamProject/_git/GitRepository#refs/heads/master"}: List remote refs failed: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target 

Your corporate certificate is not known by TeamCity. This is how you fix the problem.

Save your corporate root certificate into CompanyRoot.cer file

For server side checkout: 

1. Login to the TeamCity server machine
2. Add this root certificate to keystore of your TeamCity 

%TeamCityInstallDir%\jre\bin\keytool.exe  -importcert -trustcacerts -file CompanyRoot.cer -alias ca_alias -keystore "%TeamCityInstallDir%\jre\lib\security\cacerts"

(replace %TeamCityInstallDir% with actual path to your TeamCity)

If you haven't ever changed it, password for the keystore is changeit :)

For agent side checkout:

1. Install Git on agent machine to lets say C:\Programs\Git\
2. Run on agent machine git config --global http.sslCAInfo D:\Programs\Git\bin\curl-ca-bundle.crt
3. Edit curl-ca-bundle.crt manually, add your Company root certificate to the end of the file

Mor information can be found here: 

http://blogs.msdn.com/b/phkelley/archive/2014/01/20/adding-a-corporate-or-self-signed-certificate-authority-to-git-exe-s-store.aspx

TeamCity and multiple VCS root - be aware

Recently we hit the problem, that in some cases when using multiple VCS roots in TeamCity build configuration, files from some roots were missing.

Well, of course we did "something" - we changed server-side checkout to agent-side checkout. And that dramatically changed how the checkout directory looks like after fetching files.

The catch is:

• On server-side checkout, all VCS roots are processed separately by the server and then files are sent to the agent. 
• On agent-side checkout VCS roots are processed one after one in the SAME agent checkout directory. 

Disaster scenario:
One of VCS roots (lets say RootX) is set up to map files into root og agent checkout folder.
Checkout changes from server-side to agent-side.
TeamCity sees that as clean checkout for all VCS roots.
Agent fetches root1 into Folder1, root2 into Folder2, RootX ... into root of agent checkout directory. And prior to fetch it does cleanup files, which in this case means - delete Folder1 and Folder2.

Result: 
Files from some roots are missing.

Solution: 
Do not map files to root of checkout directory, if you have more then one VCS roots.
Or use server-side checkout.