Wednesday, October 8, 2014

Howto: Set up TeamCity to use TFS Proxy

TeamCity usage of TFS as a source provider is not exactly perfect, but setting up fetching data using TFS Proxy worked surprisingly well!

So, we have some TFS server and a configured TFS proxy with address, let's say http://tfsProxy:8081
and now we need to explain TeamCity that it should fetch files using it.

Configure VCS root pointing to TFS server as ususal, do not think about proxy yet.

And now - all you need to do is to set up an environment variable TFSProxy with the value of TFS Proxy address.
Then restart TeamCity agent - if you use agent side checkout. Or restart TeamCity server - if you use server side checkout.

NB! Sending TFSProxy as an environment variable to build will not work. It MUST be an environment variable on a machine which is fetching code.

Testing scenario:
Set up a build to fetch some data from TFS.

1. Clean up TFS Proxy cache by deleting all under: %TFS_Proxy_install_directory%\Version Control Proxy\_tfs_data\Proxy

2. Clean up TeamCity cache:

  • Server side checkout - delete all under %TeamCity Data Directory%\system\caches\sources
  • Agent side checkout - delete all under %localappdata%\Microsoft\Team Foundation\5.0\Cache
3. Clean up build directory from source files, if not empty.
4. Start build and see Cache folder getting populated on TFS Proxy machine :)

Hope that helps!

Wednesday, May 14, 2014

Git problem connecting to TFS on-premise from Bamboo agent

That was a weird case - thus even more interesting to solve.

We needed to connect our on-premise TFS instance with Git projects to build on Bamboo build environment (there will come a full howto-step-by-step post about it later).
And besides everything  was set up correct, build was still failing trying to fetch code from repository. Error looked like:

fatal: https://{buildUser}{Collection}/{TeamProject}/_git/{RepositoryName}/info/refs?service=git-upload-pack not found: did you run git update-server-info on the server?

More interesting - testing the same stuff against my own VisualStudio repository in Azure went smooth.

Narrowing the scope showed up that the command failing was git-ls-remote:

git ls-remote https://{buildUser}:*****{Collection}/{TeamProject}/_git/{RepositoryName}

The problem was (weird!) that the password for the build user contained ! at the end. After I changed the password - everything went fine. Even more - after I changed the password back it still goes fine...

For info: it is git version running at Amazon elastic cloud build instanse.

Wednesday, April 16, 2014

3 things to know to set up DC in Azure

I assume some have already configured domain controller on-premise. It is almost the same, when you do it in Azure, accept for few things. But those are important things.

1. Set a static IP for your domain controller
Actually your IP address stay persistent for 45 years if you do not stop your VM. But if you do stop\deallocate your VM - you might get another IP address after start. And that is (softly said) not so good for domain controller and DNS server.
So make sure to confiure a "static" IP for your DC, like this, using powershell:
(If my vm name is myTestDC and corresponding cloud service name is myTestDC)

$dcvm = Get-AzureVM -ServiceName myTestDC -Name myTestDC
Set-AzureStaticVNetIP -VM $dcvm -IPAddress | Update-AzureVM

2. Configure location of NTDS and SYSVOL to be on Data disk, not OS disk
Azure provides us with 2 types of disks - OS and Data. Where as OS disks are using caching by default, and Data - not. So make sure to attach an extra Data disk to your DC and configure your ntds and sysvoll to be placed there.

3. Modify you network DNS to include new DNS server
Existing vm-s will need restart to pickup new DNS settings.

x. And another, rather tiny but important thing :) Administrator username is not allowed to use in Azure, so - say my administrator name is superAdmin, password superPassword, and I created domain superDomain.Test with netbios name superTest.  Logging into my DC after restart is superTest\superAdmin with password superPassword.

PS: Howether, if you demote your DC - suddenly you will need to login with .\administrator and password superPassword. ...Assume it is a bug, not a feture.

Wednesday, January 22, 2014

Managing TFS Git branches in Visual Studio

About Branches
First of all, there are 2 general types of branches you can have - Unpublished(local) and Published.

 Published is accessible to all your team members and is also visible in Team Web Access (TWA).

So as you can see - my LocalBranch is not visible in list of branches from TWA. 

To be able to work with Published branch you need a local copy of it - local branch. So you have - totally local branches - yours only, and local branches which are local working copy of Published branches.

All local branches are accessible to check-out in Branches list:

Creating a Branch

To create a branch in Visual Studio go to Branches menu:

Select New Branch and provide name of the new branch and which branch it should be taken from.
Afrer pressing "Create Branch" the new branch will be created and should appear in Unpublished Branches list (if the original was Local) or in Published Branches (if the original was on server - has name like origin\something).

You can publish any unpublished branch by right-click -> "Publish Branch" or keep it local if you want to.

A bit more...

You can see yourself which branches are connected to which published version -
From Branches explorer in Visual Studio select Actions and "Open in File Explorer":

From there navigate to a hidden folder named .git and open for edit file config. This file has list of branches created and also information if the branch is published. That is how mine looks like:
[branch "PublishedBranch"]
    remote = origin
    merge = refs/heads/PublishedBranch
[branch "master"]
    remote = origin
    merge = refs/heads/master
[branch "LocalBranch"]
[branch "NewBranch"]
    remote = origin
    merge = refs/heads/master

Monday, December 2, 2013

Visual Studio and Git - handling deleted repository

I created a Git repository in TFS and then renamed it.
The problem was Visual Studio still insisted to clone with the old name! And in attempt to connect I recevied :
An error was raised by libgit2. Category = Net (Error).

Of course, the repository didn't exist any more. So I tried to refresh, restart and etc, to let Visual Studio figure out that repository has a new name. Nope, no way.

So I found where those properties are locally saved -
C:\Users\my.user\AppData\Roaming\Microsoft\VisualStudio\12.0\Team Explorer\GitTeamProjects.config

Inside it looks like:
<project name="Inmeta" projectUri="...">
      <repository name="OldRepositoryName" guid="896f85d7-b840-48ed-92d5-74b223b34f66" />


I just deleted the whole line staring with <repository... and restarted Visual Studio.
And voila - the new repository name was discovered and shown correctly to clone :)

Wednesday, November 27, 2013

TFS: Error deleting Team Project

Recently I needed to cleanup a collection, removing a number of team projects from it. Generally all went fine, but one team project refused to be deleted, saying something like that:

Error logs said not much:
Executing step: 'Delete the team project data from Version Control' VersionControl.DeleteTeamProject (2 of 11)
[Error] Violation of PRIMARY KEY constraint 'PK__#yourLoc__FEE84A3672E9B4C6'. Cannot insert duplicate key in object 'dbo.#yourLocalPendingChanges'. The duplicate key value is (51843).

What I understood from it, is that it failed to delete team project from the source control. Oh well, thought I, I can try to delete from command line. So I did and hit:
C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE>tf destroy "$/MyProject" /collection:http:\\localhost:8080\tfs\defaultcollection
TF246021: An error occurred while processing your request.
Technical information (for administrator):
SQL Server Error: 2627

At this moment things started looking very dark. I tried of course deleting again from Admin console, and of course with no luck.

But indeed I found the solution :)

I used the destroy command to drop things under source control one by one. Like :
C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE>tf destroy "$/MyProject/Subfolder1" /collection:http:\\localhost:8080\tfs\defaultcollection
C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE>tf destroy "$/MyProject/Subfolder2" /collection:http:\\localhost:8080\tfs\defaultcollection
At first to folders I hit the same error, but I kept going and then, after some folders got successfully deleted, all others, even those wich were giving error - also got deleted. 

So when all subfolders was deleted I ran again:
C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE>tf destroy "$/MyProject" /collection:http:\\localhost:8080\tfs\defaultcollection
And it all went good!
So, back to admin console and  delete project again - voila!

I am certain I am not the only one who hit the problem and desperately googled without any result :)
So hope this solution will work for others as well!

Wednesday, November 20, 2013

TFS to Git migration: step by step

It has been some posts about migrating to Git from TFS, unfortunately none of them worked for mas expected.

So, my task was: migrate a source from TFS to Git, preserve all history, but do not include dlls or other heavy files. Last part requires naturally not only removing those files from the latest source, but also removing it from the entire history.
On my way I met some hickups, which I will mention, so the road for others after me can be smooth :)

So, lets start!
You will use GitBash and Git-tf:

1. Install Git-TF. Follow this link and download latest release:
 Installation instruction is very easy, I chosen not to use Chocolatey, but changed PATH environment variables, worked perfectly.

2. Start GitBash. If you do not have it - can be downloaded from here

Note! All things from now are done from under GitBash.

Check if git-tf was installed successfully - just type "git tf", it should return usage description of git tf command.

Navigate to the place where you have you Git repositories. I use C:\Git\Sources\Repos folder. So my git promt looks like this now:

3. Clone my TFS source to some temporary Git repo.
git tf clone {collection url} {team project path} {temp repo name} --deep

like this:

Note --deep - that means I want to take history as well.

4. Create Git repository which you will use later as your Git source. So far I will not link it to TFS Git repository, just create it.
git init {repo name}
like this:

get into newly create repository by simply
cd {repo name}
you can see you are in a master branch of your newly created repository:

5. Being there, now pull the source from you temp TFS git to your future Git place:
git pull ../{temp repo name} --depth=100000000

teoretically we could push it to our TFS git already now, but first lets cleanup dlls or other unneeded files, so when we put it to TFS git it is all clean and lightweight.

In my example I had 2 installation files located in folder wix/wixDownloads. So I want to leave those behind.

git filter-branch --force --index-filter 'git rm -rf --cached --ignore-unmatch wix/wixdownloads'
Command works only from the root, so paths must be relative from there.
If I would need to delete only one file, I would type:
git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch wix/wixdounloads/wix35.msi' 
It is important it says was rewritten in the end, that means stuff was actually removed. Otherwise - you probably misspell something.

If you type wrong, sometimes you get error "cannot remove directory '..../.git-rewrite" : directory is not empty. Dont panic, just remove it manually.

6. We are almost there! Now just clean up so all deleted files are physically deleted:
git reflog expire --all
git  gc --aggressive --prune
Now you can compare size og original forlder with clone from TFS, and size of cleaned up folder.
If everything is correct, the second one is much smaller!

7. And finally - push up to your TFS git:

git remote add master {your  TFS Git repository address}
git push -u master --all
like this: 

8. Now, when I compare hisoty on my original TFS source to history on my new TFS git source, I see no wixDownloads folder, and even not a corresponding checkin (well, this is because checkin contained JUST this folder, and no other files).

To the left - my new Git repo history, to the right - my original TFS history.

Hope it saves somebodys time!

Update 1: there is a possibility to hit  error when pulling source to the temp repository, if full filenames on temp repository exceed 256 symbols:

 error: unable to create file ...{verylongfullpath}... (Filename too long)

ignoring the error gets us on with being unable to do filter-branch and error:
fatal: Neeeded a single revision

To get around that make sure that the folder name for temp repo is short enough. In my case I just deleted temp repository, created a new one with a very short name and did git pull again.