Enable TFS 2018 Code search on multiple branches

TFS Code search is a great feature. Our developers love it; they can easily search for anything within the millions lines of code we have in our various repos.

However, by default, this feature works only on the repo’s default branch. This can be nice for a general-purpose search, but when working on multiple branches with considerable differences between them, it is not good enough.

It took a lot of digging to find out that you can add up to 5 additional branches to be included in the code search. Go to the project’s settings > Version Control > select the repo and go to the Options tab. there you will find this well-hidden option.

Adding branches should trigger auomtatic re-indexing of the repository. To monitor the status, or force re-indexing in case of need, see the Code Search Admin page.

Merge from master overrides a revert commit

First time I encounter a Git behavior which, on first look, seems to defy common sense.

Scenario:
1. create a feature branch off master
2. perform some changes and commit (in this example – add a new line to foo.c)
3. merge the changes to master using a pull request with squash
4. on the feature branch, revert the previous commit (in this example – remove the line from foo.c)
5. merge master to the feature branch

One would expect that the reverted content will remain or that at least a manual merge conflict will occur. But surprise – the merge overrides them with the content on master (the content before the revert).

So why it happens? because how three-way merge works when there is a single common ancestor.

In the diagram, you can see that C1 is the common ancestor of C3 and C4. When doing the merge, git compares C4 to C1 and C3 to C1.
Since C1 and C4 are identical, it causes git to “understand” that the feature branch did not change the file at all, so it should take the change introduced in C3 and apply it as the merge result.

How it differs from a regular (non squashed) merge?

In this case, the common ancestor changes. It is now C2. So git compares C2 and C3, find that they are identical, hence selecting the change between C2 and C4 as the “interesting” change to apply as the merge result.

 

Conclusion: if you merge with squash, do not use the same branch to introduce additional changes…

 

ClearCase Windows Quick Tip: Find all symbolic links

Our ClearCase repository make heavy use of ClearCase symbolic links. We began an initiative to get rid of them, since they cause lots of problems – from snapshot view size, to tools not supporting them (e.g. WebPack), and Git migration efforts. See also my previous post on this matter.

The first step, of course, is to find all of them. One may think that cleartool find command will provide this functionality, but it’s lackcing. So we came up with this simple command-line:

cleartool ls -l -r <folder> | findstr /C:"symbolic link"

Folder must be a VOB, or a folder inside a VOB. To iterate on all VOBs, you can run:

for /d %i in (<view root>\*) do @cleartool ls -l -r %i | findstr /C:"symbolic link"

Now, we just need to figure out what to do with all of them…

Git Virtual File System – Finally, a solution to large Git repos?

Microsoft is all about Git in the last couple of years, and a few days ago they announced an exciting new feature they are working on that should really take Git to the enterprise level.
The GVFS (Git Virtual File System) is a way to make Git “believe” that it is working with the entire repository clone, while in reality all files reside on the server, and are only downloaded the first time something tries to access them.

This means that even for very large repos, clone time is next to nothing, and commands like git status can take a few seconds instead of several minutes. The only payment is the time it takes to run a build, or to open a solution, for the first time.

Somehow I can’t shake the feeling that it was inspired by ClearCase MVFS (Multi-Version File System), which at the time was an excellent way of working against large repositories without spending hours of copying data to the local machine. Of course MVFS has a whole set of problems, but the basic idea of ‘download content only when you actually need it’ was innovative at the time, and apparently has its uses today as well.

I’ll keep a close watch on how GVFS is shaping up in the coming months. It can be a real life saver for companies like the one I work for, which are struggling with the idea of switching to Git, considering a 150 GB repository and over 200,000 files in a typical workspace.

CI with ClearCase Base: The Road Less Traveled

We spent the last year or so building and maintaining a Continuous Integration (CI) system for our major development group. And it was incredibly daunting and frustrating, considering we had to build it around a legacy and problematic version control system like ClearCase Base.

In the textbooks it seems pretty simple: every commit (check-in) to the repository triggers a build, which in turn checks out the latest version of the repository from source control, and run the build process. This way, if a build fail, you know exactly who broke the build – the person who performed the last commit – and if it the build passes, you have full traceability between the build artifacts and the source code they were built from.

Now, let’s try that with ClearCase Base.

Continue reading

Git in Visual Studio: Prevent Fast-Forward Merge

A useful feature of git merge is the –no-ff flag. What it does is always generate a merge commit, even if it was a fast-forward (trivial) merge.

The main point of using this feature is avoid losing information about the historical existence of a certain branch, and (to some degree) being able to associate commits with the branches they were created on.

Turns out, this feature existed in Team Explorer for quite some time, but you would never guessed it if you didn’t stumble upon this User Voice entry.

By unchecking the “commit changes after merging” option, you’ll prevent a fast-forward merge and have the ability supply a custom commit message.

Why not just call it ‘prevent fast-forward merge’? 🙂

 

Branch in Code (a.k.a. Config Flags)

Inspired by an article on CM Crossroads I stumbled upon lately, I’d like to give a high-level overview on this concept, commonly use in web site developments and can also be applied to cloud applications. But most people (like me) who work with ‘installable’ software never heard of it, and if they did, they regard it as an alien concept that just can’t work.

The short version is this:

  • Your software is a service. There is only a single deployment of that software, which means a single relevant codebase.
  • Hence there is no point of doing branching and merging in your source control. Everyone commits to the mainline
  • Instead of branching, developers wrap every piece of new code with some condition that can be controlled externally (e.g. a flag in a config file, user properties, server environment, etc.)

This way you can “switch versions in runtime”. Which lets you do things like:

  • Get rid of feature branches and merges – just deploy the “dark code” (partial/untested code) for a new feature ahead of time and, when the feature is ready, you can easily turn it on (and off if problems appear)
  • Roll out an update gradually and selectively
  • Rollback an update in a matter of seconds
  • Offer different UI experience to different users
  • Test new code on production servers (by conditioning based on e.g. user id or IP range)
  • And more…

Challenges:

  • The code can become somewhat “ugly”. Although many of those flags eventually get removed from the code once they don’t have a use in production.
  • ‘config file explosion’ – there are several different approaches regarding controlling the amount and lifespan of configuration flags
  • ‘mysterious flags’ – here too, there are some ideas about maintaining proper documentation as well as performing cleanup/refactoring as needed.

 

Monitoring ClearCase Performance

ClearCase is a complicated product, and when you encounter performance issues it’s quite difficult to figure out the reason.
IBM has some technotes and articles on the subject, but most of them involve running all sorts of commands and scripts and meticulously analyzing their output (see one example here).

A few months ago it came to my attention that ALMToolBox are working on a web-based, graphical monitoring tool which provides a simple yet powerful overview on system performance, both on the infrastructure level (CPU, I/O etc.) and on the application level (ClearCase objects and commands), and correlate them as seen below.

clearmon-performance

You can also watch a live demo on this page.

It’s still early in development but it looks promising, and I urge every ClearCase administrator to keep a close eye on this product.

Edit: Even better – it can be used to monitor virtually anything, not just ClearCase servers. In this page you can see a “vitality report” which also includes ClearQuest, JIRA and Bamboo servers.