IBM ClearCase MVFS Dynamic View Linux Patch
This is about getting ClearCase (a source configuration management system) dynamic views to work with a Unix client and a windows host. The documentation says you can't, maybe they should have looked into the issue a little first.

snapshots vs dynamic views
ClearCase supports snapshot views and dynamic views. The snapshot views are what every other source control system I've ever used does, it gives you the requested file in a local directory and it's a normal file. Dynamic views are a virtual file system that you mount that provide the same set of files that a snapshot view provides, only it is like it updates the file before every open, so you have the latest version. It also lets you add @@/main/2 to the end of a file to get the specified file version from any program that you can give a file name to.

There is a table 'Table 1. Rational ClearCase VOB access across platforms of different types' at the bottom of the following web page About interoperation between Windows computers and UNIX workstations, that can be summarized as follows. Every combination of ClearCase client or server running windows or Unix works with both dynamic views and snapshot views, except if the client is Unix and the server is windows, dynamic views don't work. It's right in the documentation and I was told that, so fine, we have windows servers, with a Linux client, I'll just use snapshot views.


Installing ClearCase, and getting it to work (skip if you have it working).
Work has been pushing IBM ClearCase for source control. After using CVS for years, including work projects, I got another taste of ClearCase. Some of us have Linux as our main desktop computer. First, they said ClearCase doesn't support Linux, then when another ClearCase admin said it does I got the next set of excuses, ClearCase won't work behind a firewall, for ClearCase to work the computers had to be on the Windows active directory. At the time the Linux computers were on a local private network with a firewall between then and the corporate network. Now, for reasons I kind of understand but aren't logical, the windows desktop computers are on the same private network behind the same firewall, and they work, so that excuse is out. As to the active directory, I thought I just needed kerberose to work, except ClearCase doesn't use kerberose, so I don't know what the active directory mention was about. Without any more excuses, or more likely figuring I would try and give up, they gave me the install files to ClearCase on Linux, along with a warning that it would only work on Red Hat Enterprise Linux and not the Fedora Core 3 I had installed. Whatever, the installer script was perl, and I'm a programmer, I got it to install.

After the learning curve on how to make a Unix VOB (where ClearCase stores its data) tag /vobs/IOS_SDT because the existing windows VOB tag \IOS_SDT wouldn't work because it doesn't start with a Unix slash, and I had to have the right group setup, I did get snapshot views to work. You can't register the same VOB multiple times in one region. You can register the same VOB in multiple regions, so we have one region for Windows and another for Unix with the appropriate path separators.

While the system is running Fedora Core 3, I've upgraded the kernel to 2.6.24.4, and ClearCase doesn't support so new of a kernel, so no MVFS or dynamic views. To not go back to the older kernel and loose suspend to disk I did the rest of the playing around under a supported Linux kernel in qemu. I did get the mvfs to compile under but it paniced when I went to use it. If anyone else has it working please let me know.


But why don't dynamic views work Unix to windows?
You can almost get dynamic views to work by the following. Mount the windows SMB or CIFS share in a local directory that matches the Unix VOB tag with the normal smbmount or mount command. Use the cleartool mount command to let the mvfs know it is available. Create a dynamic view, start the dynamic view, go to the right directory in that view and VOB, and ... ls, directory listing! All find a dandy until you open a file, which fails.

At some point in trying to get things to work was I using tcpdump to dump the communcation between the client and server, and I took another look. When a file is requested the response includes a file and a set of directories that start with c\1\2\3\4\4321.1234 I don't remember the exact sequence, but something like that, strange. Go to the CIFS mount point and if you substitute / for \ the file exists, and the contents is the file you requested. Which I thought was odd, because every other configuration management system I know if stores file differences not complete files. I guess when a dynamic view client requests a file it generates the file, tells the client the name, and lets the normal network based file system handle the details like seeking and reading the file as a program requests it.

Do you suppose a big company like IBM with their "enterprise" source control software, could possibly just need the path separators switched over and it would just work? It couldn't be that easy, otherwise they wouldn't spend so much time documenting and educating their users that it doesn't work.

So, a couple hours of digging in their Linux kernel module source code and I find the function that looks like it returns the file I saw in the tcpdump, add a printk, compile, load, read file, yup that's the string. Add a for loop to switch \ to /, compile, load, and it works! One loop and that was it. What I don't understand is why they didn't just return a forward slash from windows computers in the first place, most all the windows APIs will accept them, and it clearly accepts the forward slashes from Unix servers. It should have not been an issue. I'll give IBM this much, their code is GPL'ed, so here's the patch, enjoy. Now if I didn't have the source code, I couldn't fix it. If it wasn't Free Software, I couldn't put it up on my home page, who knows, maybe it will even make its way back to IBM.

patch for ClearCase version 7.0.1,
mvfs_clnt.c.diff
03-01-2011, patch modified for ClearCase version 2003.06 by Alred Jim,
mvfs_linux_mvops.c.diff

With IBM not only Linux friendly, but pushing it, "IBM, Canonical/Ubuntu, Novell, Red Hat to Deliver Microsoft-Free Desktops Worldwide", will they accept external patches? Time will tell.


Submitted 08-08-2008, id 973, IBM Rational Request For Enhancement
http://www.ibm.com/developerworks/support/rational/rfe/execute?use_case=viewChangeRequest&CR_ID=973
Headline: [PATCH] Linux MVFS dynamic view, windows interoperability
Submitter's raning of priority: CP3 - Lack of the RFE functionality is a minor road block to deployment/adoption
Product: Rational ClearCase
Component: MVFS
Operating System: Linux

Description
While the documentation lists dynamic views on windows will work with VOBs hosted on Unix, the opposite is documented to not work, and in fact it doesn't. To reproduce, mount the windows VOB share with CIFS on the local Linux system, create a VOB tag with that mount point for the global path, use cleartool to mount the VOB tag, create and start a dynamic view with that VOB, and try to read files in the view. It gives an error. Listing directories works, checking out, checking in, and editing checked out files works, but files that aren't checked out don't.

The problem is very basic. When MVFS in the kernel requests a file from the windows based ClearCase host, the returned file string looks like the following with dos backslashes for directory separators.
c\cdft\11\19\042e52ae895c4e66ae12725f9e13a7d5
and with the back slashes it isn't found, it just needs to be changed to the following with forward slashes for Unix directory separators.
c/cdft/11/19/042e52ae895c4e66ae12725f9e13a7d5
Preferably this would happen on windows before the string was returned (after all dynamic views on windows using VOBs hosted on Unix work, and presumably Unix is returning forward slash path separators). Being done on windows would allow the fix on one platform rather than each Unix platform. As the MVFS on Linux has the GPL license and that was all the source code I had available I fixed it there and dynamic views now work.

The following patch fixes the bug that dynamic views on Linux can't access VOBs that are hosted on windows.
ClearCase version 7.0.1 (Wed May 30 17:04:58 EDT 2007)
7.0.1.1-RATL-RCC-IFIX02 (Tue May 13 14:43:13 EDT 2008)

Index: mvfs_clnt.c
===================================================================
RCS file: /var/adm/rational/clearcase/mvfs/cvs_root/mvfs_src/mvfs_clnt.c,v
retrieving revision 1.1
diff -u -p -r1.1 mvfs_clnt.c
--- mvfs_clnt.c	2 Jul 2008 18:09:25 -0000	1.1
+++ mvfs_clnt.c	2 Jul 2008 22:26:40 -0000
@@ -1980,7 +1980,15 @@ CRED_T *cred;
          * into a minimum size buffer, so we can free up the maxpathlen
          * buffer allocated at this level.
          */
+	int i;
 	mnp->mn_vob.cleartext.isvob = rrp->vob;
+	//printk("mfs_clnt_cltxt_locked: original rrp->text %s\n", rrp->text);
+	for(i=0; i<rrp->text_size && rrp->text[i]; ++i)
+	{
+		if(rrp->text[i]=='\\')
+			rrp->text[i]='/';
+	}
+	//printk("mfs_clnt_cltxt_locked: modified rrp->text %s\n", rrp->text);
 	error = mfs_set_cpname(mnp, rrp->text, rrp->text_size);
     } else {
 	MFS_CHK_STALE(error, vp);

Use case: This feature would be used when an organization has Unix or Linux clients and wish to access a dynamic view where the VOB is hosted on Windows.

Business Justification (private): We have an existing windows VOB server for a piece of software that has been windows only, but now is being ported to Linux. Switching to a Linux VOB server isn't an option at this time, and not having dynamic views on Linux is frustrating development without the above patch.


IBM response, November 23, 2008 "Uncommitted Candidate"
I got an e-mail today (June 19, 2009) asking about the patch, so I looked up the request for enhancement. The good news is it hasn't been rejected, but it's in a favorable limbo. At least it passed the initial and followup review.

Status: Uncommitted Candidate
August 20, 2008 This RFE is consistent with our strategy and product roadmap and IBM is continuing to evaluate.
Most recent IBM developer update: 23 Nov 2008 This RFE is consistent with our strategy and product roadmap.

Uncommitted Candidate: This request will not be delivered within the next year, but the theme is aligned with our two year strategy. IBM is soliciting feedback for this request, and within one year from submission, status will be updated to either Rejected or Planned for Future Release.

You don't need to wait on IBM to get dynamic views to work on a Linux client from a Windows VOB. The MVFS Linux kernel module source comes with clearcase, apply the patch, recompile, insert the module, and have at. A couple notes on how to use it. As outlined above the vob tag must start with a Unix style slash and must have a matching directory on the system (which should be empty). The remote vob must be mounted locally so that the VOB_NAME.vbs matches the vob global path entry, such as /mnt/remote_server/VOB_NAME.vbs, and it can't be in the vob tag directory. Then cleartool mount will then be able to activate the VOB and it can be used normally.


RFE Voted +1 by a non-IBM developer

Date: 21 Sep 2009 03:53 AM Eastern Time (ET)
Comments: Have rebuilt mvfs.ko from 7.0.1.6 on RHEL 5.3 and works fine.

Good, a vote of success on the IBM community page, maybe a little bit of activity will bring some more attention to this RFE and get it fixed.


Update April 7, 2010, and wikipedia
The status is still "This RFE is consistent with our strategy and product roadmap." Looking at their top twenty most voted for it takes ten or eleven votes to make it. I see it now has 4 votes and is on 3 user watchlists, at least it is getting some attention.

On a side note I posted on the ClearCase talk page on wikipedia asking if anyone had a problem with me putting a reference to this site hilighting this issue. After a month I edited the main page, that was August 2009, and the paragraph has been unchanged since. Here's the link and paragraph.

http://en.wikipedia.org/wiki/ClearCase
Unix/Windows Interoperability: VOBs hosted on *nix (Solaris, Linux, AIX, HP-UX, IRIX primarily) servers can be accessed with dynamic views, snapshot views, or the new web protocol based client: the CCRC on Windows clients. VOBs hosted on Windows servers can be accessed with snapshot views or CCRC from Unix clients, but not dynamic views due to the Windows server returning file paths with backslashes as the path delimiter. There is a 3rd party patch [9] for Linux to enable dynamic views from Windows VOBs.

9. ^ http://david.fries.net/thoughts/IBM_ClearCase_mvfs_patch.php


Helpful clearcase commands

Here are some commands for getting a dynamic views on Linux from Windows VOB. For snapshot or dynamic views to work the VOB tag must use Unix / path separators and not DOS \ separators. If the VOB tag already has the Unix style path separators you might still need to remove and recreate the VOB tag if the global path is incorrect. If you don't have an existing Unix VOB tag keep in mind that you'll have to create the tag in a region different from the existing Windows VOB tag, the same VOB can't have multiple tags in the same region. Once you have a VOB tag make sure you can create and use a snapshot view. If you can't, figure out why and fix it. Unless someone tells me otherwise, a dynamic view requires everything that a snapshot view requires to work and more. If the snapshot view doesn't work you might as well forget about the dynamic until you get the snapshot view working.

  • mount the VOB (read-write is required for derived objects, which we weren't using, so I mounted it read-only), I have an entry in /etc/fstab to mount it on a local directory, do a search on mounting CIFS if you don't already know how.
  • $ mkdir /vobs/TEST_VOB /mnt/TEST_SHARE
  • $ chown my_user /vobs/TEST_VOB /mnt/TEST_SHARE
  • $ mount /mnt/TEST_SHARE
  • /mnt/TEST_SHARE must be manually mounted after each boot if it isn't automatically mounted. Look for TEST_VOB.vbs in /mnt/TEST_SHARE, that is the VOB directory, and what is used later for making the tag and the global path, in this case it is /mnt/TEST_SHARE/TEST_VOB.vbs
  • Create the VOB tag giving the Windows VOB hostname and local path path to the VOB files just mounted.
  • $ cleartool mktag -vob -tag /vobs/TEST_VOB -region REGION_unix -host win_vob_host -gpath /mnt/TEST_SHARE/TEST_VOB.vbs /mnt/TEST_SHARE/TEST_VOB.vbs
  • Do a ClearCase mount of that VOB tag. This will most likely be a manual step after each reboot.
  • $ cleartool mount /vobs/TEST_VOB
  • Make a dynamic view, strip_cr will probably help in a mixed environment.
  • $ cleartool mkview -tag dfries_SDT_Linux_dyn -tco 'Test in Linux dynamic' -tmode strip_cr /home/my_user/view_storage/dfries_SDT_Linux_dyn.vws
  • Start it to make it usable.
  • $ cleartool startview dfries_SDT_Linux_dyn
  • This is where the view files reside and were you go to edit files in that project..
  • $ ls /view/dfries_SDT_Linux_dyn/vobs/TEST_VOB

For reference this is the `cleartool lsvob -long /vobs/TEST_VOB` entry.
Tag: /vobs/TEST_VOB
   Global path: /mnt/TEST_SHARE/TEST_VOB.vbs
   Server host: win_vob_host
   Access: public
   Mount options:
   Region: REGION_unix
   Active: NO
   Vob tag replica uuid: ...
Vob on host: win_vob_host
Vob server access path: I:\TEST_SHARE\TEST_VOB.vbs
Vob family uuid: ...
Vob replica uuid: ...

If you are working on multiple clearcase projects with multiple default group users you'll want to learn about the newgrp command. Some commands like mkelem require the project group to match your primary group to execute. You can be a member of multiple groups at one time, but only one can be the primary group. The command `newgrp group` will make that group be your primary group if you are already a member of that group.


Update February 23, 2011
The status is still "This RFE is consistent with our strategy and product roadmap." It gained another vote to 5, still 3 comments, and 3 user watchlists. It's been two and a half years from the initial submission, so much for their two year strategy comment.

Switching to Git

My group at work has been increasingly fed up with IBM Rational ClearCase, so much that we've been replacing ClearCase with Git, git-scm.com the distributed configuration management program. I've been using Git for a while (about three years) and it fits how I like to develop, it's fast, branches are fast and lightweight, when you're working on a set of changes you can go back and edit a previous revision which can be very useful at times. In the transition one of the guys was doing a huge checkin and merge to two branches of 4,000 some files. In ClearCase after a few failed merges he did a clearfsimport which was taking 3 hours a piece, so that's minimum 6 hours + failed attempts. Git took 8 minutes. If we weren't going to Git anyway I would have told him to go back and do the merge because just doing clearfsimport will make all those files be evil twins, and that's a mess I wouldn't want to sort out in the future, but it was going to Git anyway, so whatever works I suppose.


Written by David Fries <david@fries.net>

Valid HTML 4.0! Valid HTML 4.0!
My pgp key is available and the fingerprint follows. See
http://www.gnupg.org/ for details.
pub 1024D/CB1EE8F0 2001-08-21 David D. Fries <david@fries.net>
Key fingerprint = 7079 F7EA D7EA 8E93 5B84 1900 008F 39D9 CB1E E8F0
sub 1024g/D9B8B029 2001-08-21