So, occasionally as a web developer you might be called upon to do some weird things. I don’t mean dress up as a marmoset for some strange European film studio, but more in the vein of working with binary files on OS X and trying to copy them to a different machine as part of your pseudo web app. Sometimes web development is just full of the crazy awesome in that way.

This also serves as a bit of friendly help from past!me to future!me – I hope this solves my problem again!

There are a two different sorts of programs on OS X. The most common type a normal user will encounter is an application – big binary file, copied into the Application folder. Well, it’s not actually a file, it’s really a package. And by package I mean a special combination of directories and files. Pick an application you like the look of, take it out for dinner and show, and when you get home, bring up some menu options, and then select ‘Show package contents’. This will open up a directory, which contains a standard directory structure that OS X recognises as an Application it can run.

The other type is a Linux style program which is interacted with via the terminal. This pretty much how all the Linuxy stuff in OS X works. And, if you decide that you can’t be bothered to compile everything from source, you can use a packaging system such as Brew or MacPorts to install new programs too.

As a simple example, on the build machine we have installed the graphviz package, a tiny program for turning descriptions of graphs into images. In this demo scenario, we want to copy one of the executables, dot, to the deployment machine without rebuilding – in the real world this would be your very important application.

Obviously to do this you will need 2 different machines running the same version of OS X – I’m running a fresh copy inside a VM because my Mac Mini is busy doing dutiful service as a TV.

So, to follow along at home, you’ll probably want the graphviz package installed. To do that, you’ll probably need to get Homebrew or MacPorts installed. And they’ll need the Xcode command line tools, which you’ll definitely need.

So, first install that. Then, install the graphviz package.

Go ahead, install it, I’ll just be drinking coffee & watching TV while you wait.

Time passes

Right, now I’m back from watching TV, and you’re done installing, I suppose I should get to the meat of this steak-and-no-veg posting.

So, as a test – let’s copy the file across and see what happens.

$ ./dot
dyld: Library not loaded: /usr/local/Cellar/graphviz/2.32.0/lib/libgvc.6.dylib Referenced from: /Volumes/VMware Shared Folders/otool/dot Reason: image not found Trace/BPT trap: 5

So, it didn’t work. If it did, I wouldn’t be writing this thing, now would I?So, the file has dependencies. To search an executable for dependencies, we use a command line program called otool.

otool has a variety of options, print libraries, print data sections, get cirrhosis, but we want to call otool with ‘-L’ to look at any of the shared libraries used.

$ otool -L ./dot
 ./dot: /usr/local/Cellar/graphviz/2.32.0/lib/libgvc.6.dylib (compatibility version 7.0.0, current version 7.0.0)
 /usr/local/Cellar/graphviz/2.32.0/lib/libxdot.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /usr/local/Cellar/graphviz/2.32.0/lib/libcgraph.6.dylib (compatibility version 7.0.0, current version 7.0.0)
 /usr/local/Cellar/graphviz/2.32.0/lib/libpathplan.4.dylib (compatibility version 5.0.0, current version 5.0.0)
 /usr/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 7.2.0) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)
 /usr/local/Cellar/graphviz/2.32.0/lib/libcdt.5.dylib (compatibility version 6.0.0, current version 6.0.0)

So, these are the libraries that dot needs around to be able to run: 5 that were installed into brew’s Cellar and 3 that come with the system.

One, horrible way, of fixing this is to copy the dependencies into the exact same location on the deployment machine. Doing that will put you into the running for the award for “Worst Sysadmin of the Year”. If you’re going to do that, you might as well install it properly. A better way is to change dot to look for its libraries in a different place.

To do that, we use a friend of O’Tool’s with a triple-barrelled-name – install_name_tool.

This does a lot of weird relinking stuff, so we’re going to keep it simple for now, and just change the path for the libraries to be where our little dot binary already is.

 $ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libgvc.6.dylib @executable_path/libgvc.6.dylib dot

That’s basically saying, change the /usr/local path setting to now be the currently executable path, and do it inside the dot binary.

Running our friend otool again gives us this changed line:

 @executable_path/libgvc.6.dylib (compatibility version 7.0.0, current version 7.0.0)

So, rinse-and-repeat on the other libraries from the Cellar.

$ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libxdot.4.dylib @executable_path/libxdot.4.dylib dot $ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libcgraph.6.dylib @executable_path/libcgraph.6.dylib dot $ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libpathplan.4.dylib @executable_path/libpathplan.4.dylib dot $ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libcdt.5.dylib @executable_path/libcdt.5.dylib dot 

Now when we run Mr O’Tool again we get:

$ otool -L ./dot
 dot: @executable_path/libgvc.6.dylib (compatibility version 7.0.0, current version 7.0.0) @executable_path/libxdot.4.dylib (compatibility version 5.0.0, current version 5.0.0) @executable_path/libcgraph.6.dylib (compatibility version 7.0.0, current version 7.0.0) @executable_path/libpathplan.4.dylib (compatibility version 5.0.0, current version 5.0.0) /usr/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 7.2.0) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0) @executable_path/libcdt.5.dylib (compatibility version 6.0.0, current version 6.0.0) 

Great – all the libraries are reassigned, so now dot will work, right?

$ ./dot
 dyld: Library not loaded: /usr/local/Cellar/graphviz/2.32.0/lib/libxdot.4.dylib Referenced from: /Users/mlion/Documents/otool/attempt1/./libgvc.6.dylib Reason: image not found Trace/BPT trap: 5

Ah, but libraries reference other libraries, and so on. So, you need to fix these references in the other libraries too. And so on, until you have all of them, their friends, and anyone they’ve ever met.

I’ll fast forward through the run, relink, reload, retry cycle and skip to the good bits.

$ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libxdot.4.dylib @executable_path/libxdot.4.dylib libgvc.6.dylib
$ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libcdt.5.dylib @executable_path/libcdt.5.dylib libgvc.6.dylib
$ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libcgraph.6.dylib @executable_path/libcgraph.6.dylib libgvc.6.dylib
$ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libpathplan.4.dylib @executable_path/libpathplan.4.dylib libgvc.6.dylib

$ install_name_tool -change /usr/local/Cellar/graphviz/2.32.0/lib/libcdt.5.dylib @executable_path/libcdt.5.dylib libcgraph.6.dylib

 

$ ./dot There is no layout engine support for "dot" Perhaps "dot -c" needs to be run (with installer's privileges) to register the plugins?

Huzzah! That looks like a successful command! Well, one that need configuring – but it’s actually executing!

Everyone has the newest and greatest OS X, as decried by the ghost of Steve Jobs, right?

Not so fast – many organisations are still stuck on older versions of OS X. So, so when you copy binaries across, you need to make sure they’ll be backwards compatible. The way to do that, is to either make sure your binaries are explicitly built for older version, or do this process on an older machine. If you try this and it works, great – they’re backwards compatible. Obviously, if not, you’ll just have to find an older machine to do this on.

Finally,  given how simple input-output this is, it is not that hard to knock this up in a shell script in whatever flavour of language you prefer. The steps are pretty easy, run otool to find any linked libraries – filter out the ones you know are going to be there in the new machine, then pass on this data to install_name_tool to loop over. Simples. In fact, in a previous incarnation, I wrote an awful bash script, that I turned into a slightly less cryptic Ruby script, and now I’ve started writing an even less cryptic Ruby command line utility.