How to Get Dependencies from /showIncludes
Our GNU make whitepaper mentions that, if you’re using Visual C with GNU make, you can use the /showIncludes
compiler option to help generate .d
files, much like you can with the -MD
option to gcc. I thought I’d post a Python code snippet to illustrate.
One thing that’s important to remember is that, as the Visual C documentation points out, /showIncludes
prints its results to stderr
and not stdout
. This means that you cannot simply redirect stdout
to a file and then parse it. You generally don’t want to redirect stderr
to a file, because this will hide other error messages. Also, there’s no sense in creating a bunch of temporary files that you’ll just have to delete later.
Fortunately, the Python subprocess
module gives us extensive control over redirection of stdin
, stdout
, and stderr
, so we can simply write a wrapper script around the compiler that takes care of everything. We’ll create a script named cl.py
that wraps cl.exe
. It will automatically add the /showIncludes
option and merge the stdout
and stderr
from cl.exe
into a single pipe that we can parse for /showIncludes
information.
cl.py
‘s usage is very simple. Normally, you might run the command cl /c foo.c
to create foo.obj
. Now you run python cl.py /c foo.c
to create both foo.obj
and foo.d
.
Without further ado, here’s the script. Enjoy!
# cl.exe wrapper to create .d file from /showIncludes # Python 2.5 or later required from __future__ import with_statement import sys, os, subprocess # Determine path for .obj and .d files # This is probably not 100% robust; you may need to tweak it depending on how # you are invoking cl.exe. cmdline = sys.argv[1:] source = None target = './' # default is current directory for arg in cmdline: if arg.startswith('/') or arg.startswith('-'): # A compiler flag if arg[1:3] == 'Fo': target = arg[3:].replace('\\', '/') else: # This must be the source file name (assume there is only one) source = arg if target.endswith('/'): # Default object file name is source file with extension changed to .obj target = target + source[0:source.rfind('.')] + '.obj' if target.startswith('./'): target = target[2:] # chop off unnecessary ./ prefix # Name of .d file is name of object file with extension changed to .d depend_file = target[0:target.rfind('.')] + '.d' # Build cl.exe command line with /showIncludes # Assumption: cl.exe is in the user's PATH cmdline = ['cl.exe', '/showIncludes'] + cmdline # Launch the cl.exe process with stdout/stderr merged into a single pipe p = subprocess.Popen(cmdline, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) # Parse the cl.exe output to build up a set of dependencies deps = set() for line in p.stdout: if line.startswith('Note: including file:'): dep = line[21:].strip() dep = os.path.normpath(dep) dep = os.path.normcase(dep) dep = dep.replace('\\', '/') # use forward slashes for path separators dep = dep.replace(' ', '\ ') # escape spaces in paths with a backslash deps.add(dep) else: sys.stdout.write(line) # Wait for cl.exe to exit, then return an error if it failed ret = p.wait() if ret != 0: sys.exit(1) # Write the .d file, one dependency per line with open(depend_file, 'wt') as f: for dep in sorted(deps): print >>f, '%s: %s' % (target, dep)
One warning: spawning an extra Python process on each source file can hurt performance. Ideally, something along the lines of the above would be built directly into GNU make, but I digress.