DdkBuild

 

 


Introduction

A long, long time ago in a far away place I started building NT operating system components. Much to my horror I discovered that rather than a nice gui'd set of tools, the standard kernel build environment looked a bit like this:

Figure 1. I don't know, I'm sure it is all subjective, but the word "fugly" comes to mind. 

The standard NT kernel build environment reminds one of the ultra-sophisticated (and I suppose oh so retro-trendy these days, freakin penguin heads everywhere,) pre-X, pre-Emacs Unix development environment of vi and make

I really love that NT 'shell' (and I use the term loosely) and in fact the idea of groveling through the scrolled-away output of build, or those lovely build log files using notepad, searching for error messages, combined with whatever code editor floats your boat (as in you need three applications with no clue of what the others are doing,) just thrills me. I'm sure you feel the same way.

I have a few simple requirements when it comes to software development:

  • Editor/Compiler integration.
  • Source code browsing (as in Unix tags.)

That isn't a very big list, is it?

 In addition, I don't want my development environment to do anything that might even possibly compromise the actual output of the process. In other words, I want to be absolutely (or at least very highly) certain that whatever tools I use are producing valid results.

So faced with what to me was the unacceptable NT DDK way of doing things, I set out to develop my own tools that would integrate the VisualStudio IDE with the NT DDK, getting me most of what I find useful in the IDE without compromising at all the integrity of the build process. Thus was born ddkbuild.bat.

DdkBuild

DdkBuld has gone through three incarnations. It was born at Sequoia Systems, in the midst of a rather painful corporate collapse. It traveled from Sequoia to OSR with its creator, and there became the standard VIsualStudio/Build integration tool for OSR and, rumor has it, quite a few other places as well. The hardest part of using ddkbuild was in fact trying to download it from OSR's web site, a problem I never quite understood the genesis of, but resolved by departing from OSR, taking my little friend ddkbuild with me. (Oh a version is still available from OSR (www.osr.com,) but as far as I know it remains near-impossible to find.)

The latest incarnation of ddkbuild, the one described in detail in this article, has been changed to accommodate the evolution of the NT DDK into the Windows 2000 DDK (there are some differences.) And represents all the author ever really wants to know about writing NT command shell scripts. 

What is ddkbuild? It is an NT command shell script. A bat file. 

What does ddkbuild do? It does a lot of things, but mostly it does exactly what that crufty checked or free build environment ddk window does, that is it sets up the correct environment for executing build, and then in invokes build.

Huh? What's the point? Oh, yes well ddkbuild does things in such a way that it is really easy to set up a VisualStudio project that will, through the vehicle of ddkbuild, integrate with the NT build process.

Oh, fine but I'll just use that SrcToDsp thing from Numega. Ok, that's a really neat tool. It does have its limits though. I'm still looking for the DspToSrc utility :-) Remember that what I want is a process that uses the standard NT build tools and integrates with VisualStudio. Ddkbuild does not preclude using the DDK shell windows for building your projects, in fact it encourages it.

For those of you who are just looking for the latest version of DdkBuild.bat. Look no further.

Get On With It Already

Ok, so here is the step-by-step algorithm for moving an existing NT DDK project into the IDE with DdkBuild. We will also discuss very briefly how to create a new build project in the IDE.

But first a word from our sponsor: folks please remember that what drives the creation of a NT DDK Build based software project is the Sources file that describes the project components. The process I'm describing here works with the DDK build process it does not replace it. You want a Driver Wizard?  Try  http://www.bluewatersystems.com or http://www.uw.com/drivercentral/examples/demos.shtml for one of those puppies.

Let's take an existing W2K driver, in fact we will use the one pictured way up as the fugly figure 1, the acpi ec driver.

1. Start your engines

Fire up VisualStudio, and pull down the File/New PropertyPage. Select the Projects tab and under the list of project types select Makefile. Now you need to carefully  select a project name and location. Keep in mind that what VisualStudio would like to do is create a new directory for you and your new project. Also remember that the Project Name field and the Location field are deeply entwined with each other. What you type into Project Name is appended to whatever happens to be in Location. This is very frequently not what you want or expect! 

What we want is a project named ec located in the directory path G:\NtDDK\src\wdm\acpi\ec. So we type in or browse to G:\NTDDK\src\wdm\acpi in the Location field, and we type in ec in the Project Name field. Our Projects Tab in the New Property Page Dialog should look like this:

Figure 2. New Dialog (less fugly)

2. The Project Wizard

We are creating an external makefile project (which by the way is all documented in the help docs for VisualStudio,) so the next thing that happens is that we get confronted by the External Makefile Project Wizard.

Remember: wizards are your friends!

Clock OK on the Projects tab. Here comes the wizard, he/she/it looks like this:

Figure 3. Makefile Project Wizard Step 1 of 2

You need to make some changes to the defaults.

The Command Line field needs to be replaced with: ddkbuild -W2k checked .

Lets parse this change. 

ddkbuild    this is the shell script (which you may download from here) that will run build.exe for us in the appropriate environment. Installation of ddkbuild is covered later.

-W2K        this option, which must be the first option on the command line if it is used, indicates that this is a Windows 2000 project. Ddkbuild must do some special processing to get the Windows 2000 DDK build environment working properly. Once again there is more on this subject when we discuss installation of ddkbuild.

checked    you have to tell ddkbuld what type of driver you are building, a checked driver or a free driver. If -W2K is specified, then checked/free must be the second parameter, otherwise checked/free must be the first parameter.

.            huh? That's a dot, a period. Actually this parameter is the location of the target project. Very conveniently, VisualStudio sets the current working directory to the location of the active project. Consequently, dot (.) is the location of the project. 

The next field, Output, can be ignored. If this bothers you, you can put your name here, or the date, or what you think of your boss, or anything else that comes to mind.

The final field, Rebuild All Switch, can also be left as it is. It is however, not ignored. The contents of this field are fed to ddkbuild as the last command line parameter if you select rebuild all as an IDE action in VisualStudio. Ddkbuild understands /a and will do the right thing with respect to invoking build.

When you are finished modifying this wizard panel it should look like this one:

Figure 4. Step 1 of 2 - Ready To Go!

As soon as you click the Next button you will proceed to Makefile - Step 2 of 2.

This panel is identical to the previous one, except that it wants you to specify the settings for a release build of your project. All you need to do is change checked  to free. Otherwise the settings are exactly the same as panel one.

Punch the finish button and you are done with this step.

3. Filling in the project details

I do have to stress once again that what we have here is a manual process that integrates the DDK build process with the VisualStudio development environment. So after the foolish wizard is done what we have for a project is exactly nothing. Oddly enough it will build just fine (that is it will build just fine after you have installed ddkbuild.bat correctly, more on that later.)

However it would be nice to have access to the project files. So what we need to do is add the appropriate files to the project. To do this click on the FileView tab in the project workspace window. You should see something like this:

Figure 5. An Incomplete Project. (Semi-fugly.)

If you point the mouse cursor at the ec files line and right click on the mouse you will get a pop-up menu that includes an option to Add Files to Project... Select that option.

This will bring up the Insert Files into Project dialog. Make sure that Files of type is set to al files (*.*). Now select all the .c, .h .rc files, plus the Sources file and the Makefile, and the .inf file and the .htm file if they exist. Click on OK.  Your FileView should now look like this:

Figure 6. A Project Fulfilled.

For the most basic features of ddkbuild, this is all that needs to be done to create a VisualStudio project from an existing DDK project.

4. Installing Ddkbuild.bat

In order to actually test drive ddkbuild it has to be installed. The download is an NT command script, a bat file. It needs to be located in a location where VisualStudio will find it. I installed my copy in the VisualStudio\common\Tools directory, but you can install it anywhere that VisualStudio will find it.

Warning: the PlatformSdk and VisualStudio come with different and slightly incompatible linkers. You can build a driver correctly with either one, but you should also only use one of these linkers or the other. Set your paths appropriately.

Oh, there is this wee change in the Windows 2000 DDK.

You see prior to one of those W2K betas (rc1 or 2) ddkbuild use the environment variable BASEDIR to figure out where the DDK was installed. Note the singular nature of the environment variable. The W2K DDK does not believe that it is the one and only true DDK installed on your system. There is no BASEDIR environment variable as far as this DDK is concerned. or more specifically BASEDIR is constructed on the fly every time you open one of those fugly shells (see figure 1.) 

Small problem for ddkbuild. Resolved by the addition of the -W2K flag, and a little work by you. To build a W2k project you must set the -W2K flag in the command line that invokes ddkbuild.bat and you must create an environment variable named W2KBASE that locates the W2K DDK. My W2K DDK is located at G:\NTDDK so W2KBASE on my system is set to G:\NTDDK.

5. Test Drive

You should now be able to simply fire up a build from the VisualStudio IDE. The steps above will give you the ability to build a project within VisualStudio or outside VisualStudio using the standard fugly process (see figure 1.) If there are compilation errors, VisualStudio will take right to the chapter and verse of your source code that offends the compiler, just like any other VisualStudio project.

Not only that, but if you look at the ClassView tab you will discover that a lot of your data structures and functions are now browsable from the ClassView window:

Figure 7. Classview Doesn't Suck!

Now this isn't half shabby, but it isn't all there. Read on, oh searcher for a functional kernel development environment, for soon we will have browsing.

Advanced Topics- AKA enabling Browse.

Classview is ok but it doesn't do anything for data structures or functions that aren't defined in the current project. In fact, despite its pretty gui look, ClassView is pretty damn stupid as far as source code browsers are concerned. What I require (see requirements above) is something equivalent to Unix tags (or more exactly, what I had using Xemacs and etags.) I want to point at some source object and click on some widget and end up at the definition of that thing. Not only that I need to be able to do what Cscope did for me: show me all the references to an object, and show me the call tree for any function or the inverse call tree for any function. The VisualStudio Browse facility does all that. so all we need to do is make it work with build and our new friend ddkbuild.

No problemo.

The massively non-existent documentation for the DDK build process clearly spells out how to enable generation of browse (.bsc) files for a Sources based project. If the documentation existed it would say add the following lines to a Sources file:

#
# setup visualC++ source browsing
#
BROWSER_INFO=1
BROWSERFILE=$(TARGETNAME).bsc -n

The first line enables generation of .sbr files, the intermediate file generated for each .obj file to create a .bsc browse file.

The second line sets the name of the .bsc file to our project target name. We also sneak in the '-n' option to BscMake here so that Bscmake does NOT truncate .sbr files. This is just the opposite behavior from the default VisualStudio behavior, but unless you have a very slow processor and a very small hard drive I strongly suggest you follow what I'm doing up above. You will be much happier.

To go back to our example, the ec driver from the ddk, we open the Sources file and add the browse enabling lines. Our sources file now looks like this:

Figure 8. Now we have browsing!

Now we do a clean build of ec by selecting the menu option Build/Rebuild All.

The Build output window should indicate that good things have happened:

Figure 9. Rebuild the ec driver with browsing.

We need to do one more thing. Select the Project/Settings  menu option. Fill in the Browse info file name field with the name of the .bsc file created by build. In this case the file is named acpiec.bsc. You may need to build the project again to convince VisualStudio that this project is browsable.

Now you should be able to point at any data structure type or function or constant and right click on it and go to its definition. In addition, a CallGraph or CallersGraph is available for all functions as well as a reference list for all data structures and functions. Life is complete, all is bliss.

Advanced Topics - Project Management

If you add or remove source files you have to modify both the VisualStudio project and the Sources file. Consequently if you are going to create a new project, you have to remember to create a Sources file, a makefile, and anything else you might need. VisualStudio will not do these steps for you as it doesn't know anything at all about how to build a device driver or what build.exe is and what its requirements are.