10. Utilities

This section describes how to use ADB's utilities, as well as a large number of "GNU" (Free Software Foundation) utilities.

None of these were written specifically for ADB; their whole purpose is to enhance the DOS command line, and/or to make DOS batch files more effective.

You'll get a lot more out of the examples in this section if you've already read the DOS Primer.

Feel free to mail your comments to me at: rog@NOSPAM_rs-freeware.org.

Return to the global table of contents
Frames mode, or No frames


1:  Introduction to ADB's Utilities ... and more about the "GNU" utilities.

2:  GNU Utilities: Cat, Cp, Ed, FEcho, FMkDir, FSFGrep, FTouch, Ls, Mv, Rm, Tr

3:  My utilities: BE2, Create0, Delim2CQ.Bat, DOSDate, ErrMsg, Excel.Awk, FDateCmp, FileBrk, FileIncl (AKA FI), GetFld.Bat, GetVal, Sleep, StripEOF, StrRepl, and View:

4:  My utility DBCvt: The commas-n'-quotes <--> fixed length converter.

5:  My utility FileMux: the friend of programmers . . . and files with multiple personalities!

6:  My utility FileStat: The Paranoid Programmer's Pal

7:  My utilities OSQLRF1 and OSQLRF2: extracting ODBC DBMS data and converting it to commas-n'-quotes

8:  My utility Repl: The file/string replicator


1:  Introduction to ADB's Utilities ... and more about the "GNU" utilities.

(To repeat:) You'll get the most out of this section if you already know how to code DOS batch files (or have read the DOS Primer.

WARNING:
There are actually two different types of utilities provided with ADB.  The first category are utilities that I wrote myself.  The second group are "GNU" utilities, i.e. those obtained from the Free Software Foundation's CD of utilties for DOS and Windows.  There are some important differences between the two types of utilities.

Why are my utilities so different from "GNU" utilities?  The answer is that the GNU utilities were "ported" from Unix, so the conventions are different.  Think of the difference between British and American English: you wouldn't say "parking lot" to a Brit: instead, you'd make reference to the "car park."


Let's start with my utilities:

  • Running instructions for my utilities

    Just enter their names from the DOS command line (with no command-line arguments).

  • How to enter command-line arguments for my utilities

    The command-line arguments are case-INsensitive for my utilities.

    If a value is required, DON'T put an extra space between the parameter name and the value.  (I.e. don't use Unix conventions.)

  • File name requirements for my utilities

    All of my utilities require DOS compatible "8 and 3" file names (and that includes path information).

    In effect, that means that your file (and folder names) should consist exclusively of no-more-than eight letters (or digits, or underscores, and possibly dashes), plus a dot followed by an extension of no more than three letters (or digits, or underscores, and possibly dashes).

  • Documentation for my utilities

    You'll find it right here, in this section  (or, in the case of ADB itself, in the other sections).

  • What's execluded in the list of non-GNU utilities' documentation

    RPSort (Robert Pirko's excellent ASC text file sort) isn't written by me, but it follows the same general conventions, and is not a GNU product.

  • Source code for my utilities

    This is available in ADB_C.Zip.  It's all free code - do whatever you like with it.


Let's compare the GNU utilities to my utilities:

  • Running instructions for GNU utilities

    Just enter their names from the DOS command line, followed by a space, and "--help".

  • How to enter command-line arguments for GNU utilities

    The command-line arguments are case-sensitive for GNU utilities.

    If a value is required, DO put an extra space between the parameter name and the value.  (I.e. use Unix conventions.)

  • File name requirements for GNU utililities

    If you want to use "long" (i.e. non-DOS) file names for the GNU utilities, you're most definitely on your own.  My experience suggests that most work just fine with Windows 9x, but seem to be rather cranky when run on more advanced MS operating systems, such as NT, or 2K (particularly NT).  This is particularly true for CP, MV, and RM.

  • Documentation for GNU utilities

    Each GNU utility is documented in the file GNUHelp.Doc

  • What's execluded in the list of GNU utilities' documentation

    GAwk (see the Awk Primer) has its own documentation, contained in GAwk.Doc.

  • Source code for the GNU utilities

    This is available from the Free Software Foundations' "GNU Software for MS Windows and MS DOS" CD at: http://www.gnu.org/order/order.html

    (Note: the above URL will take you to the main ordering page for GNU.  As of this writing, the following one will take you directly to the CD-ROM that I just mentioned: http://www.gnu.org/order/windows.html

    If you're desperate for GNU source code, you can also send me e-mail at: rog@NOSPAM_rs-freeware.org, and I'll try to send you the source for any modules you specify.  After all, GNU stuff is freeware too!  :)

Return to local table of contents
Return to global table of contents
Frames mode, or No frames

2:  GNU Utilities: Cat, Cp, Ed, FEcho, FMkDir, FSFGrep, FTouch, Ls, Mv, Rm, Tr

Remember, each of these utilities is documented in the file GNUHelp.Doc ... you can also receive help by entering the name, followed by a space, and --help, e.g.


Cat --help

Also note that all of these may be problematic with "long" file names, particularly on NT, 2K, and XP.

Finally: they all provide a return code, which in turn lets you use the DOS If ErrorLevel ... construct.


Cat lets you concatenate a series of files, and/or the "standard input" to the standard output.

For example, the following lines will take the contents of the file in1.txt and in2.txt to the file result.txt:


Cat in1.txt in2.txt >result.txt

Why would you want to use Cat instead of DOS's built-in Copy command?  For one thing, Cat will return a return code of 0 if all went well, or nonzero if it didn't.  (See the DOS primer for more information about return codes.)  This can be important, if you're worried about cases in which a copy may not succeed  (such as a full disk, or a file that's being "held" by another process.)

Cat also doesn't need the "/b" command-line argument, in order to copy binary files.


Cp is a more powerful version of DOS copy/XCopy.


ad is the command-line editor that's used in every version of Unix: it's very primative, but it also allows you to code a series of commands in a text file.  As far as I know, this is the must powerful command-line interface for automatically editing files from the DOS prompt.

Here's a batch file that you can run from the DOS prompt, which illustrates both the power of Ed as well as the issues associated with two other GNU utilities (FEcho, and Rm):


@Rem Delete both temporary files
@Rm -f Ed_Test.Dat Ed_Cmd.Dat
@Rem Build the Ed command file ... note that it ends with a "wq"
@Echo 1,$s/whatever/whatever2/g> Ed_Cmd.Dat
@Echo 1,$s/Whatever/Whatever2/g>> Ed_Cmd.Dat
@Echo wq>> Ed_Cmd.Dat
@Rem Build the test file that will be modified
@Echo Whatever happens is whatever happens> Ed_Test.Dat
@Echo whatever happens is Whatever happens>> Ed_Test.Dat
@Ed Ed_Test.Dat < Ed_Cmd.Dat
@If ErrorLevel 1 Echo There was an error
@Rem Display the results, using FEcho
@FEcho Here\'s Ed_Test.Dat, after applying the Ed commands:
@Type Ed_Test.Dat
@Rem Delete both temporary files using GNU Rm
@Rm -f Ed_Test.Dat Ed_Cmd.Dat

This batch file will build two files.  Ed_Test.Dat consists of these two lines:


Whatever happens is whatever happens
whatever happens is Whatever happens

Ed_Cmd.Dat consists of these three lines:


1,$s/whatever/whatever2/g
1,$s/Whatever/Whatever2/g
wq

The first line says that all occurances of "whatever" should be changed to "whatever2".  The second line says that all ocurances of "Whatever"  (note the capitalization) should be changed to "Whatever2".

The last line says to write the file, and exit.

Note also that Rm is invoked to delete more than one file at a time  (you can't do this with DOS Del, unless you use "wildcard" file names).

Finally, note that FEcho, when used with a single quotation (AKA "apostrophe"), requires a leading backslash.

Ed, and the other Unix utilities, are sometimes tricky to use, especially when "special" characters are involved.  But they all provide you with a lot more power than you might expect ... and Ed, in particular, is much more capable and flexible than you might initially imagine.

Caveat: you could implement the above example with my utility StrRepl which is documented below, but there are many other things that you can do with Ed that can't be done with StrRepl.

As I said before, the GNU utilities are all documented in GNUHelp.Doc, or you can enter the utility name at the DOS prompt, followed by a space, and --help


FEcho is the generic GNU utility which does what DOS Echo does, although even more of it.  For one thing, FEcho issues a return code!  For another, it will let you embed new lines, and other special characters.  Unlike DOS Echo, you don't have to have a new line at the end of each line: FEcho can be quite useful for building log files.

Note that the original GNU executable was called "echo" ... but the renaming was required, so as not to interfere with the DOS's "built-in" of the same name.


FMkDir will not "complain" if the folder already exists  (you have to use the -p command-line argument), and will build folders "recursively" ... DOS's "mkdir"  (AKA MD)  is inferior in all of these ways.

Note that the original GNU executable was called "mkdir" ... but the renaming was required, so as not to interfere with the DOS's "built-in" of the same name.


FSFGrep lets you find file that have certain strings in them ... but it's much more powerful than DOS's "find" command.

It's particularly helpful for cases in which you want to autmatically find/change a particular value across many file names.

Note that the original GNU executable was called "grep" ... but the renaming was required, so as not to interfere with other versions of grep.


FTouch resets the update date on a given file.

This is an obscure utilitiy, but you may find it useful when combined with my utility FDateCmp.

Users of my JavaScript MakeFile System  may also find it especially helpful.

Note that the original GNU utility was called "touch", but I renamed it in order to avoid conflicts with other programs of the same name.


Ls lets you list files in a directory, but it's much more powerful than DOS's built-in "dir" command.

I'm not sure how well it works with "long" file names, on all Windows operating systems  (i.e. you may have trouble on 2K or XP).


Mv is better than the DOS equivalent (move), only because of its ability to make backups.  You may encounter trouble with "long" file names on NT, or 2K.


Rm is more powerful than the DOS equivalent for two reasons: first, it can removed (AKA delete), multiple files.

Second, it can delete folders: although Microsoft suppored deltree on Windows 9x, it didn't do so on NT, 2K, or XP.


Tr ("translate characters") is a very obscure Unix utility, but one that DOS users may welcome: it's very advanced and specialized capabilities help you manipulate "unprintable" (AKA "hexadecimal") characters.

Return to local table of contents
Return to global table of contents
Frames mode, or No frames

3:  My utilities: BE2, Create0, Delim2CQ.Bat, DOSDate, ErrMsg, Excel.Awk, FDateCmp, FileBrk, FileIncl (AKA FI), GetFld.Bat, GetVal, Sleep, StripEOF, StrRepl, and View:

(These are all very straightforward.)

BE2 is my substitute for the old Norton Utilties [B]atch [E]hancer.  Basically it lets the user enter a key and sets the return value.  You can use it to create option menus in batch files (run it for an example).

You can also use the DOS utility Choice.Com for this purpose: Choice will let you supply a "timeout" default (i.e. a key that is to be assumed if the user does nothing), however unlike BE2, it's possible for the user to hit control-c (or control-break) to exit.


Create0 takes one argument which is a file name.  It creates a file of length 0 (if one doesn't already exist).  Create0 is particularly useful for building "marker" files that take up no space on a hard drive.


Delim2CQ.Bat uses Repl and StrRepl to take a file that has a particular delimiter convention (e.g. the bar character, "|" that is often used in Perl), and convert it to commas-n'-quotes.

For example, to convert infile.dat to outfile.dat, where infile.dat has the bar character as the delimiter:


Delim2CQ InFile.Dat OutFile.Dat "|"

You don't need double quotes around the delimiter if it doesn't contain any special characters, although I recommend using them anyway, for safety.


DOSDate simply writes this string:


Set DOSDATE=YYMMDD

to standard output, where YYMMDD is the obvious (the 2-digit year/month/day).

You can use it to create a batch file that can be CALLed.  E.g.:


DOSDate > Date.Bat
CALL Date.Bat
PkZip %DOSDATE%.Zip ...

Now the zip file has the current date in its name.  You can then copy it to a backup folder, or even ask the user to insert a floppy disk.  This is a very simple and inexpensive way to secure a set of files that goes through periodic (e.g. weekly or monthly) updates.


ErrMsg is a very simple program that writes all of its command line arguments to standard output, and then prompts the user to enter a space.  It will return 1.  For example:


If ErrorLevel 1 ErrMsg *ERROR* This run FAILED!!
If ErrorLevel 1 Goto FAIL_LABEL

ErrMsg also instructs the user to get help, if the word "error" appears in the message.

ErrMsg has a second mode, which it enters if the first command line argument is "-E".

In this mode, it returns 0, and does nothing more than print all command line arguments (after the first) to both standard output and standard error.

This mode is useful for broadcasting a message to the user who's running a batch file . . . while simultaneously writing a message to a log file.  For example:


ErrMsg -E Step 2 is beginning . . . >>ADBLog.%ADBPROJ%


Excel.Awk is a little Awk script that I wrote which lets you convert Excel files into commas-n'-quotes format (after you've pasted them into a Windows Wordpad window, and saved the results as MS-DOS text.

Excel.Awk will then convert it to commas-n'-quotes format for you.  See Excel.Awk for more detailed running instructions.


FDateCmp is a program that lets you compare the update date of one file against one or more other files.  With FDateCmp, you can tell whether a particular updating process is required.

The syntax is very simple:


FDateCmp [-Q] Main_File Other_File1 ... Other_FileN

FDateCmp will return 0 if the Main_File exists, and is newer (i.e., more recent) than any of the others.

It will return 1 if the Main_File doesn't exist, or is older (i.e. less recent) than any of the others.

It will return 2 if any of the "Other" files doesn't exist.

It will return 3 if you don't specify at least 2 files.

If you code -Q as the first command-line parm, it will not write to the standard output, i.e. it will be "quiet."


FileBrk lets you split up enormous files into chunks.  You specify the file name prefix of the new files, and the chunk size (in bytes).  It will then read your input file and split it up.

FileBrk will also do the reverse.

FileBrk is perfect for situations in which (say) a log file has become enormous, or you have a file that has to be moved across the 'net which is too big to go "all at once".  Or perhaps you want to move a huge file from one computer to another, but it's too big to fit on a floppy or a Zip-100 disk, or whatever.

(Note: in all these situations, you should use an archiver to zip the file up first.  Then break up the archive with FileBrk, and put the chunks back together on the other system.  This isn't always possible, of course.)


FileIncl is a "file inclusion preprocessor" for text files (this is sometimes referred to as FI in other documentation).

Essentially what it does is to take a text file which has special strings inside it that reference other text files, and then it "includes" those other files in the final result.

Here's a quick example.  Suppose the file Root.Txt looks like this:


Blah
#include:Stuff.Txt
Blah blah blah

Suppose that Stuff.Txt looks like this:


Blah blah

Now we execute:


FileIncl -S#include: < Root.Txt > Expanded.Txt

Expanded.Txt now looks like this:


Blah
Blah blah
Blah blah blah

Note that FileIncl can process as many "levels of inclusion" as you wish; however it doesn't check for "circular" inclusions (i.e. cases in which a file is included in a file that indirectly includes itself.)

FileIncl is extremely useful if you need to maintain complex "structured" sets of ADB definitions files, and/or large Awk library files.  You can also use it for myriad other purposes (such as preprocessing HTML).

FileIncl will also let you process "variable" lines.  If a line begins with a percent-sign (it must be in the first column), FileIncl will then check to see whether there's an equals sign, followed by a single character, and a percent sign.

For example, it will look for something like this:


%VARNAME=X%This is the rest of the line

If it finds that a line begins with a percent sign, and the remainder of the line contains at least one character following the initial percent sign, followed by an equals sign, one more character, and a second percent sign ... then it will then check to see whether VARNAME is a defined (DOS) variable, with the value of "X" (only single-character values will be recognized).

If VARNAME doesn't exist, or it isn't equal to "X", then the rest of the line will be ignored.  Otherwise, the rest of the line (after the second percent sign) will be included.  Both VARNAME and the value will be converted to upper case, prior to any comparison.

This method is a primitive substitute for "conditional inclusion."

Note: in the documentation for my "JavaScript MakeFile" system, available at http://rs-freeware.org/jsm, the very same program is called FI.Exe.  I apologize for any resulting confusion.


GetFld.Bat is a simple batch file that extracts one fixed length field from a record which is (presumably) fixed length.  The paramters are the input file, the output file, the field offset (measured from 0), and length.  You can think of it as a "column extractor."  It's roughly analogous to the rarely-supported Unix utility called "cut."


GetVal is my generic solution to the problem of multiple programming languages, although it can also be very useful for a system that's stitched together with DOS batch files, and written in one programming language.

Let's suppose you have a VBS file that defines a value, like so:


Const adMovePrevious = &H00000200

Now, what you want to do is to be able to extract that value for any number of purposes: perhaps you want to use it in a DOS batch file, or you want to pass it to a program written in another language.

The problem is that there may be many other variables defined in this file, and it may have remarks that refer to that variable, etc.

Let's say that what we can to do is to extract this value as a DOS variable.  We don't want the "&H" part, since this is VB's way of indicated a hexadecimal value.

Finally, let's assume that the VBS file is called adovbs.vbs, and we want to create a variable assignment in a DOS batch file called Temp.Bat.

The commands we execute are:


GetVal -q "-cConst adMovePrevious = &" -bH "-p@Set VAR="  < adovbs.vbs > Temp.Bat
Call Temp.Bat
Del Temp.Bat

The -c operand is what preceeds the definition of the variable value that we wish to extract.  The -p operand is what preceeds the output.  As with all my freeware utilities, you can learn more about this program by typing its name from the DOS prompt, and pressing enter.

After this, the file Temp.Bat has this code:


@Set VAR=00000200

So, we've successfully transferred the value from a *.VBS file into the DOS environment.  If we need to convert it into a variable value that can be read by some other programming language interface (such as C/C++, JScript, SQL, etc.), then we'll have to do some more work.

The bottom line is that GetVal lets you migrate information from a "global variables file" (or a particular programming language) into the DOS environment, and/or another programming language.  

GetVal is best thought of as of as a kind of "transit lounge" in a busy international airport ... such as Dubai or Heathrow: "passengers" (i.e. variable values) from all sorts of origins (i.e. programming languages) are "migrated" to any number of destinations (i.e. other programming languages).

To really learn to use this powerful tool, you should type GetVal from the DOS prompt, and press enter.  Try a few experiments: you'll soon see just how powerful it is.


Sleep just sleeps for a certain amount of time and "wakes up". The default is 60 seconds.  You put the number of seconds on the command line.

You can use Sleep to help create a DOS batch file that runs "forever" on a server which is (say) collecting data from the web via a CGI process.  (In Unix terms, this is known as a "daemon").

For example:


:WAKE_UP
@Rem Process collected data every 10 min.s
@Sleep 600
@Goto WAKE_UP


StripEOF removes the Control-Z (decimal character 26) from DOS files.  Many DOS editors automatically put these at the end of text files.  (That includes Breeze, the program I've supplied in ADB's miscellaneous zip file as a possible choice for a text editor.  DEdit, the other alternative, won't do this.)

Here's an execution example:


StripEOF MyFile.Txt


StrRepl allows you to do string replacement in a whole bunch of files at once.  It takes two files as its command line parms: a "change list" file which has your "old" and "new" strings (up to 1,000 of length up to 150 bytes each).

The second file is a list of file names.  You can even put remarks in each file, as well as any control character that can be recognized in an ASC file (basically everything except the carriage return, the line feed, and control-Z).

If you preceed the second file name by a slash, then StrRepl presumes that you have just one file to change, and that's the file's name.

So, in effect, StrRepl operates in two modes:


STRREPL Change_List_File File_List_File
STRREPL Change_List_File /File_Name

StrRepl doesn't do anything fancy (e.g. there's no support for "regular expressions" or changes that occur across lines).  But it handles a lot of quick jobs fairly well.

StrRepl excells at is in allowing you to use DOS Echo to create files that contain characters that DOS doesn't normally permit (such as less-than and greater-than signs).  For example, if you're generating HTML in a DOS batch file, you can use substitutes (e.g. the left curly brace instead of the less-than, similarly for the right).  Then run StrRepl to do the replacing.

With a little creativity, you can use StrRepl with ADB to allow you to put your Awk scripts in the very DOS batch files where they're used.  You can just come up with temporary replacments for less-than, greater-than, plus, and bar ("|").  The result is that almost all the executable code in a complex ADB job is "visible" to the reader of the DOS batch file.

StrRepl changes files "in place," and makes only one change per input line.

Ordinary messages are written to standard output, so you can suppress them by adding ">NUL" to the end of the command line.


View is the large file viewer.  It can handle just about any ASC text file around, because unlike a typical editor, it doesn't store the file's data.

Although you can view large files in Windows Wordpad, it won't show you line numbers (and the ruler is relatively useless for columms.  View.Exe isn't perfect: it's not possible to align the ruler very usefully for every line in a short file, nor for the last page's worth of lines in a long one.

You run view with the file name and an optional screen size (the default screen size is 25).  E.g.:


View Temp.Txt 50

(For a 50-line screen..  If the screen value is larger than your actual screen (or DOS window) size, then you won't see certain lines in the file.  If you really need to see the column ruler aligned right next to one of the last few lines in a big file, use a very small screen size (the minimum is 5, but there are 2 "overhead" lines used to print the ruler and the options.)

Return to local table of contents
Return to global table of contents
Frames mode, or No frames

4:  My utility DBCvt: The commas-n'-quotes <--> fixed length converter.

Although ADB runs DBCvt as a subprocess, you can use it for a number of other purposes.

The most obvious one is to take data from a DBMS in commas-n-quotes format and convert it to fixed length, so that some program you've written in a standard programming language can process the records.

Or the reverse: you may have a program that outputs a file in fixed-length format, and you might find it easiest to load the data into a DBMS in commas-n-quotes format.

DBCvt is also very useful for the situation in which someone sends you a file that's supposedly in commas-n'-quotes format . . . but which in fact contains various errors (maybe the DBMS "hiccuped" or the data got corrupted).

Return to local table of contents
Return to global table of contents
Frames mode, or No frames

5:  My utility FileMux: the friend of programmers . . . and files with multiple personalities!

I've already described FileMux in the Awk section on writing multiple output files.

But here's a rehash:

FileMux.Exe is a utility that I wrote for taking a single input file and dividing the records into _mutually exclusive and mutually exhaustive_ subsets.

(In simple language, it splits the records in the file up into groups :-)

FileMux.Exe can split records up based on the data in the input records.

Let's see a quick example:


A This is record 1
B This is record 2
A This is record 3

Now suppose you run FileMux like so (constants are in capital letters):


FILEMUX -INmyfile.inp -PREFout -EXT.dat -COL1 -LEN1

That says to split MyFile.Inp up into different parts, based on the data in column 1.  Two output files will be created: "OutA.Dat" and "OutB.Dat".  "OutA.Dat" will have records 1 and 3. "OutB.Dat" will have record 2.

FileMux's default is to keep the field that you use to split the records up (in this case, the field beginning at column 1, which goes for 1 character).

If you want to remove that field, code the "-NOKEEP" argument.

(FileMux, like all the utilities supplied with ADB, prints its command line arguments when you run it from the DOS prompt with no arguments.  Note also that FileMux's command line parameter names are case-INsensitive, like all ADB's utilities.)

If you have a background in computer science (or in hardware), you'll recognize the term "Mux" as short for "multiplexor".  A multiplexor is a hardware device that has one input and multiple outputs.  Based on a specific set of bits in an incoming word (AKA a set of bytes), a multiplexor will "route" the rest of the word to a specific output.

Return to local table of contents
Return to global table of contents
Frames mode, or No frames

6:  My utility FileStat: The Paranoid Programmer's Pal

FileStat's name comes from the Unix "fstat" (I only decided to change the name because I felt it would be more intuitive this way.)

FileStat has a fairly complex parameter structure.

The first command-line parameter may be any of "-q" (quiet mode), "-t" (trace mode), or "-p" (pause mode).  I'll talk more about this special first paramter in a moment.

All the rest of the parmeters are file names.  FileStat does different things, depending on a prefix you append to the front of the file name:


Prefix:  Extension:           Action:
-------  ----------           -------
!        Irrelevant           Must be writeable, erased if exists
@        Irrelevant           Must exist, and be writeable
/        Irrelevant           Must exist and be of length 0
:        Irrelevant           Directory name: must terminate in backslash
%        Irrelevant           A DOS Environmental variable name
None     .com/.exe/.bat       Must be available in the DOS path
None     None of the above    Must exist and be readable

There are two special rules that apply to files:

Either path information or wildcards are allowed in filenames of the first type.  But not both.  Files of the first type are the only ones that allow wildcards.

File names of the fifth type (.com/.exe/.bat) *can't* have path information (i.e. drive, and/or folder names).

All the others can.

Note that FileStat will also check to see whether DOS environmental variable names have been assigned: use a prefix of '%' on the variable name (but don't use one on the end.)

FileStat also reads file names from the standard input.

If you don't want it to read file names from the standard input, just run it like this:


FileStat (other parms) < NUL

FileStat is extremely useful for a wide variety of purposes.

First, if you're very paranoid (as I am) about DOS batch processes that get run remotely, then you should probably put FileStat in every directory (AKA folder) in which a DOS batch file executes.

Then you can start every DOS batch file off with something like this:


If Not Exist FileStat.Exe Echo FileStat.Exe not found, tell Rog!!
If Not Exist FileStat.Exe Pause
If Not Exist FileStat.Exe Goto ABORT_RUN

Why do this?

Because FileStat is your foundation for ensuring that all the files will exist in the batch job that's about to run.  If FileStat itself doesn't exist, then the user will see all sorts of "bad command or file name" messages . . . but you'll have no way of knowing what the problem is (unless you can actually sit down at their computer and check, which is often not an option.)

Now if you believe in "paranoid programming" as much as I do (and/or in "Muphy's Law," which is that: if something can go wrong, then it will), then you should probably use FileStat to validate the existence of every executable that you're going to need:


FileStat ADB.Exe GAwk.Exe RPSort.Com ErrMsg.Exe FileMux.Exe < NUL
>>ADBLOG.%ADBPROJ%
If ErrorLevel 1 Echo *ERROR* Some executables not found, tell Rog!!
If ErrorLevel 1 Pause
If ErrorLevel 1 Goto ABORT_RUN

(I've taken the first line and broken it up into two physical lines, for ease of display.  That has to be one line in your DOS batch file.)

Note that in the above execution of FileStat, I wrote the results to ADB's log file.  If that's going to get zipped up (at label ABORT_RUN), then I'll have all the information I need to do remote debugging.

Generally speaking, you don't need to check for the existence of ADB's files, since ADB will automatically do this for you and write the results into the log.

But if you're running other programs (such as those you've written in some other compiled or interpreted language), then FileStat can save you from having to verify that files exist, and it can make sure that the proper error messages go right into ADB's log file.  Once again, remote debugging is simplified.

FileStat's also handy for temporary files.  Suppose you want to make sure that the files Temp*.Dat are erased before a run:


FileStat !Temp*.Dat < NUL >> ADBLOG.%ADBPROJ%
If ErrorLevel 1 ErrMsg Filestat couldn't erase temporary files!
If ErrorLevel 1 Goto ABORT_JOB

The reason why you'd probably prefer to ensure that temporary files are all erased before the job begins is that there are many situations in which the windows versions of DOS will not allow files to be overwritten (or they may insist on prompting the user).

Worse, the file may be claimed by another process.  If so, all the user will see is "access denied".  That won't give them much of clue as to what's wrong.

Nor will you have the slightest notion either when they call you . . . and they say what all users tend to say, no matter what the problem is, or how many very specific error messages they may have seen.

(I'm talking about those immortal words that describe every computer error in the history of the human race with 100% accuracy, precision, and completeness: "Didn't work."  No technical person or programmer worth his or her salt needs any more information to get to the bottom of any problem encountered by a user, client, or customer.)

You might be wondering why the third type of parameter (file names that begin with a slash) is provided.  These files must exist and be of length 0.

I added this option especially for ADB.  In a typical case in which you're using (say) an invoice file as the "transaction" file for ADB and a customer file as the "master" file, you'd probably expect that every invoice has a valid customer number.

But what if there are some that don't.  Simple.  You create a T-M output file for ADB.  If any invoices are in this file, they'll be unmatched invoices.  That means they have customer numbers that don't exist in the customer file (presuming that you're matching on the customer number).

At the end of the ADB run, you use FileStat to verify that this file of unmatched invoices has length 0.

So FileStat is your "Murphy's Law Detector" when it comes to files that can't be executed, erased, written to, or which (erroneously) contain data.

As the term "Dectector" suggests . . . FileStat can't stop Murphy's Law from wreaking havoc.  But it can certainly make it a lot easier and quicker to clean up the mess.

What about the optional first paramter.  You can code "-Q" (quiet) if you want FileStat to do its work silently (i.e. not to write anything to standard output).  If you code "-P" (pause), FileStat will ask the user to hit the space bar to continue, whenever it encounters errors.

The optional first argument of "-T" (trace) is for those special situations in which Windows gives you "abort, retry, fail" errors, and there's no way of determining which paramter (i.e. file) is causing the problem.

Finally, you may wonder why FileStat reads its parameters from the standard input as well as the command line.  The reason is that the DOS command line is limited to 128 characters, and if you have a lot of files to check, it may be much more convenient to use DOS Echo commands to build a file that's a list of file names (or more precisely: each line in the file is one of the standard command-line arguments, other than the three special and optional "first" parms.)

I have to admit that it's slightly irritating to always have to code "< NUL" whenever I want to use FileStat with nothing other than command-line paramters, but the extra flexibility of being able to read the options from a file has come in handy for some larger jobs.

You can use FileStat's ability to read file names from the standard input in order to document the purpose of a file, since FileStat doesn't understand Windows 95 file names, and will stop reading a file name when it sees a space.  So you can just skip a space (or a tab) and start writing comments.  The only thing is that you do need to put at least one file name (or one of the three special parms described above) on the command line.  You can also just use FileStat.Exe as a "dummy" parm if you wish:


@Echo @FileIn1.Dat This is my first input file   >FileList.Dat
@Echo @FileIn2.Dat This is my second input file >>FileList.Dat
@FileStat FileStat.Exe &llt;FileList.Dat

WARNING:
if you exceed the 128-character limit on the DOS command line, DOS will truncate the line without issuing an error message.  That's why FileStat lets you put file names in via the standard input.

WARNING:
The "-p" parm in order to pause on errors is incompatible with redirecting the standard input!  FileStat doesn't check for this condition, although the on-line help does warn you about this.  If there is an error, the DOS command line will simply "freeze" and you'll have to force windows to close the DOS prompt.

FileStat's one of the most effective ADB utilities: it's worth spending some time to learn how to use it.

Return to local table of contents
Return to global table of contents
Frames mode, or No frames

7:  My utilities OSQLRF1 and OSQLRF2: extracting ODBC DBMS data and converting it to commas-n'-quotes

The OSQL batch interface lets you extract data from ODBC compatible DBMSs in "commas-n'-quotes" format via batch processes.  For example: Access/SQL/Foxpro, etc. Here's a sample DOS batch file that runs the two OSQL programs:


OSQL -U sa -P -i Query.Dat -d WebBoard -s ¯ | OSQLRF1 -Q >Raw.Dat
OSQLRF2 CQ.Dat -S175 -FLayout.xx -Nfldsetname -T256
DBCvt CQ.Dat Fixed.Dat -NOQUOTES


This execution example shows three steps.

In step #1, OSQL is used to run the file Query.Dat against the database called "WebBoard."  The DBO (DB owner) name for this is "sa", which is the default MSDE DBO, and the password is nothing (blank).  Your sysadmin should know the DBO name and password for your data.

A typical query might be something like this:


select * from users

(If you don't know SQL, you can enter "+SQL +tutorial" in Yahoo's search engine to get a fairly long list of free tutorials.)

The -s parm is followed by a space and a single character that will be used to separate the columns.  In this example I've chosen decimal 175, which doesn't come out the same way in HTML as it does in plain text.  That character can't appear in the data.  You may also use a single character (other than a digit).  If you don't have a text editor that will let you type control characters, use DOS EDIT at the DOS command line.  To get that character, hold down the alt key on your keyboard and type "175" on the numeric keypad (be sure that scroll lock is off.)

The first step is all that needs to be run by a sysadmin who has server access: the output file (Raw.Dat) is relatively compressed in the sense that it doesn't have the unnecessary spaces, etc. that OSQL outputs.  It should be even smaller after being zipped.


In the second step, OSQLRF2 converts Raw.Dat into commas-n'-quotes format.  (That's the file CQ.Dat).

This format is ready to be loaded into many different kinds of interfaces.  The -F specifieds a file that will contain both FLD (field) and FLDSET (field set) definitions in ADB's format.  The name of the fieldset is given by the -N parm.

The -T parm allows you to truncate fields automatically: in this case the fields are truncated at 256 bytes, which just happens to be the default.


The last step converts CQ.Dat (commas-n'-quotes format) into fixed-length format using DBCvt (another ADB utility; described above).


OSQLRF1 isn't very "smart" about how it computes the lengths of fields: it simply looks at the column headers that OSQL supplies.

For that reason, if the name of the field is longer than the field value, the former will be used in the layout file produced by OSQLRF2.


Leading and trailing spaces aren't preserved in the data.


You can run exactly one query (SELECT) per execution of OSQLRF1 and OSQLRF2.  If you need to extract the results of more SELECTs, you'll have to clone the example code for each one.


Your syaadmin may be (understandably) reluctant to run an executable on your server, unless s/he is absolutely convinced that it doesn't contain a virus, "Trojan horse," or some other method involving unauthorized entry.  Of course, it's possible to run OSQL by itself and reformat the output on another system.  The output file will be suitably small, if zipped.  Nevertheless, OSQL writes very large files, and if you can use the C source or this web page to persuade your sysadmin to run OSQLRF1, then you'll see significant efficiency improvements.  Note that the string "open" doesn't appear in the C source (because all the I/O is done though standard input and standard output).

My hope is that most sysadmins will appreciate the fact that they don't have to start any special software: they just click once on a DOS batch file icon and send out one file . . . this makes it easier and more convenient to set up periodic extraction processes.


OSQLRF2 will convert double quotes to single quotes by default, or use the replacement character of your choice.  You can use the -D parm for this: the syntax is the same as for the -S parm (a 1-3 digit number for decimal hex values, or a single character, other than a digit).

Return to local table of contents
Return to global table of contents
Frames mode, or No frames

8:  My utility Repl: The file/string replicator

Repl ("replicate") is the humble DOS equivalent of the rarely-supported Unix paste command.

Let's see a quick example.

Suppose File1.Dat looks like this:


AAAAA
BBBBB

Suppose File2.Dat looks like this:


111
222

This command:


Repl @File1.Dat @File2.Dat >NewRepl.Dat

will write this file to NewRepl.Dat (the output file):


AAAAA111
BBBBB222

You can think of Repl as the reverse operation to GetFld.Bat. GetFld.Bat will extract fields in files (based on length and offsets).

So, for example, if I took the output file NewRepl.Dat and ran:


GetFld NewRepl.Dat File1.Dat 0 5

then File1.Dat would be *unchanged*.  That's because I told GetFld to read the file called NewRepl.Dat, and output the field starting at offset 0 (remember, with GetFld, the leftmost column is 0), and going for a length of 5 . . . and the output went to File1.Dat.

You might wonder why I didn't name "Repl" as "FileRepl".

There's a good reason for that.  Repl can also use constants.

Let's try that Repl example again, but with a different set of parameters:

Repl @File1.Dat "$ a constant value " @File2.Dat >NewRepl.Dat

NewRepl.Dat now looks like this:


AAAAA a constant value 111
BBBBB a constant value 222

See what happened.  The constant value got inserted.

Repl is particularly useful when it comes to building DOS batch files.  In particular, you often want a DOS batch file to build another (temporary) DOS batch file, which will then be executed.

Typically you're collecting a file of file names.  I.e. a file in which each record is a file name.

For example, suppose you type the DOS command:


Dir C:\*.Txt /s /b >C_Txt.Dat

This collects all *.Txt files on the C drive, and writes their names (by themselves) into the file "C_Txt.Txt".  In other words, each line of C_Txt.Txt is the name of a text file on C.

Note that the path names *are* included.  And also that "long" Windows 9x file names are listed.

Now if you execute:


Repl  "$Copy \""  @C_Txt.Dat  "$\" D:\C_Txt\*.*"  > CopyCD.Bat

CopyCD.Bat will consist of lines that look like this:


Copy "C:\program files\a long textfile name.txt" D:\C_Txt\*.*

This is tricky to see, but the first argument to Repl is: "$Copy \""

That argument is quoted, and what Repl "sees" is a constant string: which begins with the word "Copy" and is followed by one space, and a double quote.

For simplicity, let's use the bar ("|") as a quoting character.

In other words, if I refer to a string as |" ab"c| then the string consists of a double quote, followed by a space, followed by the letters a and b, and finally by a double quote.

The second argument to Repl is the file C_Txt.Dat, which means that each line of the output will contain a line from C_Txt.Dat.

The last argument is (using our convention):


|" D:\C_Txt\*.*|

Okay.  So let's put it all together.  Suppose a typical line from C_Txt.Dat looks like this:


|c:\program files\ms world control plans.txt|

It's going to prefixed with:


|Copy "|

And followed immediately by:


|" D:\C_Txt\*.*|

Let's put them all together: |Copy "||c:\program files\ms world control plans.txt||" D:\C_Txt\*.*|

And finally remove the bars, because they were only used for our quoting convention:


Copy "c:\program files\ms world control plans.txt" D:\C_Txt\*.*

The result of that command is that the file: D:\C_Txt\ms world control plans.txt

is created.  So we have effectively extracted all the text files on drive C (no matter where they're stored) and put them into one directory (AKA folder) on drive D.

Repl is especially useful for writing DOS batch files that call themselves (recursively).  The idea is to generate a series of lines that look like this:


%COMSPEC% /e:1024 /c SomeFile.Bat Arg1 Arg2

and write them into a temporary batch file.  Chances are, that Arg1 is a file name and Arg2 is a constant (perhaps one that's been set as a result of user choices made via ADB's utility BE2).

So your Repl command will look something like this:


Repl "$%COMSPEC% /e:1024 /c SomeFile.Bat \"" @Files.Dat "$\" %SOMETHING%"

(I haven't shown the file to which Repl's results are written.)

Repl always echos its arguments to standard error.  To suppress this, code the "-Q" argument.  This must be the first argument on the command line.


You can also use Repl to get less-than signs or greater-than signs into a text file.  Also remember that if you want to pass a double-quote to a program, you simply put a backslash in front of it.


Repl "$<A HREF=\"link.htm\">%% Salary Increases</A>" >index.html

Now the file index.html looks like this:


<A HREF="link.htm">% Salary Increases</A>

Return to local table of contents
Return to global table of contents
Frames mode, or No frames

Next documentation section