How to get more out of Akiva's WebBoard versions 4.x-6.x with Rog's user DB batch processor (version 1.1)

This describes freeware that I wrote which allows any virtual board manager (or WebBoard administrator) to:

  • Add or delete users all-at-once from the WebBoard user database.

  • Add/delete a specified subset of users all-at-once from private conferences

  • Add/delete a specified subset of users all-at-once from conference mailing lists.

  • Extract user profile information for all users or a subset selected based on any specified criteria.

  • Change user profile information for all users or a subset selected based on any specified criteria, while incorporating free-form data obtained from other sources which may be associated with the user whose profile is being changed.  For example: loading the department names of users into the country field; or storing up to 10,000 bytes of data in WebBoard's "userinfo" fields.

  • Send out e-mails for all users or a subset selected based on any specified criteria.

  • "Replicate" board membership status, i.e.: take any set of users that are users of one board and make WebBoard "think" that they're users of another as well.

  • Perform any WebBoard operation repeatedly.

    The "special operations" interface can be configured as desired; however a JavaScript programmer is needed to handle these tasks.  In this documentation, I provide an example of how to use this interface to automatically post messages (e.g. those migrated from a "ListServ.")

(For some ideas on how these operations can be combined to get the most out of your WebBoard, see section 23.)

No administrator access is required to use this product--it's enough to be a virtual board manager.

No server access is required either--it's sufficient to be able to use the system via a telephone modem.

Because it does everything through WB's standard user interface, there's no risk of corrupting the WB user database, nor is there any specific dependency on the type of WB database interface that you're using.

You don't have to be a programmer or a "technical person" to use this product.  (Caveat: if you want to change user profile information, you will have to find someone who can write some simple JavaScript.  None of the other applications other than the "special operations" interface require programming.)

Nevertheless, you'll get the best results if you have a basic grasp of how to use a database manager (e.g. DBase or Paradox), or at least a product similar to Access.  Essentially, what you need for the most advanced applications is a piece of software that can "mirror" WB's user database and import/select/export records in "commas and quotes" format.

You must know how to cut-and-paste, and be comfortable with basic WebBoard concepts (such as: what it means for a board to "use real names" instead of "login names.")  It's strongly recommended that you be able to use a text editor such as Window Notepad.

If you're a virtual board manager or a WebBoard administrator who lacks any of the above skills, chances are fairly high that someone does have them within your office/organization or any WebBoard hosting provider that you may be using.

You must use this with Microsoft Internet Explorer version 5.0+.  It most definitely won't work on any other browser.

I've only tested for it use on a Windows PC (it may very well work on a Macintosh or on Unix-based systems.)

You must have both JavaScript (JS) and "session cookies" enabled.  (Any "firewall" that you're behind must support session cookies.)

This is for WB 4.x, 5.x, and 6.x only.  WebBoard 3.5/3.0 users can contact me if they need a version that works with WB 3.0 or 3.5.

This product is specially designed for use via an unreliable telephone modem connection running at speeds as low as 4800 baud; nevertheless a reliable 14.4 or higher telephone modem connection, a cable modem, ISDN, and/or T* access (etc.) is recommended if the number of users you wish to manipulate is on the order of 1,000 or more.

This code is freeware.  You use it at your own risk.  No warrany of fitness for use or any other form of guarantee inheres.

You may use it and/or the source code in whole or in part for any otherwise legal purpose that you deem fit; however you may not claim legal ownership of same for legal purposes against anyone who's obtained it from an "independent source" (such as this web page).


1:  Basic information

2:  Input, output, and operations (overview)

3:  Basics of batch processing

4:  Batch processing, IE5+, and the "same origin" policy.

5:  Special considerations: adding users

6:  WebBoard's support for adding users

7:  Special considerations: user data list searches

8:  Special considerations: the two methods for replicating memberships

9:  Special considerations: replicating members onto a "technical operations" board.

10:  Special considerations: changing user profile info.

11:  Special considerations: changing the "userinfo" fields

12:  Special considerations: sending e-mails

13:  Local searches versus global searches

14:  "Mirror, mirror, on the wall . . . "

15:  Special considerations: unexpected JS errors, restart failures, and large volume applications

16:  User list operations: how to clone private conference memberships, e-mail subscriber lists, replicate board memberships, or extract user info. QUICKLY

17:  Install (I): Complete Install Zips

18:  Install (II): Standard Install

19:  Install (III): "Fast" version

20:  Install (IV): Rebuilding the source files, and/or making modifications

21:  Install (VI): Changing the defaults

22:  The "special operations" interface

23:  Why client-side batch processing?  (Survey of applications)

24:  Bug reports, comments, and change requests, etc.

25:  Other freeware products

26:  Version history information

27:  Acknowledgements


1:  Basic information

Before you begin: if your primary purpose is to "populate" the membership of boards and private conferences, and/or conference e-mail lists, you may prefer my new User List system, available at: http://www.rs-freeware.org/ul.  Although this system still has a stronger interface for adding users (indeed, I recommend it for anyone who has to add more than a few thousand users), the User List system shines when it comes to "populating" boards and conferences, and it also contains an excellent interface for adding conferences en mass.


You need to keep the following facts in mind when you create input for this system (specifically: when building what I refer to as user data list format, and augmented user ID# format.

  • Lost Internet "packets"

    If you've spent a lot of time on the 'net, you probably know that sometimes pages refuse to load for an inordinate amount of time, and that hitting the refresh button can often make them load immediately.

    This is frequently due to a the fact that a "packet" has been lost because a server along the "pathway" has gone down.  If you just sit there without doing anything, nothing may ever occur.

    If you see this phenomenon occuring (once the processing actually begins), just close the little window (not the frameset), and the system will "think" that an "IE5+ timeout" has occured and will try accessing the page again.

    (If you're a programmer and wish to know more about IE5+ timeouts and how they're handled here, please see section 4.)

  • When commas-and-quotes are used as the field separators, this program is more restrictive ("pickier") than the usual standard.

    Normally, two commas in a row can be used to indicate an empty field, spaces are acceptable outside of double quotes, and no commas are required around fields with numeric values.

    In this system, commas-and-quotes is implemented as if the fields were separated by "," (double-quote, followed by comma, followed by double-quote) and each record is "wrapped" within double quotes.

    Specifically, the following isn't acceptable:
    "field-value"<space>,<space>"field-value"  (because excess spaces are included)
    "field-value",,"field-value"  (two commas in a row not legal)
    "field-value",numeric-field-value  (double quotes not around all fields)

    The way around this is to define the only relevant numeric field (numeric user ID#) as having character values, and to make sure that every field outputted by your database manager (DBMS) is either equal to some dummy value (e.g.: "no-value") or a single space.  Some DBMS systems will allow you to output a null field as "" (two double-quotes in a row), and this is perfectly acceptable.

    In later releases, I may implement a better parsing algorithm which correctly handles the full commas-and-quotes standard.

  • Tabs, carriage returns, double quotes, and line feeds are never acceptable as an input field value or a field separator.

    The only exception to this rule occurs when you use a field separator other than commas-and-quotes and there are double-quotes in a field value.  These double-quotes are always converted to single quote (AKA apostrophes).

  • Plain ASCII text format should always be used for input.

    It's okay if you're using a word processor to create the input, but you must save values as "MS DOS Text" or whatever the equivalent is.  This system won't understand rich text format, MS-WORD format, Excel format, etc.

    Some DBMS systems will output data as one long "line," this is also not acceptable.  Whenever multiple records are to be inputted, they must be separated by (at least) new lines.  (Carriage returns aren't required, so the format is Unix compatabile.)

  • Your input and output field separators are always the same.

    If you provide the input with a field separator of "***" (three asterisks), your output will be formatted in exactly the same way.  The same goes for commas-and-quotes.

  • Spaces on the left and on the right of each field are stripped.

    For example: it's impossible to specify that someone's last name will be set to exactly 3 spaces.

  • You must determine whether your board uses "real names" or "login names."

    Virtual boards that use real names can be readily identified by the results of a search users operation.  In that case, every user is listed as lastname,firstname.

    On boards that use login names, the search results show the users' login names.

  • Never minimize the main (frameset) window, once the process begins to run.

    For reasons unknown to me, IE5+ performs badly if the window is minimized or un-minimized during certain critical periods.

    If you need to get it off your desktop, resize it to the smallest possible size instead.

    If you make a mistake and end up minmizing it anyway, try to restore it immediately.

    If you get a JavaScript error, see section 15, for a recovery workaround that works in most cases.

  • Extracting data with my OSQL system, and using my userinfo system to store your own information in WebBoard's database.

    If you have server access, please see the link in the section 25 for my other freeware products: the OSQL system described there can help you extract the data you need to run some of the operations described here.  The userinfo system also shows you how to take full advantage of WebBoard's DB when it comes to storing your own data in the built-in userinfo fields (this doesn't require server access).

Return to table of contents

2:  Input, output, and operations (overview)

(This table is explained immediately below.)

Operation Input Output
"Generic" search An optional keyword, just as in WebBoard's "search users" page. For boards that use real names, a list that looks like this for all found users:
"User ID#","E-mail","FirstName","LastName",
"City","State","Nation"
For boards that use login names, a list that looks like this for all found users:
"User ID#","E-mail","Login",
"City","State","Nation"
In both cases, a list of UserID#s for all found users, with one user ID# on each line, surrounded by double-quotes
"Directed" search by e-mail (all boards), login (only for boards that use login names), or last name (only for boards that use real names). User data list input For all boards:
"User ID#","E-mail","Password",
"FirstName","LastName","City","State","Nation"
Boards that use real names will "inherit" the login from the input.  Boards that use login names will "inherit" the first and last names from that input.  All boards will "inherit" the password from the input.
For all boards, a list of UserID#s for all found users, with one user ID# on each line, surrounded by double-quotes.
For all boards, unfound users will be returned.  The format will be identical to the input format.
Add users User data list input Both successful adds and unsuccessful adds are outputted when users are added.  If no password is included in the user data lists, then one will be generated and added in the TEXTAREA field that outputs successful adds.  Other than that, the successful adds and unsuccessful adds are returned in user data list format.
No "reason" is given for failures; WB has simply refused to add them.
The most likely causes for failures are: (a) a duplicate login, or (b) an invalid e-mail address.
Remember, Only a WebBoard administrator or a virtual board manager can add users.
Send E-mail User data list input Only the login is used from the user data list.  The user is sent the e-mail me my password file, which is: emailpass.txt.  Presumably, you've either created a "dummy" board for this purpose, or you've temporarily removed that button from the login page.  A list of unfound user logins is returned (these users have changed their login names.)
"Replicate membership" on another board (replication method #1) User data list input The login and password are used from the user data list.  If they're correct, the user is then made into a "user" of the board from which the batch processor is run.  This can make it particularly convenient to use the generic search and "directed" search operations on this new board, if the users are drawn from multiple virtual boards, or if you wish to use the "fast" versions of certain pages to improve the processing speed (see section 19).  A list of unfound user logins is returned (these users have changed their login names and/or passwords.)
Delete users Numeric user ID# lists Nothing is outputted, because a user# which isn't found is one which has already been deleted.  Remember, only a WebBoard administrator or a virtual board manager can delete users.
Harvest user profile info. Numeric user ID# lists The output format for all found users is:
"userID#","login","firstname","lastname",
"email","location","homepage","first login (date)",
"last login (date)","#logins","#posts"
Unfound users are returned as well, in the input format.
At this point, the userinfo fields aren't included, but they could be with minor changes.
Add or delete users from conference e-mail lists Numeric user ID# lists The numeric user ID list is pasted into TEXTAREA boxes in the appropriate WebBoard page (not into the User DB batch processor).
No special output is delivered, other than what WebBoard returns after the submit button is clicked.
Add or delete users from private conferences. Numeric user ID# lists The numeric user ID list is pasted into TEXTAREA boxes in the appropriate WebBoard page (not into the User DB batch processor).
No special output is delivered, other than what WebBoard returns after the submit button is clicked.
"Replicate" membership onto another board (replication method #2) Numeric user ID# lists The numeric user ID list is pasted into TEXTAREA boxes in the appropriate WebBoard page (not into the User DB batch processor).
No special output is delivered, other than what WebBoard returns after the submit button is clicked.  Unlike the other replication method, this requires administrator access.
Change user profile data Augmented user ID# list input Numeric user ID#s are outputted for both users who were successfully found and those who weren't in a TEXTAREA field.
Only a WebBoard administrator or a virtual board manager can change user profile data.
Perform any number of "cutting and pasting" operations with the membership lists of private conferences, the subscribers to conference e-mail lists, or the members of a board. (Not a batch processor operation.) When you go to "manage mailing lists" to manage the subscriber list for a conference e-mail list, or you click on the users link when managing conferences, or you're an administrator, and you click on the "Add users to board" link on the administrator menu, you have the option of extracting a list of user ID#s for this particular conference or board (even open boards).  You may take this list of user ID#s and combine it with any other such list and (re-)create the membership list for a board, or a priate conference, or the subscriber list for a conference e-mail list.  This option doesn't require the use of the batch processor per se.  For more details, please see see section 16.

There are five basic "excution pathways" through the system, depending on the sort of input that's required.

Return to table of contents

3:  Basics of batch processing

If you're not comfortable with computers, you can skip this section completely: the batch processor's defaults should work well for most applications.  However, if your system is unusually slow (e.g. an unreliable 9600 baud telephone modem connection) or fast (you're running on the server), or if you're a technical person, you might benefit from reading this section.

The key thing to note here is that pages are being "requested" from a server.

For example, every time you inspect a user's profile, a page has to load from your WebBoard server.  Each time you add or delete a user, a page must also load.  And if you're doing a search, WB's search page has to load for every "chunk" of 25 displayed users.

The proper way to do this is to have a system that "sleeps" most of the time, and periodically "wakes up" to see if the page it's "looking for" is present.

Depending on the speed/reliability of your connection, and your WebBoard server, the best choice of "sleeping time" is going to vary.

The default is 2 seconds, which works well for my 28.8 modem connection.  If you're on ISDN or a cable modem, you might want to drop the sleeping time down to 1 second.

When you run the HTML file, the "sleeping time" is referred to as the "clock tick" time.

Every time the clock "ticks", the HTML file will "wake up" and see whether the page has loaded.

Now, what happens if your phone line gets disconnected, or your server goes down?

Obviously, the page isn't going to load.

To handle this problem, I've got another paramter that you can set, which I call the "maximum number of failures in a row."

A "failure" occurs when the HTML file "wakes up" (i.e. the clock "ticks") and the page hasn't loaded.

If you have a fast, but not always perfectly reliable connection, then you want the clock to "tick" very rapidly (e.g. every 2 seconds), but you want the maximum number of failures in a row to be fairly high (say: 1000, which is the default).  That means that if the page hasn't loaded in 2 x (1000+1) = 2002 seconds, then the "batch operations" will stop.  And well they should, because there's probably some sort of serious problem.

Similarly, if you have a slow, but reliable connection, then you want the "tick" ("sleep") time to be relatively high (5-10 seconds), but the maximum number of failures to be low (2-5).

What happens when there are too many failures?  Nothing terrible occurs, but an "alert" will pop up to let you know that the maximum number of failures has been exceeded, and therefore that you'll have to start again.  The starting point is automatically reset to the last unsuccessful operation.

The progress of the "job" is displayed in the bottom frame, and you can easily figure out where you want to restart the process if your computer "hangs."

In section 21, I tell you how to set the default values, so you don't need to inspect them each time.

Section 21 also explains how you can set the variable C_TickSecIncr so that you get many clock "ticks" per second, if you're running the system from the server.  Theoretically, you can get tens or even hundreds of thousands of operations per hour on a fast server by doing this.

If you're seeing the IE5+ "page cannot be displayed" error in the little (execution) window, and the system gives you a JavaScript error when you try to restart the processing, see section 15) for the recovery workaround.

At least one user reported getting an IE5+ "object doesn't support this property or method" error that references this line of code:


Fr.document.writeln(H);

If you're also getting it, then you should check the box called fresh execution window.  This option slows down the processing, but it may be your only way of getting around a tendency of some versions of IE5+ to enforce the "same origin policy" with excessive zeal.  This option is off by default, but you can change the default if you wish (see section 21).  You shouldn't use this option unless you see the error described above.

Return to table of contents

4:  Batch processing, IE5+, and the "same origin" policy.

(You don't need to read this section if you don't wish to.  I recommend that you simply leave the box that says "Automatic IE5+ reset clears #failures?" checked.  But if you're a programmer and/or you know JavaScript, you might find this section interesting.)

MS Internet Explorer (version 5) contains a number of unusual characteristics.

One of these is a "hair trigger" that operates whenever there's the slightest delay from the ISP or the server.  Although this can be changed in the registry, my approach is to try to work with it (since it can also indicate that the phone line is down, or the server is down).

Note that the offline.html page is also treated as an IE5+ timeout.  (This page loads when WebBoard is alive, but the server has been paused.)

Such delays force an entire frameset to be replaced with a page from the viewer's hard disk that contains the message: "The page cannot be displayed."  Typically the viewer can click on the back button to resume the processing.  Unfortunately this won't always preserve the values of any interactive process that was in progress.

Because IE5+ is so distinctive in this particular way, web developers sometimes refer to "IE time-outs."   By that they mean that IE is showing the "The page cannot be displayed" page, even though the server is up (the user's ISP or the web server might be just a little slower than IE5+ expects).

(The separate window that this software uses in which pages are loaded is required due to IE5+'s endearing tendency to clear the frameset when timeouts occur.)

This software gives you two basic "controls" to deal with the problem.

First, you can specify whether IE "timeouts" will reset the maximum-number-of-failures-in-a-row to zero.

I've configured the software to do this automatically, but you can change this when you're running it (if you plan to be watching the computer as it performs the tasks you assign).

If you decide not to use the option to reset the consecutive-failures-count to zero whenever IE times out, then you'll probably have to invoke the restart option very frequently if your ISP and/or WebBoard server are less than 100% perfect.

Second, you can change the number of seconds that the software waits when IE times out.  I recommend that you leave it at the default, which is 10 seconds. The reason for this has to do with how IE timeouts interact with the "same origin" policy. (Specifically: the execution window has to be closed and reopened to a "dummy" page before the WebBoard form HTML can be written back into it.)

Hence, if you run this software over the weekend at your office (e.g. to dump out all users' profile information), and your server goes down or your ISP drops you, or your phone line causes the modem to disconnect, then there could be a lot of unnecessary disk activity if you change this second default to (say) 1 second.  (In the worst case, you could damage and/or reduce the expected lifetime of your hard disk.)

The first option (whether IE5+ timeouts reset the number-of-failures-in-a-row to zero), is a checkbox that can be set at run time (there's also a variable which controls the default).

The second option (the number of seconds that we wait when an IE5+ timeout is detected) can only be set by modifying a variable.  In section 21, I tell you how to set both these values in advance (this is the only way to change the second option).

If you're a programmer and/or you know JavaScript, you might be curious about how I detect IE5+ timeouts.  The trick is to exploit the "same origin" policy, which prohibits two web pages on different domains from exchanging data directly.

Of course two pages on different domains can pass data through the "query string," viz. the "?" that one often sees on web pages, but that can only be set when a URL is invoked.  There is no other browser-independent mechanism for two client-side pages on different servers to share information.

(Irrelevant diatribe:)

This feature of the web, in tandem with Microsoft's unforgiveable hobbling of Netscape's cookie standard (MS limits the total space used by client-side cookies for a single domain to 4K), has "forced" many applications quite unnecessarily into server-side mode.  The result?  Increased development costs, slower servers, and a lower overall societal return from the Internet.

In fairness to Netscape, at least it has tried to redress the harshness of the "same origin" policy with it's "data tainting" mechanism.  But MS has always striven to un-do or otherwise vitiate every convenience offered to consumers and developers by Netscape, as part of MS's world domination strategy.

In this way, MS stands alone as the one firm that's single-handedly succeeded in raising costs, lowering performance, and reducing the value of the Internet to all companies, consumers, developers, and governmental entities that benefit from the web.  This is yet another reason why many technical people would love to see MS broken into a thousand snivelling quivering fragments as a result of the pending antitrust litigation.

I exploit the "same origin" policy by setting a variable equal to the URL of the little execution window.  If IE5+'s "The page cannot be displayed" page has loaded, it has loaded from the client's computer (i.e. from your computer).

That means it's on a different "domain" as far as IE5+ is concerned (i.e. a domain that's unequal to the WebBoard server), and IE5+ would normally report a JavaScript error.

However, I've "protected" this assigment with IE5+'s try . . . catch construct so that no error's "reported" to the user.

If you wish to write a process similar to this which will load pages on a web server under the command of JavaScript running on your own PC, then that method won't work.

If you're running on a windows machine, what you do is to reverse the strategy.  You assign a variable to the URL of the window where the external page (the one from the web server) has loaded, and if an error is caught by the try . . . catch construct, then the page has loaded.

If, on the other hand, the URL begins with the string "res" . . . then IE5+ has loaded its "page cannot be displayed" page, and that means that the web server which you were trying to get the page from has either gone down or failed to deliver the page you were trying to access for some other reason.  At that point, you can retry the operation (you probably don't want to keep retrying forever, so you also have to implement something analogous to the maximum number of failures concept that I described in section 3).

The code that I use to implement this operation is in the routine called IE5FailChk, and as you can see, if _TESTFLAG is set, I actually reverse the strategy because the page is loading from my computer during testing.  If you want to be completely "correct" in your code, you should check for error #70, although I haven't bothered to do this in my test version (I do show you how to compute the error number.)

You can contact me if you need more guidance on writing batch processors for your own applications.

Return to table of contents

5:  Special considerations: adding users

In section 2, I describe the user data list format which is used to add users.

That section tells you most of what you need to know, but here are a few additional details that might be relevant to your situation:

  • Do you need numeric user IDs?

    If you're going to add users and then immediately sign them up for conference e-mail lists, or private conferences, then you'll have to do two operations--even if you don't care about cases in which the attempt to add a new user has failed (e.g.: duplicate login).

    (Important note: if you're a WebBoard administrator, see section 16.)

    This is because the numeric user ID is required whenever you sign users up for conference e-mail lists or private conferences, and WebBoard gives you no easy mechanism for discerning the numeric user ID after a user has been added.

    What you need to do is to use the user data list search options right after you've added the users.

    For best results, I recommend that users be added in such a way that their login names (on boards with login names) are easy to look up.  For example, if you just assign numbers, such as User1, User2 . . . etc., then you probably want to "pad" zeros on their names (e.g. User001 instead of User1) so that they can be located immediately by a search (see section 7, below.)

    This is because the user data list searches (AKA the "directed" searches) only inspect the first search page, and if you have more than 25 qualifying users, you might miss the user(s) that you're looking for (because they'll be on a page after the first one).

    On boards that use real names, it's impossible to look them up via their login names.  In that case, I recommend that you either: (1) use their last name for the user data list search (you may need to make that the same as their login name), or; (2) you use their e-mail address as the basis for the search.

    If you can't do either of those, you're stuck with dumping out all the users' profile information.  You start with the generic search and then use the numeric user ID# list to get to the profile info.  Finally, you load those into your database manager, and "match" that against the list of users whom you added.

    I wish there was a better way to handle this (somewhat obscure) case.

  • Timing issues: sending welcome e-mails

    If you're going to add users and then subsequently collect numeric user ID#s via last name (on boards that use real names), then I recommend that you refrain from sending a welcome e-mail at the time you add users.

    This is because a user can log in and change all of their profile information before you have a chance to look up their numeric user ID#.

    What you should do instead is to send the welcome e-mails later, after you've added all the users and looked up their numeric user ID#s (see section 12, below, for more information on sending out bulk e-mail messages).

Return to table of contents

6:  WebBoard's support for adding users

If you have more than (say) 10,000 users to add, you may be tempted to use the "authentication" approach, as described in their sample *.VBS scripts.

This method relies on putting your users in some other database in the server and then validating the user's name and password at login time--if validated, the users are added one-at-a-time to WebBoard.

While this may be effiecient, provided you know VBS (or can translate this into JavaScript or Perl), I've found it impractical in most contexts.

For one thing, these strategies don't handle situations well in which users are permitted to change their login names and/or passwords.  (Specifically: you can end up with two users who have identical logins, beacause the UserExists function requires a correct password.)

They also require direct access to the server in order to update the user database.  And if you're "migrating" posts as well as users from (say) an e-mail list, you have to have all the users in the database "at once" (at least those with migrated posts).

It's tempting to think that you can load users in via "authentication" scripts with the auth.adduser call (as described in the WebBoard documentation (chapter 10).

Unfortunately, this approach won't work for large numbers of users, because the internal "webserver" has a ScriptTimeout default value of (?) 90 seconds.

So if you want to use auth.adduser with WebBoard "out of the box," you'll have to "chunk" your users into groups of about 50, create an auth.js (or auth.pl, or auth.vbs) script for each chunk.

Then you have to FTP over each script in turn (under the proper name), and login.

This is clearly unsuitable for applications involving many thousands of users, and it's not at all clear to me that the processing runs that much faster than it does with this product, when the "fast" version of adduser.html is installed (see section 19).

Return to table of contents

7:  Special considerations: user data list searches

In section 2, I describe the user data list format which can be used to look up users.

(Important note: if you're a WebBoard administrator, see section 16.)

This list-based search is particularly useful if you have a list of (say) last names (on boards that use real names) or login names (on boards that use login names) or e-mail addresses (all boards).

The most important thing you need to keep in mind about this form of seach is that only the first page of results will be scanned.

For that reason, you need to be particularly careful when (e.g.) seaching for users with very common last names or e-mail addresses that are likely to be prefixes of other addresses (for example: lee@hotmail.com).

When running user data list searches, you may insist upon an exact match for the first name and/or last name (on boards that user real names), or the login name (on boards that use login names).

Either of these "filtering" criteria can be combined with an insistence that the e-mail address match precisely.

Finally, you may specify that all matches are to be output, or just the first one.

If you've ever used a database query langauge, you probably realize how tricky these "filtering" issues can sometimes become.

One way to get around this is to use the generic search option to dump out all the likely-looking numeric user ID#s, and load them into a PC database manager (along with the other information returned by WebBoard's search users page), and then write highly specific queries to extract what you need.  Alternatively, if that doesn't give you enough information, you can dump the numeric user ID#s out and run them through the user profile harvesting option.

Return to table of contents

8:  Special considerations: the two methods for replicating memberships

If you don't have administrator access and the board isn't closed, you can only replicate memberships via the batch processor itself.

That means you need to build the "user data list" format, and paste it into the main execution module (see section 2 for more information on user data list format).

You actually only need the first 3 fields of this format, and the e-mail address isn't used, so the input format looks like this:
"login","","password"

To facilitate the membership replication, you may wish to temporarily prevent users from changing their passwords on the other boards while you collect their user profile information.

Note that this method for membership replication won't work for basic authorization boards (you need to use cookie authorization).


If you do have administrator access, or the board's already closed, the second replication method is both faster and easier.  It's also the only way that you can remove users from a board's membership.

Go to the administrator menu, and look for the "Add users to board" option.

If you've installed the appropriate files, you can select that option and use a list of numeric user ID#s to add and remove users from the board (see (?install_wb)section (#?install_wb) for the install information.)

(Note that you have to click on the "show user list" link to see the page come up in the left frame.)

Note that this is a very quick operation for fewer than (say) 1,000 users, and could therefore be done during working hours with minimal confusion.

For more information about how to accumulate list of numeric user ID#s, see section 13.


You may also wish to read the section on using a "technical operations" board (section 9.)

Return to table of contents

9:  Special considerations: replicating members onto a "technical operations" board.

The fastest way to get information about members is by using the standard user search page, usersearch.html.

Even over a typical 28.8 modem, you should get about 10 page accesses per minute, especially with the "fast" version of the search page (see section 19).

That means you're getting the information for 250 users per minute, or about 15,000 per hour.

Unfortunately, usersearch.html only returns the users who have logged into the board on which its being run, and if the board uses login names, it won't show the users' real names (although it will show their login names).  If the board uses real names, it won't show the users' login names (although it will show their real names).

If you're a manager of just one virtual board, there isn't much you can do about these limitations.

But if you're a WebBoard administrator, or you're a manager who has access to more than one board, you can do a great deal.

The trick is to make WebBoard "think" that all the users from one group of boards (or a single board) have logged into a special board that you designate as a kind of "work area" or "technical operations" board.

There's a drawback to this, and that's that you'll have to collect the users' passwords as well as their login names.

So you can't escape an initial round of dumping user profile information for each user.

But, after that, you can maintain accurate lists of the users' numeric ID#s, real names, login names, e-mail addreses, and geographical information (city, state, nation), and do so very efficiently.

If you're an administrator, you can always set the board to use either real names or login names, depending on the information you wish to collect.  Virtual board managers will have to enlist the aid of their adminstrators to perform this change, but it just takes the administrator a second to do this.

You probably also want to use the "fast" version of the search page (see section 19).

The ability to "replicate memberships" is particularly useful in contexts in which the boards are all used by members of a firm, association, educational institution, etc.  (I provide more details about membership replication in section 8).

Since you know when new users will be coming in, you can start by adding the new ones to the main board(s), and then replicate their membership on your "technical operations" board.

Finally, you can use the e-mail sending facility to deliver welcome messages afterwards, so they don't have a chance to change their passwords in the meantime.

This is a relatively convenient and efficient discipline for cases in which you're regularly adding a small number (say, 2,000 or fewer) users every week or month.

I haven't tested this facility with boards that use "basic authentication," so I suggest that you make your "technical operations" board use cookie authentication, if at all possible. )).

Return to table of contents

10:  Special considerations: changing user profile info.

If you have server access, please see the link in the section 25 for my other freeware products: the OSQL system described there can help you update the user profile information.

The userinfo system also shows you how to take full advantage of WebBoard's DB when it comes to storing your own data in the built-in userinfo fields (this doesn't require server access).


In section 2, I describe the agumented user ID# format which can be used to change user profile data.

This is the only application that requires you to write JavaScript code.

There's simply no way that I can specify all the different changes that you could wish to make to user profiles . . . via any means other than having someone write code.

However, the coding involved is minimal in scope, and with any luck, your "office geek" either already knows how to write the JavaScript needed to change a form value, or can learn to do so with relatively little effort.  I've suggested that any bright high-school student could learn to use this product . . . well, there are probably millions of such kids who know enough JavaScript to set a few values in the user profile form, according to your specifications.

It's also the only application that requires that you use cookie authentication in the virtual board from which you're executing it (see the WebBoard documentation book, pp. 120-1).

If you can't create a board with cookie authentication, you can still use this feature of the user DB batch processor, however you can't change the userinfo fields (see the next section.)

It goes without saying (but I'll say it anyway :-) that you should probably execute this application when few if any users are likely to be changing their profiles.  Otherwise, they may have that page open at the same time as this process is running, and could accidentally "cover over" your changes.  One possible strategy here is to temporarily remove the profile editing option from more.html (the "more options" page) and restore it when you're finished.

For the rest of this section, I'm going to presume that you're someone who can write a little JavaScript.

As you may recall, agumented user ID# format consists of the numeric user ID# plus an optional field consisting of free-form data.

When you set up an operation to change user profile data, you modify the third block of code in the function Profile_Update_Fnc (contained in my version of useredit.html).

This function is passed the original free-form data field as its only argument.

If the user profile data is modified, the function returns true, and this will result in the saving of the modified data.  Otherwise, if the user profile data is acceptable in its current form, the function should return false  (the user profile won't be saved).

If you wish, your version of this function can always return true, but the processing will be more inefficient (because useredit.html will be loaded again to save the data).

(Similarly, if your function returns false when it should return true, then any changes you've made won't be saved.)

The example that I've provided shows you how to set the digest field, although you can obviously change any other field.

There's no checking to ensure that your changes were acceptable to WebBoard in the current version.  In fact, the processing will halt if you (for example) try to set the e-mail address to a value that WebBoard regards as illegal, or you cause any of the error messages listed at the beginning of useredit.html to come up.

Naturally, any changes that you make to the user profile data can incorporate information that's passed to Profile_Update_Fnc (via the only argument to this function).

If you're wondering about the use for the variable C_UIFlag, please see the next section.

BTW, if you remove that variable, or you change its name, you'll get a JavaScript error.

If you believe that there's any data stored in the userinfo1 . . . userinfo5 fields of any user whose profile you'll be modifying, you must read the next section, otherwise you could end up accidentally overwriting those fields.

Of course user profile changes require two server accesses for each user whose profile was successfully changed.  This is (obviously) because useredit.html has to load once in order to initiate the change, and a second time in order to save the change.

Return to table of contents

11:  Special considerations: changing the "userinfo" fields

(Please read the previous section before reading this one.)

WebBoard offers five fields in the user profile which are "undefined."  (to use the term of art that WebBoard documentation uses).

These fields are called: userinfo1, userinfo2 . . . userinfo5.

By "undefined," what I presume they mean is that these aren't used by WebBoard for any required processing, and that WebBoard administrators and virtual board managers can use them to collect and store information associated with a particular user.

Each of these fields can store as many as 2,000 bytes of data (see the WebBoard documentation book, pp.255-6).

You can use the User DB Batch processor to set and/or change values for these fields, but there are several things you need to do in order to ensure that the processing works properly.

Even if you don't wish to change any of these values, you must follow the procedures in this section if there is any information currently stored in the userinfo1 . . . userinfo5 fields for the users that you'll be affecting.  Otherwise, you could unintentionally set the values of these fields to '' (the null string).

First, you must be using cookie authentication on the virtual board from which you're running this process (see pp. 120-1 of the documentation book).  If none of your boards have this form of authentication, and you're an administrator, you can create a special board which you use soley for the purpose of changing user profile data.

Obviously you have to make sure that the userinfo fields are actually included in the form that's submitted by useredit.html.

If you don't want the user to see them, then you can add statements like this to the HTML:
<INPUT TYPE="hidden" NAME="userinfo1" VALUE="<wb-userinfo1>">

You need to be careful to make sure that none of those fields contains double quotes, of course.

Second, you need to make sure that the variable C_UIFlag is set to true.

This variable is needed to control the processing, due to a bug in WebBoard 4.x.  (However, users of WebBoard 5 and 6 must still set this variable.)

Specifically: the userinfo fields which will load (when an administrator or virtual board manager edits a user profile), belong to the administrator or virtual board manager, not the user-in-Q.

To get around the bug, this product will "log in" the user by setting the WB-User and WB-Pass cookies to the user's value.  Obviously that can only be done after useredit.html loads, since the user can change their password and/or login name anytime (unless you've prohibited that, and in my judgement, it's extremely unwise to do so when it comes to the password.)

Note that the user DB batch processor will "collect" your login name and password when it first comes up, and it will use the same strategy to "log you back in" any time that there's the slightest chance that the processing will be interrupted.  When you run the user profile changing option, you'll see a button which is called VERIFY-ID.  That should convince you that the user DB batch processor has correctly ascertained your login and password.

After the user's been "logged in," the profile page will load again, with WebBoard's profile?0 URL.  This ensures that the correct userinfo fields will be loaded.

This second load operation will occur iff the variable C_UIFlag is set to true.

If you forget to make this change, you could unintentionally "clobber" the userinfo fields for every user whose profile is saved.

Finally, keep in mind that three server accesses are required each time you alter the userinfo fields for a user.  First, their profile page must load.  Next, the page must be reloaded after the user has been "logged" in (via the change of the WB-User and WB-Pass cookies).

If your Profile_Update_Fnc returns a value of true, then useredit.html will load a third time, after the changed profile information has been saved.

Although it's a little tricky to handle the userinfo fields, the extra effort required may well be worthwhile, due to the large amount of data (10,000 bytes) which can be stored there.

Return to table of contents

12:  Special considerations: sending e-mails

This feature only works for boards that use cookie authentication, i.e. it won't work for boards with basic authentication.

In section 2, I describe the user data list format which can be used as the basis for sending out e-mail messages to all users or a specified subset of users.

Since nothing is used here besides the login name, that's all you have to specify for the user data list format.

This feature "hijacks" the e-mail me my password feature from WebBoard and lets you use it for your own purposes.

Obviously the first thing you need to do is to make sure that none of your users will unknowingly invoke this feature while you're sending out the e-mail messages.

You can avoid that unhappy scenario in several ways.

First, you can simply include the user's password in any bulk e-mail that you're sending out, as a friendly reminder (e.g. in a postscript.)

That may not be the best choice, if you're concerned about security, or if you feel that it might offend some security-conscious users.

As an alternative, you can put a little warning message on forgot.html which mentions the fact that the "e-mail me my password" feature is temporarily down, or you can simply (temporarily) remove the "forgot" link on the login page.

Third, if you're a WebBoard administrator, you can create a dummy board specifically to support bulk e-mails.

Finally, you can simply run the bulk e-mail processing at a time when you think the board isn't being used very much, and put a little note on the e-mailed message which explains that the "e-mail me my password" feature is temporarily down, and will be restored shortly.

What if you're on a board that uses real names, and you don't have an easy way of getting to the login name from (say) a user's e-mail address or last name?

In that case, you're basically stuck with doing two operations: you have to start by doing a user data list search (see section 7), to pick up the numeric user ID#s.  Then you use that to extract user profile information, which will contain the login name.

Regardless of whether your board uses real or login names, you may benefit from "mirroring" techniques (see section 14).

Note that the text for the message is contained in:
emailpass.txt

It goes without saying (but I'll say it anyway), that I personally abhor junk e-mail.  I've implemented this feature in the hopes that virtual board managers and administrators will use it responsibly  :-)

One more detail: if you have an older DOS editor that puts in a control-Z at the end of the file, WebBoard will choke, and send you a message saying that your emailpass.txt file wasn't found.

Return to table of contents

13:  Local searches versus global searches

I want to emphasize that both the "directed" search (via user data lists) and the "generic" search rely on WebBoard's search users mechanism.

That's just dandy if you're only interested in the users for a particular virtual board.

What if you want to access the information for all users on all boards?

One way to do this is simply to login to all the boards and use the generic search capability on each board.

Then you dump the results into your database manager and remove the duplicates.  But if there are more than (say) just a small handful of boards, this is going to be a little awkward.

(If you haven't read the second on replicating memberships to a "technical operations board", I suggest you go back and review section 9).

Now, if you have that section, you realize that there are basically two choices: you can dump their user profile information every time (which is what I'll describe starting in the next paragraph), or you can replicate their memberships onto a "technical operations board", and use the "generic" search facility.  But note that you do still need their passwords to perform the replication.

Fortunately, WebBoard's user profile page will load profile information for a user who's never visited the current board.

All you need is the numeric user ID#.

Uh . . . wait a second: how are you going to get to that?

Alas, there's no easy solution that I know of, although there is a way.

First, register a new user, and login as that user.  Find the user using WebBoard's search machanism and then view their profile by opening it in a new window (you need to right-click on the link to do this).

The last part of the URL at the top of the page look something like this:
userpeek?1234

The 1234 part is the user's numeric user ID#.

Since numeric user ID#s increase sequentially, you now know that (for this example) the highest numeric user ID# on the board is 1233 (because you just added 1234).

Now you can use the user profile search to look up the profile information for users 1 through 1233.

To do that, you need a numeric user ID# list.  If you're a programmer, generating such a list should be no problem.

If not, you can use the form in the box below.

Enter the highest numbered user below:

Then click the button below:

Finally, copy the list below to your clipboard:

You may also cut and paste this code out into a .HTM file that loads from your hard drive:

Note that unfound user numeric ID#s are always outputted into a separate TEXTAREA box at the end of the run by the user profile seach.

If you're "mirroring" the users on the board with a database manager, you can capture the unfound numeric user ID#s so that you never end up searching them twice.

This is particularly useful if you're going to periodically "purge" users who haven't logged in recently.

Note that once you've "replicated" a user's membership into any special "technical operations" board, they'll always be considered users of that board.

So, under the replication strategy: if you need to dump information out periodically (say: monthly), then you should be logging the highest user ID# present at the time of the last extraction, and then use this as your basis for dumping user profile information (i.e. for all users who've been added since then).

Keep in mind that you'll have to prevent users from changing their passwords in between the time that you've extracted them from the user profile search, and the time that the membership replication process is run.

Return to table of contents

14:  "Mirror, mirror, on the wall . . . "

As you may recall, I stressed that you'd get the most out of this product if you have a basic working knowledge when it comes to using a database manager such as Paradox, DBase, or even a simple tool such as Access.

If you're going to be regularly adding users and/or removing them from conference e-mail lists and/or private conferences, or regularly sending out e-mail messages to a specific subset of users (or all users), then you really do need to think in terms of mirroring the WebBoard user database.

Mirroring is even more useful if you're going to be adding new users every now and then.  For example, suppose your organization has a quarterly or monthly meeting, and you're using the WebBoard as a way of keeping the attendees informed of the agenda and other aspects of the upcoming meeting.  In that case, you may be adding members all the time, and your "mirrored" database is what you'll use for generating the login names of fresh members (so that the fresh members' login names don't overlap with existing members' login names).

Essentially what "mirroring" means is that you use the generic search capability to dump out all users periodically.  This works fine if you're only interested in the users who patronize a particular virtual board (and it's much faster than looking up user profile information, because 25 users are shown on each page).

Of course, if you're interested in all users on all boards, or the information displayed by WebBoard's search users page doesn't do the trick, then you will have to lookup user profile information, possibly with techniques similar to those I described in section 13.

If you're on a board that uses real names, you really don't have a choice about using the user profile information, because WebBoard's search users page has been specifically designed not to display the login name for boards with real names.  (One way around that is to prevent users from changing their login names, which strikes me personally as being a desireable change in any event.)

For example: suppose you want to send out a monthly e-mail to all users who are known to live in the state of New York.

Unless you've been rather careful to restrict the users during their sign-up process, you'll probably find that some users have listed their state as "NY," whereas others have listed it as "N.Y." and still others have typed "New York".

With mirroring you can load their information into your PC database manager, reformat the state name, and then dump out their logins whenever you need to send the e-mail.

Return to table of contents

15:  Special considerations: unexpected JS errors, restart failures, and large volume applications

This section is particularly relevant if you're working with large-volume applications.

You may also with to read section 19, to find out how to install the "fast" versions of the files.

And don't forget to turn JavaScript errors on, otherwise you won't be able to take full advantage of the information in this section!

  • "Objected expected" JavaScript error

    If you're doing something else on the machine that's running the process, you'll occasionally (and I mean very rarely) see an "object expected" JavaScript error.

    Simply click on "OK" to close the grey JavaScript error alert.

    Then you'll need to manually restart the run by selecting the current operation from the top frame's drop down box (not the operation type, but rather the current record that's being processed), checking the restart after fail box, and clicking on the GO button.

    This happens to me about once every dozen runs or so, usually when I'm compiling large C++ programs at the DOS prompt--I ascribe it to an IE5+ memory issue.

  • "Object doesn't support this property or method" JavaScript error

    At least one user reported getting an IE5+ "object doesn't support this property or method" error that references this line of code:

    
    Fr.document.writeln(H);
    

    If you're also getting it, then you should check the box called fresh execution window.  This option slows down the processing, but it may be your only way of getting around a tendency of some versions of IE5+ to enforce the "same origin policy" with excessive zeal.  This option is off by default, but you can change the default if you wish (see section 21).  You shouldn't use this option unless you see the error described above.

  • Turn off HTTP logging and SMTP tracing

    If you're a WebBoard administrator, or a virtual board manager who can obtain the administrator's cooperation, I recommend that you turn off both HTTP logging and SMTP tracing--both of these options cause data to be appended quite unnecessarily to log files on the server's hard disk during processing.  (These are under the "site management" section under the adminstrator menu, also see pp.96-8 of the 4.0 documentation book.)

    You can always have these options turned back on later, if desired.

  • Lost Internet "packets"

    If you've spent a lot of time on the 'net, you probably know that sometimes pages refuse to load for an inordinate amount of time, and that hitting the refresh button can often make them load immediately.

    This is frequently due to a the fact that a "packet" has been lost because a server along the "pathway" has gone down.  If you just sit there without doing anything, nothing may ever occur.

    If you see this phenomenon occuring (once the processing actually begins), just close the little window (not the frameset), and the system will "think" that an "IE5+ timeout" has occured and will try accessing the page again.

    (If you're a programmer and wish to know more about IE5+ timeouts and how they're handled here, please see section 4.)

  • The browser isn't really dead (1)

    Pasting a large number of lines into a TEXTAREA box can make it seem as if you're browser has ceased to execute.  None of the commands may work properly, and if you use ctrl-alt-del to bring up the task manager, it may be listed as "[not responding]".

    This is a misleading impression--the browser is simply "digesting' your data, and will "come back to life" in a few minutes.

    You can experience the same phenomenon when you click on the relevant buttons that process what's been loaded into the TEXTAREA boxes.

    If you're (say) adding more than 1,000 users at a time, you may have to wait fifteen minutes or even up to an hour or two for the process to complete.  On a P2-233MMX, for example, it takes about 30 minutes to "digest" 4,000 users.

    FWIW, I know of no effective limitations on the amount of data that you can "shove" into the batch processor--I've heard of people adding as many as 12,000 users at once.

  • The browser isn't really dead (2)

    If you're going to be extracting more than (say) 5-10,000 users at a time via the "generic search" operation, you'll also see the browser appear to "die" at the end of a run.

    This is due to the fact that JavaScript is very inefficient when merging strings.  I've done the best I can to minimize these problems.

  • "Generic" searches: keep to about 6-10,000 users.

    To maximize your speed when extracting large numbers of users via the "generic" search, I suggest you use the start and stop fields to limit yourself to around 250-400 pages of results at a time.

    This limitation should affect only the largest virtual boards.

  • Accidental minimization of the main window during processing

    As I explained in section 1, you should never attempt to minimize the main window when the batch processor is running.

    But accidents do happen.

    If you find that the processing stops, and you see a JavaScript error on your window, you'll need to restart the process.

    Simply click on "OK" to close the grey JavaScript error alert, select the current operation from the top window's drop down box (not the operation type, but rather the current record that's being processed), check the restart after fail box, and then click on the GO button.

    Sometimes, you have to throw away the entire run when the window is accidentally minimized.  I.e. it will never un-minimize.  I can't do anything about this: it's a bug in IE5+ itself.

  • Can't restart--IE5+ page cannot be displayed

    In some cases, you may find that the IE5+ "page cannot be displayed" error has cropped up in the little (execution) window, and a restart is impossible.  This is a very rare occurance, but it does happen from time to time.

    The reason is that the page has been partly fetched from the server, and the batch processor has been "fooled" into thinking that the page is indeed available.

    But because the server doesn't deliver the rest of the page quickly enough for IE5+, or the phone line has been disconnected, the IE5+ "page cannot be displayed" page replaces the partly-loaded page, and a JavaScript "permission denied" error appears on the main window.

    Click on "OK" to clear the JavaScript error box.

    Then click on the little execution window, and type a control-R.  (Note: if your phone connection was broken, you should dial back in first.)

    Finally, select the current operation from the top frame's drop down box (not the operation type, but rather the current record that's being processed), check the restart after fail box, and then click on the "GO" button.

Return to table of contents

16:  User list operations: how to clone private conference memberships, e-mail subscriber lists, replicate board memberships, or extract user info. QUICKLY

WebBoard uses three very similar pages to manage the user list for private conferences, the subscribers for conference e-mail lists, and the users in a board (the latter is accessible only to WebBoard administrators).

The key here is that lists of user ID#s are the coin of the realm.

To see how this works, create a private conference (if you don't have one already), and then add a few users to the conference.

Note that in the right frame, you can extract all the user ID#s for that conference (the users have to be currently enrolled).  If you click on the show user list link and scroll down in the left frame, you'll see a place where you can paste in a list of user ID#s, to be added to this conference's user list.

In the right frame, you can also delete users by using a list of user ID#s.

Finally in the right frame, you can also extract a list of user ID#s, login names, and first/last names: this list can be formatted in "commas-and-quotes" format, or in "tab-delimited" format.

Caveat: the detailed list of users won't necessarily be accurate, if some users have placed tabs, commas, parentheses, or double quotation marks within their login names, first names, or last names.  Although WebBoard 5 and 6 are fairly careful about allowable login names, they exert very little control over first and last names.

If you're a WebBoard administrator, you can access the users on the board, by going to the administrator menu, and selecting the "Add users to board" option.  Assuming that none of the users on your board have any of those forbidden characters (tabs, commas, parentheses, or double quotes) in their login names, first names, or last names, you can also get a fairly informative user list.

Note that the useredit page, and the new user registration pages have been altered, in order to prevent these illegal characters from appearing in login names, first names, or last names.

The same hasn't been done for the Add users option: so you can manually add users with these prohibited characters in their login names, first names, or last names.  Obviously, I don't recommend doing this.

If you're a WebBoard administrator, and you want to use the detailed user info. feature, then I recommend that you use the batch processor to extract the information for all the users on the board, paste the results into a text editor, and then check for users who appear to have illegal login names, first names, or last names.  Unless you have a very large number of users, you may not find any.  If you do find some, then edit their profile information and remove those illegal characters.  From then on, you'll be able to rapidly extract detailed user information for any current board.

Return to table of contents

17:  Install (I): Complete Install Zips

If you want this documentation, as well as all the included zip files, WebBoard 4/5 users can download bu4.zip instead.  WebBoard 6 users should download bu6.zip

Once you download the complete zip file, you'll still have to follow the rest of the install instructions.

Return to table of contents

18:  Install (II): Standard Install

For WebBoard 4 and 5 users, the install file is: bu_rel4.zip.

WebBoard 5 users should use useredit5.html instead of useredit.html, and sysadmin5.html instead of sysadmin.html.  (These extra files don't exist in the WebBoard 6 install.)

For WebBoard 6, use: bu_rel6.zip.

There are two subfolders in the install, which represent two different "flavors" of the system: the bu subfolder is the standard version.  I'll discuss the contents of the bufast folder in section 19.

For now, go ahead and move the files in the standard version (the bu folder) to a test board's subfolder.  Be sure that the files on the help subfolder end up on the board's help subfolder: if no such folder exists already, you'll need to create it.

Once you install the system, log in as a WebBoard Administrator, or a virtual board manager, and go to the "more" menu.  You'll see a new option there for the User DB Batch Processor.

For the batch processor to work properly, you must be running Internet Explorer, version 5.0 (or higher).


Please note that if you've customized any of the WebBoard pages that are in my install, you'll have to either propagate your customizations to my pages, or my alternations to your customized page.  However, you may not want all the files.

Here's a list of each file, along with the role it plays in the various operations:

WebBoard name: Needed for:
asknew.html Used for replicating memberships (method #1)--WebBoard will return to this page on boards that use cookie authentication whenever the user name isn't valid.
authfail.html Used for replicating memberships--WebBoard will return to this page on boards that use basic authentication whenever the user name and/or password isn't valid.  I'm not at all sure that this works under all circumstances, so for this reason I suggest that you use cookie authentication for the "target" board (i.e. the one from which you're running the batch processor) when replicating members onto the board.
help/bu_frsh.html This is only necessary if you use the "fresh execution window" running option, which shouldn't be necessary for most users.  (See sections 3 and 15).  Note that this goes on your board's help subfolder.
help/bu_main.html This is the main module of the User DB Batch processor.  It's required for all operations.  It's invoked from the "more options" menu via the URL: "help?bu_main".
help/bu_mainc.html This is the version of the main module with remarks.  You can't actually execute it, because there are bugs in how IE5+ handles comments in JavaScript.  Nevertheless, the URL "help?bu_mainc" will bring this file up, in case you wish to save it to disk, in order to diagnose any bugs that there might be in the batch processor.  If you find a bug, please contact me  You can also fix the bug by rebuilding the source code, as explained in section 20.
bu_tag.js This is a simple tag script that returns the current version of WebBoard.  That's important, because the mechanics of deleting are different for WebBoard 4/5 vs. WebBoard 6.  And since it's a tag script, you absolutely must have it in order to run the User DB Batch processor.
confselect.html Used for replicating memberships--WebBoard brings up this page whenever the user attempts to post to an invalid conference.  The membership replication strategy attempts to post to conference 0, thus yielding this page, provided that the login name and password of the user are both correct.  At that point, WebBoard will "think" that the user has indeed logged into this virtual board.  I chose to use this page because of its small byte size.
emailpass.html Used for sending e-mail.  (This is the "e-mail me my password" confirmation page, which comes up after the user has entered a valid login and the password has been e-mailed.)
login.html Used for sending e-mail.  (This is the login page, and is only needed to detect situations in which the e-mail isn't sent because the login name is invalid.)  Note that the e-mail interface only works for boards that use cookie authentication.
This is also used for replicating memberships: WebBoard will return to this page whenever a user name is valid, but a password isn't (on board that use cookie authentication.)
more.html This is the "more options" menu, which has been changed to include a link to the User DB Batch processor.  Only WebBoard administrators and virtual board managers can "see" the link.  This page is essential for all functions  (unless you want to provide an alternative way of accessing the link: "help?bu_main").
newuser.html Blocked users from using tabs, commas, parentheses, or double quotees in their login names or first/last names when registering as new users.
newuser-e.html Same as newuser.html, above, but for boards that use e-mail address verification.
offline.html This is the "server off line" page, and it sets the WB_Batch cookie to 9.  This is treated as an IE5 timeout by the batch processor.  (For more details, see section 4).  If you never expect your server to be placed offline (i.e. paused), then you don't need to use this page.
postmsg-fx.html This is really postmsg-f.html, which is needed only for the demonstration of the "special operations" interface explained in section 22.  You don't need it, unless you're a JavaScript programmer, who intends to "hack within" the user DB batch processor.
readx.html This is really read.html, which is needed only for the demonstration of the "special operations" interface explained in section 22.  You don't need it, unless you're a JavaScript programmer, who intends to "hack within" the user DB batch processor.
select_forum.html For deleting a list of numeric user ID#s all-at-once from a private conference, or extracting user ID# lists from that conference  (or even a detailed list of users' ID#s, login names, and real names).  This is the page that comes up on the right frame when you maintain the user list for a private conference.  For more information about this page, please see section 16.
select_boardusers.html For deleting a list of numeric user ID#s all-at-once from a board's user list, or extracting user ID# lists from that board's user list  (or a detailed list of users' ID#s, login names, and real names).  This is the page that comes up on the right frame when a WebBoard administrator selects the "Add users to board" option on the administrator menu.  For more information about this page, please see section 16
select_listusers.html For deleting a list of numeric user ID#s all-at-once from the subscribers to a conference e-mail list, or extracting user ID# lists from the e-mail list's subscribers  (or even a detailed list of users' ID#s, login names, and real names).  This is the page that comes up on the right frame when you manage the mailing list for the conference.  For more information about this page, please see section 16.
sysadmin.html Allowed access to select_boardusers.html for open boards.
sysadmin5.html Same as sysadmin.html but for WebBoard 5.  This file only exists in the WebBoard 4/5 install.
useradd.html For adding users.  (This is the page that comes up when you ask to add users without the "wizard.")
userdelete.html For deleting users.  (This is the page that comes up after you've confirmed that a user should be deleted.)
useredit.html Used for changing user profile information.  There are four JavaScript code sections in this module, and one is specifically intended for you to modify if you wish to alter user profile information.  There's an example there which shows how to set the user's e-mail list option to digest format (if the user hasn't already selected digest/zipped).  For more information, please see section 10 and section 11.  Also prohibited users from using tabs, commas, parentheses, or double quotes in their login names, first names, or last names.
useredit5.html Same as useredit.html, above, but for WebBoard 5.  This file only exists in the WebBoard 4/5 install.
userlisting.html For adding a list of numeric user ID#s all-at-once to a private conference, a conference e-mail list, or the second method of replicating board membership.  (This is the page that comes up on the left frame when you ask maintain the user list for a private conference, a conference e-mail list, or adding users to a closed board.  It's also used to select conference moderators, virtual board managers, and administrators, but my code won't hurt those other functions.)
user-profile.html For looking up user profile information, on boards that use real names.  (This is the user profile display page, of course.)
user-profile-l.html For looking up user profile information, on boards that use login names.  (This is the user profile display page, of course.)
usersearch.html For "directed search" of users based on user data lists or the "generic" search.  This is WebBoard's standard search users page.
Warning:
This page won't work if the user's browser can't handle JavaScript (e.g., WebTV), or the user has JavaScript turned off.  For these users, you may wish to use <NOSCRIPT> ... </NOSCRIPT> tags to redirect them to WebBoard's Address Book page, which has identical search capabilities.  This page also has a TEXTAREA box hidden in the lower right which some may find less than completely appealing (however most users aren't likely to notice it.)

Return to table of contents

19:  Install (III): "Fast" version

As I explained in section 18, there's a second subfolder in the release files, known as bufast

This contains modified WebBoard files that can make the User DB Batch processor even more useful: but with one important caveat.

Thanks in advance for being patient, while I explain the rationale and purpose behind this additional set of files!


For reasons unknown to me, WebBoard has a tendency to "serve up" the first few bytes of each page relatively quickly, but then to "dawdle" for a while when returning the rest.  

(My theory is that WebBoard's built-in web server only delivers somewhere between 128 bytes to 1K at a time, and there's a long enough delay between each "chunk" that the communications software will "go on to the next server."  This accounts for the often-agonizing wait over phone lines which the user experiences when trying to load WebBoard pages.)

Whatever the cause, this delay matters a great deal when using the batch processor: if you're adding users over a 28.8 connection, it can mean the difference between putting 300 users through in an hour, or 1200 (i.e., as much as 4-to-1.

If you're working with fewer than 1,000 users, this isn't likely to matter in most cases.

But for larger appications, these "fast" versions can save you tremendous amounts of time when searching for users, adding users, or harvesting/changing user profile information.

The disadvantage is that none of these pages can serve their "original" function.


In at least one case (adduser.html), I see little reason to maintain WebBoard's original page, since it's faster to add users with the batch processor (unless you have just one or two, in which case you can use the "wizard"). But adduser.html is the version without the "wizard".

But for the rest of these pages, you probably do need to make some arrangments if you're to get the faster throughput available from the batch processor.

If you're a WebBoard administrator, you may wish to set up a special "technical operations" board with the "fast" versions of these pages, no conferences, and a note on the intro.html page which informs your members that the board is of no use to them.  Do not make this a closed board.  (Also see section 9).

If you're a virtual board manager, you might want to temporarily use the fast versions when doing (say) an overnight run, and then restore the standard versions after the job is complete.  A note on the intro.html page should do the trick: in my experience, people are pretty good about tolerating minor service defects, particularly when warned in advance.

In any event, if you absolutely must have a "local" search (see section 13), then you'll have to temporarily close down the search users option on that board.  You might want to redirect them to the address book page, which contains the same functionality (and more).

Note: if you're interested in maximizing effeciency even further, and you're a WebBoard administrator, or a virtual board manager who can obtain the administrator's cooperation, then I recommend that you turn off both HTTP logging and SMTP tracing--both of these options cause data to be appended quite unnecessarily to log files on the server's hard disk during processing.  (These are under the "site management" section under the adminstrator menu, also see pp.96-8 of the 4.0 documentation book.)  You can always have these options turned back on later, if desired.


Here's a list of the "fast" files on the bufast subfolder, which are different from the corresponding files on the bu subfolder of the install:

WebBoard name: Needed for:
confselect.html Used for replicating memberships--WebBoard brings up this page whenever the user attempts to post to an invalid conference.  The membership replication strategy attempts to post to conference 0, thus yielding this page, provided that the login name and password of the user are both correct.  At that point, WebBoard will "think" that the user has indeed logged into this virtual board.  I chose to use this page because of its small byte size, and the "fast" version is only about 200 bytes.
emailpass.html Used for sending e-mail.  (This is the "e-mail me my password" confirmation page, which comes up after the user has entered a valid login and the password has been e-mailed.)
useradd.html For adding users.  (This is the page that comes up when you ask to add users without the "wizard.")
useredit.html Used for changing user profile information.  There are four JavaScript code sections in this module, and one is specifically intended for you to modify if you wish to alter user profile information.  There's an example there which shows how to set the user's e-mail list option to digest format (if the user hasn't already selected digest/zipped).  For more information, please see section 10 and section 11.
user-profile-l.html
and
user-profile.html
For looking up user profile information, on both boards that use real names, as well as those that use login names.
usersearch.html For "directed search" of users based on user data lists or the "generic" search.  This is WebBoard's standard search users page.

Return to table of contents

20:  Install (IV): Rebuilding the source files, and/or making modifications

If you want to modify the system, you'll need to download the source code.  For WebBoard 4/5 users, this is in: bu_src4.zip.  For WebBoard 6 users, this is in bu_src6.zip

Note that these zip files are contained within the "complete" install download, as described in section 17.

You'll need to unzip these files into a new folder on your hard drive.

After you do so, you'll have to install my "JavaScript MakeFile" system onto that same folder: http://www.rs-freeware.org/jsm.

Within the JavaScript MakeFile system, I offer specific instructions for WebBoard users, at: http://www.rs-freeware.org/jsm/jsmdoc.htm#WebBoard_Regenerate.

In a nutshell: after installing the "JavaScript MakeFile" system, you'll have to modify the "make" file: bu.mak.

The two lines that you'll have to change are these:


DEST=f:\\w\\html\\bu
DESTF=f:\\w\\html\\bufast

DEST should have the folder name of the User DB Batch files for the "standard" install.

DESTF should have the folder name of the User DB Batch files for the "fast" install.

Note that double backslashes are required to separate folder names.

For more information about folder names, please see my DOS primer.

Return to table of contents

21:  Install (VI): Changing the defaults

If you've never written any code in a programming language before, I recommend that you find someone who has done so . . . if you wish to change any of the values below.  JavaScript, like all programming languages, is very "picky" about the "syntax" it uses.

In the event that you've never written any code in a programming language before and you can't find anyone who has, then I suggest you keep a backup copy of the main file (bu_main.txt) just in case you make a mistake.

For those of you who aren't programmers:

"Boolean" values are those which are set to either true or false.  (There should be no quotes around the value in the actual code.)

"Integer" values should be set to "whole" numbers that are greater than 0.  (I.e., don't use a decimal point or a minus sign.)

There's one item there described as a "float", which means you can use a decimal point (but don't use a negative number!).

(If you're a programmer, there are a few other values in the code which may also be modified if you wish; see the commented version which is on the help subfolder as bu_mainc.html, but do not try to install and run this version: due to a bug in IE5's interpretation of comments, it will give you a JS error.  You can rebuild the entire system by following the instructions in section 20).

Variable name: Value type (current value): Comments:
C_DefAddEmail String This is the default e-mail address used for user data list format.  It's only relevant if you're adding users.
If you change this value, it should have an at-sign in it, and a period after that, and it should be at least 7 characters long.
The installation default is:
'need.emai@ddress.com'
C_DelimDef String ('c') Field separator: 'c' is the value used for commas-and-quotes, although you should keep in mind that the version here is more restricted (see section 2).  If it's not 'c', then it's whatever field separator you wish to have, although you shouldn't use tabs, carriage returns, new lines, double quotes, or anything that will appear in the data.
C_RealNamesDef Boolean (false) For true, the box which asks whether the board uses real names is checked by default.
C_WelcEmailDef Boolean (false) For true, the box which asks whether added users get sent welcome e-mails is checked by default.
C_FindAllMatchDef Boolean (false) For true, the box which asks whether all matches should be found under a "directed" search (i.e. a search from user data lists) is checked by default.
C_FNameMatchDef Boolean (false) For true, the box which asks whether the first name in a "directed" search must match (in order for the user to be "found") is checked by default.  Only affects boards that use real names.
C_LNameMatchDef Boolean (false) For true, the box which asks whether the last name in a "directed" search must match (in order for the user to be "found") is checked by default.  Only affects boards that use real names.
C_LoginMatchDef Boolean (true) For true, the box which asks whether the login name in a "directed" search must match (in order for the user to be "found") is checked by default.  Only affects boards that use login names.
C_EMailMatchDef Boolean (true) For true, the box which asks whether the e-mail address in a "directed" search must match (in order for the user to be "found") is checked by default.  Affects all boards.
C_FailMaxDef Integer (1000) Default maximum number of failures allowable in a row (see section 3).
C_FailMaxRetry Integer (15) This affects IE5.5+ exclusively (i.e. not IE5.0), and it causes something to happen only if the number of failures-in-a-row is at least equal to this number, and 0 modulo this number.  If the number of failures-in-a-row is less than or equal to four times this number, we'll try to rewrite the execution HTML into the execution frame.  If the number of failures-in-a-row is greater than four times this number, we'll try to close the execution window, thus simulating an IE5 timeout (see section 3).
C_IE5FailResetDef Boolean (true) For true, the box which asks whether IE5 timeouts should reset the number-of-failures-in-a-row to 0 is checked by default (see section 4).
C_IE5FailDelay Integer (10) This is the number of seconds to wait after an IE5 "timeout" is detected (also see section 4).
C_FreshExeWinDef Boolean (false) This is the default status of the Fresh execution window option.  Don't use this option until you've read about how it blocks the "object doesn't support this property or method error" in section 15).
C_TickSecDef Integer (2) This is the default increment for the time to wait between clock ticks, as measured in units of C_TickSecIncr (which is normally 1).
C_TickSecIncr Float (1) See above.  This isn't a default.  If you're running on the WebBoard server or you have a very fast connection, you may wish to set this to 0.5 or even 0.25, to achieve throughputs exceeding 3,600 operations per hour.  Theoretically, on a fast server, you can perform tens or hundreds of thousands of operations per hour.
C_PwdChars String (digits plus lowercase letters) This is the character set from which any randomly generated passwords are chosen.  This is only relevant when users are being added and no password is specified (see section 5).
C_PwdLen Integer (5) Length of randomly generated passwords for added users (see section 5).
C_Unknown_Value String ('*Unknown*') Value of fields that are unknown, when producing output from a directed search.

Return to table of contents

22:  The "special operations" interface

I wrote the "special operations" interface to facilitate the performance of any repeated task with WebBoard.

This interface can only be used by a JavaScript programmer who understands enough about WebBoard to set up an automated task cycle; essentially what it does is to bring the batch processor's "overhead" and "control structure" to bear on the task-in-Q; nevertheless the programmer is responsible for organizing the data in such a way that the task can be repeatedly performed as a "batch" process.

Here are the "hook" functions that have to be written:

You can see an example of how the "generic" operations interface can be used for posting by looking at the file bu_p.js in the source file zip (that's bu_src4.zip for WebBoard 4/5 users, and bu_src6.zip for WebBoard 6 users).

To actually run this example, you'll also have to rename postmsg-fx.html to postmsg-f.html, and readx.html to read.html.

The understanding is that your JS file will be physically appended to the User DB Batch processor, and it holds both the "hook" functions described above, as well as the "data" needed to do several posts.  (The testing data is commented out.)

Each post has two variables associated with it: _PostData and _PostText.  The former is divided into 6 parts, using a delimiter of #%^&$@!# (see the SP_BldHTM function).

The global array _SubjHeadPostNos holds the "lead" post# (i.e. message #) for each thread that is to be "migrated" to the WebBoard, and a value of 0 means that the thread is new.

The global array _SubjNewHeadPostNos also holds the "lead" post# for each thread, but it's filled in as the processing goes forward (and the contents are returned in a TEXTAREA box by SP_ResultsHTM1.

This allows for incremental posting of threads, i.e. for an application in which all the posts from (say) a ListServ mailing list are to be (gradually) moved over to WebBoard.  Note that replies are always formatted as replies to the header post, so the "tree structure" of replies isn't perfectly preserved.

Note that _SubjHeadPostNos is indexed by the relative thread number (i.e. the first thread in this job cycle is 0) and the array _AbsSubjNos is indexed in the same way, but the value of the latter is the "absolute" thread number (i.e. the thread sequence number in some other location).

At EOJ, the header post number is returned for each thread in a TEXTAREA box.

Specifically, the absolute thread number is followed by a semicolon and the header post number for the thread.  These could, in turn, be passed back into whatever DBMS or other permanent data structure is used to keep track of threads (and which builds the input data lines shown at the end of bu_p.js).

While this application is complex, it demonstrates the power and flexibility of the "special operations" interface.

Return to table of contents

23:  Why client-side batch processing?  (Survey of applications)

(This section is last because it doesn't tell you anything specific about how to run the software, although it does give you a better feel for its uses.  It's mainly an "advertisement" for the technique, which many computer professionals may view as "low tech.")

Those of you who have "deep pockets" and a corresponding amount of time, energy, and patience, don't need this software.

Everyone else benefits from the fact that it presents no risk to the "integrity" of the WebBoard user database, that it's accessible to just about anyone with a certain amount of basic knowledge, and that it requires no special privileges to run (other than those already possessed by a virtual board manager).

Return to table of contents

24:  Bug reports, comments, and change requests, etc.

You can e-mail me at rog@NOSPAM_rs-freeware.org if you encounter problems, or think that you've found a bug.

I can also be reached by telephone in the U.S. from about 9 AM U.S. central time to 5 PM U.S. central time at: 765-742-6705.  If you don't get an answer, you can use my numeric pager at: 765-417-0664 ... I'll try to call you back, if you're in the U.S, Canada, or Mexico.


Please supply me with as much information as you can about your server, your version of WebBoard, any browser or Operating System that was involved, including the WebBoard Server's Operating System  (9x, NT, 2K, XP?).  Also, please provide any configuration files, or customized WebBoard files that you were using.  Keep in mind that I might actually have to have the opportunity to try what you were doing, in order to diagnose the problem.


BTW, I don't gaurantee to answer all e-mail or fix all bugs, etc.  Please remember that this is freeware, and that my time and resources are limited.

That said, I've put a lot of work into designing, coding, testing, and documenting this product, and I'm probably going to be fairly interested in any comments anyone has, fixing any bugs, and/or extending the scope to applications that strike me as being potentially valuable to a large number of users.


Naturally, if you're willing to hire me to make changes for your special-purpose application, I'm certainly willing to consider your offer.  My standard rate is $75/hr., but I may charge less if the work is to be done for a small business (less than 25 employees), or a nonprofit organization (in the latter case, I might even do it on a gratis basis  :-)  I may also consider charging you nothing if you're suggesting an improvement that I feel is of value to a large number of other users.

You can find out more about me, including references, and a list of clients/projects at: http://www.rs-freeware.org/rog.

Return to table of contents

25:  Other freeware products

WebBoard users will find lots of WebBoard freeware at: http://www.rs-freeware.org/freeware.htm.

Anyone who writes JavaScript  (regardless of whether this is written on the client or server side),  deals with SQL, or who happens to be interested in obtaining the full power available from DOS may wish to check out http://www.rs-freeware.org/free2.htm

Return to table of contents

26:  Version history information

This is version 1.2, released on Dec. 5, 2002.

On Sept. 19, 2000, I added the fresh execution window option, to block an erroneous enforcement of the "same origin policy" in some versions of IE5.

On Sept. 21, 2000, I added the model ("fast") install.

On Nov. 16, 2000, I made some changes to patch bugs in IE5.5: these were forcing some IE5.5 users to use the fresh execution window option . . . also fixed an IE5.5-related problem that was causing some form-based operations to fail (these errors were confined to the generic search).

On Feb 23, 2001, I added the second (faster) method for replicating membership.

On Nov 1, 2002, I totally revised the system to create version 1.1: new features include the source code, as well as separate support for both WebBoard 4/5 and 6.

On Dec 5, 2002, I added the features that give you even more power to manipulate the subscribers for conference e-mail lists, the memberships of private conferences, and the user lists for boards.

This documentation was generated by Rog's FAQHack: a DOS/Windows-based freeware program that handles simple macro preprocessing with special support for FAQs and other structured HTML documents.

Return to table of contents

27:  Acknowledgements

I'd like to thank Sheryl McKenna, the Information Services Department manager at the Academy of General Dentistry in Chicago, IL (USA) for her cooperation in testing and developing this product.

Alan Dean, the Computer Officer at the School of Earth Sciences in the University of Birmingham (U.K.) also assisted me greatly in helping to track down a difficult IE5 bug (and this lead to the fresh execution window option).

Don Dwiggins, of Competitive Knowledge, alerted me to some IE5.5 compatibility issues.

Kate Thylander came up with the "trick" for putting pages on the /help subfolder, which is used in the install sections.

Return to table of contents