This appendix is intended to help you understand the Microsoft Exchange Server Load Simulator program, commonly referred to as LoadSim. If you expect to use LoadSim, you should definitely read this appendix.
LoadSim is quite a powerful tool, and along with that power comes a degree of complexity. This appendix explains the major aspects of the program so you can start using it.
First, the appendix introduces the LoadSim program and explains some key concepts about it, such as user profiles and suggested number of users you can simulate with it. The idea is to give you some explanation of the when, where, and why about using the program.
Next, you walk through a sample LoadSim scenario and look at an example of how to install, configure, and run LoadSim. The sample gives you a good idea of how to actually use the program, and from there you can adapt it to your own needs.
Finally, you learn the data collection aspect. LoadSim wouldn't be useful without a way to collect data about the effects it has on Microsoft Exchange Server. You look at proven techniques for gathering accurate data.
After you finish reading this appendix, you should be able to set up LoadSim yourself on your own machines and start simulating.
Here is some common terminology referred to throughout the appendix.
Finally, you should be aware that LoadSim is not officially supported software. Although it was designed and written by Microsoft, it is shipped with Exchange Server as a tool that is used "as is."
LoadSim has a single purpose in life: to provide a way to simulate a user load on a Microsoft Exchange Server machine. That's it. Everything about LoadSim is focused on the purpose of simulating a specific number of users on an Exchange server. And if you want to do performance testing on Microsoft Exchange Server, your main goal is to disturb the system under test as little as possible. LoadSim makes this possible.
So say you want to simulate a hundred simultaneous users on a server. To do it without LoadSim, you have to have 100 identically configured Exchange client machines, each running some kind of script that regulates the activity. Then you must have a way to measure all the response times and keep track of the activities of each user.
Using LoadSim, you can accomplish this 100-user simulation with a single powerful machine. A Pentium/90 with 48MB RAM, a fast hard drive, and a good network card will do perfectly.
LoadSim does not run on the Microsoft Exchange Server machine itself. Instead, it runs on a separate machine. You don't want to run it on your server machine because running LoadSim puts extra stress on the server that isn't there normally. This disturbs the server being tested and skews the results. Again, you want to keep from disturbing the system under test as much as possible.
As clear and simple as this purpose sounds, there is much to the notion of simulating a user load in this way.
For starters, how can you actually create a load representing many users using one LoadSim client without causing a bottleneck on the machine?
If the LoadSim client is simulating multiple users on a single machine, how can you be assured that each simulated user is delivering the same load as all the others?
How many users can you realistically simulate on a single machine?
How will the test be structured so it has some bearing on the real world? For that matter, is it even important to make the test relate to the real world? Why not make a benchmark that gives a number to compare one Exchange Server machine to another?
Is it possible to configure the program to simulate a specific customized user profile?
How do you track the LoadSim user response time?
How do you collect data and make the administration of the tests manageable?
Under what circumstances do you use a program like LoadSim?
These are just some of the questions that might come to mind when you are considering a program like LoadSim. Rest assured the engineers at Microsoft, and others who beta tested LoadSim, have invested much time and effort into making LoadSim address all these issuesand more.
The next few sections cover topics that answer these questions either directly or indirectly. Hopefully, it will give you some insight into how LoadSim works.
LoadSim takes full advantage of Windows NT's capabilities to be able to reliably simulate multiple users. It is a multi-thread, multi-process program that enables a single machine running Windows NT to simulate one user or many users. Each LoadSim process has one or more threads in it, with each thread performing the task of a single LoadSim user. Because LoadSim relies on the inner workings of Windows NT to handle running each thread, you can be assured each thread, or user, gets equal attention from NT. That means each simulated user produces the same load on the targeted Exchange server.
LoadSim requires Windows NT Workstation (or Server) 3.51 or higher. It uses the Microsoft Exchange Client software for Windows NT to talk to Exchange server, just like a real Exchange user does. So you must install the Microsoft Exchange client for Windows NT on the LoadSim client before you get underway.
The same version of LoadSim also runs under Microsoft Windows 95. However, due to the way processes are implemented in Windows 95, LoadSim only supports simulating a single user. If you want to simulate multiple users with a single machine, run LoadSim under Windows NT Workstation or Windows NT Server. You can run only one instance of LoadSim for each LoadSim client machine.
By design, LoadSim does not disturb the Microsoft Exchange Server being tested. The whole idea is to make Exchange Server think it is seeing a real user load so you can measure how it performs. Because the LoadSim client is a separate machine from the Exchange Server computer, and because LoadSim uses the Exchange Client software to talk to Exchange Server, the LoadSim users are treated like real users by Exchange Server.
There are other features incorporated into LoadSim that make it act like real users. The idea is to minimize the number of performance artifacts caused by LoadSim itself. You want the load imposed to be as natural as possible.
For example, there are realistic "think times" built into the LoadSim users. These delays simulate the time a user spends typing, reading, or whatever, before they actually perform a task such as sending mail or opening a folder.
Also, the artificial user names produced by LoadSim are not in any particular pattern. This ensures there are no unfair efficiencies gained by Exchange Server processing names that are similar or follow a pattern.
Finally, there is also a random element in the LoadSim users, making the load they generate less regular so LoadSim doesn't cause predictable patterns of performance. But not so much that LoadSim won't generate reproducible results.
These are just a few features LoadSim has, but they illustrate the fine line to walk in designing such a program.
OK, enough LoadSim theory. Let's move on to some of the more practical aspects of the program.
In LoadSim, there is the concept of a user profile. The profile of a user determines how much load that simulated user places on Microsoft Exchange Server. There are three user profilesLow, Medium, and Highthat represent three classes of users which come "stock" in LoadSim. They are listed in Table 6.1.
LoadSim User Attribute |
Low |
Medium |
High | ||
LENGTH OF A DAY (hours)
|
8
|
8
|
8
| ||
READING MAIL
| |||||
New mail (times/day)
|
12
|
12
|
12
| ||
Existing mail (times/day)
|
5
|
15
|
20
| ||
AFTER READING MAIL
| |||||
Reply to it
|
5%
|
7%
|
15%
| ||
Reply All to it
|
3%
|
5%
|
7%
| ||
Forward it
|
5%
|
7%
|
7%
| ||
Move it
|
20%
|
20%
|
20%
| ||
Copy it
|
0%
|
0%0%
| |||
Delete it
|
40%
|
40%
|
40%
| ||
Do nothing to it
|
27%
|
21%
|
11%
| ||
RUN/LOAD MAIL ATTACHMENT (if one exists)
|
25%
|
25%
|
25%
| ||
INBOX SIZE LIMIT (# messages)
|
20
|
125
|
250
| ||
SENDING MAIL
| |||||
New mail (times/day)
|
2
|
4
|
6
| ||
Save a copy in Sent Mail Folder?
|
YES
|
YES
|
YES
| ||
Number of random recipients
|
3
|
3
|
3
| ||
How often to add a Distribution List
|
30%
|
30%
|
30%
| ||
Message Priority
|
Normal
|
Normal
|
Normal
| ||
Delivery Receipt?
|
No
|
No
|
No
| ||
Read Receipt?
|
No
|
No
|
No
| ||
NEW MAIL MESSAGE CONTENT
| |||||
Text-only, no attachment
| |||||
1KB body (ups1K.msg)
|
90%
|
64%
|
50%
| ||
2KB body (ups2K.msg)
|
0%
|
17%
|
10%
| ||
4KB body (ups4K.msg)
|
0%
|
4%
|
5%
| ||
1KB mail body, with attachment
| |||||
10KB attachment (ups10Kat.msg)
|
10%
|
5%
|
10%
| ||
Embedded bitmap object (upsBMobj.msg)
|
0%
|
2%
|
5%
| ||
Word attachment (upsWDatt.msg)
|
0%
|
2%
|
5%
| ||
Excel attachment (upsXLatt.msg)
|
0%
|
4%
|
5%
| ||
Embedded Excel object (upsXLobj.msg)
|
0%
|
2%
|
10%
| ||
SCHEDULE+ CHANGES
| |||||
Changes per day
|
1
|
5
| |||
Update Free/Busy information
|
No
|
No
|
No
| ||
Schedule File Size (Avg)
|
22K
|
22K
|
22K
| ||
PUBLIC FOLDER ACTIVITY
|
None
|
None
|
None
| ||
CALCULATED DAILY USER LOAD (based on defaults)
| |||||
Total mail received
|
22.94
|
66.30
|
118.89
| ||
Total mail sent
|
4.70
|
14.18
|
30.67
| ||
Mail sent as New mail
|
2.00
|
4.00
|
6.00
| ||
Mail sent as a Reply
|
1.05
|
3.76
|
13.03
| ||
Mail sent as a Reply to All
|
0.60
|
2.67
|
5.82
| ||
Mail sent as a Forward
|
1.05
|
3.76
|
5.82
| ||
Average # recipients for each mail
|
4.88
|
4.68
|
3.88 |
Don't be confused by all the attributes and their associated values. If you don't know what an entry means, don't worryit should become evident by the end of the appendix. Each of the attributes is needed in order to properly simulate an Exchange user.
These three user profiles give you an idea of the different loads you can place on Exchange Server with LoadSim. More important, you can now analyze these user loads and determine which one fits best into your organization.
The calculated daily user load figures are based on the values in the table. For example, if you increase the amount of new mail sent per day, the calculated numbers go up accordingly.
The High profile is most useful as a stress test rather than as an actual user test. It does not represent a realistic user profile because its load is so intense. However, if you need to pound on your server to test its limits or to see if something in software breaks, the High profile is a good way to do it.
The Medium profile most closely reflects users in "e-mail intensive" environments who rely on their e-mail systems. A good example of this profile is a Fortune 500 corporate e-mail user who depends upon e-mail as a regular, integral part of corporate communications. This also makes it a good practical upper limit, or worse-case scenario, for estimating the number of users LoadSim can support. In other words, if the Exchange Server can support 300 Medium users with acceptable client response time and server performance, you can plan to support 300 real users on the server in real life.
The Low profile most closely reflects those users who employ e-mail sometimes or irregularly. A good example is a user at a smaller company or at a company that has just started using e-mail. The users do not rely heavily on it yet, so their usage pattern is light. Converse to Medium users, Low represents a good practical best case scenario for many circumstances.
Of course, you aren't stuck with only three canned profiles in LoadSim. They're mainly there to get you started and to provide a common frame of reference for comparing LoadSim data to other LoadSim data. Arguably, the most powerful feature of LoadSim is that user profiles can be configured to mirror your particular users. In this sense, LoadSim is infinitely configurable. Well okay, maybe not infinitely. But there are many combinations.
For example, say your users receive about 30 e-mails per day and they sent around seven. Most of the messages are short but once in awhile, there is an attachment. Looking at the LoadSim users, you find your users are close to the low LoadSim profile. In fact, a few tweaks of parameters, and you can simulate your users very closely. All you have to do is configure LoadSim to reflect those exact characteristics, and you can generate a realistic picture of your user load. This helps you to predict server performance and assists you in selecting proper server hardware.
So your next question might be, "How many users can I simulate?" A good question. There are two answers.
First, the limit to the total number of users you can simulate is dictated only by the number of LoadSim clients you have and by the number of actual users supported by Microsoft Exchange Server. It is no problem, for example, to simulate 20,000 usersif you have enough LoadSim machines and a powerful enough Exchange Server.
Second, the actual number of users a single LoadSim client can simulate depends upon the class of hardware it's running on. As I said before, you have to be running Windows NT, so factor that in. Table 6.2 shows some typical hardware recommendations for a LoadSim client.
User Type |
Quantity per Client |
Minimum Recommended Hardware |
Low
|
Up to 50
|
486/66, 32MB RAM, 1 disk
|
Low
|
Up to 100
|
Pentium/ 60, 32MB RAM, 1 disk
|
Low
|
Up to 200
|
Pentium/66, 64MB RAM, 1 fast disk
|
Medium
|
Up to 50
|
Pentium/ 66, 32MB RAM, 1 disk
|
Medium
|
Up to 100
|
Pentium/ 90, 48MB RAM, 1 fast disk
|
Medium
|
Up to 200
|
Pentium/100, 64MB RAM, 1 fast disk
|
High
|
Up to 50
|
Pentium/ 90, 32MB RAM, 1 fast disk
|
High
|
Up to 100
|
Pentium/100, 64MB RAM, 1 fast disk |
These estimates are conservative, but I don't recommend exceeding these numbers of simulated users per LoadSim client if you want a valid user load to be produced on the Exchange Server computer.
Furthermore, if you plan to rely on the score produced by a particular LoadSim client, do not simulate more than 100 users of any profile on that client. If you attempt more users, your score will likely be skewed higher than it should be. The higher score will make Exchange Server appear as though it is performing worse than it really is. However, if you stay within the guidelines listed in Table 6.2, the user load placed on the server will still be accurate. The result? While the score on the LoadSim client is inaccurate, the Exchange Server performance data remains valid. In fact, this is a way to squeeze more simulated users out of fewer machines. See the sections, "Configuration Example 1" and "Configuration Example 2" later in this appendix for more on this idea.
Finally, you should realize that when you simulate a significantly higher numbers of LoadSim users on a single clientsay, 300 or morethe LoadSim client itself will eventually bottleneck. If this happens, LoadSim will not produce the intended load on the Exchange Server computer. The result is that both the LoadSim score and the Exchange Server performance data will be completely unusable.
Say you want to simulate 500 light (Low) users on your Microsoft Exchange Server. There are a couple of ways to go about properly configuring your LoadSim clients and gathering data. These techniques are discussed further in the section, "Techniques for Gathering Useful Data" later in the appendix.
If you want to rely on the score produced by each LoadSim client in the test, you should not exceed 100 LoadSim users per client. Therefore, according to Table 6.2 you would need five LoadSim clients, each configured with at least a Pentium/60, 32MB RAM, and a hard drive. This is an optimal configuration for testing 500 Low users, although it might not be optimal for your budget. However, with this configuration you would be able to use the score from all five LoadSim clients, so the score would be more reliable statistically. See "Use Scores from Multiple Clients" later in the appendix for more about this approach.
Alternatively, you could get by with using only three LoadSim clients for this test. Using the preceding guidelines from Table 6.2, to simulate 500 Low users you need at least three LoadSim client machines: two to support 200 users each with a Pentium/66, 64MB RAM, 1 fast hard disk; and one to support 100 users with a Pentium/60, 32MB RAM, 1 hard disk. All three LoadSim clients would place the proper load on the server, but you would use only the score from the LoadSim client running 100 users. The score produced by the other two LoadSim clients will be higher, so disregard them. See "Use a Dedicated Monitor Client" later in the appendix for more about this approach.
Bear in mind these are not hard and fast rules, but guidelines based on experience. You can always monitor your LoadSim client with PerfMon to see if it is getting overloaded. The two most crucial areas to watch on the LoadSim client are network utilization and CPU utilization, in that order. Because LoadSim is simulating usage for many Exchange users, these system resources are usually the first ones to get used up on the LoadSim client. When they do, LoadSim no longer generates the load you think it is. As a result, your performance result is skewed.
There are basically two items to observe while running LoadSim against Microsoft Exchange Server.
Observing the load placed on Exchange Server is the more complex of the two. Windows NT Performance Monitor is the best overall tool to accomplish this. There are PerfMon objects and counters that are generic to Windows NT and those that are specific to Microsoft Exchange Server. These tell a large part of the story about how your server is doing under the load.
Of course, you don't have to run LoadSim to monitor these counters. They are also useful for observing a server under a real user load. There are some pre-configured PerfMon workspaces that are installed with Microsoft Exchange Server. These are covered in more detail in Chapter 22, "Monitoring and Preventing Problems." If you need to keep an eye on things, become familiar with them.
The second task to do is analyze the score on the LoadSim client. This represents LoadSim's main performance metric. The score is simply an indication of the average LoadSim client response time, in milliseconds. This is covered later in the section called "Analyzing the LoadSim Score."
Figure A.1 depicts a LoadSim screen during execution. The screen shows a snapshot of the current score as LoadSim runs.
Figure A.1. A typical LoadSim screen during a run contains much useful information.
As you can see, there is a lot of information presented to you here.
Each line shows the activity detail for each LoadSim user. For example, in the last entry you see the date and time (hh:mm:ss) of the activity, followed by the LoadSim user name, followed by the activity detail. In this entry, at 22:58:53, the user called EXCHANGE-30 performed a task where it read one message in and deleted one message from the inbox folder.
During a LoadSim run, the default is to write all this data to log files, LOADSIM.SUM, LOADSIM.OUT, and LOADSIM.LOG. There is a utility that comes with LoadSim called LSLOG that enables you to analyze the scores from a LoadSim log. There is more on this later in the section called "Analyzing the LoadSim Score."
So far, this appendix has covered basic LoadSim topics in order to give you a foundation for the coming section. However, because the best way to learn something is usually to jump in and try it, I have set up an example scenario in the next section. Here, you look at the details of how to install, configure, and run LoadSim.
In the example, Microsoft Exchange Server is up and running. But before you deploy it and put real users on it, including your boss, you think it's a good idea to see how it holds up against the required user load. I tend to agree.
The first requirement is to ensure a single server sufficiently handles 300 users that closely match LoadSim's medium user profile. Performance is not an exact science, so close is good enough.
This example looks at the simplest case where there is only one server in the Exchange organization. There are parameters in LoadSim that enable you to do multiple server tests, but I focus on the single server scenario because that covers the majority of the parameters. If you understand those, it is an incremental step to configure LoadSim to do a multiple server test.
The other major requirement is to have response times of one second or less. This is a standard value and users generally don't get impatient if response time stays in this range.
To collect the user profile parameters, I review the ones introduced earlier in Table 6.1. Table 6.3 shows the required parameters for a medium user, which is the target. I go through the areas in LoadSim where all these parameters are set, so if you want to change some of them for your specific requirements you'll know where to do it.
A good idea is to create a blank form with each of these parameters on it. Then you can fill in the blanks with the proper parameters for your user requirements. That makes it easier to configure LoadSim for your specific user profile.
LoadSim User Attribute |
Medium | ||
LENGTH OF A DAY (hours)
|
8
| ||
READING MAIL
| |||
New mail (times/day)
|
12
| ||
Existing mail (times/day)
|
15
| ||
AFTER READING MAIL
| |||
Reply to it
|
7%
| ||
Reply All to it
|
5%
| ||
Forward it
|
7%
| ||
Move it
|
20%
| ||
Copy it
|
0%
| ||
Delete it
|
40%
| ||
Do nothing to it
|
21%
| ||
RUN/LOAD MAIL ATTACHMENT (if one exists)
|
25%
| ||
INBOX SIZE LIMIT (# messages)
|
125
| ||
SENDING MAIL
| |||
New mail (times/day)
|
4
| ||
Save a copy in Sent Mail Folder?
|
Yes
| ||
Number of random recipients
|
3
| ||
How often to add a Distribution List
|
30%
| ||
Message Priority
|
Normal
| ||
Delivery Receipt?
|
No
| ||
Read Receipt?
|
No
| ||
NEW MAIL MESSAGE CONTENT
| |||
Text-only, no attachment
| |||
1KB body (ups1K.msg)
|
64%
| ||
2KB body (ups2K.msg)
|
17%
| ||
4KB body (ups4K.msg)
|
4%
| ||
1KB mail body, with attachment
| |||
10KB attachment (ups10Kat.msg)
|
5%
| ||
Embedded bitmap object (upsBMobj.msg)
|
2%
| ||
Word attachment (upsWDatt.msg)
|
2%
| ||
Excel attachment (upsXLatt.msg)
|
4%
| ||
Embedded Excel object (upsXLobj.msg)
|
2%
| ||
SCHEDULE+ CHANGES
| |||
Changes per day
|
5
| ||
Update Free/Busy information
|
No
| ||
Schedule File Size (Avg)
|
22K
| ||
PUBLIC FOLDER ACTIVITY
|
None
| ||
CALCULATED DAILY USER LOAD (based on defaults)
| |||
Total mail received
|
66.30
| ||
Total mail sent
|
14.18
| ||
Mail sent as New mail
|
4.00
| ||
Mail sent as a Reply
|
3.76
| ||
Mail sent as a Reply to All
|
2.67
| ||
Mail sent as a Forward
|
3.76
| ||
Average # recipients for each mail
|
4.68 |
Now that you've collected the required data for your test user profile, you are ready to configure LoadSim.
LoadSim provides the capability to simulate public folder tasks. The focus in this example is configuring for mail tasks, but configuring for public folders is very similar and uses similar principles. Along the way, where appropriate, I mention specific public folder options so they aren't totally unfamiliar when you're finished.
There are two parts to installing LoadSim.
The LoadSim files are located on the Microsoft Exchange Server CD in the \SUPPORT directory. There you find a LOADSIM directory with subdirectories under it for different machine types. I assume an Intel-based LoadSim client machine, so the correct LoadSim files are in the \SUPPORT\LOADSIM\I386 subdirectory.
Copy these files to a directory on your LoadSim client's hard drive, for example, \LOADSIM. For convenience, you can make an icon for LoadSim in a location of your choice in Windows NT. I usually put it in the same group as the other Microsoft Exchange client stuff.
There are several main steps in configuring LoadSim. This section goes through them one by one to get LoadSim up and running.
When you bring up LoadSim for the first time after installing it, you see a screen like Figure A.2.
Figure A.2. The initial LoadSim opening screen looks like this.
Fortunately, the way LoadSim is structured, you go through the Configuration, Test, and Preferences menus in order to get up and running. So although there's a lot of information in some of the menus, bear that in mind and you won't go wrong.
At this point you have a blank slate with LoadSim. From here it's a matter of filling in the menus and saving your work.
The test topology is the very first thing to configure in LoadSim. This is just a fancy name for specifying the name of the server(s) under test. This is also the basis for the remainder of the test configuration.
From the Configuration menu, select Test Topology. Figure A.3 shows the resulting dialog box.
Figure A.3. The test topology is ready to be configured.
Right now it's empty, so you're going to put something in it. Select the Add Server button, and the Server Properties dialog box appears, as in Figure A.4.
You must enter the information about the Exchange Server in the Server Properties dialog box. It's self-explanatory, but realize that the organization, site, and server text boxes must be entered exactly as you configured Microsoft Exchange Server.
If you are setting up for public folder tasks, you need to put some information in the Public folders / replication section of this dialog box. In a single server test, the only items to fill in are the Total public folders and Root folders text boxes. If you have multiple Exchange Servers hosting public folders, the other boxes need to be filled in appropriately.
After you fill in this dialog box, it looks like Figure A.5.
Figure A.5. The Server Properties dialog box has been completed for our test.
After you click OK, you return to the Test Topology dialog box. The entry you just made is listed as shown in Figure A.6.
Figure A.6. The Exchange Server to be tested is now in the test topology.
Notice it shows the full directory name of the server. Also, notice the number of users shows up as well. This becomes important later because the number of users entered here dictates the number of users available when you are configuring the test. You see that later in this section.
Click OK to close the dialog box, and you are ready for the next step.
Using distribution lists (DLs) is an optional part of any LoadSim test. But because the example user profile says the user adds a DL to the address list 30 percent of the time, you need to ensure use of DLs is enabled. You don't configure the 30 percent usage herethis is only where you turn DLs on and configure their content.
Open the Configuration menu, and select Distribution Lists. Figure A.7 shows the dialog box.
Figure A.7. Distribution lists are configured through this dialog box.
You see the default is for DLs to be enabled, so leave the Use distribution lists box checked.
Also, you find some customization parameters for DLs in the Distribution list specification section.
Click OK and move to the next step.
In the example scenario, you want to simulate 300 medium users. Based on the requirements introduced earlier in Table 6.2, there are at least three LoadSim clients for this test. Although listing LoadSim clients is technically optional, it is much easier to manage the test by specifying all the clients used here. Then LoadSim takes care of distributing the 300 users among all listed clients. Otherwise, you have to set up a test configuration for each of the three LoadSim client machines and that can get unwieldy, especially if you start increasing the number of LoadSim users and LoadSim clients.
However, if you plan to use a dedicated LoadSim client for your score, as discussed later in the section "Use a Dedicated Monitor Client," you would not use this option. That's because you would not want the LoadSim users to be evenly distributed among the LoadSim clients.
LoadSim saves its settings in a .SIM file in the directory of your choice. Using this option, all three LoadSim clients share the same .SIM file. If you don't enable this option, you have to configure a .SIM file for each LoadSim client.
Open the Configuration menu, and select Client Machines. Figure A.8 shows the resulting dialog box.
Figure A.8. Multiple LoadSim clients can be configured and managed easily.
Now it's time to enter your LoadSim clients. Click the Use multiple client machines check box to enable this feature, and then click the Add Client(s) button. This brings up a dialog box as shown in Figure A.9. Type in the name and click OK.
Note the entry in the site list box at the bottom is the same as the one you listed in the server topology before. It is also the same as the client site list box in Figure A.8.
Figure A.9. Adding LoadSim clients is a matter of typing in the names.
Call the LoadSim client machines something creative, like LOADSIM1, LOADSIM2, and LOADSIM3. When you install Windows NT on your LoadSim clients, these are the computer names you must use. The NT user name it logs in as does not matter. When you run LoadSim, the program checks the names in this list against the Windows NT computer name running LoadSim to see if the computer is supposed to be in the test. If it matches, you're off and running. If it does not match, a dialog box appears informing you that the LoadSim client is not supposed to participate in the test, and LoadSim terminates.
When you finish, the dialog box looks similar to Figure A.10.
Figure A.10. LoadSim is configured for multiple client machines.
All the LoadSim clients are listed, each with both a site entry matching the site name of the Exchange Server and an associated weighting. Now you're ready for the next step. Click OK and move on.
The weighting is used to determine how LoadSim distributes the load to the LoadSim clients. In the example, you have configured three LoadSim clients to simulate 300 users. Because you have enabled Use multiple client machines and have given them each a weighting of 1, LoadSim divides the users equally among the three machines. However, if you had set the weighting to 2 for LOADSIM1, and left it 1 for both LOADSIM2 and LOADSIM3, LOADSIM1 gets more users than the others. When the test runs, LoadSim assigns LOADSIM1 half of the users, 150, and LOADSIM2 and LOADSIM3 each one quarter of the users, 75. Use this if you have unequally equipped LoadSim clientsin this case, one powerful machine to handle 150 and two lesser machines to handle 75.
Having said all that, I rarely use this feature because I find it better if all LoadSim machines are configured as equally as possible. One less variable to worry about that way.
The next thing to do is create and import the user list. LoadSim does not use any users or mailboxes you might have created in your Exchange Server. Instead, it creates its own user names, mailboxes, and (optionally) distribution lists. It also places these in its own recipient container in the Exchange site called LoadSim. That keeps things nice and tidy so as not to interfere with any existing names.
Open the Configuration menu, and select Generate Directory Import Files. There is no further configuration to do; the files are generated automatically. Figure A.11 shows the output.
Figure A.11. Three directory import files are created by LoadSim.
The directory import should happen almost instantaneously in this case, because you are only importing 300 users. Even if you are doing more users the process is still very fast.
After the import completes, three files are created and listed on the screen, each of which is a .CSV (comma-separated variable) ASCII text file. These files are the appropriate format for importing into the Microsoft Exchange Server directory. They are located in the directory where you installed LoadSim.
Now you are ready to import the files. There are two ways to do this:
Both methods produce the same result, but with the second method you don't have to worry about setting site permissions for the LoadSim client to allow the import.
Providing you don't change the test configuration, each time you run Generate Directory Import Files, it creates identical output files. For example, if you run it for 300 users, then again for 400 users, the first 300 entries for both sets of files are identical. This is because LoadSim uses a consistent algorithm to generate user names. If you examine the .CSV file you see user names derived from the Exchange Server name. It's usually best to generate the maximum number you plan to use, import them all, then run your tests.
All right, you're ready to leave the Configuration menu and move to the Test menu. This is the heart of LoadSim and once you're up and running, it is probably the menu you use the most.
Now that there are names in the Exchange Server directory, you need to "user initialize" (UserInit) the database. Currently, there is nothing in the mailboxes of the users who are imported, so UserInit is a way to populate the mail databasecalled the private store or PRIV.EDBwith some data. This simulates having mail in each mailbox in the database as though the LoadSim users have been using it all along.
If you recall in the profile, the LoadSim users read existing mail 15 times a day. The UserInit process is where that existing mail comes from. Also, having some data in the database doesn't give the Exchange store an unfair performance advantage of dealing with an empty database.
Running UserInit adds data to PRIV.EDB. If you plan to run LoadSim against a live database, be aware that this adds data to your live database in the LoadSim container. You can always delete the data, but just be aware of it.
Open the Test menu, and select User Initialization Parameters. You see a dialog box with some tabs on it as in Figure A.12.
Figure A.12. The User Initialization dialog box.
Basically, when you run UserInit, the LoadSim client begins methodically stuffing mailboxes on the server with mail according to the parameters specified in these dialogs.
The Parameters tab is the main tab to focus on. The other tabs contain information very similar to that found in tabs covered in the next section, "Creating and Configuring a LoadSim Test." So most of them are self-explanatory after reading the next section.
Number of processes per server is a bit misleading because it doesn't have anything to do with Exchange Server. Rather, it dictates how many simultaneous LoadSim users log on and run from this LoadSim client during UserInit. Leave it at 2 or some value less than 5. Making it higher doesn't help things go faster. In fact, it can make the UserInit process take longer.
The Mailbox setup section of this tab has to do with configuring how the LoadSim user's mailbox looks. Click any of the three usage buttons to change the numbers in this section. Because you are doing a medium user run, click Medium Usage. That is where the numbers that appear in Figure A.12 come from.
Under the Test menu, there is a corresponding entry for public folder initialization, Public Folder Initialization Parameters. This is where you configure how the public folders are initialized. The menus are straightforward and self-explanatory.
At this point, save the simulation configuration. From the File menu, select Save and you are presented with the usual dialog box. Give the file a name. For this example, name it LSTEST.SIM. Now your .SIM file is saved for later use.
Notice the default is not to save the file in the LoadSim directory. Instead, the default is the %SystemRoot%\SYSTEM32 directory. If you think your .SIM file is lost, look there first.
Now it's time to run UserInit. Open the Test menu, click Run, and click UserInit. LoadSim starts initializing the database, two users at a time. You can run this on a single LoadSim client, walk away, and come back when it's finished.
At the Medium Usage setting, UserInit increases the PRIV.EDB database by about 2MB for each user. So for 300 users, expect to end up with about a 600MB database. The Light Usage setting is about 500KB for each user.
Configuring a test is the central part of LoadSim. So pay attention here.
From the Test menu, select Load Simulation Parameters. LoadSim tells you there are no user load tests specified and asks if you want to create a new one. Click Yes and fill in the name of a test. For this example, use LSTEST.
Figure A.13 shows the dialog box for configuring the user profile properties used in the test.
Figure A.13. The main dialog box for configuring LoadSim tests.
There are several tabs on this dialog box but most of the parameters in the tabs is left as default. You only need to change them when you want to customize something. Some of the tabs pertain to public folders, which you aren't concerned with in this test.
Probably the best way to understand all the tabs is to go through them and discuss each one.
The first tab, shown in Figure A.13, is called General. It is a very important tabone you see often if you use LoadSimand it always comes up first. This is where you select which test you want to configure, where you create new tests, and where you set overall test properties.
Note that if your length of day is supposed to be 8 hours, select For 8 hours in the Test duration option. After 8 hours, LoadSim will automatically shut down. If you choose 4 hours here, for example, LoadSim crams an 8-hour day into 4 hours, effectively doubling the stress your LoadSim users impose on the server. Alternatively, you could select Forever, and you would have to shut down LoadSim manually. However, the load imposed by LoadSim would be correct.
Next is the Logoff tab. Most of the time it is sufficient to leave the settings default. This is where you set the duration of the artificial day and night and how to log on and off.
Figure A.14 shows the Logoff tab.
Figure A.14. The Logoff tab of User Profile Properties.
Next is the Read Mail tab. This is a key section that controls part of the user load. The numbers in these boxes directly correspond to the values in the user profile that started with the user profile in Table 6.3. They also change if you press the low, medium, and high usage buttons in the General tab.
Figure A.15 shows the Read Mail tab.
Figure A.15. The Read Mail tab of User Profile Properties.
Next is the Send Mail tab. This is another key tab in configuring the user load. The values in these boxes directly correspond to the values in the user profile. They also change if you press the low, medium, and high usage buttons in the General tab.
Figure A.16 shows the Send Mail tab.
Figure A.16. The Send Mail tab of User Profile Properties.
Note that the upper limit of 300 is derived from the number of users originally specified in the server topology.
Next is the Schedule+ tab. This tab is very simple and it doesn't require much attention. It is self-explanatory especially if you are familiar with Microsoft Schedule+.
Figure A.17 shows the Schedule+ tab.
Figure A.17. The Schedule+ tab of User Profile Properties.
If you uncheck the Schedule+ Actions Task Option in the General tab, these settings are disregarded.
Next is the Public Folders tab. The settings in this tab are disregarded in the test because you didn't check the Public Folders Task Option in the General tab.
Figure A.18 shows the Public Folders tab.
Figure A.18. The Public Folders tab of User Profile Properties.
Most of the parameters function similar to sending mail. For example, there is the number of times each day to open public folderssort of like how many times to send new mailalong with actions for browsing and for after reading.
If you uncheck the Public Folder Actions Task Option in the General tab, these settings are disregarded.
Next is the Test Report tab. This tab is different from all the others because there are no user configurable settings here. Its purpose is informational.
Figure A.19 shows the Test Report tab.
Figure A.19. The Test Report tab of User Profile Properties.
The values shown here represent the load each LoadSim user in the test produces. The numbers are calculated based on your entries throughout the LoadSim configuration process, and they appeared at the bottom of the user profile in Table 6.3.
For example, if you click the Light Usage button on the General tab, these numbers change. Or if you change the average number of recipients in the Distribution lists dialog box, the numbers change. There are several parameters in LoadSim that affect the calculations on this page.
Next is the Messages tab. This tab is also a bit different from the others; the only other one like it is the Pub Fld Msgs tab. Because they work identically, this explanation applies to both tabs.
Figure A.20 shows the Messages tab.
Figure A.20. The Messages tab of User Profile Properties.
When a user composes an e-mail or sends an attachment, these are the source files that are used. Basically LoadSim pulls them into the mail message when it needs them.
How does LoadSim know when it needs a file? If you notice, each file is assigned a weighting at the bottom. As you highlight each file in the list, its associated weighting appears. This determines how often the file is used in e-mail.
Each file was originally created using the Microsoft Exchange Client and it is saved in message format. This format is specific to Microsoft Exchange and it contains everything in the mail message including text, formatting, attachments, embedded objects, and so on. With this in mind, you don't have to explicitly specify to LoadSim how many messages have attachments, embedded objects, rich text, and so on. Because all these attributes are included in the .MSG file, specifying a weighting for the file implicitly handles this all in one location.
If you add the weightings for all the .MSG files, they should equal 100. They don't have to, but that makes it easy to figure the percentage of how often each file is used by LoadSim. This percentage maps directly to the original user profile parameters in Table 6.3.
If you look closely in this case, the default medium weightings actually add up to 95, but assume they add up to 100 for simplicity's sake.
You can use your own files if you want. Simply create them with the Microsoft Exchange Client and save them in message file format. Then add them into this list with the Add File(s) button and assign them a weighting.
Next is the Pick Senders tab. This tab is probably the most important as to how the test runs. Its contents determine how many actual LoadSim users are simulated in this LoadSim test, regardless of the number of LoadSim clients. It also controls which users send e-mail.
Figure A.21 shows the Pick Senders tab.
Figure A.21. The Pick Senders tab of User Profile Properties.
Let me repeat that. The contents of this tab determine how many actual LoadSim users are simulated in the entire LoadSim test for all LoadSim clients, regardless of whether you have one LoadSim client or twenty. That is, assuming Use multiple client machines is selected.
This is not real obvious to the casual observer. You might have been expecting a parameter somewhere that says, Number of users to simulate. There's not onethis is where you specify it. More folks get confused about this than anything else.
So, in this example you see the full server name listed in the top list box along with the total number of users specified in the Server topology section. Below, you see all 300 users selected (0-299) on the Exchange Server called Exchange. Because you specified three LoadSim client machines earlier, these 300 LoadSim users are divided between the three LoadSim clients based on the weightings assigned to each client. Because all three have equal weightings of 1, they are each automatically assigned 100 LoadSim users to simulate. This concept is very important to understanding how to configure a LoadSim test.
Just for another example, if you had four LoadSim clients each one would be assigned only 75 LoadSim users. Furthermore, if you didn't check the Use multiple client machines option in the Client Machines dialog box, all 300 would attempt to run on a single LoadSim client. That's not what you want.
Next is the Pick Recipients tab. This tab is also very important as to how the test runs. Its contents determine which users receive e-mail from the senders.
Figure A.22 shows the Pick Recipients tab.
Figure A.22. The Pick Recipients tab of User Profile Properties.
Unlike the Pick Senders tab, this tab does not have anything to do with the number of LoadSim users that are simulated. Instead, it specifies how many users receive e-mail from the senders. However, it is usually a good idea to have the range selected here match the range selected in Pick Senders. If you think about it, that is the way it is in real life. For example, if a company has 300 employees, all 300 employees can send mail to any of the 300 employees.
The Active Pub Flds tab functions the same as the previous two tabs, and you must have specified a number of public folders earlier in the Server topology configuration to use it. Similarly, the Pub Fld Msgs tab works the same as the Messages tab.
Whew, you made it. That finally brings you to the end of the User profile property tabs. Now move to the LoadSim options and start wrapping this up.
You're almost finished configuring the test, but take a minute and review the LoadSim program options. You might find some of these useful.
There are two entries under the Preferences menu that enable you to configure various LoadSim parameters. These values are stored in LOADSIM.INI, which is kept in the %SystemRoot% directory.
The first is Options, as shown in Figure A.23.
Figure A.23. Set general LoadSim options using this dialog box.
Here you can specify general operating parameters for LoadSim.
The second is Logging, as shown in Figure A.24.
Figure A.24. Set LoadSim logging options using this dialog box.
Now you are completely through configuring LoadSim, so don't forget to save your work. Here's an overview of where LoadSim keeps different information.
A .SIM file contains the specific test information, such as the number of LoadSim clients, names of tests, test parameters, etc. It's format is like that of an .INI file. The filename doesn't need to be the same as a test name. For example, you can have a test name of MYTEST saved in a .SIM file called LSTEST.SIM. LoadSim defaults to saving this file in %SystemRoot%\SYSTEM32, but you can save it anywhere.
.SUM, .LOG, and .OUT files contain the information described earlier. They are usually stored in the same directory as the LoadSim program files.
LOADSIM.INI contains overall program settings set in the Options and Logging menus such as the number of boxcars, maximum simultaneous logons, recently opened files, and so on. Note this file is not stored in the LoadSim directory; it is stored in %SystemRoot%.
Now you're ready to run the test. Assuming your Exchange Server is up and running, it should go OK.
Open the Test menu, select Run, and then LSTEST. LSTEST is the name given to the test when it was created. LoadSim starts churning away, and you see processes start up and LoadSim users start to log on. This activity shows as increasing numbers on the screen beside Processes Started and Clients Logged On. There isn't any other activity on the screen at first unless LoadSim needs to create Exchange profiles for its users. When all the users are logged in and profiles are created, you might see something like Figure A.25.
Figure A.25. All the LoadSim users are logged in and about to start their tasks.
Depending on what you want to accomplish, you let the test run for a fixed amount of time. If you want reliable scores, let LoadSim run for at least five hours. Then you can use the LSLOG utility to parse the LoadSim logs. The next section goes into this in more detail.
By now, at least five hours have passed and your LoadSim run is over. Now it's time to analyze the data LoadSim has gathered so you can see how the Exchange Server performed.
LSLOG is the utility to parse the LoadSim logs and give you the score information for the run. It is included with LoadSim as one of the program executables. See Appendix B, "Command Reference" for full command syntax of LSLOG.
By default, when you run LSLOG against a LoadSim log, it shows you the 95th percentile score. So if you are simulating 100 users and the score shown by LSLOG is 1100, that means 95 of the simulated users are experiencing an average response time of 1.1 seconds or better.
It is best to use LSLOG to produce a score if you're trying to capture accurate, response-time performance data for a LoadSim run. Although there is a score on the LoadSim screen while it is running, that score is not as accurate as the score derived from the LoadSim logs by LSLOG.
This becomes useful if you want to see how fast Exchange Server is servicing client requests. Furthermore, if you impose performance goals of sub-second client response time, you expect the score to show around 1000 or less during a LoadSim run.
Figure A.26 shows a typical output of LSLOG.
Figure A.26. LSLOG provides a summary of the LoadSim score for a run.
Assuming this is the output for the test run, you can see from the Weighted Avg line at the bottom that the 95th percentile score is 270. That is to say that 95 percent of the users can expect to experience a response time of .27 seconds or less. Reality translates this into saying most users have less than half second response time.
This figure is well below the 1000 imposed in the example criteria, so the conclusion from the run is that the Exchange Server is providing adequate response time for the users. Assuming there is nothing weird in the NT Performance Monitor data on the Exchange Server computer, you are ready to deploy 300 users on the serverwith confidence.
There are other scores listed here. They are broken out for each activity type in LoadSim. You can see the weighting each task is given and the number of times the task occurred (hits), as well as the respective 95th percentile score. The final average is most definitely a weighted average.
About LoadSim Data
Due to both the ways LoadSim operates and the fact it is simulating many clients, allow LoadSim to run for more than an hourpreferably five hoursbefore using the score as a meaningful number. If you throw away the first hourreferred to as hour 0and use the second through fourth hours (hours 1-3), the scores are the most usable. LSLOG makes this process easy. See Appendix B for the syntax.
Even so, there is still some margin of error in the score. Typically, the longer you let LoadSim runup to a point the more accurate the score is. Identical runs don't produce identical scores, but they are within a hundred milliseconds. Often they are within 50 or less. The theoretical minimum score (depending on the Exchange Server hardware) is in the 100-200 range.
As LoadSim runs, conditions tend to degrade over long periods of time. For example, if you allow a test to run for days on end, you see fluctuations in the response time trends. The idea in taking the first five hours of data is to get a good sampling of how the system is expected to run with no degraded conditions.
There are different ways to use LoadSim to gather data that are useful to you. Three are summarized in this section, but as you work with LoadSimor even as you read thisyou might think of other methods. Try them out and see what works for you.
One technique for gathering accurate scores is to run multiple, identically configured LoadSim clients. The idea is to ensure each LoadSim client is producing accurate scores, then average them for more accuracy.
Using the example of testing 300 users and based on the recommendations in Table 6.2, you would need three LoadSim clients. Each machine is a Pentium/90 with at least 48MB RAM and a fast disk. Each LoadSim client can easily handle 100 medium users, and the score reflected is reliable because the LoadSim client is not bottlenecked.
Run LSLOG as explained, and take the average score produced by each LoadSim client to produce the final score for the run. If things are running correctly, each score should be within 50 points, plus or minus, of the average.
This technique is probably the best, but it requires a lot of hardware resources, especially if you intend to simulate many clients. It is also more trouble to keep track of all the .LOG files as the number of LoadSim clients increases. However, the scores produced using this technique are probably as accurate as they can be.
Another technique for gathering accurate LoadSim scores is to dedicate a LoadSim client as a score machine. The trick here is to ensure the LoadSim client is not overloaded. In fact, it should be under-loaded. This machine is always assured of having a fair score regardless of whether the other LoadSim clients are overloaded.
This technique is based on the idea that although a heavily loaded LoadSim client might not produce an accurate score, it still produces an accurate load. Up to a point this is true. That is, you probably can't simulate 500 users on a single LoadSim client, but you can squeeze more users out of a LoadSim client if you don't care about the score the client produces.
For example, run a test with multiple LoadSim clients, each one simulating a number of users greater than Table 6.2 recommendsbut still a reasonable amount. In the example scenario with 300 medium users, simulate 250 of the 300 users with two slightly lesser machines, say Pentium/66 machines with at least 32MB RAM and a fast disk. They don't even have to be configured the same. That's 125 users for each LoadSim client rather than 100. Then, with a separate LoadSim client, say a Pentium/100 with 64MB RAM and a fast disk, simulate the remaining 50 users. This is the monitor client. Only use the score from this monitor client and disregard the scores from the other two.
This technique is useful if you don't have quite enough hardware to configure multiple identical high performance LoadSim clients. Because the LoadSim clients that produce the bulk of the load don't have to be identical, and because you only need to have one high performance machine from which to get the score, this is a good option. However, you don't get the benefit of averaging scores. Still, scores compared from identical runs should be within 100 points or so.
Of course, the most useful piece of information that can be gleaned from this entire exercise is the number of users a server supports. That's the question everyone wants to answer anyway. So use the techniques outlined and plot your own graph.
Taking the sample scenario one step further, say you want to see how much "headroom" the server has. You already know the example Exchange Server yields a score of 270 for 300 medium users. But how does it do all the way up to 800 users? Simple. Reconfigure the test for 100 users, repeat the process to get another data point, and so on until you test through 800 users. Skip 300, of course, because you already have it. Then plot the data points. The resulting graph might look like Figure A.27.
Figure A.27. A typical users per server graph might look like this.
You can get information from this at a glance. It's clear to see that the current hardware on the Exchange Server easily takes you to 600 users before it starts encroaching the sub-second response time requirement.
These data points for a users per server graph take time to generate, but it's a good picture of how the server performs. And combined with the Exchange Server's PerfMon data from each run, you can begin to spot trends for which subsystems in the server are beginning to get used up. Then you're well on your way to mastering the performance of your server.
One final word about conducting your performance tests. Stay consistent. If you select a method of gathering data, producing scores, or running tests, stay with it. This is the single most important thing. Any variables introduced into the system can cause havoc and produce results that have you scratching your head. Remember, performance is not an exact science. There's some subjectivity to it as well. If something does not make sense, it's probably because something is wrong somewhere. Run the test again just to be sure. That's always much better than proceeding on a false premise. And if the results look good, and they're repeatable, you are probably on the right track. Good luck!
This appendix covers a lot of ground.
You see that LoadSim is quite a powerful tool, but along with that power comes a degree of complexity. However, it is plain that if you are willing to invest the time and effort, the results can be quite useful. You also see that LoadSim is not really as complex as it appears at first glance. After examining the program, each item is there for a reason and it makes sense.
After finishing this appendix, you are ready to set up LoadSim on your own machines and start running simulations with minimal problems. As an additional resource, you could refer to a helpful Microsoft white paper that covers LoadSim and the concept of users per server. The paper is titled, "Performance: Concurrent Users per Server," and it is available beginning with the May 1996 edition of Microsoft TechNet.