Web Page Rendering

Below are the step-by-step descriptions of how page get displayed in your browser when you simply enter some URL in address bar:

  1. You type an URL into address bar in your preferred browser.
  2. The browser parses the URL to find the protocol, host, port, and path.
  3. It forms a HTTP request
  4. To reach the host, it first needs to translate the human readable host into an IP number, and it does this by doing a DNS lookup on the host
  5. Then a socket needs to be opened from the user’s computer to that IP number, on the port specified (most often port 80)
  6. When a connection is open, the HTTP request is sent to the host
  7. The host forwards the request to the server software (most often Apache) configured to listen on the specified port
  8. The server inspects the request (most often only the path), and launches the server plugin needed to handle the request (corresponding to the server language you use, PHP, Java, .NET, Python?)
  9. The plugin gets access to the full request, and starts to prepare a HTTP response.
  10. To construct the response a database is (most likely) accessed. A database search is made, based on parameters in the path (or data) of the request
  11. Data from the database, together with other information the plugin decides to add, is combined into a long string of text (probably HTML).
  12. The plugin combines that data with some meta data (in the form of HTTP headers), and sends the HTTP response back to the browser.
  13. The browser receives the response, and parses the HTML (which with 95% probability is broken) in the response
  14. A DOM tree is built out of the broken HTML
  15. New requests are made to the server for each new resource that is found in the HTML source (typically images, style sheets, and JavaScript files). Go back to step 3 and repeat for each resource.
  16. Stylesheets are parsed, and the rendering information in each gets attached to the matching node in the DOM tree
  17. Javascript is parsed and executed, and DOM nodes are moved and style information is updated accordingly
  18. The browser renders the page on the screen according to the DOM tree and the style information for each node
  19. You see the page on the screen

And we get annoyed why the response time is very high but now at least I have some documentation to look at, while waiting the remaining fractions of a second before the page renders.

Splunk Overview

Splunk is powerful and versatile IT search software that takes the pain out of tracking and utilizing the information in your data center. If you have Splunk, you won’t need complicated databases, connectors, custom parsers or controls–all that’s required is a web browser and your imagination. Splunk handles the rest.
Use Splunk to:

  • Continually index all of your IT data in real time.
  • Automatically discover useful information embedded in your data, so you don’t have to identify it yourself.
  • Search your physical and virtual IT infrastructure for literally anything of interest and get results in seconds.
  • Save searches and tag useful information, to make your system smarter.
  • Set up alerts to automate the monitoring of your system for specific recurring events.
  • Generate analytical reports with interactive charts, graphs, and tables and share them with others.
  • Share saved searches and reports with fellow Splunk users, and distribute their results to team members and project stakeholders via email.
  • Proactively review your IT systems to head off server downtimes and security incidents before they arise.
  • Design specialized, information-rich views and dashboards that fit the wide-ranging needs of your enterprise.

Index new data

Splunk offers a variety of flexible data input methods to index everything in your IT infrastructure in real time, including live log files, configurations, traps and alerts, messages, scripts, performance data, and statistics from all of your applications, servers, and network devices. Monitor file systems for script and configuration changes. Enable change monitoring on your file system or Windows registry. Capture archive files and SNMP trap data. Find and tail live application server stack traces and database audit tables. Connect to network ports to receive syslog and other network-based instrumentation.
No matter how you get the data, or what format it’s in, Splunk indexes it the same way–without any specific parsers or adapters to write or maintain. It stores both the raw data and the rich index in an efficient, compressed, filesystem-based datastore–with optional data signing and auditing if you need to prove data integrity.

Search and investigate

Now you’ve got all that data in your system…what do you want to do with it? Start by using Splunk’s powerful search functionality to look for anything, not just a handful of predetermined fields. Combine time and term searches. Find errors across every tier of your IT infrastructure and track down configuration changes in the seconds before a system failure occurs. Splunk identifies fields from your records as you search, providing flexibility unparalleled by solutions that require setup of rigid field mapping rulesets ahead of time. Even if your system contains terrabytes of data, Splunk enables you to search across it with precision.

Capture knowledge

Freeform searching on raw data is just the start. Enrich that data and improve the focus of your searches by adding your own knowledge about fields, events, and transactions. Tag high-priority assets, and annotate events according to their business function or audit requirement. Give a set of related server errors a single tag, and then devise searches that use that tag to isolate and report on events involving that set of errors. Save and share frequently-run searches. Splunk surpasses traditional approaches to log management by mapping knowledge to data at search time, rather than normalizing the data up front. It enables you to share searches, reports, and dashboards across the range of Splunk apps being used in your organization.

Automate monitoring

Any search can be run on a schedule, and scheduled searches can be set up to trigger notifications or when specific conditions occur. This automated alerting functionality works across the wide range of components and technologies throughout your IT infrastructure–from applications to firewalls to access controls. Have Splunk send notifications via email or SNMP to other management consoles. Arrange for alerting actions to trigger scripts that perform activities such as restarting an application, server, or network device, or opening a trouble ticket. Set up alerts for known bad events and use sophisticated correlation via search to find known risk patterns such as brute force attacks, data leakage, and even application-level fraud.

Analyze and report

Splunk’s ability to quickly analyze massive amounts of data enables you to summarize any set of search results in the form of interactive charts, graphs, and tables. Generate reports on-the-fly that use statistical commands to trend metrics over time, compare top values, and report on the most and least frequent types of conditions. Visualize report results as interactive line, bar, column, pie, scatterplot and heat-map charts.

Searching in Splunk

The first time you use Splunk, you’ll probably start by just searching the raw data to investigate problems — whether it’s an application error, network performance problem, or security alert. Searching in Splunk is free form — you can use familiar Boolean operators, wildcards and quoted strings to construct your searches. Type in keywords, such as a username, an IP address, a particular message… You’re never limited to a few predetermined fields and you don’t need to confront a complicated query builder, learn a query language, or know what field to search on. You can search by time, host and source.

Go to the Search app

After logging into Splunk, you will see either the Welcome view or Splunk Home view.

  • If you’re in the Welcome view, select Launch search app.
  • If you’re in Splunk Home, select Search.
  • If you are in another app, select the Search app from the App menu, which is located in the upper right corner of the window.

This takes you to the Summary dashboard of the Search app. For more information about what you will find in the Search App.

Start with simple terms

To begin your Splunk search, type in terms you might expect to find in your event data. For example, if you want to find events that might be HTTP 404 errors, type in the keywords:

http 404

Your search results are all events that have both HTTP and 404 in the raw text; this may or may not be exactly what you want to find. For example, your search results will include events that have website URLs, which begin with "http://", and any instance of "404", including a string of characters like "ab/404".
You can narrow the search by adding more keywords:

http 404 "not found"

Enclosing keywords in quotes tells Splunk to search for literal, or exact, matches. If you search for "not" and "found" as separate keywords, Splunk returns events that have both keywords, though not necessarily the phrase "not found".
You can also use Boolean expressions to narrow your search further.

Add Boolean expressions

Splunk supports the Boolean operators: AND, OR, and NOT; the operators have to be capitalized. You can use parentheses to group Boolean expressions. For example, if you wanted all events for HTTP client errors not including 404 or 403, search with:

http client error NOT (403 OR 404)

In a Splunk search, the AND operator is implied; the previous search is the same as:

http AND client AND error NOT (403 OR 404)

This search returns all events that have the terms "HTTP", "client", and "error" and do not have the terms "403" or "404". Once again, the results may or may not be exactly what you want to find. Just as the earlier search for http 404 may include events you don’t want, this search may both include events you don’t want and exclude events you want.
Note: Splunk evaluates Boolean expressions in the following order: first, expressions within parentheses; then, OR clauses; finally, AND or NOT clauses.

Search with wildcards

Splunk supports the asterisk (*) wildcard for searching. Searching for * by itself means "match all" and returns all events up to the maximum limit. Searching for * as part of a word matches based on that word.
The simplest beginning search is the search for *. Because this searches your entire index and returns an unlimited number of events, it’s also not an efficient search. We recommend that you begin with a more specific search on your index.
If you wanted to see only events that matched HTTP client and server errors, you might search for:

http error (40* OR 50*)

This indicates to Splunk that you want events that have "HTTP" and "error" and 4xx and 5xx classes of HTTP status codes. Once again, though, this will result in many events that you may not want. For more specific searches, you can extract information and save them as fields.

Search with fields

When you index data, Splunk automatically adds fields to your event data for you. You can use these fields to search, edit the fields to make them more useful, extract additional knowledge and save them as custom fields. For more information about fields and how to use, edit, and add fields.
 Splunk lists fields that it has extracted in the Field Picker to the left of your search results in Splunk Web. Click a field name to see information about that field, add it to your search results, or filter your search to display only results that contain that field. When you filter your search with a field from the Field Picker, Splunk edits the search bar to include the selected field.
Alternately, you can type the field name and value directly into your search bar. A field name and value pair can be expressed in two ways: fieldname="fieldvalue" or fieldname=fieldvalue.
Note: Field names are case sensitive.
Let’s assume that the event type for your Web access logs is eventtype=webaccess and you saved a field called status for the HTTP status codes in your event data. Now, if you wanted to search for HTTP 404 errors, you can restrict your search to the specific field:
status=404

Use wildcards to match multiple field values

If you’re interested in seeing multiple values for the status field, you can use wildcards. For example, to search for Web access events that are HTTP client errors (4xx) or HTTP server errors (5xx), type:

eventtype=webaccess status=40* OR status=50*

Use comparison operators to match field values

You can use comparison operators to match a specific value or a range of field values.

Operator

Example

Result

=

field=foo

Field values that exactly match "foo".

!=

field!=foo

Field values that don’t exactly match "foo".

<

field<x

Numerical field values that are less than x.

>

field>x

Numerical field values that are greater than x.

<=

field<=x

Numerical field values that are less than and equal to x.

>=

field>=x

Numerical field values that are greater than and equal to x.

Note: You can only use <, >, <=, and >= with numerical field values, and you can only use = and != with multi-valued fields.

P.S. Thanks to Manik

HTTP Status Code Definitions

Each Status-Code is described below, including a description of which method(s) it can follow and any metainformation required in the response.

Informational HTTP 1xx

This class of status code indicates a provisional response, consisting only of the Status-Line and optional headers, and is terminated by an empty line. There are no required headers for this class of status code. Since HTTP/1.0 did not define any 1xx status codes, servers MUST NOT send a 1xx response to an HTTP/1.0 client except under experimental conditions.
A client MUST be prepared to accept one or more 1xx status responses prior to a regular response, even if the client does not expect a 100 (Continue) status message. Unexpected 1xx status responses MAY be ignored by a user agent.
Proxies MUST forward 1xx responses, unless the connection between the proxy and its client has been closed, or unless the proxy itself requested the generation of the 1xx response. (For example, if a proxy adds an "Expect: 100-continue" field when it forwards a request, then it need not forward the corresponding 100 (Continue) response(s).)

HTTP 100 Continue

The client SHOULD continue with its request. This interim response is used to inform the client that the initial part of the request has been received and has not yet been rejected by the server. The client SHOULD continue by sending the remainder of the request or, if the request has already been completed, ignore this response. The server MUST send a final response after the request has been completed.

HTTP 101 Switching Protocols

The server understands and is willing to comply with the client’s request, via the Upgrade message header field, for a change in the application protocol being used on this connection. The server will switch protocols to those defined by the response’s Upgrade header field immediately after the empty line which terminates the 101 response.
The protocol SHOULD be switched only when it is advantageous to do so. For example, switching to a newer version of HTTP is advantageous over older versions, and switching to a real-time, synchronous protocol might be advantageous when delivering resources that use such features.

Successful HTTP 2xx

This class of status code indicates that the client’s request was successfully received, understood and accepted.

HTTP 200 OK

The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:
GET an entity corresponding to the requested resource is sent in the response;
HEAD the entity-header fields corresponding to the requested resource are sent in the response without any message-body;
POST an entity describing or containing the result of the action;
TRACE an entity containing the request message as received by the end server.

HTTP 201 Created

The request has been fulfilled and resulted in a new resource being created. The newly created resource can be referenced by the URI(s) returned in the entity of the response, with the most specific URI for the resource given by a Location header field. The response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. The origin server MUST create the resource before returning the 201 status code. If the action cannot be carried out immediately, the server SHOULD respond with 202 (Accepted) response instead.
A 201 response MAY contain an ETag response header field indicating the current value of the entity tag for the requested variant just created.

HTTP 202 Accepted

The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place. There is no facility for re-sending a status code from an asynchronous operation such as this.
The 202 response is intentionally non-committal. Its purpose is to allow a server to accept a request for some other process (perhaps a batch-oriented process that is only run once per day) without requiring that the user agent’s connection to the server persist until the process is completed. The entity returned with this response SHOULD include an indication of the request’s current status and either a pointer to a status monitor or some estimate of when the user can expect the request to be fulfilled.

HTTP 203 Non-Authoritative Information

The returned metainformation in the entity-header is not the definitive set as available from the origin server, but is gathered from a local or a third-party copy. The set presented MAY be a subset or superset of the original version. For example, including local annotation information about the resource might result in a superset of the metainformation known by the origin server. Use of this response code is not required and is only appropriate when the response would otherwise be 200 (OK).

HTTP 204 No Content

The server has fulfilled the request but does not need to return an entity-body, and might want to return updated metainformation. The response MAY include new or updated metainformation in the form of entity-headers, which if present SHOULD be associated with the requested variant.
If the client is a user agent, it SHOULD NOT change its document view from that which caused the request to be sent. This response is primarily intended to allow input for actions to take place without causing a change to the user agent’s active document view, although any new or updated metainformation SHOULD be applied to the document currently in the user agent’s active view.
The 204 response MUST NOT include a message-body, and thus is always terminated by the first empty line after the header fields.

HTTP 205 Reset Content

The server has fulfilled the request and the user agent SHOULD reset the document view which caused the request to be sent. This response is primarily intended to allow input for actions to take place via user input, followed by a clearing of the form in which the input is given so that the user can easily initiate another input action. The response MUST NOT include an entity.

HTTP 206 Partial Content

The server has fulfilled the partial GET request for the resource. The request MUST have included a Range header field indicating the desired range, and MAY have included an If-Range header field to make the request conditional.
The response MUST include the following header fields:

  • Either a Content-Range header field indicating the range included with this response, or a multipart/byteranges, Content-Type including Content-Range fields for each part. If a Content-Length header field is present in the response, its value MUST match the actual number of OCTETs transmitted in the message-body.
  • Date
  • ETag and/or Content-Location, if the header would have been sent in a 200 response to the same request
  • Expires, Cache-Control, and/or Vary, if the field-value might differ from that sent in any previous response for the same variant

If the 206 response is the result of an If-Range request that used a strong cache validator, the response SHOULD NOT include other entity-headers. If the response is the result of an If-Range request that used a weak validator, the response MUST NOT include other entity-headers; this prevents inconsistencies between cached entity-bodies and updated headers. Otherwise, the response MUST include all of the entity-headers that would have been returned with a 200 (OK) response to the same request.
A cache MUST NOT combine a 206 response with other previously cached content if the ETag or Last-Modified headers do not match exactly.
A cache that does not support the Range and Content-Range headers MUST NOT cache 206 (Partial) responses.

Redirection HTTP 3xx

This class of status code indicates that further action needs to be taken by the user agent in order to fulfil the request. The action required MAY be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD. A client SHOULD detect infinite redirection loops, since such loops generate network traffic for each redirection.
Note: previous versions of this specification recommended a maximum of five redirections. Content developers should be aware that there might be clients that implement such a fixed limitation.

HTTP 300 Multiple Choices

The requested resource corresponds to any one of a set of representations, each with its own specific location and agent- driven negotiation information is being provided so that the user (or user agent) can select a preferred representation and redirect its request to that location.
Unless it was a HEAD request, the response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content- Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.
If the server has a preferred choice of representation, it SHOULD include the specific URI for that representation in the Location field; user agents MAY use the Location field value for automatic redirection. This response is cacheable unless indicated otherwise.

HTTP 301 Moved Permanently

The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible. This response is cacheable unless indicated otherwise.
The new permanent URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).
If the 301 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.
Note: When automatically redirecting a POST request after receiving a 301 status code, some existing HTTP/1.0 user agents will erroneously change it into a GET request.

HTTP 302 Found

The requested resource resides temporarily under a different URI. Since the redirection might be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.
The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).
If the 302 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.
Note: RFC 1945 and RFC 2068 specify that the client is not allowed to change the method on the redirected request.  However, most existing user agent implementations treat 302 as if it were a 303 response, performing a GET on the Location field-value regardless of the original request method. The status codes 303 and 307 have been added for servers that wish to make unambiguously clear which kind of reaction is expected of the client.

HTTP 303 See Other

The response to the request can be found under a different URI and SHOULD be retrieved using a GET method on that resource. This method exists primarily to allow the output of a POST-activated script to redirect the user agent to a selected resource. The new URI is not a substitute reference for the originally requested resource. The 303 response MUST NOT be cached, but the response to the second (redirected) request might be cacheable.
The different URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).
Note: Many pre-HTTP/1.1 user agents do not understand the 303 status. When interoperability with such clients is a concern, the 302 status code may be used instead, since most user agents react to a 302 response as described here for 303.

HTTP 304 Not Modified

If the client has performed a conditional GET request and access is allowed, but the document has not been modified, the server SHOULD respond with this status code. The 304 response MUST NOT contain a message-body, and thus is always terminated by the first empty line after the header fields.
The response MUST include the following header fields:

  • Date, unless its omission is required

If a clockless origin server obeys these rules, and proxies and clients add their own Date to any response received without one, caches will operate correctly.

  • ETag and/or Content-Location, if the header would have been sent in a 200 response to the same request
  • Expires, Cache-Control, and/or Vary, if the field-value might differ from that sent in any previous response for the same variant

If the conditional GET used a strong cache validator, the response SHOULD NOT include other entity-headers. Otherwise (i.e., the conditional GET used a weak validator), the response MUST NOT include other entity-headers; this prevents inconsistencies between cached entity-bodies and updated headers.
If a 304 response indicates an entity not currently cached, then the cache MUST disregard the response and repeat the request without the conditional.
If a cache uses a received 304 response to update a cache entry, the cache MUST update the entry to reflect any new field values given in the response.

HTTP 305 Use Proxy

The requested resource MUST be accessed through the proxy given by the Location field. The Location field gives the URI of the proxy. The recipient is expected to repeat this single request via the proxy. 305 responses MUST only be generated by origin servers.
Note: RFC 2068 was not clear that 305 was intended to redirect a single request, and to be generated by origin servers only.  Not observing these limitations has significant security consequences.

HTTP 306 (Unused)

The 306 status code was used in a previous version of the specification, is no longer used, and the code is reserved.

HTTP 307 Temporary Redirect

The requested resource resides temporarily under a different URI. Since the redirection MAY be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.
The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s), since many pre-HTTP/1.1 user agents do not understand the 307 status. Therefore, the note SHOULD contain the information necessary for a user to repeat the original request on the new URI.
If the 307 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

Client Error HTTP 4xx

The 4xx class of status code is intended for cases in which the client seems to have erred. Except when responding to a HEAD request, the server SHOULD include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. These status codes are applicable to any request method. User agents SHOULD display any included entity to the user.
If the client is sending data, a server implementation using TCP SHOULD be careful to ensure that the client acknowledges receipt of the packet(s) containing the response, before the server closes the input connection. If the client continues sending data to the server after the close, the server’s TCP stack will send a reset packet to the client, which may erase the client’s unacknowledged input buffers before they can be read and interpreted by the HTTP application.

HTTP 400 Bad Request

The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.

HTTP 401 Unauthorized

The request requires user authentication. The response MUST include a WWW-Authenticate header field containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field. If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication".

HTTP 402 Payment Required

This code is reserved for future use.

HTTP 403 Forbidden

The server understood the request, but is refusing to fulfil it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.

HTTP 404 Not Found

The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.

HTTP 405 Method Not Allowed

The method specified in the Request-Line is not allowed for the resource identified by the Request-URI. The response MUST include an Allow header containing a list of valid methods for the requested resource.

HTTP 406 Not Acceptable

The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.
Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.
Note: HTTP/1.1 servers are allowed to return responses which are not acceptable according to the accept headers sent in the request. In some cases, this may even be preferable to sending a 406 response. User agents are encouraged to inspect the headers of an incoming response to determine if it is acceptable.
If the response could be unacceptable, a user agent SHOULD temporarily stop receipt of more data and query the user for a decision on further actions.

HTTP 407 Proxy Authentication Required

This code is similar to 401 (Unauthorized), but indicates that the client must first authenticate itself with the proxy. The proxy MUST return a Proxy-Authenticate header field containing a challenge applicable to the proxy for the requested resource. The client MAY repeat the request with a suitable Proxy-Authorization header field. HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication".

HTTP 408 Request Timeout

The client did not produce a request within the time that the server was prepared to wait. The client MAY repeat the request without modifications at any later time.

HTTP 409 Conflict

The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem; however, that might not be possible and is not required.
Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the entity being PUT included changes to a resource which conflict with those made by an earlier (third-party) request, the server might use the 409 response to indicate that it can’t complete the request. In this case, the response entity would likely contain a list of the differences between the two versions in a format defined by the response Content-Type.

HTTP 410 Gone

The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. Clients with link editing capabilities SHOULD delete references to the Request-URI after user approval. If the server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) SHOULD be used instead. This response is cacheable unless indicated otherwise.
The 410 response is primarily intended to assist the task of web maintenance by notifying the recipient that the resource is intentionally unavailable and that the server owners desire that remote links to that resource be removed. Such an event is common for limited-time, promotional services and for resources belonging to individuals no longer working at the server’s site. It is not necessary to mark all permanently unavailable resources as "gone" or to keep the mark for any length of time — that is left to the discretion of the server owner.

HTTP 411 Length Required

The server refuses to accept the request without a defined Content- Length. The client MAY repeat the request if it adds a valid Content-Length header field containing the length of the message-body in the request message.

HTTP Precondition Failed

The precondition given in one or more of the request-header fields evaluated to false when it was tested on the server. This response code allows the client to place preconditions on the current resource metainformation (header field data) and thus prevent the requested method from being applied to a resource other than the one intended.

HTTP 413 Request Entity Too Large

The server is refusing to process a request because the request entity is larger than the server is willing or able to process. The server MAY close the connection to prevent the client from continuing the request.
If the condition is temporary, the server SHOULD include a Retry- After header field to indicate that it is temporary and after what time the client MAY try again.

HTTP 414 Request-URI Too Long

The server is refusing to service the request because the Request-URI is longer than the server is willing to interpret. This rare condition is only likely to occur when a client has improperly converted a POST request to a GET request with long query information, when the client has descended into a URI "black hole" of redirection (e.g., a redirected URI prefix that points to a suffix of itself), or when the server is under attack by a client attempting to exploit security holes present in some servers using fixed-length buffers for reading or manipulating the Request-URI.

HTTP 415 Unsupported Media Type

The server is refusing to service the request because the entity of the request is in a format not supported by the requested resource for the requested method.

HTTP 416 Requested Range Not Satisfiable

A server SHOULD return a response with this status code if a request included a Range request-header field, and none of the range-specifier values in this field overlap the current extent of the selected resource, and the request did not include an If-Range request-header field. (For byte-ranges, this means that the first- byte-pos of all of the byte-range-spec values were greater than the current length of the selected resource.)
When this status code is returned for a byte-range request, the response SHOULD include a Content-Range entity-header field specifying the current length of the selected resource. This response MUST NOT use the multipart/byteranges content- type.

HTTP 417 Expectation Failed

The expectation given in an Expect request-header field could not be met by this server, or, if the server is a proxy, the server has unambiguous evidence that the request could not be met by the next-hop server.

Server Error HTTP 5xx

Response status codes beginning with the digit "5" indicate cases in which the server is aware that it has erred or is incapable of performing the request. Except when responding to a HEAD request, the server SHOULD include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. User agents SHOULD display any included entity to the user. These response codes are applicable to any request method.

HTTP 500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

HTTP 501 Not Implemented

The server does not support the functionality required to fulfil the request. This is the appropriate response when the server does not recognize the request method and is not capable of supporting it for any resource.

HTTP 502 Bad Gateway

The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfil the request.

HTTP 503 Service Unavailable

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.
Note: The existence of the 503 status code does not imply that a server must use it when becoming overloaded. Some servers may wish to simply refuse the connection.

HTTP 504 Gateway Timeout

The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server specified by the URI (e.g. HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed to access in attempting to complete the request.
Note: Note to implementors: some deployed proxies are known to return 400 or 500 when DNS lookups time out.

HTTP 505 HTTP Version Not Supported

The server does not support, or refuses to support, the HTTP protocol version that was used in the request message. The server is indicating that it is unable or unwilling to complete the request using the same major version as the client, other than with this error message. The response SHOULD contain an entity describing why that version is not supported and what other protocols are supported by that server.

Performance bottleneck symptoms

CPU Bottleneck Symptoms:

Symptoms for CPU bottlenecks include the following,
The Processor(_Total)\% Processor Time(measures the total utilization of your processor by all running processes) will be high. If the server typically runs at around 70% or 80% processor utilization then this is normally a good sign and means your machine is handling its load effectively and not underutilized. Average processor utilization of around 20% or 30% on the other hand suggests that your machine is underutilized and may be a good candidate for server consolidation using Virtual Server or VMWare.
Further to breakdown this %processor Time, monitor the counters – Processor(_Total)\% Privileged Time and Processor(_Total)\% User Time, which respectively show processor utilization for kernel- and user-mode processes on your machine. If kernel mode utilization is high, your machine is likely underpowered as it’s too busy handling basic OS housekeeping functions to be able to effectively run other applications. And if user mode utilization is high, it may be you have your server running too many specific roles and you should either beef hardware up by adding another processor or migrate an application or role to another box. The System\Processor Queue Length(indication of how many threads are waiting for execution) consistently greater than 2 or more for a single processor CPU is a clear indication of processor bottleneck . Also look at other counters like ASP\Requests Queued or ASP.NET\Requests Queued as well.

Tips to find out Application server bottlenecks:

  1. A high increase in application server processing time when the load is increased.
  2. One or more page components take more time when the same request db call is taking less execution time.
  3. The Static files are having less response time whereas the dynamic contents (servlets, jsp, etc) take more time.
  4. Network delay is negligible.
  5. Home Page gets displayed in few seconds even during the stress period(as it is fetched from the web server).
  6. Hits/sec & Throughput remains less.
  7. If the CPU/ Memory/Disk of the App server has any bottleneck symptoms.
  8. If the HTTP / HTTPS connections established doesn’t increase proportionally with the load.
  9. If the new connections established is very higher & the reused connections are very less.

Tips to find out Web server bottlenecks:

  1. Increased ‘Server Time’ breakup
  2. One or more page components of transaction takes more time where in the DB query is having less execution time.
  3. The static files are having high response time than the dynamic contents (servlets, jsp, etc)
  4. Network delay is negligible.
  5. Home Page takes more time for display.
  6. Hits/sec in the web server is very less.
  7. If the CPU/ Memory/Disk of the web server has any bottleneck symptoms.

Hardware Malfunctioning Symptoms:

  1. System\Context Switches/sec (measures how frequently the processor has to switch from user- to kernel-mode to handle a request from a thread running in user mode). If this counter suddenly starts increasing however, it may be an indicating of a malfunctioning device, especially if you are seeing a similar jump in the Processor(_Total)\Interrupts/sec counter on your machine.
  2. You may also want to check Processor(_Total)\% Privileged Time Counter and see if this counter shows a similar unexplained increase, as this may indicate problems with a device driver that is causing an additional hit on kernel mode processor utilization.
  3. If Processor(_Total)\Interrupts/sec does not correlate well with System\Context Switches/sec however, your sudden jump in context switches may instead mean that your application is hitting its scalability limit on your particular machine and you may need to scale out your application (for example by clustering) or possibly redesign how it handles user mode requests. In any case, it’s a good idea to monitor System\Context Switches/sec over a period of time to establish a baseline for this counter, and once you’ve done this then create a perfmon alert that will trigger when this counter deviates significantly from its observed mean value.

Memory Bottleneck Symptoms:

When it comes to the System memory, there are 3 things to monitor:

  1. Monitor Cache (Hits/Misses),
  2. Monitor Memory (Memory Available/sec, Process/Working Set),
  3. Monitor Paging (Pages Read/Sec, Pages Input/Sec, Page Faults/Sec, % Disk Processing)Memory\Available Bytes,

If this counter is greater than 10% of the actual RAM in your machine then you probably have more than enough RAM and don’t need to worry. The Memory\Pages/sec counter indicates the number of paging operations to disk during the measuring interval, and this is the primary counter to watch for indication of possible insufficient RAM to meet your server’s needs. You can monitor Process(instance)\Working Set for each process instance to determine which process is consuming larger and larger amounts of RAM. Process(instance)\Working Set measures the size of the working set for each process, which indicates the number of allocated pages the process can address without generating a page fault. A related counter is Memory\Cache Bytes, which measures the working set for the system i.e. the number of allocated pages kernel threads can address without generating a page fault. Finally, another corroborating indicator of insufficient RAM is Memory\Transition Faults/sec, which measures how often recently trimmed page on the standby list are re-referenced. If this counter slowly starts to rise over time then it could also indicating you’re reaching a point where you no longer have enough RAM for your server to function well.

Disk Bottleneck Symptoms:

A bottleneck from a disk can significantly impact response time for applications running on your system. Physical Disk (instance)\Disk Transfers/sec counter for each physical disk and if it goes above 25 disk I/Os per second then you’ve got poor response time for your disk. By tracking Physical Disk(instance)\% Idle Time, which measures the percent time that your hard disk is idle during the measurement interval, and if you see this counter fall below 20% then you’ve likely got read/write requests queuing up for your disk which is unable to service these requests in a timely fashion. In this case it’s time to upgrade your hardware to use faster disks or scale out your application to better handle the load. Look for the Physical Disk (instance)\Average Disk Queue length & Physical Disk (instance)\Current Disk Queue length parameters to get more details on the queued up requests.

Network Performance/Bottlenecks:

The first step in monitoring is to monitor the network performance, to make sure your network performance is good. There are some simple ways to do so. First monitor whether you are getting the same bandwidth which you are supposed to get. The easiest way to find out is to check the current bandwidth counter with your expected bandwidth. Also verify the rate at which the server sends and receives the data. Network performance depends on 2 factors, network cards and interfaces (Switches/Routers) configured on the servers.

Here are some of the counters to find network bottlenecks:

Network Interface: Current Bandwidth
This counter determines your current bandwidth of the network interface. Capture this counter value and correlate with bytes receives/sec, bytes send/sec and bytes total/sec.
If the bytes total should be at least half of your total bandwidth .If not so then we can confirm a network bottle neck

Network Interface: Bytes Total/sec:
To determine if your network connection is creating a bottleneck, compare the Network Interface: Bytes Total/sec counter to the total bandwidth of your network adapter card. To allow headroom for spikes in traffic, you should usually be using no more than 50 percent of capacity. If this number is very close to the capacity of the connection, and processor and memory use are moderate, then the connection may well be a problem. To determine the network utilization (throughput on a server’s network cards), you can check the following counters:

  1. Network\Bytes Received/sec
  2. Network\Bytes Sent/sec
  3. Network\Bytes Total/sec
  4. Network Current Bandwidth

If the total byte per second value is more than 50 percent of the total network utilization under average user/work load, then your server is having some problems under peak load conditions. Make sure you compare network counter values with Physical Disk\% Disk Time and Processor\% Processor Time utilization. If the disk time and processor time values are low but the network values are very high, there might be a problem with your network.
There are 2 ways to solve this problem:

  1. By optimizing the network card settings
  2. By adding an additional network card.

Analyzing IIS logs with LogParser

When users access your server running IIS, IIS logs the information. The logs provide valuable information that you can use to identify any unauthorized attempts to compromise your Web server.
Depending on the amount of traffic to your Web site, the size of your log file (or the number of log files) can consume valuable disk space, memory resources, and CPU cycles. You might need to balance the gathering of detailed data with the need to limit files to a manageable size and number. Logging information in IIS goes beyond the scope of the event logging or performance monitoring features provided by Windows. The IIS logs can include information, such as who has visited your site, what the visitor viewed, and when the information was last viewed.

IIS log file format:

IIS log file format is a fixed (meaning that it cannot be customized) ASCII format. This file format records more information than other log file formats, including basic items, such as the IP address of the user, user name, request date and time, service status code, and number of bytes received. In addition, IIS log file format includes detailed items, such as the elapsed time, number of bytes sent, action (for example, a download carried out by a GET command), and target file. The IIS log file is an easier format to read than the other ASCII formats because the information is separated by commas, while most other ASCII log file formats use spaces for separators. Time is recorded as local time.

IIS log file location:

The IIS logs provide a great deal of information about the activity of a Web application. You can find the IIS logs in

systemroot\System32\LogFiles\W3SVCnumber, where number is the site ID for the Web site.

LogParser:

LogParser is a command line utility. The default behaviour of LogParser is it works like a “data processing pipeline”, by taking an SQL expression on the command line, and outputting the lines containing matches for the SQL expression.

Download LopParser from:
http://www.microsoft.com/downloads/en/details.aspx?familyid=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en

W3C Extended Logging Field Definitions


Prefix Meaning
s- Sever actions
c- Client actions
cs- Client-to-server actions.
sc- Server-to-client actions.


Field Appears As Description
Date date The date that the activity occurred.
Time time The time that the activity occurred.
Client IP Address c-ip The IP address of the client that accessed your server.
User Name cs-username The name of the authenticated user who accessed your server. This does not include anonymous users, who are represented by a hyphen (-).
Service Name s-sitename The Internet service and instance number that was accessed by a client.
Server Name s-computername The name of the server on which the log entry was generated.
Server IP Address s-ip The IP address of the server on which the log entry was generated.
Server Port s-port The port number the client is connected to.
Method cs-method The action the client was trying to perform (for example, a GET method).
URI Stem cs-uri-stem The resource accessed; for example, Default.htm.
URI Query cs-uri-query The query, if any, the client was trying to perform.
Protocol Status sc-status The status of the action, in HTTP or FTP terms.
Win32® Status sc-win32-status The status of the action, in terms used by Microsoft Windows®.
Bytes Sent sc-bytes The number of bytes sent by the server.
Bytes Received cs-bytes The number of bytes received by the server.
Time Taken time-taken The duration of time, in milliseconds, that the action consumed.
Protocol Version cs-version The protocol (HTTP, FTP) version used by the client. For HTTP this will be either HTTP 1.0 or HTTP 1.1.
Host cs-host Displays the content of the host header.
User Agent cs(User-Agent) The browser used on the client.
Cookie cs(Cookie) The content of the cookie sent or received, if any.
Referrer cs(Referer) The previous site visited by the user. This site provided a link to the current site.



The following is an example of a record in the extended log format that was produced by the Microsoft Internet Information Server (IIS):
——————————————————————————–
#Software: Microsoft Internet Information Server 6.0
#Version: 1.0
#Date: 2011-05-09 22:48:39
#Fields: date time c-ip cs-username s-ip cs-method cs-uri-stem cs-uri-query sc-status sc-bytes cs-bytes time-taken cs-version cs(User-Agent) cs(Cookie) cs(Referrer)

2011-05-09 22:48:39 192.168.1.5 – 173.201.216.31 /GreenBlue.jpg – 200 540 324 157 HTTP/1.0 Mozilla/4.0+(compatible;+MSIE+4.01;+Windows+95) USERID=CustomerA;+IMPID=01234 http://www.punebids.com

Procedure:

  1. First note the date and timings of the test run.
  2. Collect the IIS log files of that particular timeframe from the Web Servers (specific to your Web Application) and put it in a single folder
  3. Write query for data collection from logs and run it in LogParser

To find URL Hit count:



[cce lang=”c”]
logparser “SELECT cs-uri-stem AS Url, count(cs-uri-stem) AS Hits FROM ‘C:\Logs\WebServers1-4\*.*’ WHERE date=to_timestamp(‘2011-05-10’,’yyyy-MM-dd’) and time>’01:55:00’ and time<’02:35:00’ and cs-uri-stem like ‘%asp%’ GROUP BY Url ORDER BY Hits DESC” –i:IISW3C –o:csv >”C:\Logs\CollectedUrlHits.txt”
[/cce]


Output:

Url,Hits

/Framework/website/Default.aspx,15678

/isvs/consulting/userprofile.asp,897

/DNA/Common/Portal/ClientHome.aspx,75

/DNA/Common/Clients/Clientlist.aspx,75

/DNA/Common/portal/DNAuserlist.aspx,75

Statistics:

Elements Processed: 245646

Elements output: 5

Execution time: 5.80 seconds

To find the HTTP Error count:



[cce lang=”c”]
logparser “SELECT cs-uri-stem AS Url, count(cs-uri-stem) AS Hits  FROM ‘C:\Logs\WebServers1-4\*.*’ WHERE date=to_timestamp(‘2011-05-10’,’yyyy-MM-dd’) and time>’01:55:00’ and time<’02:35:00’ and cs-uri-stem like ‘%asp%’ and sc-status=500  GROUP BY Url ORDER BY Hits DESC” –i:IISW3C –o:csv >”C:\Logs\Collected500Errors.txt”
[/cce]

Output:

Url,Hits

/Framework/website/Default.aspx,1

/isvs/consulting/userprofile.asp,2

/DNA/Common/Portal/ClientHome.aspx,1

Statistics:

Elements Processed: 245646

Elements output: 3

Execution time: 3.80 seconds





P.S. : Thanks to Rahul D.

Managing Application Pools

When you run IIS 6.0 in worker process isolation mode, you can separate different Web applications and Web sites into groups known as application pools. An application pool is a group of one or more URLs that are served by a worker process or set of worker processes. Any Web directory or virtual directory can be assigned to an application pool.

A worker process is user-mode code whose role is to process requests, such as processing requests to return a static page, invoking an ISAPI extension or filter, or running a Common Gateway Interface (CGI) handler.

Every application within an application pool shares the same worker process. Because each worker process operates as a separate instance of the worker process executable, W3wp.exe, the worker process that services one application pool is separated from the worker process that services another. Each separate worker process provides a process boundary so that when an application is assigned to one application pool, problems in other application pools do not affect the application. This ensures that if a worker process fails, it does not affect the applications running in other application pools.

Use multiple application pools when you want to help ensure that applications and Web sites are confidential and secure. For example, an enterprise organization might place its human resources Web site and its finance Web site on the same server, but in different application pools. Likewise, an ISP that hosts Web sites and applications for competing companies might run each company’s Web services on the same server, but in different application pools. Using different application pools to isolate applications helps prevent one customer from accessing, changing, or using confidential information from another customer’s site.

Application Pool Actions:

To manage Application Pools go to IIS Manager and click on application pool which you want to configure for recycling.

Tab and their descriptions:

Element Name Description
Add Application Pool Opens the Add Application Pool dialog box from which you can add an application pool to the Web server.
Set Application Pool Defaults Opens the Application Pool Defaults dialog box from which you can set default values that apply to all application pools that you add to the Web server.
Start Starts the selected application pool.
Stop Stops the selected application pool. This causes the Windows Process Activation Service (WAS) to shut down all running worker processes serving that application pool. An administrator must restart a stopped application pool or else requests made to applications in that application pool will receive HTTP 503-Service Unavailable errors.
Recycle Stops and restarts the selected application pool. Restarting an application pool causes the application pool to be temporarily unavailable until the restart is complete.
Basic Settings Opens the Edit Application Pool dialog box from which you can edit the settings that were specified when the application pool was created. This action is available only when an item is selected from the list on the feature page.
Recycling Opens the Edit Application Pool Recycling Settings wizard from which you can specify conditions under which to recycle an application pool and configure how recycling events are logged.
Advanced Settings Opens the Advanced Settings dialog box from which you can configure advanced settings for the selected application pool.
Rename Enables the Name field of the selected application pool so that you can rename the application pool.
Remove Removes the item that is selected from the list on the feature page.
View Applications Opens the Applications feature page from which you can view the applications that belong to the selected application pool.

Edit Application Pool Recycling Settings:

Use the Recycling Conditions page of the Edit Application Pool Recycling Settings Wizard to configure IIS to periodically restart worker processes in an application pool. This can help you to recover valuable system resources and to better manage faulty worker processes.

Tab and their descriptions:

Element Name Description
Regular time intervals (in minutes) Select this option to specify a time interval, in minutes, at which you want IIS to recycle the worker process. You might choose this option if you have an application that causes problems when it runs for an extended time. Based on what you know about the application, you should set the value to be less than the length of time elapsed before application failure.
Fixed number of requests Select this option to specify the number of requests after which you want IIS to recycle the worker process. You might choose this option if you have an application that causes problems after reaching a certain number of requests. Based on what you know about the application, you should configure the value to be less than the number of requests processed before application failure.
Specific time(s) Select this option to specify a time or times at which you want IIS to recycle the worker process in a 24-hour period. For example, to recycle a worker process at 4:30 A.M. and 4:30 P.M., enter 4:30 AM, 4:30 PM. The time that you specify uses the local time on the Web server. You might choose this option if you have an application that causes problems when it runs for an extended time and you want to recycle the application pool at a specific time, such as a time that is late at night or early in the morning, to avoid a negative impact on users. Based on what you know about the application, you should set the interval to be frequent enough to prevent application failure.
Virtual memory usage (in KB) Select this option to specify the maximum number of kilobytes of your system’s common virtual memory that can be used by a worker process before that process is recycled. You might choose this option when you notice a steady increase in the virtual memory used on your server. This might indicate that an application reserves memory multiple times, which fragments the memory heap. Entering too high a value can severely decrease system performance. At first, you should set the virtual memory threshold to be less than 70 percent of available virtual memory, and then adjust the setting if you have to.
Private memory usage (in KB) Select this option to specify the maximum number of kilobytes of privately allocated system physical memory that can be used by a worker process before the process is recycled. You might choose this option when you have an application that leaks memory. Entering too high a value can severely decrease system performance. At first, you should set this value to be less than 60 percent of the available physical memory on the server, and then adjust this setting if you have to.

Recycling Events to Log:

Use the Recycling Events to Log page of the Edit Application Pool Recycling Settings Wizard to configure IIS to log an event when a worker process is recycled.

You can configure IIS to log information for recycling events that you configure, such as at a fixed interval or for recycling events that occur at runtime, such as when an ISAPI declares itself unhealthy.

Tab and their descriptions:

Element Name Description
Regular time intervals Select this option to log an event when a worker process is recycled at a specified time interval. This option is available only when the Regular time intervals (in minutes) option is selected and a time interval is specified on the previous wizard page.
Virtual memory usage Select this option to log an event when a worker process is recycled after using a specified amount of virtual memory. This option is available only when the Virtual memory usage (in KB) option is selected and a number of kilobytes is specified on the previous wizard page.
Number of requests Select this option to log an event when a worker process is recycled after reaching a specified number of requests. This option is available only when the Fixed number of requests option is selected and a number of requests is specified on the previous wizard page.
Scheduled time(s) Select this option to log an event when a worker process is recycled at a specified time. This option is available only when the Specific time(s) option is selected and a time is specified on the previous wizard page.
Private memory usage Select this option to log an event when a worker process is recycled after using a specified amount of physical memory. This option is available only when the Private memory usage (in KB) option is selected and a number of kilobytes is specified on the previous wizard page.
On-demand Select this option to log an event when you recycle a worker process by using IIS Manager or Appcmd.exe to correct a problem.
Configuration changes Select this option to log an event when a change to configuration causes the application pool to recycle.
Unhealthy ISAPI Select this option to log an event when an ISAPI extension reports to the worker process that it is unhealthy.





P.S. : Thanks to Kranti.

Removing Zero from the Date in VuGen LoadRunner

Challenge:

Enter start date and end date and date should not include zero before actual value of month and day.

E.g.

Invalid date format: 04/27/2011

Valid date format: 4/27/2011

Solution:

//variable declaration
char month[10], day[10], year[10], startDate[10], endDate[10];
int nonZeroDay, nonZeroMonth;

/*if you want to run script for multiple interactions then you have to nullify
the concatenated string before starting new iteration otherwise it
will keep the old value*/
strcpy(startDate, “”);
strcpy(endDate, “”);

//original date which contain zero in the date
lr_save_datetime(“Today’s Date is: %m/%d/%Y”, DATE_NOW, “normalDate”);
lr_output_message(lr_eval_string(“{normalDate}”));

//for generating non zero start date

//to assign the current date parts to a parameter
lr_save_datetime(“%m”, DATE_NOW, “month”);
lr_save_datetime(“%d”, DATE_NOW, “day”);
lr_save_datetime(“%Y”, DATE_NOW, “year”);

//to remove zero from the string. e.g.getting ‘4’ from ’04’

nonZeroMonth = atoi(lr_eval_string(“{month}”));
lr_save_int(nonZeroMonth, “month”);

nonZeroDay = atoi(lr_eval_string(“{day}”));
lr_save_int(nonZeroDay, “day”);

//creating a date string using concatenation of nonzero date parts and separator ‘/’
strcat(startDate, lr_eval_string(“{month}”));
strcat(startDate, “/”);
strcat(startDate, lr_eval_string(“{day}”));
strcat(startDate, “/”);
strcat(startDate, lr_eval_string(“{year}”));

lr_save_string(startDate, “p_StartDate”);
lr_output_message(“Start Date is: %s”, lr_eval_string(“{p_StartDate}”));

//for generating non zero end date by incrementing date by one day offset
lr_save_datetime(“%m”, DATE_NOW + ONE_DAY, “month”);
lr_save_datetime(“%d”, DATE_NOW + ONE_DAY, “day”);

lr_save_datetime(“%Y”, DATE_NOW + ONE_DAY, “year”);
nonZeroMonth = atoi(lr_eval_string(“{month}”));
lr_save_int(nonZeroMonth, “month”);

nonZeroDay = atoi(lr_eval_string(“{day}”));
lr_save_int(nonZeroDay, “day”);

strcat(endDate, lr_eval_string(“{month}”));
strcat(endDate, “/”);
strcat(endDate, lr_eval_string(“{day}”));
strcat(endDate, “/”);
strcat(endDate, lr_eval_string(“{year}”));

lr_save_string(endDate, “p_endDate”);
lr_output_message(“End Date is: %s”, lr_eval_string(“{p_endDate}”));

Output:

Today’s Date is: 04/27/2011
Start Date is: 4/27/2011
End Date is: 4/28/2011

RTE protocol script recording in VuGen LoadRunner

Terminal Emulation (RTE):

Emulation of users who submit input to, and receive output from, character-based applications. VuGen LoadRunner has its own internal terminal emulator using which it connects to the application.

RTE Vuser:

  • RTE Vuser operates terminal emulator in order to load test client/server base applications.
  • Terminal emulator session is a collection of a real user’s actions.
  • RTE Vuser types character input into a terminal emulator, submits the data to a server, and then waits for the server to respond.
  • LoadRunner uses PowerTerm as a terminal emulator for emulating RTE session.
  • PowerTerm works like a standard terminal emulator, supporting common protocols such as IBM 3270 & 5250, VT100 and VT220.
  • VuGen records the terminal setup and connection procedure into the script.

Procedure:

    1. Click on New Script.
    2. Select Single Protocol Script and in All Protocols category choose Terminal Emulation (RTE).

    1. Click on Create for creating script and then click on Start Recording.
    2. PowerTerm Emulator will show up.
    3. Click on Communication menu and then click on Connect.

  1. In Connect window choose Terminal type: 3270 Display
  2. It will automatically set the default settings like Session Type, Terminal ID, and Port Number according to your Terminal Type value.
  3. You can change them according to your need.
  4. Enter Host Name, which is given in Test Plan.
  5. Click on Connect, it will connect to the server then follow the business flow given in test plan.
  6. When you are done with your business flow then stop recording.
  7. Now you can enhance your recorded script with transactions and synchronization functions.

Challenge 1: Finding blank row on Terminal screen to add user inputs

char text[100];

/* suppose top rows are reserved for function description and user can input data from 6th row onwards and total rows are 22 */

int row=6;

int flag=TRUE;

//Finding the blank row to add user inputs

while (flag){

//If it’s End of page then start from next page

if (row==22) {

TE_type(“”);

TE_wait_sync();

row=6;

}

TE_get_text_line(5,row,1, text);

if (strcmp(text,”_”)==0){

flag=FALSE;

break;

}

row++;

}

TE_set_cursor_pos(2,row);

Challenge 2: Handling transaction status using Text Check on terminal screen

lr_start_transaction(“T01_Modify_AccountDetails”);

TE_type(“{p_Amount}”);

TE_wait_sync();

iReturn=TE_find_text (“RECORD(S) PROCESSED SUCCESSFULLY.”, 1,22,40,22, &col, &row, match);

lr_output_message(“value:%d”, iReturn);

if (iReturn==0){

lr_end_transaction(“T01_Modify_AccountDetails “,LR_PASS);

}else{

lr_end_transaction(“T01_Modify_AccountDetails “,LR_FAIL);

}

Creating Custom Request in VuGen LoadRunner

There are some situations where you need to create custom request so that you can manipulate the request according to your need. Here we will see how to create custom request and tamper with it to get our job done.

Basic:

When we record the script for any web application, we generally choose HTML-based recording because it is easy and minimizes the correlations you have to handle in the script.

LoadRunner do generate custom request when some request does not fall under category of lr_web_submit_data() or lr_web_submit_form() requests category.

Challenge:

Suppose there is one request where you are sending some data items with it and every time there will be random number of data items like, first time there will be four data items second time there will be five data items in the same request. Here you cannot use same recorded request as number of data items are going to be different every time.

In below example you can see that there are only four data items in the request next time there may be five data item.


“Name=SearchResultsGrid_ctl04_ViewPackageDialogResult”, “Value=291570”, ENDITEM,
“Name=SearchResultsGrid_ctl08_ViewPackageDialogResult”, “Value=291566”, ENDITEM,
“Name=SearchResultsGrid_ctl11_ViewPackageDialogResult”, “Value=291563”, ENDITEM,
“Name=SearchResultsGrid_ctl12_ViewPackageDialogResult”, “Value=291562”, ENDITEM,

Solution:

Here you can create a custom request and use a parameter which will hold the manipulated string of data items. In that string you can save different number of data items and you can change it as required.

Procedure:

    1. First record the script with HTML-based recording and with proper transaction names for every action.
    2. Save that script and then click on Tools> Regenerate Script…

    1. Below popup will come, click on Options…

    1. It will show you the Regenerate options, Click on General> Recoding

  1. There are two options available, HTML-based script and URL-based script. Choose URL-based script.
  2. For URL-based script there is a button for advanced settings named URL Advanced, click on it.
  3. From the new popup select Use web_custom_request only.
  4. Save the settings and regenerate the script.
  5. In the regenerated script you can see that new script is generated which contain only web_custom_request.
  6. Now find the particular request which you want to manipulate with the help of transaction names which you have given while recording the script.

Below is the HTML-based recorded request, You can see that there are four data items of grid results,


web_submit_data(“managepackages.aspx_4”,
“Action=http://{p_Url}/site/secure/reportcenter/managepackages.aspx”,
“Method=POST”,
“TargetFrame=”,
“RecContentType=text/html”,
“Referer=http://{p_Url}/site/secure/reportcenter/managepackages.aspx”,
“Snapshot=t50.inf”,
“Mode=HTML”,
ITEMDATA,
“Name=SearchResultsGrid_ctl03_ViewPackageDialogResult”, “Value={c_PackageID}”, ENDITEM,
“Name=SearchResultsGrid_ctl04_ViewPackageDialogResult”, “Value=291570”, ENDITEM,
“Name=SearchResultsGrid_ctl08_ViewPackageDialogResult”, “Value=291566”, ENDITEM,
“Name=SearchResultsGrid_ctl11_ViewPackageDialogResult”, “Value=291563”, ENDITEM,
“Name=SearchResultsGrid_ctl12_ViewPackageDialogResult”, “Value=291562”, ENDITEM,
LAST);

Below is the custom request which is URL-based,

web_custom_request(“managepackages.aspx_4”,
“URL=http:// {p_Url}/site/secure/reportcenter/managepackages.aspx”,
“Method=POST”,
“Resource=0”,
“RecContentType=text/html”,
“Referer= http://{p_Url}/site/secure/reportcenter/managepackages.aspx”,
“Snapshot=t135.inf”,
“Mode=HTTP”,
“Body=SearchResultsGrid_ctl03_ViewPackageDialogResult={c_PackageID}”
“{final_string}”,
/*//below string is going to vary every time so we are going to manipulate it every time using string manipulation functions//

“&SearchResultsGrid_ctl04_ViewPackageDialogResult=291570”
“&SearchResultsGrid_ctl08_ViewPackageDialogResult=291566”
“&SearchResultsGrid_ctl11_ViewPackageDialogResult=291563”
“&SearchResultsGrid_ctl12_ViewPackageDialogResult=291562”
*/
LAST);


In custom request you can manipulate final string every time using string manipulation functions available in VuGen.

final_string= &SearchResultsGrid_ctl04_ViewPackageDialogResult=291570

&SearchResultsGrid_ctl08_ViewPackageDialogResult=291566

&SearchResultsGrid_ctl11_ViewPackageDialogResult=291563

&SearchResultsGrid_ctl12_ViewPackageDialogResult=291562

P.S. : Thanks to Kranti.

Monitoring PMI data with Tivoli Performance Viewer

Recently I got opportunity to use Tivoli Performance Viewer (TPV) for monitoring application server counters of Java base web application which was deployed on Websphere Application Server. Below are the guidelines for how to use TPV to view Performance Monitoring Infrastructure (PMI) data in chart or table form.

Basic:

TPV monitors the performance activity of all servers on a node.  Before monitoring one should confirm that one or more servers have been created and are running on the node, and that PMI is enabled.
You view performance modules when your server is experiencing performance problems. For example, a common performance problem occurs when individual sessions are too large. To help view data on a session, you can view the Servlet Session Manager PMI module and monitor the SessionObjectSize counter to make sure that Session Object Size is not too large.
Performance modules are shown in the TPV current activity settings in the administrative console.

Procedure:

  • Select PMI data to view.
    1. Click Monitoring and Tuning > Performance Viewer > Current Activity > server_name > Performance Modules (PM) in the console navigation tree.
    2. Place a check mark in the check box beside the name of each performance module that you want to view.
    3. Click on View Modules. A chart or table providing the requested data is displayed on the right side of the page. Charts are displayed by default.

If you do not see all the PMI counters you expect, or a PM you are interested in is visible but cannot be selected, PMI is not enabled for that particular counter or for any counter in that PM. Go to the PMI control area of the administrative console and enable PMI for any counters you wish to view, then return to TPV and select those PMs. To view the PMI page, click on Monitoring and Tuning > Performance Monitoring Infrastructure > server_name.

Some of the important counters you should monitor are given below:

Category Counter Description
JDBC Connection Pools CreateCount The total number of connections that are created
CloseCount The total number of connections that are closed
PoolSize The size of the connection pool.
FreePoolSize The number of free connections in the pool.
WaitingThreadCount The average number of threads that are concurrently waiting for connection.
JVM Runtime HeapSize The Total memory (KiloBytes) in Java Virtual Machine Runtime
UsedMemory The amount of Used Memory (KiloBytes) in Java Virtual Machine Runtime
Thread Pools Pool Size Average number of thread in pool
Object Request Broker Average number of thread in pool
Message Listener Average number of thread in pool
WebContainer Average number of thread in pool


Each module has several counters associated with it. These counters are displayed in a table underneath the data chart or table. Selected counters are displayed in the chart or table. You can add or remove counters from the chart or table by selecting or deselecting individual counters. By default, the first three counters for each module are shown.
You can select up to 20 counters and display them in the TPV in the Current Activity mode. The data from all the PMI counters that are currently enabled in each PM for all active instances of the PM can be recorded and captured in a TPV log file. You can view the TPV log file for a particular time period multiple times, selecting different combinations of up to 20 counters each time. You have the flexibility to observe the relationships among different performance data in a server during a particular period of time.

    1. Optional: To remove a module from a chart or table, deselect the check box next to the module and click View Modules again.
    2. Optional: To view the data in a table, click View Table on the counter selection table. To toggle back to a chart, click View Graph.
    3. Optional: To view the legend for a chart, click Show Legend. To hide the legend, click Hide Legend.
  • Scale the PMI data. You can manually adjust the scale for each counter so that the graph displays meaningful comparisons of different counters.
    1. Find the counter whose scale you want to modify in the table beneath the chart.
    2. Change the value for Scale as needed.
    3. Click Update.
  • Clear values from tables and charts.
    1. Ensure that one or more modules are selected under Performance Modules in the TPV navigation tree
    2. Click Clear Buffer beneath the chart or table. The PMI data is removed from a table or chart.
  • Reset counters to zero (0).
    1. Ensure that one or more modules are selected under Performance Modules in the TPV navigation tree
    2. Click Reset to Zero beneath the chart or table. Reset to Zero sets a new baseline using the current counter readings at the instant the button is clicked. Future data points are plotted on the graph relative to their position at the time Reset to Zero is clicked.

Analyzing Processor

Monitor below counters to find if the processor is cause of low performance:

  • Processor: %Processor Time often exceeds 85%.
  • System: Processor Queue Length is often greater than 2.
  • On multiprocessor systems, System: % Total Processor Time often exceeds 50%.

But these symptoms don’t always indicate a processor problem. And even when the processor is the problem, adding extra processors doesn’t always solve it.

Understanding the Processor Counters

It is important to understand the components of the primary processor activity counters, and to distinguish them from each other.

Counter Description
System: % Total Processor Time For what proportion of the sample interval were all processors busy?
A measure of activity on all processors. In a multiprocessor computer, this is equal to the sum of Processor: % Processor Time on all processors divided by the number of processors. On single-processor computers, it is equal to Processor: % Processor time, although the values may vary due to different sampling time.
System: Processor Queue Length How many threads are ready, but have to wait for a processor?
This is an instantaneous count, not an average, so it’s best viewed in charts, rather than reports. Unlike disk queue counters, it counts only waiting threads, not those being serviced.
The queue length counter is on the System object because there is a single queue even when there are multiple processors on the computer.
Processor: % Processor Time For what proportion of the sample interval was each processor busy?
This counter measures the percentage of time the thread of the Idle process is running, subtracts it from 100%, and displays the difference.
This counter is equivalent to Task Manager’s CPU Usage counter.
Processor: % User Time
Processor: % Privileged Time
How often were all processors executing threads running in user mode and in privileged mode?
Threads running in user mode are probably running in their own application code. Threads running in privileged mode are using operating system services.
The user time and privileged time counters on the System and Processor objects do not always sum to 100%. They are measures of non-Idle time, so they sum to the total of non-idle time.
For example, if the processor was running the Idle thread for 85% of the time, the sum of Processor: % User Time and Processor: % Privileged Time would be 15%.
Process: % Processor Time For what proportion of the sample interval was the processor running the threads of this process?
This counter sums the processor time of each thread of the process over the sample interval.
Process: % Processor Time: _Total For what proportion of the sample interval was the processor processing?
This counter sums the time all threads are running on the processor, including the thread of the Idle process on each processor, which runs to occupy the processor when no other threads are scheduled.
The value of Process: % Processor Time: _Total is 100% except when the processor is interrupted. (100% processor time = Process: % Processor Time: Total + Processor: % Interrupt Time + Processor: % DPC Time) This counter differs significantly from Processor: % Processor Time, which excludes Idle.
Process: % User Time
Process: % Privileged Time
How often are the threads of the process running in its own application code (or the code of another user-mode process)? How often are the threads of the process running in operating system code?
Process: % User Time and Process: % Privileged Time sum to Process: % Processor Time.

Key Performance Indicator


Category Counter Description Optimal Value What if not Optimal Can Cause
Processor System\Context Switches/sec This counter gives the combined rate at which all processors switch from one thread to another. This counter is indicative of hardware devices functioning properly. Context switches per request (Context Switches/sec divided by Webservice\Total Method Requests/sec) should be low. As a general rule, context switching rates of less than 5,000 per second per processor are not worth worrying about. If context switching rates exceed 15,000 per second per processor, then there is a constraint. If this counter suddenly starts to increase, it may be an indicating of a malfunctioning device. If CPU utilization is low and very low level of context switching, it could mean that threads are getting blocked. Increased %Processor Time
Processor Processor(_Total)\% Processor Time % Processor Time is the percentage of elapsed time that the processor spends to execute a non-Idle thread. Less than 85 percent. This counter suggests either there is too much load on the system or there is some other factor causing the CPU to shoot up. This counter shall be used with % Privileged Time or Processor Queue Length System Unstability
Processor System \% Processor Queue Length % Processor Queue indicates the collection of threads that are ready but not able to be executed because there are threads in execution. Less than 2 This counter indicates there is too much load on the system or application threading logic needs improvement, If CPU %Time is constantly 85 percent or higher with PQL of 2 or more, then CPU is the bottleneck. If the CPU time is low with PQL of 2 or more, then threading logic shall be investigated. System Unstability
Disk LogicalDisk\% Idle Time % Idle Time reports the percentage of time during the sample interval that the disk was idle. Greater than 5 % If the value is less than 5% means disk is really busy. The Value should be greater than 20% typically. Increased %Processor Time
Disk LogicalDisk\Average Disk sec/Read Avg. Disk sec/Read is the average time, in seconds, of a read of data from the disk. Less than .015 sec if the value is more than .025 sec that means disk is bottleneck. Response Time would be affected. Increase Response Time
Disk LogicalDisk\Average Disk sec/Write Avg. Disk sec/Write is the average time, in seconds, of a write of data to the disk. Less than .015 sec if the value is more than .025 sec that means disk is bottleneck. Response Time would be affected. Increase Response Time
Memory Memory\Pool nonpaged Bytes This counter displays the last observed value of an area in system memory that contains objects that cannot be written to disk because they are still being referenced. Steady or No more than 10 percent increase since system start up Increase of this counter indicates a potential memory leak. System Unstability
Memory Memory\Available MBytes This counter gives the amount of physical memory available to processes running on the computer in Megabytes. 25% of physical memory. It would mean the application is not getting enough memory to operate and could result in application slowness. A gradual decline in Available memory may indicate a memory leak. Memory Leak
Memory Memory\Pages/sec Pages/sec is the rate at which pages are read from or written to disk to resolve hard page faults. Multiply this value with Physical Disk\ Avg.Disk sec/Transder and if the product of those 2 counters exceeds 0.1, then this would indicatres that RAM shall be added, because paging is taking more than 10 percent of disk access time Memory Bottleneck
Network Network Interface\Bytes Total/sec Indicates the rate when data send and received on the network adapter Less than 80% of the network bandwidth This counter will indicate if the Network bandwidth on the Adapter is getting bottleneck Adapter Bottleneck
Server Server \Bytes Total/sec Indicates the rate when data send and received on the network Less than 50% of the network bandwidth This counter will indicate if the Network bandwidth is getting bottleneck Bandwidth Bottleneck

Memory Analysis

Memory analysis is important to improve web application’s performance. To know the shortage of memory you can look at the frequency of the paging.

Paging is the process of moving pages (block of data) from RAM to hard disk to free the memory for other processes. Because of paging you can use more memory than actually exists, but more paging can cause low performance.

Monitoring Memory Usage

To monitor for a low-memory condition, use the following object counters:

  • Memory: Available Bytes

The Available Bytes counter indicates how many bytes of memory are currently available for use by processes.

Low values (e.g. 10 MB) for the Available Bytes counter can indicate that there is an overall shortage of memory on the server or that an application is not releasing memory.

  • Memory: Pages/sec

The Pages/sec counter indicates the number of pages that either were retrieved from hard disk due to hard page faults or written to hard disk to free space in the working set due to page faults.

A high rate for the Pages/sec counter could indicate excessive paging.

  • Memory: Page Faults/sec

Monitor the Memory: Page Faults/sec counter to make sure that the disk activity is not caused by paging.

Page faults/sec is the sum of hard and soft page faults. A soft page fault occurs when then the requested page is found elsewhere in physical memory. A hard page fault occurs when the requested page must be retrieved from disk.

Monitoring Excessive Paging Activity

As we know paging includes disk activity so to monitor paging we can monitor hard disk activities and to do that make sure to track disk usage counters such as the following along with memory counters:

  • Logical Disk\% Disk Time
  • Physical Disk\Avg. Disk Queue Length

If a low rate of page-read operations coincides with high values for % Disk Time and Avg. Disk Queue Length, there could be a disk bottleneck. However, if an increase in queue length is not accompanied by a decrease in the pages-read rate, then a memory shortage exists.

What is 90th Percentile Response Time?

The 90th percentile response time value is the value for which 90% of the data points are smaller and 10% are bigger.

To calculate the 90th percentile value:

1.       Sort the transaction instances by their value.

2.       Remove the top 10% instances.

3.       The highest value left is the 90th percentile.

Consider the below example:

There are 10 instances of transaction “TRANSACTION 01” with the values

4 sec 2 sec 3 sec 1 sec 5 sec 17 sec 8 sec 7 sec 6 sec 10 sec



1.       Sort values from best to worst:

1 sec 2 sec 3 sec 4 sec 5 sec 6 sec 7 sec 8 sec 10 sec 17 sec



2.       Remove top 10%, In our case i.e. value “17 sec

1 sec 2 sec 3 sec 4 sec 5 sec 6 sec 7 sec 8 sec 10 sec



3.       The highest value left is the 90th percentile value i.e. “10 sec” is the 90th percentile value.

Scripting for Microsoft ASP.NET VIEWSTATE in VuGen LoadRunner

ASP.NET web applications maintain state of data by passing hidden _VIEWSTATE field encoded using base64 algorithm. The state of the page is restored based on the VIEWSTATE. If on the client side, the value of textbox has been changed, VIEWSTATE will now have the new value. (Obviously, this stage does not happen when the page is requested for the first time).

Let’s see how Web Applications developer store objects in VIEWSTATE,


// Storing a Customer object in view state.
Customer cust = new Customer (“Dilip”, “Kalyankar”);
ViewState[“CurrentCustomer”] = cust;

// Retrieve a student from view state.
Customer cust = (Customer) ViewState[“CurrentCustomer”];


Now we will see how to capture and replace VIEWSTATE in a recorded script,

1. First set the html parameter length to maximum so that it can store VIEWSTATE, You can adjust this number after examining the captured value.


web_set_max_html_param_len(“50000”);

2. Capture the VIEWSTATE value when it is returned from the server using web_reg_save_param() function

web_reg_save_param(“MyViewState”,”LB=\”__VIEWSTATE\” value=\””,”RB=\””,”ORD=ALL”,LAST);

3. Substitute the captured value with the parameter name in which you captured the VIEWSTATE:

“Name=__VIEWSTATE”, “value={MyViewState}”, ENDITEM,

4. Run the script (with enabled advanced log) in VuGen and verify that the correlation function is capturing the right value.

Writing Performance Test Plan

Below is the outline of the Performance Test Plan:

1. Document Information

  • About this Document
  • Document Approvals

2. Application Information

  • Application Name
  • Application Description
  • Application Platform/Runtime Environment
  • Application Flow
  • Application/System Technologies Employed

3. Key Business Function Definition

  • Key Business Functions
  • Performance Center Transaction Mapping
  • Key Business Function

4. Performance Test Workload Model

  • User-Based Workload Model
  • Business Function-Based Workload Model

5. Test Scenarios

  • Test Scenario Scope

6. Test User Profiles and Ids

  • Test User Profiles
  • Test Id and Entitlements Provisioning

7. Test Data

  • Test Data Requirements
  • Test Data Assumptions and Risks

8. Timeline

  • Testing Timeline

Appendix: Definitions

Appendix: Terms / Acronyms / Abbreviations

Base 64 encoding in loadRunner

There are some secure applications which provide certain functionality to protect important data. Those applications use algorithms to encode secured data before sending from the client and decoding algorithms to decode data at server side.
There are many algorithms which do encoding and decoding for you. Let us consider one,
Base-64 algorithm encodes the plain text into 64-bit encoded format.
e.g.
[cce lang=”c”]

/*Plain Text: */
“action=INIT&state=ViewTransfer_INIT&txnPayPayRefNumber=000000&txnBatchRefNumber=111111”
/*Encoded Text:*/
“YWN0aW9uPUlOSVQmc3RhdGU9Vmlld1RyYW5zZmVyX0lOSVQmdHhuUGF5UGF5UmVmTnVtYmVyPTAwMDAwMCZ0eG5CYXRjaFJlZk51bWJlcj0xMTExMTE=”
[/cce]
Secured applications use this type of encoder to encode data at client side, so that no one can read the important data. Applications use JavaScript to get this done at client side.
When you record the script for this type of applications you will find some random characters in script. If application is sending these random characters in the request body then you should know what these characters stand for. Without decoding that encoded string you cannot correlate them. To correlate them follow the below steps,
• From the generation log find out which encoding algorithm application is using.
• There are many websites available on the internet, which provide online tool for encoding and decoding. E.g. http://www.rbl.jp/base64.php
• Decode the encoded string and do correlation for those values.
• Prepare the exact string using string tokenizer
• Encode the final string and put it in the request.
[cce lang=”c”]
web_custom_request(“saveParameters.jsp”,
“URL=https://example.com: saveParameters.jsp”,
“Method=POST”,
“TargetFrame=”,
“Resource=0”,
“RecContentType=text/html”,
“Snapshot=t13.inf”,
“Mode=HTML”,
“Body=page=ViewTransfer&params=YWN0aW9uPUlOSVQmc3RhdGU9Vmlld1RyYW5zZmVyX0lOSVQmdHhuUGF5UGF5UmVmTnVtYmVyPTAwMDAwMCZ0eG5CYXRjaFJlZk51bWJlcj0xMTExMTE=
“, LAST);
[/cce]
Decode the above encoded string and capture the dynamic values accordingly. Create a string with dynamic values and pass it as parameter to the base-64 encoding algorithm function.
Put the below given functions in globals.h
[cce lang=”c”]
#ifndef _GLOBALS_H
#define _GLOBALS_H
/* Include Files*/
#include “lrun.h”
#include “web_api.h”
#include “lrw_custom_body.h”
/* Global Variables*/
#endif // _GLOBALS_H
/*Converting into Base-64 encoding*/
char *convert( char *src)
{
int dest_size;
char *deststr;
/* Allocate dest buffer*/
dest_size = 1 + ((strlen(src)+2)/3*4);
deststr = (char *)malloc(dest_size);
memset(deststr,0,dest_size);
base64encode(src, deststr, dest_size);
return deststr;
}
void base64encode(char *src, char *dest, int len)
/* Encodes a buffer to base64*/
{
char base64encode_lut[] = {
‘A’,’B’,’C’,’D’,’E’,’F’,’G’,’H’,’I’,’J’,’K’,’L’,’M’,’N’,’O’,’P’,’Q’,
‘R’,’S’,’T’,’U’,’V’,’W’,’X’,’Y’,’Z’,’a’,’b’,’c’,’d’,’e’,’f’,’g’,’h’,
‘i’,’j’,’k’,’l’,’m’,’n’,’o’,’p’,’q’,’r’,’s’,’t’,’u’,’v’,’w’,’x’,’y’,
‘z’,’0′,’1′,’2′,’3′,’4′,’5′,’6′,’7′,’8′,’9′,’+’,’/’,’=’};
int i=0, slen=strlen(src);
for(i=0;i>0x2];
*(dest++)=base64encode_lut[(*src&0x3)<<0x4|(*(src+1)&0xF0)>>0x4];
*(dest++)=((i+1)<<0x2|(*(src+2)&0xC0)>>0x6]:’=’;
*(dest++)=((i+2)
}
*dest=’\0′; /* Append terminator*/
}
[/cce]

Preparing the query string & passing to base-64 encoding
[cce lang=”c”]
char *getParam(char *str, char *param1, char *param2,char *param3)
{
char *temp;
char *src, *target;
char param[200];
temp=””;
src=””;
target=””;
temp=str;
strcpy(param,””);
strcat(param,temp);
strcat(param,param1);
strcat(param, “txnPayPayRefNumber=”);
strcat(param,param2);
strcat(param, “txnBatchRefNumber=”);
strcat(param,param3);
src=(char *) param;
target=convert(src);
return target;
}
[/cce]

Call the above functions as below in the script, capture the return value into a parameter & substitute it.
[cce lang=”c”]
lr_save_string( getParam(“action=INIT&state=ViewTransfer_”,
“INIT”,
lr_eval_string(“{p_PayPayRefNumber}”),
lr_eval_string(“{p_BatchRefNumber}”)),
“param” );
web_custom_request(“saveParams.jsp”,
“URL=https://example/saveParams.jsp”,
“Method=POST”,
“Resource=0”,
“RecContentType=text/html”,
“Referer=https://example.com:8517/ExecPendingTransfer.do”,
“Snapshot=t13.inf”,
“Mode=HTML”,
“Body=page=ViewTransferparams={param}”,
LAST);
[/cce]
Final string:
action=INIT&state=ViewTransfer_INIT&txnPayPayRefNumber=000000&txnBatchRefNumber=111111

LoadRunner Flex / amf Protocol


Flex

Flex software development kit provides user interface components like button, data grids, layouts. Flex work as a presentation layer for n-tier web applications.

It maintains a state, where it can send, receive data from server side without reloading the page. Flex works just like AJAX.

Working on Flex Protocol

1. Choose multi protocol recording and select Web, Flex or Web, AMF protocol depending on the requirement.

2. Set the Flex code generation options and don’t forget to provide path of the flex jar files. To Get valid xml we require jar files, which were used by developers like,

  • flex-messaging-core.jar
  • flex-messaging-common.jar
  • flex-messaging-opt.jar
  • flex-messaging-proxy.jar
  • flex-messaging-remoting.jar

3. Start recording the business flow. After completion of recording, in the script you can see the there are flex_amf_call with BEGIN_ARGUMENTS, then XML formatted values and END_ARGUMENTS.

4. To correlate the values in amf request, go to tree view .

5. Find the dynamic value in xml grid of the response and right click and create a parameter.

6. Provide file name and parameter name where you want to capture the dynamic value.

7. Then replace all the instances of that value in the script with the created parameter.

P.S. : Thanks to Rahul Awasthi.

LoadRunner Naming Standards

S.No Area Description
1 Script Name ApplicationName_Goal_NumberIndex_BusinessTransactionName_Version
2 Script Description Verify whether the Script included with below fields with description in Vuser_init block
Author
Script Objective
Script Description
Revision History (Version and Comments)
3 Script Enhancement Verify each business transaction corresponds to a unique action
Verify any spaces exist while defining Action names
Verify Action names are defined with first letter in Capitals
Verify While naming Action names, underscore is used as separator
Verify whether all other actions are referring to the Main Action
Verify whether all functions referred within Main Action are spelt correctly
4 Error Handling Verify whether Conditional Pass/Failure is handled with LR_PASS and LR_FAIL.LR_AUTO should not be used
Verfiy whether  rules are created to enable global error handling mechanism
5 Parameterization Verify whether ” p_ ” is used prior to the parameter name to differentiate parameter from a variable
Verify whether all dynamic values generated from or passed to the server are correlated using web_reg_save_param
Verify whether all domain URL’s, UserID/Passwored and controlled variables are parameterized using lr_save_string or as a parameter list file
6 Correlation Verify whethere While Correlating dynamic values is Correlation rule file is used?
7 Variable Declaration Verify whether all Global Variables are placed outside of Functions/Actions?
Verify whether  Vuser_init action contains all of the variable declarations for file manipulation
Verify whether all variable declarations follow below naming convention:
iName = integer
sName = Char or Char *
8 Comments Verify whether  “//” are used instead of /* */ while commenting.
Verify whether every function/action is included with descriptive comments
9 Run Time Settings Verify whether in Miscellaneous settings if the below options are checked
Run Vuser as a thread
Define each action as a transaction
Verify whether in Preferences settings, check whether below options are selected
Hits per second and HTTP Codes
Pages per second(HTML mode only)
Response byte per second
Non Critical resource errors as warning
Verify whether  text check, image and content check are used with discretion
10 Load Test Name Verify naming convention for Load Test Name is followed in the below format:
Baseline_10Vusers
11 Load Test Description Verify Load test description format as below:
Ex: Measure the system performance of application release 1.0 using 10 Virtual users of the following business transactions



Prerequisite of Load Test Execution

  1. Ensure that time slot is booked in calendar on your project name.
  2. Check that application is working manually.
  3. Check whether the entire server’s are up and in normal condition.
  4. If in SiteScope the values against summary of any server are ‘NA’ then restart those servers.
  5. If time permits run each script individually for five iterations in VuGen.
  6. Refresh the script in Performance Center with the latest version.
  7. Check the runtime settings of each script.
  8. Check the scenario settings and number of users configured for each script with workload Model.
  9. Check for IP Spoofing whether enabled or disabled and change it as per the requirement.
  10. Before starting the scenario verify that each script duration, ramp up and ramp down time is correct.
  11. Check whether Monitor profiles are configured correctly.
  12. Check whether windows profilers are added.
  13. Book the same time slot in Performance Center and manually select Load Generators and Controller.
  14. Configure number of Load Generator, Controller as per allocated in time slot and take only those Generators and Controller, which are free.
  15. When ready to run the test send an alert mail to all related groups.