Stress Testing Repository: Websites & Computers

The following is an article on load testing by Chris, chief engineer at webperformance.

"Having performance testing results from frequent points in the development timeline can help developers understand the performance impact of various code and system changes. When testing early in the development process on a test rig that is not equivalent to production, the performance numbers are not valuable in their own right, but _changes_ in the performance numbers can extremely valuable. These changes can reveal newly-introduced performance bottlenecks that should be investigated.

For each new development iteration, the previous test (which does not exercise the new functions) should be repeated to determine if the changes had an effect on the performance of previously-tested parts of the system. Then new test scenarios should be added to exercise the new functions and a second test executed to both assess the performance of the new functions under load and provide a baseline for the next test iteration.

In addition to alerting developers to unexpected performance changes, the overall performance trend during the development process can help project managers assess the overall state of progress and determine if more resources should be allocated to analyze and implement performance improvements.

Chris, Chief Engineer"

I think Chris always writes a good article.

The following is an article about a software development conference in Prague.

"Since most of our readers are software/web developers with global reach, we want to share with you info about an upcoming international conference where you can learn cool new coding techniques and hang out with some rock stars in the developer industry. We believe in the quality of this conference so much that we are offering a special bonus to attendees: 5,000 free virtual users in our stress testing tool!

If you are a registered user, simply email a copy of your registration confirmation to support@loadstorm.com. We will drop the 5,000 Storm on Demand users into your account. Oh wait...if you don't have a free LoadStorm account, you can get one easily and quickly by clicking here

Stress testing tool free sign-up

There is no credit card required. You will get an email from our system that contains a confirmation link, please click that and you are ready to go. LoadStorm gives you a free account forever - not just a trial. No feature limitations for creating stress test scenarios. The only limit is 25 concurrent users in each test.

As a registered user, you will be able to run a stress test as large as 5,000 concurrent users. Or you may choose to use them in 10 tests of 500 each. Your choice to split them up anyway you choose. They never expire.

The following is an article on load testing in the clouds.

"The forecast is …. Cloudy
And everybody is loving it. Cloud computing is changing the IT scenario as much as the web did, back in the 1990s. Now a garage setup can deploy its services on the web, on a shoe-string budget and still aspire to scale to the size of the Googles, FaceBooks and Amazons of the world. The deployment platform will scale as and when the business grows, that too, in a quantum which is fine grained. For big companies, cloud computing provides enormous advantages of consolidation, manageability, flexibility and sand-boxing. For many businesses, cloud computing can provide scale-out capability with core infrastructure being on-premise.

What’s the catch?
Not all organizations may benefit from a cloud computing platform. Not all applications may benefit. And most importantly, a legacy application ported in a naïve fashion to a cloud computing platform may not utilize any of the platform's USPs at all. Deciding the optimal cloud vendor or platform for your requirements is a complex task.

As cloud platforms continue to evolve, cloud vendors are coming up new products and adding features to their existing offerings. These cloud infrastructure providers need skilled development partners who can provide them with innovative and quality services.

Our Value Proposition

The following is a good article by Erin Harrison on load testing.

"If you attempted to scan Target’s (News - Alert) website for the latest sales yesterday, you may have been among the many that were cut off from accessing Target.com. For the second time in six weeks, Target Corp.’s website crashed interrupting online shopping for the retail giant.

According to media reports, a Target spokesman confirmed that Target.com went down at 11:50 a.m. CDT (News - Alert) Tuesday.

“We are working diligently to bring the site back online and to ensure that it is operating efficiently,” said spokesman Lee Henderson, the Chicago Tribune reported.

Target’s website also crashed in September following the release of its new line of Missoni apparel. The Associated Press (News - Alert) said that Henderson confirmed that Target's site was down for 2 1/2 hours midday Tuesday without giving a reason for the site failure.

“Minneapolis-based Target recently took over running its website from Amazon.com, and the Missoni launch was the first major initiative for the retailer since that,” according to the Tribune.

Target’s website crashes are another example of the importance of analyzing traffic data and load testing.

The following is an article about Target.com's second site crash.

" For the second time in six weeks, Target's website crashed Tuesday, and that is scary news for any major retailer heading into the holiday shopping season.

Target's site went down for about two and a half hours Tuesday afternoon, a company spokesman confirmed to Computerworld. However, he declined to say what caused the problem.

On Sept. 13, Target's site went down in a well-publicized crash caused by a flood of shoppers going online to scoop up a new line of high-end clothing and home accessories. Soon after the retailer began offering products from the Missoni Italian fashion house, the site began to waver and then crashed under the pressure of so many online shoppers.

Then, on Tuesday, the site suffered another outage, which may have Target's corporate executives anxious and putting pressure on IT to strengthen the site and work out any bugs before the major holiday e-commerce rush commences.

The following is an article on load testing by Chris Merrill, chief engineer at webperformance.

"When a load test is complete, you will be asked “How did we do?” Do you know how will you answer that question? Our customers come to us and know, for example, that they need their site to handle 1000 users, but they frequently cannot tell us what “handle 1000 users” means to them. You will need to know which metrics are important and what the goals for those metrics are – preferably long before you start testing.

The first step is to determine what you should be measuring. For websites, you will typically be interested in page duration – how long it takes to display a page in the browser. This will usually be associated with another metric for the amount of load – the most common is simultaneous users. Other common metrics for load are hits/sec, transactions/min and page-views/hour.

Once you know what should be measured, you must assign goals to them. For example “All pages must load within 4 seconds under a load of 1000 users.” Without specific, concrete goals, you will not be able to give a specific, concrete answer to the “How did we do?” question.

The following is an article by Bill Kalma that I found interesting about cloud computing and load testing.

"The first generation of cloud computing was revolutionary in that it added business value to organizations by reducing development time, eliminating the need to procure infrastructure, providing massive scaling potential, establishing scale through multi-tenancy and by allowing IT people to focus on solving business problems versus technical ones. As cloud-based applications grew in prominence, they fought to achieve parity with legacy on-premise software solutions that were feature/functionality rich. These goals were achieved in short order, and thus the new generation of cloud solutions was born.
Web 2.0 goes far beyond parity with legacy systems and leverages the power of information sharing, collaboration and the social grid to achieve value that would be impossible with a paradigm of pushing information out from a single location. The new age cloud applications are powerful because they break down silos and simplify the process of converting data into information. As Web 2.0 systems in the consumer space evolve, companies are taking notice and looking to unleash the same power in their enterprise applications.

The following is an article by Bill Kalma that I found interesting about cloud computing and load testing.

"The first generation of cloud computing was revolutionary in that it added business value to organizations by reducing development time, eliminating the need to procure infrastructure, providing massive scaling potential, establishing scale through multi-tenancy and by allowing IT people to focus on solving business problems versus technical ones. As cloud-based applications grew in prominence, they fought to achieve parity with legacy on-premise software solutions that were feature/functionality rich. These goals were achieved in short order, and thus the new generation of cloud solutions was born.
Web 2.0 goes far beyond parity with legacy systems and leverages the power of information sharing, collaboration and the social grid to achieve value that would be impossible with a paradigm of pushing information out from a single location. The new age cloud applications are powerful because they break down silos and simplify the process of converting data into information. As Web 2.0 systems in the consumer space evolve, companies are taking notice and looking to unleash the same power in their enterprise applications.

The following is a good article on load testing by Joel Weierman.

"We recently completed a very large load test of a storage platform for one of our Customers. The goal of the testing was to verify that the storage platform could handle the upload of multiple terabytes of data within a 60-minute period that simulated peak usage.

Before Cloud Computing and the advent of a true hourly computing model, this would have meant provisioning hundreds of servers in multiple data centers and ensuring the proper upstream circuits were in place to handle the traffic. This effort alone would have previously taken many weeks to turn around with all of the headaches of making sure the hardware is properly racked/stacked, memory is in stock, server builds working as expected, etc, etc.

I found this article to be a good article on load testing.

"There are four key aspects that you should know about LoadStorm:

Testimonials from our customers. Does our tool work? Are we trustworthy?
How to contact us. Get support or answer sales questions via email, telephone, or submit a form.
Our company values, products, skills, and history.
Partners and alliances with LoadStorm.

LoadStorm Testimonials
Every company will tell you how they are the best or how great their product is. So what?! It's not believable until you can know what the customers think. LoadStorm has over 6,000 happy customers. Most are small to medium web development firms or web creative agencies. Some are very large global companies. Here's one selected testimonial, but visit our testimonial page to read dozens more.

"Thanks for your help and quick response. Your website (LoadStorm) is a very useful tool that's helping me to improve my system." Alan Cheung, Manager - Technical Services, Dow Jones Publishing Company

Contacting Us for Sales Questions or Product Support
Email for immediate help: support@loadstorm.com

Call to speak with one of our team: 1 (970) 389-1899

The following is a blog about load testing by My PHP Net.

"App design starts with a notion of an idea. That idea builds into plans and concepts. From there it’s not far before the programmers get into the act and begin to really bring life to the application. The final phase would be the load testing. The difference between a successful launch and a total flop can all come down to the mobile application testing. Get that right and success can be yours for the taking.
There was a time not too long ago when mobile application testing only meant a trying out the load testing on a few models. Today there are dozens of different operating platforms using a variety of user interfaces. Not only could this include standard finger keyboard typing but also voice command activation. Then there are all the variations of application triggers like swiping or scrolling and panning and the new pinch/stretch to zoom method. Of course, there is also the very popular “shake” method. All of these various models and triggers need to go through the mobile application testing process in order to reach the widest customer base.

The following is an article on load testing written by Erin Harrison.

"All websites are vulnerable to crashing. One needs to look no further than several recent instances in which major company websites have crashed because of high traffic peaks.

Earlier this month, fans of JK Rowling’s magical stories were unable to book tickets for “The Making of Harry Potter” because of a website error.

Another case involved the Apple (News - Alert) website, which crashed a few weeks ago likely due to the high volume of traffic surrounding the iPhone 4S release announcement. Visitors to the website got an error message: “Access Denied. You don’t have permission to access ‘http://www.apple.com/’ on this server,” the International Business Times reported.

And last month, Bank of America’s homepage and online banking experienced problems the day after the company said it would start charging a $5 monthly fee for customers who make debit card purchases, according to an Associated Press (News - Alert) report.

Not only did these occurrences annoy consumers, with Harry Potter fans venting their frustration on social networking site Twitter (News - Alert), some saying it took two hours to book, according to The Mirror, they also created a temporary outage for these businesses online.

The following is an article by Compuware on load testing.

The following is an article from Compuware on load testing, written by Stefan Karytko.

"It is important to understand the benefits ramping load can have on load test results. For many applications it is unrealistic to apply a peak amount of user traffic immediately at the beginning of a test. Most real world applications do not encounter situations that cause them to reach peak users and peak load all at once.
In most cases, traffic builds up over a period of time before reaching a peak load. When a load test gradually ramps load, this process not only allows the system to warm-up and ease into heavier loads, it also provides a more realistic load scenario.
Another benefit of ramping is that the arrival timing of the virtual users being used occurs in a controlled manner which helps to provide even and consistent load during a load test. Figure 1 and Figure 2 shows an example of results from a test that was configured with no ramping phase at the beginning of the test.

Figure 1 – Test A - Virtual Users
In Figure 2 the Transactions per Minute results have a high frequency, with the transaction rate spiking up and then dipping down on a three-minute frequency. This is not desirable as the volatility of the transaction rate does not provide consistent and predictable load volume, making the interpretation of results more difficult.

Another good article on performance testing from Alon Girmonsky.

"Every good test plan should include testing your application under different load scenarios such as: gradual load, stress, endurance, etc.

However before applying any load scenario it is a good practice to follow two very important steps:

Define your performance expectations
Identify your application's performance limits
If these steps are skipped, we will run the risk of under or over testing our application.

Adaptive load scenario adjusts the load level according to your application's performance in order to identify its limits.

One way to identify our application's performance limits is by trying out numerous load levels to identify the appropriate load level that our application can support with an acceptable end-user experience. This process can require a few iterations and is not always accurate.

To simplify the process of identifying performance limitations, we have recently introduced the adaptive load scenario. This scenario adapts the load level to the performance level of the application under test. The adaptation is made according to an acceptable service level that is defined prior to the test.

The following is another good article on load testing by Alon Girmonsky.

"Running a large-scale load test involves multiple challenges. The first challenge is to create the right testing environment that can realistically simulate thousands of concurrent users.

To simulate such a load realistically requires tens of dedicated servers residing on a high bandwidth network.

Any compromise with the testing environment can cause the test to appear to have excellent results providing false confidence when in fact the testing environment did not have sufficient strength to provide a realistic simulation.

Two days ago, Tescom Singapore, one of PerformanceXpert's valued re-sellers in the far east ran a 12,000 users load test.

View Report

The large-scale test was comprised out of 45 dedicated servers scattered all over the world together simulating a load of 12,000 concurrent users.

With PerformanceXpert each dedicated server enjoys

a dual CPU, 1.7GB of memory and up to 100Mbps of bandwidth connection.

Tescom created the load script that simulated several groups of users, each executing a different business process. For example: 10% of the users will log in and view some pages, 30% will search and read articles and the rest will do general browsing. All groups will operate in parallel creating the most realistic simulation related to the website under test.

The following is an article on load testing by Alon Girmonsky.

"Web site performance testing while simulating a load scenario where the actual traffic originates from with-in the corporate Local Area Network can be compared to testing a car's performance when it is actually situated on the garage's lift. You floor the gas paddle, go into fifth gear and easily enjoy 200 mph.

Only the car is still on the lift and in fact not moving ...

There is no friction on the road, no cars you need to bypass, no traffic lights, no narrow lanes .. simply ideal conditions.

This how it goes when you test from with-in the corporate LAN. You do not go through the firewall, nor through the cache, the load balancer, the network equipment, the modem, the routers along the way. No collisions of packets or re-transmits.

In the real world there are numerous factors that affect your website performance in a load scenario. These are not included while testing with-in the corporate LAN. Testing from with-in the corporate LAN indeed contributes as it does test the functionality of your website even under load, but it certainly does not reflect the system performance under load, as it would happen in real-world load scenario.

Cloud-testing makes use of cloud-computing resources in order to simulate real users who come directly from the internet.

The following is an article about load testing by performanceXpert.

"Comparing bandwidth on a AWS Micro instance and a Rackspace 512MB server

Cloud computing is an emerging trend which enables the rapid deployment of new web based applications. While cloud computing providers do not commit to a performance level or end-user experience, they encourage their users to test and benchmark before deploying.

When you host a server at your own facility, you know according to your network interface and the bandwidth provided by your ISP what to expect in-terms of bandwidth allocation. If you find yourself asking what your bandwidth could be via a cloud provider, then this article is for you.

The goal of this article is to understand if there are any bandwidth limitations associated with the cloud providers, AWS and Rackspace, and if there are such limitations that can lead to bottlenecks, what are they.

Choosing the cloud resource

The following is a good article on load testing and performance testing from Soasta.

"Most companies – if they are testing at all – are using performance testing and load testing tools designed for a different era, when 1,000 concurrent users was high traffic and most users were employees.

SOASTA CloudTest® offers companies with consumer facing websites a transformational approach for ensuring high performance under normal, as well as extreme traffic conditions. With the advent of social media, focused marketing events, and an ever-growing stream of new mobile devices, this has become an increasingly dicey proposition – making performance testing and load testing tools more important than ever. Without performance testing, external events such as the Super Bowl, Cyber-Monday, Tax Day, Valentines Day, a significant drop in interest rates or even news about a celebrity can suddenly drive unforeseen traffic to a web site turning success into embarrassment if the site is slow, or worse, if the site crashes under the load.

Accurate, Scalable and Affordable

The following is a good article on load testing by Stephanie Klaiber.

"The Internet is ever evolving and shows no sign of stopping. According to a ComScore Q3 State of the Internet report:

215.1 million Americans used the Internet in July 2011
The average user was online 25.3 days in the month, for a total of 34.8 hours, and viewed 3,029 pages
86% of Internet visitors viewed an online video in the U.S., for an average of 18.2 hours each
Businesses need to follow and understand Internet trends so they can best serve their customers and develop business objectives that drive their bottom line. This surge in Internet users means businesses need to prepare their corporate websites and applications to handle spikes in web traffic.

Website monitoring and load testing are two essential tools that help businesses keep up with these trends. Real-time external monitoring and full-service load testing help organizations gauge website and application performance at various levels of load from the end-user perspective. This is key to identifying and fixing problems before customers ever become aware of them.