Stress Testing Repository: Websites & Computers

Testing cloud applications with an exhaustive approach to performance, security and robustness is a more complex undertaking than many managers and users might at first estimate it to be. Three reasons to test in the cloud: 1. "Load testing a cloud application on-premise never truly reflects the impact of real-world data volumes as they spike erratically. Actually, this is pretty much true for non-cloud-based applications, but it is a great truism for cloud apps too. 2. Testing inside the cloud model should generally always be quicker, more cost-effective and should pave the way toward higher-quality applications. 3. Performance validation inside post production environments can deliver "consumption-based" performance testing of Web 2.0 and mobile apps charged in an on-demand pay-per-use model. Yes - testing becomes more service-based."

What happens if I don't test?

"Failure to follow testing procedures commensurate with the "mission criticality" of the application (and data) in question can see the IT shop hit performance bottlenecks, which in turn can have a direct impact on business performance and the customer's "time-to-market" (to use the marketing manager's favored term) and make profits."

I think this blog tells why it is so important to stress test and load test which is what performance testing is all about.

http://www.sys-con.com/node/2162380

Can you imagine having millions invested in advertising for the Super Bowl and the online ad fails? That is what happened to several of the big companies who had not performance tested their websites correctly.

"The Coke Polar Bears Facebook page and main website was unavailable in five languages."

"Act of Valor had a very impressive ad, but their website was less so. According to Yottaa's monitoring, it seemed every time the spot ran the site crashed, with more than six outages of five minutes each. The site was also five times slower yesterday during game time."

"Acura cars had problems too. "The launch of their new performance car was met with poor performance from their website." Their home page was fine, but the call to action pages were saturated with traffic on Sunday."

"Cars.com, TaxACT.com, GoDaddy.com and History.com all fared really well during the Super Bowl, according to Bob Buffone, writing on Yottaa's blog. He emphasizes that live stress testing is critical at this big event-driven moments, and instrumenting what is going on during the event is essential if the sites are going to meet the anticipated demand."

"$3.5M for 30 seconds and can't find ways to keep website running?

I have a good blog on load testing that I think many people will find interesting. It talks about getting the right people involved when load testing. Your testing results will be more valuable as a result of their involvement.

"If you are using an outside testing resource service or if you are running one big test on a production system, then it is crucial to have the right people involved in the planning and execution of the test. This list includes:

Network administrators
System developers and/or integrators
Web- and app-server administrators
Database admins
Architects
Content Experts
3rd-party services, such as your hosting companies

During the load test, we recommend making live data available during the test to all of the participants (e.g. via a web-share of your desktop). This will give them an opportunity to view test data alongside other metrics that they may be monitoring during the test. We use this approach extensively in our testing services and find that it also gives the participants a better view into the testing process that improves their understanding as well as their contribution to the effort.

Each of these groups brings a specific view of a web-based system and a unique set of talents for identifying, diagnosing and fixing the performance problems that are encountered."

I found this to be an interesting article on how important stress-testing, load-testing, performance testing can be. Can you imagine the amount of hits Domino's website got on Sunday for the Super Bowl? Those people knew their website had to be strong, able to handle a large number of callers, and perform exceptionally well.

Domino’s, a system of more than 9,541 worldwide restaurants, including at least 4,891 in the United States, monitors all aspects of its online operation during one of the largest pizza ordering and delivery days of the year, from pre-game load testing to after-the-game trash talking.

"What has Domino's done, IT systemwise, to prepare?

Every year we go through and do extreme load testing. We set up a bunch of synthetic transactions that basically emulates a human placing an order on our system. Because the process is automated, it gives us the ability to ramp up the number of orders to the point where it would be greater than what we’d expect to see during the Super Bowl. We do that because while you might have great estimates [of game-related business], you never know exactly what to expect and you definitely don’t want [systems] to be down during the Super Bowl because about 30 percent of all our orders come through the online channel."

"What makes the Super Bowl special?

The business owner needs to take a look at the different options and decide which of them to take advantage of to get the most out of the system. Cloud systems almost always require some sort of load testing and stress testing to maximize efficiency.

Businesses that test their cloud services on their own often save money, but the quality of load testing will usually make their customers suffer in the long run.

Given the scale of many testing companies, there is a remedy for every server setup that needs to be tested. In order to be ready for any situation that happens to come along, server testing companies have every load test program that they could possibly need.

For those of you out there who are new to website development, I think this article tells in pretty simple terms what you need to know about load testing.

FUNDAMENTALS OF LOAD TESTING
"Load testing is required at every stage in the life cycle of an application, in particular web applications. Load tests help identify performance problems during system design and development stages. For existing applications, the simulation becomes an aid to maintain, fine-tune and forecast future system needs.

Developing a mobile and web application can be great for any business. This product needs to perform at high performance levels at all times. Testing is the wisest place to spend your money. Don’t waste your budget or your time on a faulty application. Perform as many tests as possible before you upload it. Your app and your business will get profitable results.

website and mobile applications are a big business. For an application to perform well it must function properly under extreme loading. To compete in a big market, your application must run correctly. You need to feel 100% confident that your product is ready to be released before uploading it for users. With a comprehensive testing service, you will know if your application is running right. With thorough testing, your application will be ready for customers.

It is hard to believe mobile computing could sneak up on anyone. Blackberrys have been used by businesses for quite awhile now. The Apple iPhone is just months away from being on the market for five years. There is plenty of evidence to show that businesses in the IT world are having a hard time keeping up with capabilities to make it possible to manage and optimize traditional PCs and applications. Based on recent research only 18% of organizations are currently monitoring application performance on mobile devices

But there is hope that companies and vendors are catching up and providing the tools and capabilities needed to monitor, test and manage applications on mobile devices. In our recent research, 45% of businesses plan to add mobile performance monitoring in the next year and 50% of businesses that were identified as Best in Class are investing in mobile application performance testing.

Expect to see more innovation in solutions for mobile performance management, as well as interesting strategies by successful businesses who are dealing with mobile application performance issues.

With the start of the new year, now is the time to performance test your website to make sure you have everything going as it should. The last thing you want to happen is to be surprised by an unknown issue that was somehow able to sneak up on you and do immeasurable harm to your business by either crashing your website, or by slowing its performance to the extent that your visitors are turned away in the process.

You may stay on top of your web page’s functionality by administering a periodic website load test, and recognizing warning signs before any big issues present themselves, and view how different levels of traffic affect your website’s speed and efficiency, giving you all of the tools necessary to be active, so that you could focus on other, more relevant aspects of your business without fear of the unknown. Your website is an investment, and a part of your business, and just as you would take precautions to protect any other part of your work, the same care should be paid to this aspect as well, and in the event of a problem, you will be glad that you were ready.

The following is a blog from Apica CEO Sven Hammar on his top five cloud optimization predictions for 2012. These predictions are based on the company's interactions with customers, partners, analysts and industry thought-leaders.

"1. Social manifestation moves to .Net With social media setting the stage for more and more of the Internet's overall web traffic, a lot of the typical social manifestation will be moved to .Net this year. Product demonstrations and such from companies will become a much more frequent event on the web.

2. Cloud providers will offer customers both guarantees and independent validation of performance. This trend will then escalate to applications and services. In short, merely having cloud "uptime" will not suffice.

3. WebSocket provides exceptional performance Anyone serious about achieving exceptional performance will start to use WebSocket for transaction and data-intense applications. Updating real-time graphs and dealing with large volumes of transactions will be greatly improved by WebSocket. HTTP is great for content, but comes up short for frequent data updates.

I found an interesting blog, written by Scott Price at LoadStorm.com, about how business executives will place a high priority on web performance in 2012.

Executives are finding out how important it is to load test before the website is put out to consumers after seeing some of the biggest companies websites crash because their websites could not handle all the hits. Company executives don't want that to happen to them. Also, they have found faster websites make more money.

People were spending much more money this holiday season than they were in 2010. Department stores online sales were up 18 percent over December 2010. Christmas Day 2011 online shopping grew by 16.4 percent over 2010.

Money is the lifeblood of business. Better web performance results in higher sales. Web performance is hot.

As we have blogged before, the website crashes in 2011 were amazing. While the web was big in retailing, so were the website outages. Being prepared for a rush of Web traffic will be an important factor for retailers to focus on in 2012. When a website goes down, an online business has effectively shut its doors and is left wondering 'Will that visitor ever come back?' Website failures and performance problems can be minimized greatly by simply, conducting on-going testing and monitoring to avoid being the next big failure story.

"The following high profile events and related website crashes made the list:

1. Tickets for London 2012 Olympics Available for Purchase: In June, the Olympic committee announced 2.3 million tickets were available for purchase for the 2012 London Olympics. Excited fans rushed the site to find a "Sorry, we can not process your request at this time" message because the website could not handle the rush of visitors. London has gone to extraordinary lengths to win and host the Olympics, only to have its website crash when it opens its doors to the public.

SmartBear Software put together a list of the biggest web outages of 2011. Among the biggest web fails in 2011 were Bank of America, Netflix, Target and Reddit. Though a number of minor website outages and performance declines in 2011 were seen, there were fewer significant websites outages in 2011 than in 2010.

In 2010, web outages included Facebook, Twitter, MasterCard, Visa, JC Penney.

"Netflix, under fire for its price increases, "had a number of short website interruptions this year, which collectively resulted in prolonged service disruption to millions of Netflix subscribers." The first outage was March 22; "On this day, we tested Netflix's home page 20 times between 6:22 p.m. ET and 7:59 p.m. ET. Of those samples, only two successfully loaded the home page, revealing that the site was unusable for most users during that time frame," says SmartBear.
Another outage lasted several hours June 19, and the site "also experienced performance issues" July 20 and Aug. 22. (We recall Netflix having some log-in issues with its site in late November, as well.)
Reddit, Foursquare, Hootsuite, "et al":

Load testing is the process of putting demands on a software system. It can also put demands on an application or a device. It measures the response of either of those three. Load testing is done to determine how the software system, application or device, will respond under the normal load condition and anticipated peak load condition.

Load testing helps to determine the maximum degree of the operating capacity of the software system or application along with any bottlenecks and to check which element or error and bug is causing degradation of the software system or the application.

Sometimes the load levied on the software system or application is increased beyond the normal usage limits, in order to test the response and behavior of the software system at unusually high and peak loads. This kind of testing is known as stress testing. The load is usually kept so high that the expected result is nothing more than loads of errors.

Load testing is a term used with reliability testing, software performance testing, and volume testing.

Load testing can be said to refer to the practice of determining the expected usage of a software system, program or application by simulating multiple user situation in which many users are accessing the program at the same time.

Performance testing is defined as the technical investigation done to determine or validate the speed, scalability, and/or stability characteristics of the product under test. Performance-related activities, such as testing and tuning, are concerned with achieving response times, throughput, and resource-utilization levels that meet the performance objectives for the application under test.

Technically we do performance testing to prove speed. To see if the system is quick enough for end users to access without delays and frustrations. Also, we do performance testing to prove capacity. We want to see if infrastructure is capable enough to handle the load on the system. We do performance testing to check scalabilty. That means to ensure the system is capable enough to scale to the growth anticipated. we performance test for stability. To ensure the behavior of the system does not change under a heavy load of users.

Any product/project being released for the first time or a major release being released, performance testing has to be done.

As everyone knows, who has anything to do with building websites, performance is key to how successful the website will be. That is where cloud performance testing comes in. Typical cloud benefits are most notably lowered capital and operational costs, and support for distributed development teams. But cloud-based testing also changes the way the tests themselves are performed. These changes come at a time when more and more organizations are looking at software as their competitive differentiator. "Performance will make or break whether or not someone is going to use your app. If you think about the type of apps you use – enterprise or personal apps – performance is the determining factor, so make sure that performance is there and that you’re able to test appropriately for performance.” Cloud-based performance and load testing tools make it easier than ever before to ensure that internal enterprise apps as well as external customer-facing applications can handle user demand.

I believe everyone can see how performance testing in the cloud will be beneficial for their websites.

In order to have a good website, where you put your product out for sale, you must first load test the website to see if it can handle the amount of hits that you hope to get. If a website is slow, or if it should crash, you have most likely lost your customers to a competitor who has load tested his website before putting it out for customers to use. The key to good load testing is ensuring meaningful data that helps you identify ways to improve your system performance. In order to prevent negative impacts on our customers when implementing new services or launching new products, you need to first and foremost find potential problems in a testing environment.

Always remember to load test first!

With Black Friday and Cyber Monday done, the holiday season has truly begun. At this point, many companies have already done their prep for handling the high volume holidays. However, the holidays don't just represent the overwhelming onslaught of customer traffic and requests. They are also a great time for measuring just how well your service center and other departments are performing. Why?

"Because it’s basically your day-to-day business activities on max speed. So you had 72 percent first-call resolution rates in August. Great. What about the day after Christmas?" says Lauren Carlson, CRM analyst at Software Advice, a company that provide help desk software reviews.

Carlson recently wrote an article that discusses the benefits of using the holidays as a sort of stress test for your contact and service center. She suggests five areas to monitor:

• Employee onboarding
• Internal collaboration
• Peak load management
• Emergency response processes
• Customer satisfaction

You can read about each of these in more depth on her blog, but by analyzing these areas and identifying both high and low performing aspects in each, you can go into the new year with improved operations, resulting in improved overall performance.