Stress Testing Repository: Websites & Computers

Nobody likes to wait. In the past several weeks, web performance has gotten a lot of attention because of the time some websites take to load.

Why do we want faster pages? Slow pages cause web stress: Poorer concentration and increased agitation.

Why do we stress? It is mainly because of our short-term memory. Information stored in our short memory evaporates quickly. That is why we don't perform as well when we have to wait. Even after just 10 seconds, we have lost our chain of thought.

At any moment, there are three basic types of memory processing at work in your brain. There are sensory, short term memory, and working memory. Sensory memory works in 100ms bursts.
There are three types of sensory memories; iconic memory, sound memory, and touch memory. Iconic memory is every time you see something, this visual information is taken in by photoreceptor cells in your eyes and they send to the occipital lobe in your brain. Your short term memory can store information for 10-15 seconds, enough time for your working memory to process, manipulate, and control it.

The goal is to get page load times down to 100 millisecond to keep us from losing information through the cracks in our iconic memory. Also, this gives our short term and working memories time to soak up all they can before they start losing information.

Online retail sales increased by 14 percent last year. It is predicted that sales this year will be even higher. More and more shoppers are choosing to spend their money online. That should make website developers want to make sure their customers have a good web experience.

Five ways retailers can improve the internet shopping experience for consumers:

1. Know your customers, their profiles and their usage patterns.

2. Continually check on your customers’ experience.

3. Know what contributes to your customer’s experience.

4. Benchmark.

5. Connect the web experience to business results.

If you build a business-to-business website, customers might come. If they do, you must take important steps to ensure they get the service they need, make the purchases they came for, continue using the cost-effective web channel, and enjoy an experience that strengthens your valuable brand. After all, a lot is at risk: Revenue, customers, the brand itself, the business – and then what else is there?

http://www.retail-digital.com/online_retailing/top-five-best-online-prac...

Testing discrete applications using the Cloud platform helps not only to host and run the test infrastructure and test applications but also to use test-bed which can help projects/products reduce the overheads in setting up world class test facility. Cloud infrastructure along with Cloud test platform benefit in terms of reduced time, effort and cost in setting up the various test environments. Automated provisioning of test infrastructure on cloud enables to achieve instantaneous high scale as needed for the target test application helping in more accurate capacity planning and improved user experiences.

A Cloud Based Load Simulation platform offers to the enterprises a full service catalogue to test a range of real-time production scenarios. This platform with provisioning and de-provisioning on-demand cloud infrastructure shrinks test cycles from months to weeks by eliminating the procurement time and infrastructure setup time drastically. Configurable tools using macros, simplifies the testing process, early stage analysis of weak links and ensures business continuity. This Cloud platform reduces the complexity of using cloud infrastructure for a developer/tester by providing those as part of the feature of platform itself.

Stress testing is absolutely necessary for a website that will be getting a lot of hits. This was so true with Target, Bank of America, the Super Bowl advertisments, and other website crashes of 2011.

Experience has shown that an effective way to discover defects is to design tests around the limits of the system. In performance testing, this means stressing the system by making demands that are outside the design limits of the software. This is known as ‘stress testing’. Stress testing is particularly relevant to distributed systems based on a network of processors. These systems often exhibit severe degradation when they are heavily loaded. The network becomes swamped with coordination data that the different processes must exchange. The processes become slower and slower as they wait for the required data from other processes. Stress testing helps you discover when the degradation begins so that you can add checks to the system to reject transactions beyond this point.

To get the best results from software performance testing, you should begin your tests while the system is in development. This will allow you to highlight any faults early in the development process and address them immediately. If they are not removed from the system they could create further problems, which will require more time to fix, and ultimately more money.

Every day web developers make the decision that load and stress testing is NOT necessary for their site or application. This is very unfortunate for the website developers. The money they spend for load and stress testing is money well spent. To go to all the trouble to develop a website and not see how it is going to perform is nonsense.

Performance of your site has a direct correlation to your success. Slow sites lose revenue. Sites crash under heavy traffic every day because they got a favorable review on Slashdot. Unexpected volume comes from unlikely sources and blindside your company. Digg, Reddit, Twitter, and hundreds of other social media sites can immediately pour tens of thousands of users to your URL. How will your site handle it?

If you don't load test, you won't know. If you don't measure the performance of your system under large traffic, then you will have no opportunity to tune it to make it more scalable.

If you ignore the facts about load testing, then you will pay the price - some day and some way. I strongly recommend that you run a stress test against your system to see when and where it breaks.

Work with your marketing team to figure out their goals for large volume. Share with them real metrics regarding the number of concurrent users or the requests per second or the throughput that your site can handle. It's not that hard, really!

Capacity planning is a process to help plan for an organization's future. Capacity planning is for determining the best way to complete business requirements such as increases in the amount of work to be done, while at the same time meeting service level requirements.

"The basic steps involved in developing a capacity plan are:

1. To determine service level requirements

a. Define workloads

b. Determine the unit of work

c. Identify service levels for each workload

2. To analyze current system capacity

a. Measure service levels and compare to objectives

b. Measure overall resource usage

c. Measure resource usage by workload

d. Identify components of response time

3. To plan for the future

a. Determine future processing requirements

b. Plan future system configuration

By following these steps, we can help to ensure that your organization will be prepared for the future, ensuring that service level requirements will be met using an optimal configuration.

A good capacity management plan is based on monitoring and measuring load data over time and implementing flexible solutions to handle variances without impacting performance."

http://blogs.hexaware.com/quality-assurance-and-testing-services/capacit...

With so many web applications and websites moving to the cloud, it has caused a great amount of problems with regards to load capacity, application performance and overall reliability. Tuning and scaling systems and being ready for extreme traffic spikes is critical for any successful company doing business online and in the cloud.

Business leaders agree that the payoff for diligent performance load testing is huge. Every day the likes of major retailers, iconic technology brands and multi-national airlines experience websites failures. When these websites crash, their businesses are at least partially closed. As more and more businesses move to the cloud, it’s imperative that these businesses keep a close watch on quality and continually test, monitor and validate application performance to consider: What is the potential cost of poor performance of my cloud application? Is there a bottleneck that could hamper performance or worse, cause the site to crash? What is the load limit? Are applications fine-tuned for maximum success?

Apica LoadTest portal squarely addresses this and simplifies continuous load testing and also supports DevOps-based lifecycle testing, helping companies to best optimize their systems and succeed with new application release rollouts,” said Sven Hammar, CEO for Apica.

http://www.apicasystem.com

Software performance testing is critical to a new website. If the website is not load tested before putting it on the internet, there could be major problems ahead.

"Software performance testing is a way of determining whether a system, such as a website, can remain responsive and stable under a certain workload. This is vital for anyone with an online business, as it ensures that visitors to your website can browse safely and securely and make hassle-free purchases. In the most basic terms, PCs are used to emulate a number of simultaneous users, while a separate PC records performance data.

By using computers that can acts as multiple users of a system and a separate computer to record results, software performance testing has a number of ways to test the system. Load testing is the most basic – this simply makes sure that the software can cope with the expected load. To make sure that this load can be coped with on a long-term basis endurance testing may also be implemented.

However, often a system will have to process an unusually high number of users. This is why stress and spike testing is essential, otherwise you could find yourself missing out on unexpected business. It is vital that software performance testing makes sure a system can cope with both its expected workload over a period of time and a sudden burst in workload.

Stress testing a website is very important before putting the website out for consumers to see. "Stress testing is a complex test in which testing is conducted under reduced machine and system resources conditions. The resources that undergo stressing may include internal memory, CPU availability, disk space, and network bandwidth. Stressor utility is generally used to reduce internal resources.In principle, stress testing tries to break the software by overwhelming the software with too many tasks. The concept behind stress testing is to let the system fail intentionally to see how it will recover over time. This test will let the software developer know about ability of the software application to recover from adverse situations.

This article is about stress testing, load testing, and performance testing. The question is "can you get away with not doing these things when you are ready to launch a new website?"

"Can you get away with not implementing software performance testing?
Software performance testing exists to make sure that any system your business uses is able to cope with the demands placed on it by having multiple users engage with it simultaneously. If your business depends on a stable website or piece of software to operate then software performance testing is absolutely crucial. Overlooking the importance of testing could lead to very costly problems further down the line.

Software performance testing techniques such as endurance testing makes sure the systems you use to keep your online business running can handle its expected workload on a continual basis. However, this is only the first step of testing, as there are certain times when a system will be pushed to its limits – in the weeks before Christmas for example.

Quality services to software testing is a rarity these days. Many companies offer testing services but their services are simply not up to the mark. Stress test is usually ignored by many, if they make software testing services. Software vendors end up paying a lot of money for services that are simply not worth the cost to compute it. Some service-load tests are ignored, as this requires a complex process of attention. It has happened.

First, what is load testing. This is a type of software testing technique that tests the software by simulating multiple users accessing a single application. This is useful when a company tries to use software that is used by many users to simultaneously build. It is a kind of non-functional tests.

Benefits of load testing software:

Load the software testing answers questions such as:
· Is my software does not work properly supports multiple users
· Supports to meet my performance requirements?
· How many users can access both at my request
Other advantages are:
· It reduces the possibility of downtime.
· It improves the quality of deploying an application.
· It allows you to find the performance bottlenecks.
· If a program has passed its test with flying colors, you can client satisfaction can be ensured.
· It provides specific statistics for software developers.
· It improves the scalability of the software.

As more businesses move toward e-commerce, the ability to monitor and manage Web-based applications will no longer be an advantage, but a necessity. "If you look at 10 years ago, very little business in comparison was done on the Internet. Now the Internet is just a complete way of life, and it's a matter of survival for businesses to make sure they have the absolute best performance that they can provide."

Valentine's Day is today, and those who have not bought gifts for their loved ones are scurrying around to find the right gift as quickly as possible. This is where the Internet comes in to play. If you have a slow website, most people will give up and go on to another site. That is why every website needs to have stress testing, load testing, and performance testing done before the website is put out on the Internet. "It all comes down to performance because any slowdowns impact the revenue of a company," Robinson Schoeller says. "It used to be OK if you had, say, a six-second response time to load a page. But now it's really a second or less to load. Otherwise you're going to lose those customers to another site, because the competition is just a click away. Keeping customers on the website, and the ability to resolve the issues that are driving them away, becomes significantly more important around high-traffic periods like Valentine's Day."

Testing cloud applications with an exhaustive approach to performance, security and robustness is a more complex undertaking than many managers and users might at first estimate it to be. Three reasons to test in the cloud: 1. "Load testing a cloud application on-premise never truly reflects the impact of real-world data volumes as they spike erratically. Actually, this is pretty much true for non-cloud-based applications, but it is a great truism for cloud apps too. 2. Testing inside the cloud model should generally always be quicker, more cost-effective and should pave the way toward higher-quality applications. 3. Performance validation inside post production environments can deliver "consumption-based" performance testing of Web 2.0 and mobile apps charged in an on-demand pay-per-use model. Yes - testing becomes more service-based."

What happens if I don't test?

"Failure to follow testing procedures commensurate with the "mission criticality" of the application (and data) in question can see the IT shop hit performance bottlenecks, which in turn can have a direct impact on business performance and the customer's "time-to-market" (to use the marketing manager's favored term) and make profits."

I think this blog tells why it is so important to stress test and load test which is what performance testing is all about.

http://www.sys-con.com/node/2162380

Can you imagine having millions invested in advertising for the Super Bowl and the online ad fails? That is what happened to several of the big companies who had not performance tested their websites correctly.

"The Coke Polar Bears Facebook page and main website was unavailable in five languages."

"Act of Valor had a very impressive ad, but their website was less so. According to Yottaa's monitoring, it seemed every time the spot ran the site crashed, with more than six outages of five minutes each. The site was also five times slower yesterday during game time."

"Acura cars had problems too. "The launch of their new performance car was met with poor performance from their website." Their home page was fine, but the call to action pages were saturated with traffic on Sunday."

"Cars.com, TaxACT.com, GoDaddy.com and History.com all fared really well during the Super Bowl, according to Bob Buffone, writing on Yottaa's blog. He emphasizes that live stress testing is critical at this big event-driven moments, and instrumenting what is going on during the event is essential if the sites are going to meet the anticipated demand."

"$3.5M for 30 seconds and can't find ways to keep website running?

I have a good blog on load testing that I think many people will find interesting. It talks about getting the right people involved when load testing. Your testing results will be more valuable as a result of their involvement.

"If you are using an outside testing resource service or if you are running one big test on a production system, then it is crucial to have the right people involved in the planning and execution of the test. This list includes:

Network administrators
System developers and/or integrators
Web- and app-server administrators
Database admins
Architects
Content Experts
3rd-party services, such as your hosting companies

During the load test, we recommend making live data available during the test to all of the participants (e.g. via a web-share of your desktop). This will give them an opportunity to view test data alongside other metrics that they may be monitoring during the test. We use this approach extensively in our testing services and find that it also gives the participants a better view into the testing process that improves their understanding as well as their contribution to the effort.

Each of these groups brings a specific view of a web-based system and a unique set of talents for identifying, diagnosing and fixing the performance problems that are encountered."

I found this to be an interesting article on how important stress-testing, load-testing, performance testing can be. Can you imagine the amount of hits Domino's website got on Sunday for the Super Bowl? Those people knew their website had to be strong, able to handle a large number of callers, and perform exceptionally well.

Domino’s, a system of more than 9,541 worldwide restaurants, including at least 4,891 in the United States, monitors all aspects of its online operation during one of the largest pizza ordering and delivery days of the year, from pre-game load testing to after-the-game trash talking.

"What has Domino's done, IT systemwise, to prepare?

Every year we go through and do extreme load testing. We set up a bunch of synthetic transactions that basically emulates a human placing an order on our system. Because the process is automated, it gives us the ability to ramp up the number of orders to the point where it would be greater than what we’d expect to see during the Super Bowl. We do that because while you might have great estimates [of game-related business], you never know exactly what to expect and you definitely don’t want [systems] to be down during the Super Bowl because about 30 percent of all our orders come through the online channel."

"What makes the Super Bowl special?

The business owner needs to take a look at the different options and decide which of them to take advantage of to get the most out of the system. Cloud systems almost always require some sort of load testing and stress testing to maximize efficiency.

Businesses that test their cloud services on their own often save money, but the quality of load testing will usually make their customers suffer in the long run.

Given the scale of many testing companies, there is a remedy for every server setup that needs to be tested. In order to be ready for any situation that happens to come along, server testing companies have every load test program that they could possibly need.

For those of you out there who are new to website development, I think this article tells in pretty simple terms what you need to know about load testing.

FUNDAMENTALS OF LOAD TESTING
"Load testing is required at every stage in the life cycle of an application, in particular web applications. Load tests help identify performance problems during system design and development stages. For existing applications, the simulation becomes an aid to maintain, fine-tune and forecast future system needs.