Stress Testing Repository: Websites & Computers

An email from Jovana Milutinovich from Serbia, formerly Yugoslavia, is impressed and has found value in a blog from LoadStorm.

Jovana found value in our post about web performance tuning. In this blog one of the things I pointed out is about how improving your web application performance can directly impact your bottom line by 10% or more. It has been proven that speeding up websites improves online business. There is so much evidence that better web performance can produce a 16% higher conversion rate and a 5% higher order value. The facts are clear - poor performance of a website is one of the worst practices for any business. If a site responds slowly or fails under load, then money is going to be lost. Bad performance leads to other bad things happening.

In conclusion, I am honored that Jovana found some value in our blog on web performance. Just remember to stress test, load test, and performance test your website. Performance testing early and often improves the development cycle.

http://loadstorm.com/2009/web-performance-tuning

Last year, Cyber Monday spending hit $1.25 Billion, setting the record as the heaviest U.S. online spending day in history. This was a 22% increase in spending over 2010, and retailers can only hope that even more impressive records will be set in the upcoming holiday season. However, also last holiday season, 61% of customers reported a poor experience on a retail site. Although e-retailers saw great sales success in 2011, many missed the mark when it came to delivering the optimal customer experience. The bright side to this downfall, is that e-retailers have room for improvement and greater revenue gains as they prepare for Cyber Monday 2012.

For those of you ecommerce retailers looking to make the most of the 2012 holiday season, here are some tips to consider as you prep your site for the upcoming peak traffic periods:

1. Testing, testing and more testing

To prevent business failure on your website, you must test: stress test, load test, and performance test. This must be done from beginning to end. The approach of "test early, test often" is unfortunately not adhered to by most companies, and test efforts are usually left to the last minute. Companies hence struggle to test critical functionalities and transactions before the software's release date.

It is important to understand the impact of failure on the business, and working backwards, mitigate those risks by ranking each test as early as possible so as to align these tests with the business requirement that drives them. When transactions are critical to business success, they must be thoroughly tested from both a functional and nonfunctional perspective. This is to see if the application correctly performs the transaction in a timely fashion for all users, regardless of how they access the application. Even after the software goes live, continuous and regular testing of its critical transactions is important.

Automated tests should be developed as early as possible and re-run regularly to ensure development efforts don't break the application. These automated tests should cover the developer's code, functionality of key transactions and application performance.

http://www.zdnet.com/software-testing-lacking-serious-effort-7000002324/

Nearly a million visitors are expected for the Olympic Games. The Olympics will be streamed like never before. The BBC predicts that its live footage, streamed to computers across Britain, will generate in excess of a terabit (1 trillion bits) per second of traffic at peak times – the equivalent of 1,500 people downloading a feature-length DVD-quality movie every minute. Is the Olympic website (London2012.com) ready for an onslaught of traffic? Wired Enterprise reports that The London Olympics Organizing Committee turned to cloud testing specialist SOASTA to stress test the Olympic web site. SOASTA engineers spent six months working with the committee to simulate traffic not only to the Olympics website but across the many mobile apps that tie into it. The result, according to SOASTA CEO Tom Lounibos, is that the London Olympics website is probably better tested any Olympics site in the web’s short history. Lounibos says SOASTA has ensured that the site can handle traffic from as many as 1 billion people over the course of the games.

"The familiar IP addresses that are assigned to practically every device that can access the Internet— computers, smartphones, cameras—follow the IPv4 naming scheme which was developed in 1977. The 32-bit system, which follows the xxx.xxx.xxx.xxx pattern, with each set of numbers ranging from 0 to 254, is capable of generation up to 4.3 billion possible addresses.

Vint Cerf, Google’s current chief Internet evangelist and the “father of the Internet,” has said in previous interviews that he thought 4.3 billion IP addresses was more than enough. With the explosive growth in Internet and the potential of having practically everything connected to the Internet, such as refrigerators sending out alerts when running low on milk and cars that double as wireless hotspots, 4.3 billion sounds quite minuscule.

As a replacement scheme, IPv6 has a staggeringly large number of addresses: 340 undecillion. (That's 340 followed by 36 zeros).

During load testing design, it is important to establish if the testing process would also include stress testing. The criteria for stress testing should be clearly established and the development team should come to an agreement as to what should happen when the application is subjected to the specified degree of stress.

"The terms load testing and stress testing are often used interchangeably. However, there is a fundamental difference between the two. Let’s assume that we have an application that at peak times is subjected to a load of 1000 virtual users. Let’s also assume that a response time of 5 seconds is considered acceptable.

Load Testing attempts to measure how well the application performs at a given load. In the case described above we would attempt to load the application with 1000 virtual users and attempt to measure the mean application response time and find out if it meets our criteria. The load should be as representative as possible of the actual real world situations. This means that the data should be parameterized and different virtual user ramp up scenarios should be modeled. If the application fails to meet the acceptable criteria, then it should be fixed and the load tests should be run until the application satisfies the acceptable criteria.

One of the reasons for it going badly, was that it wasn't stress tested first. The following are the nine other reasons for it not going well:

"10. It came to market too late to begin with! The last private equity round Facebook raised valued the company at about $50 billion. And the current market is taking the public shares back toward that level, which is 50 percent below the initial IPO pricing.

9. Bad news travels fast, but not far. The bad news about Facebook's declining earnings profile, traced to the fairly recent shift of more users to mobile access, which is less conducive to Facebook's advertising revenue. By the way, this transition had not yet become significant at the time of Facebook's last private equity round.

8. Large brand ad pulls. General Motors' decision to pull its Facebook advertising (but, notably, not its Facebook page) because GM is said to have judged those ads ineffective in driving car sales. This conclusion is particularly suspect, however, because the main complaint in integrated marketing circles about social media ad campaigns in general is that the tools for measuring their effectiveness are at a very primitive stage. Any judgment regarding effectiveness is currently based on pretty flimsy evidence either way.

In a technology forum, Dhruva asks, what is performance testing and what is the difference between load testing and stress testing? The following explanations should answer any questions Dhruva has about performance, load, and stress testing.

"a) Performance testing: This is a kind of testing which performed to assure how the components of the system or the applications are performing under a given specific situation. System resource usage, reliability and scalability of the product are also validated in this testing. This testing is the part of testing where performance engineering resources focused on addressing on the performance issues in the software design and architecture of the product.

Nobody likes to wait. In the past several weeks, web performance has gotten a lot of attention because of the time some websites take to load.

Why do we want faster pages? Slow pages cause web stress: Poorer concentration and increased agitation.

Why do we stress? It is mainly because of our short-term memory. Information stored in our short memory evaporates quickly. That is why we don't perform as well when we have to wait. Even after just 10 seconds, we have lost our chain of thought.

At any moment, there are three basic types of memory processing at work in your brain. There are sensory, short term memory, and working memory. Sensory memory works in 100ms bursts.
There are three types of sensory memories; iconic memory, sound memory, and touch memory. Iconic memory is every time you see something, this visual information is taken in by photoreceptor cells in your eyes and they send to the occipital lobe in your brain. Your short term memory can store information for 10-15 seconds, enough time for your working memory to process, manipulate, and control it.

The goal is to get page load times down to 100 millisecond to keep us from losing information through the cracks in our iconic memory. Also, this gives our short term and working memories time to soak up all they can before they start losing information.

Online retail sales increased by 14 percent last year. It is predicted that sales this year will be even higher. More and more shoppers are choosing to spend their money online. That should make website developers want to make sure their customers have a good web experience.

Five ways retailers can improve the internet shopping experience for consumers:

1. Know your customers, their profiles and their usage patterns.

2. Continually check on your customers’ experience.

3. Know what contributes to your customer’s experience.

4. Benchmark.

5. Connect the web experience to business results.

If you build a business-to-business website, customers might come. If they do, you must take important steps to ensure they get the service they need, make the purchases they came for, continue using the cost-effective web channel, and enjoy an experience that strengthens your valuable brand. After all, a lot is at risk: Revenue, customers, the brand itself, the business – and then what else is there?

http://www.retail-digital.com/online_retailing/top-five-best-online-prac...

Testing discrete applications using the Cloud platform helps not only to host and run the test infrastructure and test applications but also to use test-bed which can help projects/products reduce the overheads in setting up world class test facility. Cloud infrastructure along with Cloud test platform benefit in terms of reduced time, effort and cost in setting up the various test environments. Automated provisioning of test infrastructure on cloud enables to achieve instantaneous high scale as needed for the target test application helping in more accurate capacity planning and improved user experiences.

A Cloud Based Load Simulation platform offers to the enterprises a full service catalogue to test a range of real-time production scenarios. This platform with provisioning and de-provisioning on-demand cloud infrastructure shrinks test cycles from months to weeks by eliminating the procurement time and infrastructure setup time drastically. Configurable tools using macros, simplifies the testing process, early stage analysis of weak links and ensures business continuity. This Cloud platform reduces the complexity of using cloud infrastructure for a developer/tester by providing those as part of the feature of platform itself.

Stress testing is absolutely necessary for a website that will be getting a lot of hits. This was so true with Target, Bank of America, the Super Bowl advertisments, and other website crashes of 2011.

Experience has shown that an effective way to discover defects is to design tests around the limits of the system. In performance testing, this means stressing the system by making demands that are outside the design limits of the software. This is known as ‘stress testing’. Stress testing is particularly relevant to distributed systems based on a network of processors. These systems often exhibit severe degradation when they are heavily loaded. The network becomes swamped with coordination data that the different processes must exchange. The processes become slower and slower as they wait for the required data from other processes. Stress testing helps you discover when the degradation begins so that you can add checks to the system to reject transactions beyond this point.

To get the best results from software performance testing, you should begin your tests while the system is in development. This will allow you to highlight any faults early in the development process and address them immediately. If they are not removed from the system they could create further problems, which will require more time to fix, and ultimately more money.

Every day web developers make the decision that load and stress testing is NOT necessary for their site or application. This is very unfortunate for the website developers. The money they spend for load and stress testing is money well spent. To go to all the trouble to develop a website and not see how it is going to perform is nonsense.

Performance of your site has a direct correlation to your success. Slow sites lose revenue. Sites crash under heavy traffic every day because they got a favorable review on Slashdot. Unexpected volume comes from unlikely sources and blindside your company. Digg, Reddit, Twitter, and hundreds of other social media sites can immediately pour tens of thousands of users to your URL. How will your site handle it?

If you don't load test, you won't know. If you don't measure the performance of your system under large traffic, then you will have no opportunity to tune it to make it more scalable.

If you ignore the facts about load testing, then you will pay the price - some day and some way. I strongly recommend that you run a stress test against your system to see when and where it breaks.

Work with your marketing team to figure out their goals for large volume. Share with them real metrics regarding the number of concurrent users or the requests per second or the throughput that your site can handle. It's not that hard, really!

Capacity planning is a process to help plan for an organization's future. Capacity planning is for determining the best way to complete business requirements such as increases in the amount of work to be done, while at the same time meeting service level requirements.

"The basic steps involved in developing a capacity plan are:

1. To determine service level requirements

a. Define workloads

b. Determine the unit of work

c. Identify service levels for each workload

2. To analyze current system capacity

a. Measure service levels and compare to objectives

b. Measure overall resource usage

c. Measure resource usage by workload

d. Identify components of response time

3. To plan for the future

a. Determine future processing requirements

b. Plan future system configuration

By following these steps, we can help to ensure that your organization will be prepared for the future, ensuring that service level requirements will be met using an optimal configuration.

A good capacity management plan is based on monitoring and measuring load data over time and implementing flexible solutions to handle variances without impacting performance."

http://blogs.hexaware.com/quality-assurance-and-testing-services/capacit...

With so many web applications and websites moving to the cloud, it has caused a great amount of problems with regards to load capacity, application performance and overall reliability. Tuning and scaling systems and being ready for extreme traffic spikes is critical for any successful company doing business online and in the cloud.

Business leaders agree that the payoff for diligent performance load testing is huge. Every day the likes of major retailers, iconic technology brands and multi-national airlines experience websites failures. When these websites crash, their businesses are at least partially closed. As more and more businesses move to the cloud, it’s imperative that these businesses keep a close watch on quality and continually test, monitor and validate application performance to consider: What is the potential cost of poor performance of my cloud application? Is there a bottleneck that could hamper performance or worse, cause the site to crash? What is the load limit? Are applications fine-tuned for maximum success?

Apica LoadTest portal squarely addresses this and simplifies continuous load testing and also supports DevOps-based lifecycle testing, helping companies to best optimize their systems and succeed with new application release rollouts,” said Sven Hammar, CEO for Apica.

http://www.apicasystem.com

Software performance testing is critical to a new website. If the website is not load tested before putting it on the internet, there could be major problems ahead.

"Software performance testing is a way of determining whether a system, such as a website, can remain responsive and stable under a certain workload. This is vital for anyone with an online business, as it ensures that visitors to your website can browse safely and securely and make hassle-free purchases. In the most basic terms, PCs are used to emulate a number of simultaneous users, while a separate PC records performance data.

By using computers that can acts as multiple users of a system and a separate computer to record results, software performance testing has a number of ways to test the system. Load testing is the most basic – this simply makes sure that the software can cope with the expected load. To make sure that this load can be coped with on a long-term basis endurance testing may also be implemented.

However, often a system will have to process an unusually high number of users. This is why stress and spike testing is essential, otherwise you could find yourself missing out on unexpected business. It is vital that software performance testing makes sure a system can cope with both its expected workload over a period of time and a sudden burst in workload.

Stress testing a website is very important before putting the website out for consumers to see. "Stress testing is a complex test in which testing is conducted under reduced machine and system resources conditions. The resources that undergo stressing may include internal memory, CPU availability, disk space, and network bandwidth. Stressor utility is generally used to reduce internal resources.In principle, stress testing tries to break the software by overwhelming the software with too many tasks. The concept behind stress testing is to let the system fail intentionally to see how it will recover over time. This test will let the software developer know about ability of the software application to recover from adverse situations.

This article is about stress testing, load testing, and performance testing. The question is "can you get away with not doing these things when you are ready to launch a new website?"

"Can you get away with not implementing software performance testing?
Software performance testing exists to make sure that any system your business uses is able to cope with the demands placed on it by having multiple users engage with it simultaneously. If your business depends on a stable website or piece of software to operate then software performance testing is absolutely crucial. Overlooking the importance of testing could lead to very costly problems further down the line.

Software performance testing techniques such as endurance testing makes sure the systems you use to keep your online business running can handle its expected workload on a continual basis. However, this is only the first step of testing, as there are certain times when a system will be pushed to its limits – in the weeks before Christmas for example.

Quality services to software testing is a rarity these days. Many companies offer testing services but their services are simply not up to the mark. Stress test is usually ignored by many, if they make software testing services. Software vendors end up paying a lot of money for services that are simply not worth the cost to compute it. Some service-load tests are ignored, as this requires a complex process of attention. It has happened.

First, what is load testing. This is a type of software testing technique that tests the software by simulating multiple users accessing a single application. This is useful when a company tries to use software that is used by many users to simultaneously build. It is a kind of non-functional tests.

Benefits of load testing software:

Load the software testing answers questions such as:
· Is my software does not work properly supports multiple users
· Supports to meet my performance requirements?
· How many users can access both at my request
Other advantages are:
· It reduces the possibility of downtime.
· It improves the quality of deploying an application.
· It allows you to find the performance bottlenecks.
· If a program has passed its test with flying colors, you can client satisfaction can be ensured.
· It provides specific statistics for software developers.
· It improves the scalability of the software.

As more businesses move toward e-commerce, the ability to monitor and manage Web-based applications will no longer be an advantage, but a necessity. "If you look at 10 years ago, very little business in comparison was done on the Internet. Now the Internet is just a complete way of life, and it's a matter of survival for businesses to make sure they have the absolute best performance that they can provide."

Valentine's Day is today, and those who have not bought gifts for their loved ones are scurrying around to find the right gift as quickly as possible. This is where the Internet comes in to play. If you have a slow website, most people will give up and go on to another site. That is why every website needs to have stress testing, load testing, and performance testing done before the website is put out on the Internet. "It all comes down to performance because any slowdowns impact the revenue of a company," Robinson Schoeller says. "It used to be OK if you had, say, a six-second response time to load a page. But now it's really a second or less to load. Otherwise you're going to lose those customers to another site, because the competition is just a click away. Keeping customers on the website, and the ability to resolve the issues that are driving them away, becomes significantly more important around high-traffic periods like Valentine's Day."