Stress Testing Repository: Websites & Computers

The following is an article from Apica about load testing, written by Tammy Wolf.

"A developer of multi-platform electronic entertainment products, TransGaming, Inc., just launched its GameTreeTV, an on-demand gaming service running on SmartTV. But, before the system rolled out to game-hungry consumers, it needed a bit of an inspection.

That’s why the Canadian gaming company turned to Apica, a pioneer in load testing and performance monitoring for cloud and mobile applications. Ready to roll out the new service with the help of a top European broadband provider, TransGaming’s operations team first engaged with RightScale, a partner of Apica’s, to not only verify the system could handle the expected spike in TV traffic levels, but to also gain access to RightScale’s elastic cloud infrastructure.

Meanwhile, by coordinating with both TransGaming and RightScale teams, Apica initiated its LoadTest solution to assess the system’s peak load management, identify application performance hiccups throughout each tier of the system, and pinpoint failover mechanisms during peak loads.

According to company officials, TransGaming called on Apica specifically for its ability to simulate actual set-top box user scenarios in complex scripts, its ability to drive load from several geographies, and the company’s thorough technical experience with high-volume caching systems.

The following is an article on performance testing.

"SAP General Performance benchmarks have definitely been round because 1993, the purpose of these benchmarks is essentially to flex the power muscle of Hardware and Software, in this case SAP. The use of such benchmarks is for ability sizing, creating scalability of applications and compute electricity of hardware in addition to merchandising collateral.

Nonetheless these benchmarks are of not a lot use to actual SAP end customer to establish general performance of their implementation. Actual implementation seriously isn’t run in most suitable benchmark environment, SAP architecture and modules are distinct, configuration, coding, implementation, hardware, DB etc. are all distinct and it’s tremendously customized for your corporation and natural environment.

How much is SAP Operation of implementation at your site?

How does it assess to benchmarks printed by SAP/Hardware/DB vendor?

How does it assess to other individuals in Sector?

So can you merely run Loadrunner scripts and do a SAP Load Testing to be sure Operation? many of us would like!!

How much will be the response time to your end user? Is it suitable? As A Rule not. it will require some time to optimize, tune and get more advantageous general performance out of SAP.

The following is an article on load testing by Chris Merrill.

"Customers occasionally ask us “How early should we begin load testing?”

The answer is to test something, anything, as soon as the architecture is available. Performance problems have a wide variety of causes – from a single line of code to a load balancer setup; from a database schema to a server config file. Early in the development of the software you can catch simple coding problems and fundamental architectural limitations that are much easier to fix before a lot of code has been written.

Now, a word of caution: Testing against a scaled-down development or test system will not tell you how the final system will scale. So as a tester, you must be careful how you interpret these results. Interpreted incorrectly, they could raise false alarms that undermine your credibility in the future. But early testing can uncover defects that limit the capacity to unreasonably low levels. When investigated cautiously with the help of the developers and admins, a number of serious performance limitations can be caught while they are still cheap to fix.

Chris Merrill, Chief Engineer"

A good article on load testing from webperformance.

http://www.webperformance.com/load-testing/blog/2011/10/load-test-early/

The following is an article about web load testing from Compuware.

"Compuware Announces Availability of eBook: "Web Load Testing for Dummies"

"Compuware Corporation (Nasdaq:CPWR), the technology performance company, today announced the general availability of the eBook: "Web Load Testing for Dummies." In this eBook, Scott Barber, Founder and Chief Technologist at PerfTestPlus, and Colin Mason, Compuware's Web Load Testing Product Manager, explore the importance of web load testing and its impact on business.

Web applications that perform well can strengthen a company's brand and reputation, as well as create customer loyalty. Web applications that perform poorly put all of that at risk. Web load testing is a critical component to any risk management plan for web applications.

This eBook explains:

How web load testing helps enable better website performance;
When to load test; and
Why web performance directly impacts business.
Download the eBook here (registration required): http://bit.ly/mRRKKl.

The following is an article on load testing by Sergei Sokolov.

"A few of the SmartBear crew has just returned from Adobe MAX in Los Angeles - the major annual Adobe user conference. With 5 days of content, nearly 20 parallel session and lab tracks, two fascinating keynote addresses produced in the best traditions of Hollywood (watch MAX Online), and more than five thousand people attending, it was a huge event not to miss.

Following our success with adding state of the art support for Adobe technologies in TestComplete, we were compelled to bring the word about SmartBear to the biggest Adobe trade show - and we did. We showed off TestComplete and LoadComplete on the expo floor, along with 30+ other exhibitors. The SmartBear mascot was a hit of the show and definitely brought some smiles to those visiting our booth. See a picture on Facebook. We had scratch prize tickets, and one lucky attendee won an iPad 2! I finished off a great show by participating in an Adobe-sponsored panel on functional and regression testing.

The following is a very good article on load testing written by Nate Cochrane.

"It is a truism that you don't appreciate something or someone until they are gone.
Such was the case with the passing last week of entrepreneur and Apple co-founder Steve Jobs, who was one of the top search terms on the web and who recorded the biggest number of searches on his name since Google started records in 2004.
Although Jobs' passing given his health history was a subject of speculation, media outlets and social media services could not know when they would have to serve what quantum of traffic about the event.
And as sad an event as Jobs' death was, it taught lessons for those who provision servers and services for enterprises in the face of unexpected loads.
Interest in Jobs peaked massively just before 11am AEST on October 6 about the time Apple posted news of his death on its website, having informed staff earlier in the day.
Twitter buckled under the strain as users of the social media service converged online to share their thoughts and grief.
Twitter said it was handling 6049 Jobs-related tweets a second, a near-record velocity. Some analysts had expected final figures to tally 10,000 a second. Twitter's response times stalled to about 30 seconds and then its home page availability was cut in half.
Building a site that won't break

The following is a good, informative, article about stress testing.

"Definition - What does Stress Testing mean?
Stress testing refers to the testing of software or hardware to determine whether its performance is satisfactory under any extreme and unfavorable conditions, which may occur as a result of heavy network traffic, process loading, underclocking, overclocking and maximum requests for resource utilization.

Most systems are developed under the assumption of normal operating conditions. Thus, even if a limit is crossed, errors are negligible if the system undergoes stress testing during development.
Techopedia explains Stress Testing
Stress testing is used in the following contexts:
Software: Stress testing emphasizes availability and error handling under extremely heavy loads to ensure software does not crash due to insufficient resources. Software stress testing focuses on identified transactions to break transactions, which are heavily stressed during testing, even when a database has no load. The stress testing process loads concurrent users beyond normal system levels to find the system's weakest link.
Hardware: Stress testing ensures stability in normal computing environments.
Websites: Stress testing determines the limitations of any of the site's functionalities.

The following is an article on performance and load testing by Soasta.

"Most companies – if they are testing at all – are using performance testing and load testing tools designed for a different era, when 1,000 concurrent users was high traffic and most users were employees.

SOASTA CloudTest® offers companies with consumer facing websites a transformational approach for ensuring high performance under normal, as well as extreme traffic conditions. With the advent of social media, focused marketing events, and an ever-growing stream of new mobile devices, this has become an increasingly dicey proposition – making performance testing and load testing tools more important than ever. Without performance testing, external events such as the Super Bowl, Cyber-Monday, Tax Day, Valentines Day, a significant drop in interest rates or even news about a celebrity can suddenly drive unforeseen traffic to a web site turning success into embarrassment if the site is slow, or worse, if the site crashes under the load.

Accurate, Scalable and Affordable

The following is a blog from Stephanie Klaiber about load testing.

"The Internet is ever evolving and shows no sign of stopping. According to a ComScore Q3 State of the Internet report:

215.1 million Americans used the Internet in July 2011
The average user was online 25.3 days in the month, for a total of 34.8 hours, and viewed 3,029 pages
86% of Internet visitors viewed an online video in the U.S., for an average of 18.2 hours each
Businesses need to follow and understand Internet trends so they can best serve their customers and develop business objectives that drive their bottom line. This surge in Internet users means businesses need to prepare their corporate websites and applications to handle spikes in web traffic.

Website monitoring and load testing are two essential tools that help businesses keep up with these trends. Real-time external monitoring and full-service load testing help organizations gauge website and application performance at various levels of load from the end-user perspective. This is key to identifying and fixing problems before customers ever become aware of them.

The following is an article on performance testing by Deepika Mola.

"Apica, a premier load testing and performance-monitoring provider for cloud and mobile applications, has introduced a revolutionary new monitoring and performance measuring service, called the Apica AzureCheck.

The newly announced Apica AzureCheck has been designed specifically for Microsoft's (News - Alert) cloud service, Azure.

Offered at no cost for a 30 day trial period, Apica AzureCheck contains a fully integrated function check of the web application "from outside" via Apica's standard application monitoring service. It gives users control of how individual instances of Azure load and integrates support for dynamic scaling of new instances at load.

Apica AzureCheck helps to measure response times and application functionality from any place around the world.

Officials said that users will benefit from this service by combining three operations that normally require separate consultancy work or separate programs, Azure Server "Vital Signs" monitoring, application function and application response time and also dynamic scaling of new instances based on load.

Operating in the cloud, Azure is a scalable environment used to deploy and run services and/or to store data.

The following is an article on load testing by Brandon Konkle.

"If you read Part 1 of my JMeter series, you now know how to create a JMeter performance test with as much complexity as you need to hit every part of your application and push it to its limits. As mentioned at the end of the post, though, when running your test plan from your local machine you are often limited by bandwidth. The test plan may not be able to fully stress your application because it can’t transfer data fast enough for all of your concurrent connections. To really push your application hard, you need to run your load test from the same local network that your application runs within.

If you’re testing your staging or production infrastructure, you could run JMeter from a dev server within the same network. If you’re on a cloud platform, you could manually spin up an image with JMeter on-demand whenever you want to run a load test. Wherever you put it, you just want to make sure that you’re not running JMeter from one of the servers that you will be testing. The load imposed on the test machine will almost definitely skew your performance results.

On Debian-based distributions like Ubuntu, JMeter is very easy to install.

$ sudo apt-get install jmeter

The following is an article on stress testing by Sean Hull.

"Stress testing applications is like putting a car through crash tests, wear and tear tests, and performance tests. It's about finding the leaks, and bottlenecks before they become a limitation to growth. In fact, stress testing is a big part of capacity planning.

There are a few different ways to stress test a web application. You can start at the database side of the house itself, and just stress test the queries your application uses. There are benchmarking tools included with MySQL such as mysqlslap which allow you to run a query or sets of queries repeated times against the database. You can also run them in parallel and in large batches together. All of these methods are an effort to push the limit and find out when the server can handle no more.

There are tools that operate by firing off repeated url requests to the webserver like httperf and also jmeter. These can be good for hammering away at the server, but if you want to do more complex and nuanced tests a like Selenium will allow you to record a web browsing session and play it back to the server, many times or in parallel again to simulate a greater load on the servers."

A well written article on stress testing by Sean Hull.

http://www.iheavy.com/2011/06/24/stress-testing-what-is-it-and-why-is-it...

The following is an article about performance testing by Scott Price of LoadStorm.com.

"Forgive me for stating the obvious, but web applications are a critical part of global business in 2011. I see no alternative other than more dependence by companies everywhere on web software and Internet infrastructure. In my opinion, all business trend data predicts greater overall web usage, more complex application architectures, and tremendous spikes in extreme traffic volumes.

Critical Applications, Yet They Aren't Getting the Investment Needed
ComputerWorld last week made a definitive statement regarding the critical nature of web applications:

Those who are unprepared are vulnerable to service outages, customer dissatisfaction and trading losses - and often when it hurts the most. Successful businesses understand the need to assure service and application availability if they want to retain customers, deliver excellent service and take maximum advantage of the opportunity their market offers.
This is not a theoretical problem - just look at the recent challenges for the London 2012 Olympics andTicketmaster. Just when everyone wants to do business with you, you’re not available.

The following is an article on load testing by Stephanie Klaiber.

"It’s almost that time of year again. Holiday shoppers looking to stay at arm’s length from the seasonal mall madness will undoubtedly turn to the Internet to make their gift purchases. In fact, ShopperTrak predicts national retail sales will rise 3% during November and December this year as compared to the same time period last year.

In order to keep up with this expected spike in website traffic, there are a few steps ecommerce companies should take to ready their site. In order to help, we’ve rounded up the top eight load testing and website monitoring tips to help ecommerce sites stay on their “A-game” this holiday season.

1. Start load testing now. Procrastination can be a worrisome habit. Put off today what you can do tomorrow, and you may find your ecommerce site has crashed. According to IBM report, Act, Don’t React: A Proactive Business Continuity Solution Protects your Revenues and Reputation, the average revenue loss per hour of downtime is $1.01 million—a price tag your business cannot afford.

2. Set objectives. Look at industry trends and establish what is “acceptable performance” for your organization. Example Key Performance Indicators (KPI’s) before load testing can include:

90% of pages should load in 4 seconds or less

The following article is about load testing by Charles Babcock InformationWeek.

" Microsoft Azure Named Top Cloud Performer"

Microsoft's Azure leads the pack in cloud performance after 12 months of testing, according to CloudSleuth, one of the few monitoring services that's assembled a year's worth of data on cloud providers.
Google's App Engine ranked number two. GoGrid, run from a Virginia-based infrastructure-supplying data center, is number three. OpSource, now owned by NTT of Japan, was fourth and Rackspace, fifth. Amazon's EC2 cloud ranked sixth. The top three rankings are the same as those reported by CloudSleuth in late winter.

More Cloud Insights

White Papers
A Revolutionary Approach to Cloud Building
Automating User Management and Single Sign-on for Salesforce.com
Analytics
Cloud Storage: Changing Dynamics Beyond Services
SaaS and E-Discovery: Navigating Complex Waters
Webcasts
Outsourcing Security: What Every Potential Cloud Security Customer Should Know
The ABC's of Cloud Computing in the Midmarket
Videos

In an interview at Interop New York, Cisco's Justin Griffin shows how their wireless products can physically map radio sources by analyzing the spectrum. This allows you to detect rogue devices and sources of interference.

Keynote has an article about load testing.

"Keynote(R) Systems KEYN +1.62% , the global leader in Internet and mobile cloud monitoring, signed a definitive agreement to acquire privately-held Mobile Complete, Inc, doing business as DeviceAnywhere, a leading enterprise-class cloud-based platform for testing and monitoring mobile websites and apps, for approximately $60.0 million in cash plus a potential earn-out.

Umang Gupta, Chairman and CEO of Keynote, said: "Combining DeviceAnywhere's leading testing and quality assurance (QA) cloud-based solutions with our strong mobile monitoring business firmly establishes Keynote as a leader in the mobile testing and monitoring markets. The resulting increased product breadth and scale meaningfully enhance our competitive position. In addition, the acquisition will expand our addressable market into the immediately adjacent enterprise mobile testing and quality assurance space, which we project could enable us to grow this into a $100 million business."

The following is a blog about load testing and performance testing.

"Compuware Corporation introduced a load testing solution that it says enables organizations to find and fix performance and scalability problems across the entire application delivery chain. Compuware Gomez 360° Web Load Testing brings together two testing solutions - Gomez Web Load Testing and dynaTrace Test Center Edition - into one integrated offering.

This new solution is designed to rapidly identify user experience performance problems, the vendor says. Customers may test web, mobile and cloud applications, Jeff Loeb, vice president of product marketing for Compuware's Application Performance Management Business, tells 5 Minute Briefing. "We are now able to identify the problem wherever it exists across that entire web application delivery chain."

The result is that companies can achieve faster time to market for new applications and features, and also reduce the number of testing iterations, improving the productivity around the whole testing process, Loeb says.

The following is a blog about load testing and performance testing posted by Dave Murphy.

"I suppose we could have used a less inflammatory title for our recent webinar. It makes it sound like testers have been purposely doing something wrong. Perhaps we could have titled the webinar “Now you can execute more accurate and informative tests!” But the folks in marketing were right, and the intriguing title attracted our largest group of attendees ever. For those of you who didn’t attend, or if you did and would like to review the messages, you can watch the webinar here. This was the first in SOASTA’s latest webinar series, “Cloud Testing – Rewriting the Rules of Performance Testing”. Future webinars include “Run More Tests and Find More Issues” on October 27th and “Test On Your Schedule across the Lifecycle” on November 15th.

In this webinar, Scott Barber, President and CTO of PerfTestPlus, joins SOASTA’s VP of Performance Engineering, Rob Holcomb to discuss what performance engineers have done in the past to measure performance and find and fix issues; and why some of those techniques no longer reflect best practices. The focus is on web and mobile testing and why the higher scale, more distributed and often complex nature of that traffic is not well served by traditional testing tools or techniques.

The following is an article on load testing by Lee Mathews.

"A year-long independent test of cloud service providers was recently wrapped up by Compuware. Among the providers tested were the three big dogs: Amazon, Google, and Microsoft. And although the Azure cloud platform is still fairly young, it fared quite well against the competition.

In fact, it proved to be the fastest service over the past twelve months.

Compuware’s CloudSleuth monitors dozens of providers around the globe, checking load times for a simple two-page fictitious retail website. Azure’s average response time was 6.07ms, besting second-place Google App Engine by nearly half a second. That might not seem like much, but when you’re talking about millions of page loads per day those additional milliseconds add up.

Amazon’s EC2 service lagged well behind both Azure and App Engine, at 7.20 for its East Coast location and 8.11 seconds out West.

While the results of Compuware’s testing appear on the surface to be of very little interest to people outside the IT community, the fact that Azure performed so well is going to make a difference to users at home once Windows 8 lands on retail shelves.

The following is an article on load testing and performance testing.

"1. What is load testing?
Load testing is to test if the program works very well with the stress of a large number of simultaneous users, transactions and to determine weather it can handle peak periods.
2. What is performance testing?
Time to read and update transactions should be gathered to determine whether system functions are performed within a reasonable time. This should be done independently and in a multi-user to determine the effect of multiple transactions on the timing of a single transaction.
3. What is LoadRunner?
LoadRunner works by creating virtual users who take the place of real users operating client software, such as sending requests using the HTTP protocol to IIS or Apache Web server. The Load Generator generate the requests from many users to create the load on different servers to test.
These load the generator agents are started and stopped by the controller program. The Controller controls load test runs based on the scenarios, script compiled and run-time settings.
The scripts are made using mercury “Virtual User Script Generator” (named “VU gen”), it generates C-language scripting code to be executed by virtual users by capturing network traffic between clients and servers Internet applications.