Chapter 8 Load Testing
Load testing helps developers and administrators estimate how many users their application can support. If an application requires tuning, load testing and load test result analysis can be used to identify performance bottlenecks and to guide changes to infrastructure, configuration, or code.
It’s a common misconception that “Shiny doesn’t scale”. In actuality, properly-architected Shiny applications can be scaled horizontally, a fact which Sean Lopp was recently able to demonstrate at rstudio::conf 2018. We used shinycannon
to simulate 10,000 concurrent users interacting with an application deployed to AWS. You can see a recording of Sean’s talk and the load test demonstration here: Scaling Shiny
To perform a load test you’ll need two pieces of software: shinyloadtest
and shinycannon
.
shinyloadtest
is an R package used to generate recordings and analyze results. You should install it on your development machine. GitHubshinycannon
is the command-line replay tool. You can install it on your development machine for testing, but for best results we recommend installing it on a server, and preferably not the one the application under test is also on. GitHub
See the Load Testing Quickstart Here.
8.1 Optimization Loop Methodology
- Benchmark: Use
shinyloadtest::record_session()
to record interaction,shinycannon
to simulate users - Analyze: Visualize and interpret the metrics
- Recommend: Propose ways for the capacity of the application to be increased
- Optimize: Implement recommendations and benchmark again. Repeat until satisfied
Workflow:
- Use `shinyloadtest to record a session with a Shiny app
- Generate load with
shinycannon
- Analyze metrics with
shinyloadtest
- Make changes to the Shiny application
- Generate load and analyze again
8.2 Activity: Load Testing
First: Open runloadtest.R,do the pre-run checklist
Discussion:
Preparing for the load test
- First, what is up with the pre-run checklist? Any idea why these steps are (currently) necessary?
- The
runloadtest.R
file is going to start with a baseline test of 1 and then a test of 25. Why is the baseline important?
Deliverable: Run the load test
- Follow the first set of commands in
runloadtest.R
- How did the experience for 1 user compare to the experience for 25 users?
References and Resources:
Webinar:
Vignettes: