Chapter 8 Load Testing

Load testing helps developers and administrators estimate how many users their application can support. If an application requires tuning, load testing and load test result analysis can be used to identify performance bottlenecks and to guide changes to infrastructure, configuration, or code.

It’s a common misconception that “Shiny doesn’t scale”. In actuality, properly-architected Shiny applications can be scaled horizontally, a fact which Sean Lopp was recently able to demonstrate at rstudio::conf 2018. We used shinycannon to simulate 10,000 concurrent users interacting with an application deployed to AWS. You can see a recording of Sean’s talk and the load test demonstration here: Scaling Shiny

To perform a load test you’ll need two pieces of software: shinyloadtest and shinycannon.

  • shinyloadtest is an R package used to generate recordings and analyze results. You should install it on your development machine. GitHub
  • shinycannon is the command-line replay tool. You can install it on your development machine for testing, but for best results we recommend installing it on a server, and preferably not the one the application under test is also on. GitHub

See the Load Testing Quickstart Here.

8.1 Optimization Loop Methodology

  • Benchmark: Use shinyloadtest::record_session() to record interaction, shinycannon to simulate users
  • Analyze: Visualize and interpret the metrics
  • Recommend: Propose ways for the capacity of the application to be increased
  • Optimize: Implement recommendations and benchmark again. Repeat until satisfied

Workflow:

Load Test Workflow

Load Test Workflow

  • Use `shinyloadtest to record a session with a Shiny app
  • Generate load with shinycannon
  • Analyze metrics with shinyloadtest
  • Make changes to the Shiny application
  • Generate load and analyze again
Load Test Steps

Load Test Steps

Load Testing Shiny Applications

8.2 Activity: Load Testing

First: Open runloadtest.R,do the pre-run checklist

Discussion:

Preparing for the load test

  • First, what is up with the pre-run checklist? Any idea why these steps are (currently) necessary?
  • The runloadtest.R file is going to start with a baseline test of 1 and then a test of 25. Why is the baseline important?

Deliverable: Run the load test

  • Follow the first set of commands in runloadtest.R
  • How did the experience for 1 user compare to the experience for 25 users?

References and Resources:

Webinar:

Vignettes: