Blog

Decreasing VCR Dependency with Webmock

Recently, I worked on a project where we depended way too much on the VCR gem. A wise co-worker recommended using the Webmock gem in conjunction with VCR. Through using Webmock, we were able to get rid of tons of un-needed cruft VCR had generated.

 

Within a month of installing VCR, we had a large number of ‘cassettes’ (VCR’s generated snapshot of an HTTP response) in our repo that our spec suite relied on. Aside from all the un-necessary files, our spec suite was taking 5+ minutes to run, which made test suite runs quite unpleasant. Also, since one of our requests required a start date, we used Date.today as the start date. This meant we had to commit new cassettes every day for the duration of our project to keep our test suite passing. To top it off, since the timestamp was UTC (7 hours ahead of Boulder, CO at that time of year), at 5:00pm every day our spec suite would start failing. Right before the end-of-day pushes!

VCR is a great gem generally used for storing an API response so that tests can run against a snapshot of the response rather than hitting the API over and over again. This is good for a number of reasons, including: a) running your specs won’t count toward API rate limits, and b) hitting an API for each spec really slows down your test suite. But, when used in excess, VCR added lots of unnecessary files to our repository, albeit we didn’t have to worry about rate-limiting.

Anytime there was an HTTP request in our specs, the error we got was:


 VCR::Errors::UnhandledHTTPRequestError: 
 
 ================================================================================
 An HTTP request has been made that VCR does not know how to handle:
 PUT https://your_url.com
 
 There is currently no cassette in use. There are a few ways
 you can configure VCR to handle this request:
 
 * If you're surprised VCR is raising this error
 and want insight about how VCR attempted to handle the request,
 you can use the debug_logger configuration option to log more details [1].
 * If you want VCR to record this request and play it back during future test
 runs, you should wrap your test (or this portion of your test) in a
 `VCR.use_cassette` block [2].
 * If you only want VCR to handle requests made while a cassette is in use,
 configure `allow_http_connections_when_no_cassette = true`. VCR will
 ignore this request since it is made when there is no cassette [3].
 * If you want VCR to ignore this request (and others like it), you can
 set an `ignore_request` callback [4].
 
 [1] https://www.relishapp.com/vcr/vcr/v/2-9-3/docs/configuration/debug-logging
 [2] https://www.relishapp.com/vcr/vcr/v/2-9-3/docs/getting-started
 [3] https://www.relishapp.com/vcr/vcr/v/2-9-3/docs/configuration/allow-http-connections-when-no-cassette
 [4] https://www.relishapp.com/vcr/vcr/v/2-9-3/docs/configuration/ignore-request
 ================================================================================

 

Since we were using the VCR configuration shown below, it was easy to get around this error. All we had to do was insert the magical :vcr symbol in the specs.

In spec_helper.rb:

 VCR.configure do |c|
   c.configure_rspec_metadata!
 end

 

In the specs:

 describe 'some test scenario involving an HTTP request', :vcr do 
   ...
 end

 

Then, VCR would generate a cassette for the spec (whether it was beneficial to our tests or not) and not raise the error. We knew this was not an ideal solution, but after a bit of time trying to figure out how to ‘get around’ VCR, we decided this was okay for the moment. Unfortunately, this resulted in a bloated VCR file, since responses from ALL external requests got recorded.

The solution:

For the API response that needed a start date, we decided our best bet would be to hard code the date, so that we weren’t having to commit new cassettes every day to keep our tests passing. Then, we could periodically update it when needed.

Next, we decided that the best way un-bloat our repo would be to generate a VCR cassette for the ‘happy path’. For everything else, we could manipulate the data we anticipated in response to make sure that we covered all of the edge cases. In doing so, we had the additional benefit of finding a few bugs that we hadn’t thought of.

To stub the requests in our tests, we used Webmock’s stub_request method to stub any request that matched a specified regular expression, and mocked the response:


 stub_request(:any, /colorado/).to_return(
   body: {
     locations: {
       id: 1,
       city: "Boulder",
     }
   }.to_json, 
status: 200)

This code tells Webmock to stub any HTTP request with ‘colorado’ in the URL, and then we specified what the body of the response should look like. The Webmock docs are helpful and can be found here.

Stubbing with Webmock took care of the specs that were making HTTP requests outside of our ‘happy path’.  Then we gleefully purged all of our unneeded VCR files, which had the added bonus of cutting our test time significantly.