The world of testing tools is growing quickly and many tools come and go. But one of the newest tools that will stick with Sqills for some time is Cypress. Cypress has a very wide functionality that supports unit tests, front-end tests and integration tests. Sqills is currently focused on front-end tests and integration tests. The focus of this blog will be on why we chose Cypress and some tricks we like to use to keep the code maintainable.
Our life before Cypress
Dev-Z is the scrum team that develops all front-ends. This team has chosen to use JavaScript with React and Emotion as the fundament with extension to several libraries. The front-ends are modern applications using API-calls, which can give extra challenges for automated testing. Until the end of 2018, all automated front-end tests were written in the Behat language. Behat is a BDD framework that is strictly integrated with Selenium. To keep the tests readable for as many people as possible, both technical and non-technical people, the test scripts are basically split into two separate files. The first file, called feature file, describes a business case, in human (non-developer ;)) readable language. The second file, called step definitions, has the logic to really test the front-end.
An example of a Behat test is to visit click on the link of a booking and determine if you are on that page. The code would look like this:
When I click on View booking
Then I should see booking X01G3QCP
/**
* @When /^I click on ([^”]*)$/
* @param $string
* @throws Exception
*/
public function iClickOn($string)
{
$this->waitForPageToBeReady();
$this->findAndClick(‘//span[text()=”‘ . $string . ‘”]’);
$this->waitForPageToBeReady();
}
It is rather labor intensive to keep these scripts up and running. Especially since there is a hard cut between the descriptive steps of the test and the step definitions. Most of the times you need to explore the step definition to find out what is really happening if you want to change something in the feature file. This was one of the reasons to look for a new tool to help create more sustainable test scripts.
The reason Sqills works with Cypress
In the testing world, there are three major players when talking about automated front-end testing: Selenium, TestCafe and Cypress. All have their benefits and drawbacks, even when you don’t account for the development language of the tools. Selenium can be rather complicated, which requires a lot of technical knowledge of the framework. This leaves the battle between TestCafe and Cypress. Both frameworks are close, but a few features make it so that Cypress is the winner for Sqills. Cypress and TestCafe use JavaScript, just like our front-ends. The way it is implemented, Cypress connects better with our development team.
This results in more collaboration between the developers and the test engineers since they simply write the same language and in the same code-base. Besides that, the developers can also use Cypress for their unit tests. In addition, the Cypress framework can run directly from the browser, so debugging tests are even simpler. And in practice, we have seen that the framework is way faster. In fact, when compared to Behat it is faster by a factor 10. It also offers a built-in option to mock and stub all sort of services very easily, even if they are custom for your product. Contrast that to Behat which needs a lot more work to mock your services.
So far the change of test tool seems more like a new gadget, a nice-to-have feature. But the comparison between a Behat test script and a Cypress test script will blow you away:
Behat
/**
* @When /^I click on ([^”]*)$/
* @param $string
* @throws Exception
*/
public function iClickOn($string)
{
$this->waitForPageToBeReady();
$this->findAndClick(‘//span[text()=”‘ .
$string . ‘”]’)
$this->waitForPageToBeReady();
}
Cypress
cy.contains(‘View booking’)
.click()
cy.contains(‘X01G3QCP‘)
Fun fact: the Cypress test is even more complete since the Behat example doesn’t contain the check for the booking number. The biggest advance for Behat was the business case writing. But even the Cypress test is pretty easy to read, but of course that gets a bit more complex with larger test files and more difficult checks.
Define tests scenarios with it()
Thus we have to think about how to keep the test scripts readable and maintainable. Once again, Cypress is with us and helps you in many ways. When you use ‘it()’ methods you can keep tests small and isolated. For example three separate ‘it’s are created to keep the script to check the fulfilment methods panel (image below) clean, simple and independent. Every ‘it’ can run without the ‘it’ before, they are all safe to fail without further disturbing the test run.
it(‘Should not see the fulfillment methods without journey search’, function () {
cy.visit(‘/booking/create’)
cy.get(‘div‘)
.should(‘not.have.class’, ‘[data-s3-test=”panel-fulfillment-methods”]’)
})
it(‘Should see all fulfillment methods’, function () {
cy.get(‘[data-s3-test=”panel-fulfillment-methods”]’)
.find(‘span’)
.should(‘contain’, ‘Print’)
.should(‘contain’, ‘SMS’)
cy.get(‘[data-s3-test=”panel-fulfillment-methods”]’)
.find(‘svg’)
.should(‘have.length’, 2)
})
it(‘Should have preselected fulfillment SMS’, function () {
cy.get(‘[data-s3-test=”panel-fulfillment-methods”]’)
.contains(‘Print’)
.parent()
.parent()
.should(‘have.attr’, ‘aria-selected’, ‘false’)
cy.get(‘[data-s3-test=”panel-fulfillment-methods”]’)
.contains(‘SMS’)
.parent()
.parent()
.should(‘have.attr’, ‘aria-selected’, ‘true’)
})
When executing a script like this it will show the result completely burned down into individual steps. Showing you exactly which scenario and which test is failing. Besides the burn down, it remembers the state of the browser which shows the exact behaviour of the application at the moment of the failure. Resulting in even more information for debugging purposes.
Mocking the world
At first, Sqills was testing the front-end applications against testing environments. These share parts with other environments which had other purposes. That regularly resulted in false-positives, for example the moment the configuration had been changed. That means that the second issue was identified were Cypress might prove its worth. The impact for the test scripts was that the complete API had to be mocked in order to test the front-end without any other microservice. Cypress can mock it all (if you use XHR-calls, which Sqills does) and how it works is once again very easy and intuitive. Basically three steps:
Find every API-call and response you need (your browser developer tools (network tab) can help you a lot with finding the responses)
Create a response file for every API-response
Write a route() to mock the API-call
And you’re done. It result in code looks nice and clean.
cy.fixture(‘/oauth/agent-token.json’).then(response => storeTokenInSession(response))
cy.server()
cy.route(‘GET’, ‘**/api/v2/meta/passenger-types’, ‘fixture:/meta/passenger-types-response’)
cy.route(‘GET’, ‘**/api/v2/meta/stations’, ‘fixture:/meta/stations-response’)
cy.route(‘GET’, ‘**/api/v2/meta/card-types’, ‘fixture:/meta/card-types-response’)
cy.route(‘GET’, ‘**/api/v2/meta/disability-types’, ‘fixture:/meta/disability-types-response’)
cy.route(‘POST’, ‘**/api/v2/orientation/journey-search/calendar’, ‘fixture:/orientation/journey-search-calendar-request’)
cy.route(‘POST’, ‘**/api/v2/orientation/journey-search’, ‘fixture:orientation/journey-search-request’)
cy.route(‘POST’, ‘**/api/v2/booking’, ‘fixture:/bookings/confirmed-booking-request’)
cy.route(‘GET’, ‘**/api/v2/orientation/product-search?**’, ‘fixture:/orientation/product-search-response’)
cy.route(‘GET’, ‘**/api/v2/payment/methods/**’, ‘fixture:/payment/payment-methods-response’)
cy.route(‘PATCH’, ‘**/api/v2/booking/**/fulfillment-method’, ‘fixture:/bookings/confirmed-booking-request’)
The user, in this case Cypress, needs an up-and-running server to use the application. To tell Cypress to mock (in other words ‘catch all API-calls’) just add ‘server()’. Now the fun starts with creating all the API-calls. Start the route() with a method, followed by the URL of the API-call and then add the response-file. Notice that the URL can contain tricks to keep it more generic with the use of asterisks (*). That principle is borrowed from Minimatch. It results in flexible matchings, for example:
We have this API-route:
**/api/v2/booking/*/fulfillment-method
And these calls do match:
www.test.nl/api/v2/booking/1234/fulfillment-method
test.nl/api/v2/booking/abc/fulfillment-method
But these don’t match:
test.nl/api/v2/booking/abc/3123/fulfillment-method
test.nl/api/v2/booking/abc/fulfillment-method/123
Since you chose to mock everything, you can be sure what values you can expect in the API-call. But for maintenance, it is easier to replace everything that can be variable by asterisks. If you would decide that a different booking number suits better, you don’t have to change the mocked booking calls anymore.
Re-use of slightly different fixtures
As you have read, it is really easy to create mocked API-calls. Every call has its own fixture and you will even get multiple fixtures for the same API-call. That will result in a moment that you need so many fixtures that it is hard to maintain them. When you look at all the fixtures, you’ll notice that many of them are much alike. Many will have slight differences, for example to make an object ‘active’ instead of ‘inactive’ or add small information like a note to the response.
Since Sqills uses Cypress on a JavaScript architecture, we can also use other functionality from JavaScript. That’s why the functions of LoDash, specifically set() and merge(), are being used to manipulate the default fixtures. As shown in the image below, the fixture is manipulated by a set() and a merge(). After the manipulation it is used as a route. This way we can test three different situations with one fixture instead of three fixtures.
import set from ‘lodash/set’
import merge from ‘lodash/merge’
import getVoucherResponse from ‘../../fixtures/vouchers/voucher-response’
const extraNote = {content: ‘This is an extra note.’, type: ‘NOTE’, created_by: ‘jelmer.vorselman’, created_at: ‘2019-02-05T13:00:05Z’}
describe(‘Voucher details MOCKED’, () => {
it(‘Should be able to see and add notes to the voucher’, function () {
const response = set(getVoucherResponse, ‘data.voucher.notes’, […getVoucherResponse.data.voucher.notes, extraNote])
cy.route(‘POST’, ‘**/api/v2/vouchers/*/notes’, response)
cy.get(‘.s3p-block-voucher-notes’)
.should(‘not.contain’, ‘This is an extra note.’)
cy.get(‘#add-voucher-notes-notes’)
.type(‘test’)
cy.contains(‘Add note’).click()
cy.get(‘.s3p-block-voucher-notes’)
.should(‘contain’, ‘This is an extra note.’)
})
it(‘Should be able to deactivate a voucher’, function () {
cy.route(‘PATCH’, ‘**/api/v2/vouchers/*’, {})
const response = merge(getVoucherResponse, {data: {voucher: {active: false}}})
cy.route(‘GET’, ‘**/api/v2/vouchers/*’, response)
cy.get(‘.s3p-status’)
.should(‘have.class’, ‘s3p-status–active’)
cy.contains(‘Deactivate’)
.click()
cy.get(‘.s3p-status’)
.should(‘have.class’, ‘s3p-status–inactive’)
})
})
it(‘Should be able to see the voucher details page’, function () {
cy.route(‘GET’, ‘**/api/v2/vouchers/*’, getVoucherResponse)
cy.visit(‘/voucher/MOCKVOUCHER/details’)
cy.contains(‘Voucher MOCKVOUCHER’)
})
Use objects and forEach
Like most web applications, S3 also has recurring information but with slight differences. In our case, we have multiple passengers per booking. To prevent code duplication, we use objects and forEach’s. For example, we created an object called ‘Person’ which can be used to iterate through forms and fill it with the data of the objects. By creating this solution it doesn’t matter anymore if there should be one passenger or nine passengers, the code stays the same (excluding the number of people of course). Once again we split this part into separate files. We have a helper file on the left, containing the knowledge of creating a person, and on the right is a part of the Cypress test which fills a form for five different persons.
const buildRandomPerson = () => ({
firstName: randomValue(names),
lastName: randomValue(names),
email: randomValue(email),
phone: randomValue(phoneNumbers).phoneNumber
})
export const getRandomPeople = ({amount = 1}) => {
const people = []
times(amount, () => {
people.push(buildRandomPerson())
})
return people
}
const people = getRandomPeople(5)
people.forEach((person, index) => {
if (index < 1) return
const i = index + 1
cy.log(‘Should open every passenger panel apart’)
cy.get(`[data-s3-test=”passenger-A${i}”]`)
.click()
cy.get(`#passengers-a${i}-first-name`)
.type(person.firstName)
cy.get(`#passengers-a${i}-last-name`)
.type(person.lastName)
cy.get(`#passengers-a${i}-email`)
.type(person.email)
cy.get(`#passengers-a${i}-phone-number`)
.type(person.phone)
})
Debugging failing tests
As mentioned in the intro a big plus for Cypress is the capabilities of the debugging. Below you can see a screencast of a test run which contains a failing assertion. As you can see the script shows clearly when an assertion is performed, a green colour if it passes, and red if it fails. When the test is finished it is possible to ‘travel’ through the test script to see what was in view at that moment. This is especially very useful when you are debugging a failing scenario, or when you are writing a new scenario. For the nightly regression tests only screenshots are made since they run headless. It is possible to get a screencast, but Sqills rather reruns a failing test script instead of watching a whole video.
I could go on for hours and although I love to talk a lot about how awesome Cypress is for us, I also love actually using Cypress. Hopefully this blog gave you a bit more technical inside view of Cypress and gave some hooks to start using it! Sqills would love to see you spread the word how good Cypress is for test automation.
