Introduction

Why am I taking this course?

One of the things that I feel I should have at least an introductory level of experience with, among many other things as a software engineer, is testing. In all of the projects that I've built, none of them have had any level of automated testing– I've spent many iterations and potentially hours on manual testing– and decided that it was high time I learned how to write automated tests.

I'm primarily embarking on this journey to feel confident in my code and build out the skills necessary to think about what the "happy path" should look like for my users and ensure, that at a minimum, if it were to break, I would be able to quickly identify and fix whatever bug was causing the problem. This is a standard that is expected of any software engineer working at a company– something I started to do at the end of my time at Twilio– and though I'm on the job search right now, I want to hold myself to that same standard so that when I do join a team, I'm ready to rock and roll!

So let's get into the course:

• This course will be talking about the fundamentals behind tests and testing frameworks

• Distinctions of different forms of testing

• Writing unit and integration tests

• Test doubles (mocks/stubs/etc.)

• Using TDD to write new features and to find and fix bugs

• Core principles of testing to ensure my tests give me the confidence I need

Automated Testing

How do we prevent bugs?

1. Static typing like Flow or TypeScript (definitely!)

2. Linting (ESLint)

3. Testing?? ESLint and TypeScript can’t catch logic errors, so that leaves a big category of bugs we can catch with testing.

What kind of tests are there?

• Unit

• Regression

• Integration

• Accessibility (A11y)

• E2E

• Performance

• Security

• Internationalization (i18n)

• Stress

• Penetration

• Smoke

• Fuzz

• User Acceptance

• Usability

However, will be focusing on just Unit and Integration Tests in this course.

First Test Exercise

To begin, Kent gave us access to a repo with a bunch of exercises so that I’ll be able to run through them without the setup hassle. The idea is to first build an assertion library and then our my testing framework to get a foundational understanding of these tools before actually using 3rd party libraries and frameworks.

The most fundamental test

Let's write a test that reveals the bug in the code. Write code that throws an error with a helpful message about the bug, but only if there's a bug. So... if calling sum with some numbers doesn't return the right thing then throw an error with a helpful message:

const sum = (a, b) => a + b
const subtract = (a, b) => a - b

const sumResult = sum(1, 2)
const sumExpected = 3

const subtractResult = subtract(5, 2)
const subtractExpected = 3

if (sumResult === sumExpected) {
  console.log(`Function sum returned value: ${sumResult}`)
} else {
  throw new Error(
    `Sum function didn't return correct value, expected: ${sumExpected}, but got ${sumResult}`,
  )
}

if (subtractResult === subtractExpected) {
  console.log(`Function subtract returned value: ${subtractResult}`)
} else {
  throw new Error(
    `Subtract function didn't return correct value, expected: ${subtractExpected}, but got ${subtractResult}`,
  )
}

Coding an Assertion Library

Now that we’ve built a very simple test, let’s code an assertion library since this will allow us to create tests in a more reusable manner. For example, in our previous test, I had to manually copy and paste my tests and change them based on the function I was calling: sumResult and sumExpected and then subtractResult and subtractExpected.

Writing an assertion library

Now it's time to implement my own assertion library. I'll create a function called expect that accepts an "actual" and returns an object of assertions. Tip: I want to be able to use it like so: expect(actual).toBe(expected):

// writing an assertion library
function expect(actual) {
  return {
    toBe: (expected) => {
      if (actual === expected) {
        console.log('success!')
      } else {
        throw new Error(`${actual} is not equal to ${expected}`)
      }
    },
  }
}

const {sum, subtract} = require('./math')

let result, expected

result = sum(3, 7)
expected = 10
expect(expected).toBe(result)

result = subtract(7, 3)
expected = 4
expect(expected).toBe(result)

While this is easier to work with, it's still not apparent which test is the one that failed– if it were to fail. It could have been sum or subtract. Given the example, I knew that the sum function was the one broken, but this level of transparency is what I gain when using a testing framework. It can improve error messages, and help to debug much faster.

Coding a Testing Framework

Let's improve the error messages a bit by creating a function called test which can be given a title and a callback. Then add a try/catch so I can log the title with a note of success or error. Then wrap each of my tests in a test function. This also means that I can run all the tests even if one of them fails!

An example of a test function:

test(title, () => {
  // arrange
  // act
  // assert
}

// writing a testing framework
const {sum, subtract} = require('./math')

test('Should sum two numbers together', () => {
  const result = sum(3, 7)
  const expected = 10
  expect(result).toBe(expected)
})

test('Should subtract two numbers together', () => {
  const result = subtract(7, 3)
  const expected = 4
  expect(result).toBe(expected)
  expect(result).toBe(expected)
})

function expect(actual) {
  return {
    toBe(expected) {
      if (actual !== expected) {
        throw new Error(`${actual} is not equal to ${expected}`)
      }
    },
  }
}

function test(title, callback) {
  try {
    callback()
    console.log(`√ ${title} completed successfully!`)
  } catch (err) {
    console.error(`X ${title} failed!`, err)
  }
}

This is the same API that we get in Jest so we can go ahead and try it out!

Unit Tests

Jest Testing Framework

So one of the things that I have to be mindful of is the difference between toBe() and toEqual() since toBe() uses strict equality, it should be used to compare primitive values. For objects or arrays, I should use toEqual() .

A whole list of methods used to check values within an expect() function can be found in the docs here!

Here are examples of the different methods I can use to assert things:

const React = require('react')
const ReactDOM = require('react-dom')
const renderer = require('react-test-renderer')
const {getFlyingSuperHeros} = require('../super-heros')
/*
Find a full list of assertions here: <https://facebook.github.io/jest/docs/en/expect.html>
*/

test('toBe', () => {
  // similar to ===
  expect(1).toBe(1)
  expect(true).toBe(true)
  expect({}).not.toBe({})
})

test('toEqual', () => {
  // like `lodash.isEqual`: <https://lodash.com/docs/4.17.4#isEqual>
  const subject = {a: {b: 'c'}, d: 'e'}
  const actual = {a: {b: 'c'}, d: 'e'}
  expect(subject).toEqual(actual)

  const subArray = [1, 2, {three: 'four', five: {six: 7}}]
  const actArray = [1, 2, {three: 'four', five: {six: 7}}]
  expect(subArray).toEqual(actArray)
})

test('toMatchObject', () => {
  // similar to `toEqual`, but for partial equality
  const subject = {a: {b: 'c'}, d: 'e'}
  const actual = {a: {b: 'c'}}
  expect(subject).toMatchObject(actual)

  const subArray = [1, 2, {three: 'four', five: {six: 7}}]
  const actArray = [1, 2, {five: {six: 7}}]
  expect(subArray).toMatchObject(actArray)
})

test('toHaveBeenCalledTimes', () => {
  const mockFn = jest.fn()
  expect(mockFn).toHaveBeenCalledTimes(0)

  mockFn()
  expect(mockFn).toHaveBeenCalledTimes(1)
})

test('toHaveBeenCalledWith', () => {
  const mockFn = jest.fn()
  mockFn('abc', {oneTwoThree: 123})
  // NOTE: uses toEqual (not toBe) on each arg
  expect(mockFn).toHaveBeenCalledWith('abc', {oneTwoThree: 123})
})

test('toBeGreaterThan', () => {
  expect(10).toBeGreaterThan(3)
  expect(10).not.toBeGreaterThan(10)
  expect(10).toBeGreaterThanOrEqual(10)
})

test('toBeFalsy/Truthy', () => {
  expect(false).toBeFalsy()
  expect(true).toBeTruthy()
  expect(null).toBeFalsy()
  expect(undefined).toBeFalsy()
  expect(1).toBeTruthy()
  expect(0).toBeFalsy()
})

test('toEqual, toMatchObject, and toHaveBeenCalledWith matching a schema', () => {
  const birthday = {
    day: 18,
    month: 10,
    year: 1988,
    meta: {display: 'Oct 18th, 1988'},
  }
  const schema = {
    day: expect.any(Number),
    month: expect.any(Number),
    year: expect.any(Number),
    meta: {display: expect.stringContaining('1988')},
    // there's also expect.arrayContaining, or expect.objectContaining
  }
  expect(birthday).toEqual(schema)
})

test('mock functions', () => {
  const myFn = jest.fn()
  myFn('first', {second: 'val'})

  const calls = myFn.mock.calls
  const firstCall = calls[0]
  const firstArg = firstCall[0]
  const secondArg = firstCall[1]
  // could also do this on a single line
  // const [[firstArg, secondArg]] = myFn.mock.calls

  expect(firstArg).toBe('first')
  expect(secondArg).toEqual({second: 'val'})
})

// there are other ways to make mock functions/spies
// we'll cover those later.

/*

Snapshot tests below. We'll cover these later

 */

test('manual "snapshot"', () => {
  const flyingHeros = getFlyingSuperHeros()
  expect(flyingHeros).toEqual([
    {name: 'Dynaguy', powers: ['disintegration ray', 'fly']},
    {name: 'Apogee', powers: ['gravity control', 'fly']},
  ])
})

test('automatic snapshot', () => {
  const flyingHeros = getFlyingSuperHeros()
  expect(flyingHeros).toMatchSnapshot()
})

test('snapshot examples', () => {
  const object = {
    mixedArray: [1, [2, 3], {four: 5, six: [7, 8]}],
    regex: /do-not-try-to-regex-an-email/,
    date: new Date('1988-10-18'),
    error: new Error('some error'),
    someFunction: () => {},
    symbol: Symbol('symbol description'),
    set: new Set([1, 2, 3]),
    map: new Map([[{}, []], [[], {}]]),
    // and more!
  }
  expect(object).toMatchSnapshot()

  // AND DOM NODES!!!
  const div = document.createElement('div')
  const title = '<h2 class="title">Super Heros are great!</h2>'
  const content =
    '<p class="content">We can each be a super hero for someone</p>'
  div.innerHTML = `<section>${title}${content}</section>`
  expect(div).toMatchSnapshot('title of a snapshot!')

  // And react elements!
  const onClick = () => {}
  const element = React.createElement('button', {onClick}, 'Hello World')
  expect(element).toMatchSnapshot('react element')

  // and rendered elements
  const rendered = renderer.create(element)
  expect(rendered).toMatchSnapshot('rendered')

  // and DOM nodes rendered via react
  const app = document.createElement('div')
  ReactDOM.render(element, app)
  expect(app).toMatchSnapshot('react-dom')
})

For the rest of the tests, I’ll be using a frontend web app built in React that shows 10 blog posts and a server that handles auth and is already included in the repo.

Unit Test Demo

When writing a unit test, I should take a step back and think of the test I'm writing from the perspective of the user using this software. Not user-focused as in client, cause that would be E2E testing, think of it more like a user of this software is some other software– for this example my isPasswordAllowed() function or maybe another developer.

For this first example, Kent walks through it before implementing the next one:

//source code we're looking to test:

function isPasswordAllowed(password) {
  return password.length > 6 && /\\d/.test(password) && /\\D/.test(password)
    //can break this by changing to:
    //return (true || password.length > 6 && /\\d/.test(password) 
                //  && /\\D/.test(password))
}

//test files:
test('isPasswordAllowed only allows some passwords', () => {
  expect.assertions()
  expect(isPasswordAllowed('')).toBe(false)
  expect(isPasswordAllowed('ffffffffff')).toBe(false)
  expect(isPasswordAllowed('8888888888')).toBe(false)
  expect(isPasswordAllowed('asliduw_aksj982')).toBe(true)
})

One thing to be mindful of is that sometimes when testing things asynchronously, the assertions might not run. There are different ways to make sure they’re working, but a relatively straightforward way would be using the expect.assertions(#) syntax.

While this is a good way to do it, Kent suggests I could break the source code to make sure that I actually test the code I write so that those initial tests fail. This is one of the reasons why some people advocate for TDD since:

• You reveal the bug first and

• Can then see your changes fix the bug.

However, I can even further refactor my tests to make the error messages even more helpful.

Writing a Basic Unit Test Exercise

Here I'll need to create a test user object and pass that to the userToJSON function, and then assert that the test user object doesn't have any of the properties it's not supposed to. Let’s dive into the example:

test('userToJSON excludes secure properties', () => {
  // Here you'll need to create a test user object
  // pass that to the userToJSON function
  // and then assert that the test user object
  // doesn't have any of the properties it's not
  // supposed to.
  // Here's an example of a user object:
  const user = {
    id: 'some-id',
    username: 'sarah',
    // ↑ above are properties which should
    // be present in the returned object

    // ↓ below are properties which shouldn't
    // be present in the returned object
    exp: new Date(),
    iat: new Date(),
    hash: 'some really long string',
    salt: 'some shorter string',
  }

  expect.assertions(1)

  expect(userToJSON(user)).toEqual({id: 'some-id', username: 'sarah'})
})

I could do a bit better and break up the logic between a “safe user” and use that as the comparison to run.

test('userToJSON excludes secure properties', () => {
    expect.assertions(1);

    const safeUser = {
        id: 'some-id',
    username: 'sarah',
    };

  const user = {
    ...safeUser,
    exp: new Date(),
    iat: new Date(),
    hash: 'some really long string',
    salt: 'some shorter string',
  }

  const jsonUser = userToJSON(user);

  expect(jsonUser).toEqual(safeUser)
})

Additionally, I can break my source code to verify that my assertions are working and gain confidence that I'm on the right path.

Test Factories & Colocating Tests

Now, one thing that I've noticed is that for our first test, I was making a number of assertions, 4 to be exact. And any change to either assertion might not be visible immediately, which could cause a bunch of problems down the road. Something as benign as changing isPasswordAllowed() to isPasswordNotAllowed() isn’t easily detectable and very prone to error when adding more assertions. One way to combat this is through utilizing Test Factories.

What are Test Factories? Well let’s just look at an example by refactoring our first test:

//TODO: refactor

test('isPasswordAllowed only allows some passwords', () => {
  expect.assertions(4)
  expect(isPasswordAllowed('')).toBe(false)
  expect(isPasswordAllowed('ffffffffff')).toBe(false)
  expect(isPasswordAllowed('8888888888')).toBe(false)
  expect(isPasswordAllowed('asliduw_aksj982')).toBe(true)
})

//Refactored version:
describe('isPasswordAllowed', () => {
    const allowedPasswords = ['asliduw_aksj982'];
    const disallowedPasswords = ['', 'ffffffffff'****,**** '8888888888'];

    allowedPasswords.forEach(pwd => {
        it(`"${pwd}" should be allowed`, () => {
            expect(isPasswordAllowed(pwd)).toBe(true)
        })
    })

    disallowedPasswords.forEach(pwd => {
        it(`"${pwd}" should not be allowed`, () => {
            expect(isPasswordAllowed(pwd)).toBe(false)
        })
    })
})

Here I'm bundling my assertions within the describe block and then define both allowedPasswords and disallowedPasswords, loop through them and at each password, encapsulate the assertions using it blocks to then be able to show each assertion of the test.

Code Coverage

According to Kent, 100% code coverage is actually long past the point in time where the ROI of testing is achieved. This is because the last 10% of testing requires very finicky and unmaintainable tests to be created (or changing of source code to expose hooks that are only useful for those tests) and so shouldn’t be pursued at all costs.

The focus and energy of us as developers can only be achieved by developing an intuition about them. The guiding light should be “tests should resemble how your software is used” not as dogma though.

That's it for today! For next time I'll be moving into understanding what mocks are when to use them.