⬅️ Back to notes

Test Environments

First created: Thu Dec 23 2021

Topics for Discussion

  • What is a test environment?
  • How are test environments created?
  • What must be considered when designing, creating, and managing environments?

What is a test environment?

Prompt: How do you define an environment? What are some types of environments you’ve worked with?

In software deployment, an environment, or tier is a computer system or set of systems in which a computer program or software component is deployed and executed.

Examples:

  • development
  • test/QA
  • staging
  • production

A development environment is used during the process of writing software and is typically a local environment on the software developer’s machine.

A testing engineer uses test environments to test a very specific part of an application. It often involves a different configuration and data setup. It could include the entire deployed application or only a subset required for testing (i.e. API server with no front end used for API testing).

A staging environment replicates the production environment that will host the live version of your application. It’s crucial that your staging environment is an exact replica of your production environment.

While a test environment is focused on testing individual components, the staging environment is focused on testing the whole application.

How are test environments created?

Things required to create an environment:

  • A machine (physical or cloud) to host the deployed application
  • Configuration requirements for the environment
    • Machine type
    • Operating system
    • Dependencies, including browsers
  • Source code to deploy to the environment
  • Ports defined for the required parts of the application
    • Front end application
    • API server
    • Database (if needed)
  • Data for the environment
  • A set of instructions to configure the environment and deploy the application
    • YAML
    • CI configuration
    • Manual deployment

There may be specific configuration, security requirements, deployment steps, data requirements, or even differences to source code that may differ between test and production or staging environments.

Prompt: What methods have you seen for managing the creation of environments?

The best way to simplify the process of creating identical environments is by using Docker.

From Using Docker to manage and replicate test environments:

“Docker lets you build containers. Think of a container as your pre-configured, ready to use, super-fast, clone-able application runtime environment. So I’ll just have to configure my container once and I can use it to launch tens or hundreds of my application copies. And they’ll live separately.”

Example: Cypress Real World App

https://github.com/cypress-io/cypress-realworld-app/blob/develop/.github/workflows/main.yml

https://github.com/cypress-io/cypress-realworld-app/blob/develop/.circleci/config.yml

https://github.com/cypress-io/cypress-realworld-app/blob/develop/package.json#L172

What must be considered when designing, creating, and managing environments?

Designing
  • What type(s) of testing will be performed against the environment?
  • What part(s) of the application are required to perform the planned testing?
  • Will we have a single environment for multiple types of testing or multiple environments?
  • How many tests will be performed against the environment?
  • What are the performance requirements of the environment to support the planned testing?
  • What physical machine resources are required to support the testing and application?
  • What hardware, operating system, and browser requirements must be covered by the test environment(s)?
  • What security protocols will be used to ensure proper access and usage of environment(s)?
  • What data is required to seed the environment?
Creating
  • How long does it take to build/deploy a test environment?
  • How frequently will the environment(s) be created or updated?
  • How long will the environment be online or will it be continuously deployed?
  • How will we seed the environment for each deployment?
  • What environment variables will be needed or changed at run time? (Cypress, OS, CI)
Managing
  • How do we sanitize or reseed data in the environment?
  • Who is responsible for monitoring the environment stability, performance, and resource usage?
  • What is the strategy for removing environments that are no longer needed?
  • How do we audit existing machine/OS/browser requirements and ensure environments are consistent with requirements?

Prompt: What else would you add as considerations for environments? What have customers encountered that have caused issues?

Resources