Acquia Developer Portal Logo

Code Studio Integrating Lighthouse Testing Suite

  • Last updated
  • 5 minute read

Goal

Learn how to use Acquia Cloud, our CDEs, and Code Studio to run Lighthouse CI automated tests in our Drupal specific CI/CD build process.

Prerequisites

  • Basic knowledge of NPM, CI/CD workflows, automated testing, and the Acquia Cloud.

Overview

What is Lighthouse

Lighthouse CI (LHCI) is an open-source, automated tool for improving the quality of web pages. Built by Google, LHCI makes it easier to test, archive, and browse reports on SEO, performance,  and accessibility. Developers and site owners use these reports to take action against and improve their applications.

Features
  • Run LHCI reports alongside every PR.
  • Prevent regressions in accessibility, SEO, offline support, and performance best practices.
  • Track performance metrics and LHCI scores over time.
  • Set and keep performance budgets on scripts and images.
  • Run LHCI many times to reduce variance.
  • Compare two versions of your site to find improvements and regressions of individual resources.

Where should I run my Lighthouse tests

When testing web applications there are multiple areas to consider, functional vs non-functional, code standards, visual regression, performance, SEO, accessibility, and more. Taking a step back, we first need to determine which test should run in which environment and scenario.

In the CI/CD build, DEV or PROD 

Testing in the CI/CD build

Whether you are using Jenkins, Travis, CircleCI, Bitbucket, Gitlab or Acquia’s Code Studio, our Drupal-optimized CI/CD and code collaboration solution built around Gitlab. If we look at CI/CD, focused on compiling and testing the code then there are a AutoDevOps you can run in the CI from code standards, PHPcompatibility, Semgrep, SAST to unit testing, these are all good examples of the kinds of tests you would run in the CI build process. 

Using the build process to focus on testing the code will ensure your team that all code on its way to PROD has been tested and validated. Another consideration is the time it takes for each build to run. Taking into account that you might have a team of developers pushing code, how much total time do you want between pushing and the availability to deploy?

As mentioned above, the build process can play a valuable piece of your DevOps puzzle. Further in this article, I’ll explain how to use CI/CD to set up, and archive LHCI tests to run on different environments.

Testing in a temporary environment or DEV server

At this point, the code has already been deployed and is on one of the servers, DEV, Non-PROD or PROD. Having the code tested prior to this stage ensures that your team is testing a validated artifact of the application. This is a good time to start testing other aspects of the application, like Accessibility and SEO. 

Running your LHCI test here makes sense, as you have the Database, Files and Configuration ready to go. That and the fact that it's an environment with known resources to benchmark against.
 
At Acquia, we have deployed CDEs, or Continuous Development Environments that are integrated into our Code Studio product to spin up new CDE environments on any new merge requests. This existing process gives us some interesting tools to build our LHCI testing into. 

When thinking about performance testing, some customers choose to rerun these tests in PROD to benchmark against. When doing this, consider that the performance tests will not be comparable, as the Non-PROD environments usually do not have the same allocated resources as PROD environments. 

Are these real tests if I’m not running them in PROD?

What is real? Does your UAT or TEST server have the same allocated resources as PROD? Can you observe the traffic and load on the server while maintaining efficient page load speeds. 

There are things to think about when it comes to the different kinds of tests, benchmarking, measuring, and iterative. For all these examples, the goal is to improve and mitigate risk. If you keep that in mind, you can develop your own testing procedures that will lead to a successful DevOps process. 

While Lighthouse’s primary focus is performance, it can assess other things like:
  • Accessibility: Recommendations on making content more accessible. This ensures that the page is optimized for screen readers, all elements have labels, or the site is browsable with the keyboard.
  • Best practices: Scans the page issues not following best practices and can improve performance and security.
  • SEO: Runs different checks to ensure that the page is SEO-friendly

  1. The objective is to spin up a CDE and run the test each time a merge request is made.

    The goal of this is to have a new feature branch get created, which automatically spins up a new CDE in the Acquia Cloud, connected to that merge request. This allows us to write the LHCI tests around the merge request event type. 

  2. #.gitlab-ci.yml

    If you have not done so already, create a `.giltab-ci.yml` file in the root of your project. To enable it, go to https://code.acquia.com/<YOUR-USER>/<YOUR-PROJECT>/-/settings/ci_cd and expand the General pipelines section. Change the CI/CD configuration file to `.giltab-ci.yml` . 

    The code below adds a new job to the `Deploy Drupal` stage called `Create Lighthouse Test`. The job has a dependency on the `Create Acquia CDE` job and runs only on merge requests. We are allowing this test to fail and archiving the reports for a week in a folder call ./report. 

    include:
  - project: 'acquia/standard-template'
    file:
      - '/gitlab-ci/Auto-DevOps.acquia.gitlab-ci.yml'

variables:
  REPORT: '$CI_PROJECT_DIR/report'

"Create Lighthouse Test":
  stage: "Deploy Drupal"
  extends: .cache_strategy_push
  needs: ["Create Acquia CDE"]
  before_script:
   - acli update
  rules:
  - if: ($CI_PIPELINE_SOURCE == "merge_request_event" && $ACQUIA_JOBS_CREATE_CDE == "true")
    when: on_success
    allow_failure: true
  artifacts:
    when: always
    expose_as: 'Lighthouse'
    paths:
        - "$REPORT/lighthouse/"
    reports:
      junit: $REPORT/lighthouse/summary.json
    expire_in: 1 week
  services:
    - selenium/standalone-chrome
  script:
    - source scripts/ci/lighthouseci.sh

  3. #lighthouseci.sh

    At the end of the new job `Create Lighthouse Test`, we are running a script called lighthouseci.sh which will install the dependencies and invoke the test on the newly created CDE environment. The only reason we are not adding these 2 lines of code to the CI file is for the ability to add a little more messaging around the process. 

    #!/bin/bash

# Run lighthouse tests
#

echo -e "\e[0Ksection_start:`date +%s`:lighthouse_test[collapsed=false]\r\e[0K\033[1;35m[CS AutoDevOps] Lighthouse Testing\033[0m"

  printf "🔎 Running Lighthouse Test on CDE\n"

set -x
  npm ci
  npm run lhci
set +x

  4. #package.json

    To run the snippet above, the dependencies have to be built. The package.json file located in the root of your project has what’s needed to install and run the tests. LHCI gives the ability to add `fail-fast` parameters to let the CI build know when to fail, if the test does not pass the threshold. The file below sets up the basic collection thresholds and what URLs to run the test on. 

    NOTE: Make sure to run `npm install` locally prior to pushing to Code Studio. The `npm ci` command requires the `package-lock.json` file to be there in order to complete the build. 

    {
  "name": "lighthouse",
  "version": "1.0.0",
  "description": "set up and test lhci",
  "author": "flux423",
  "license": "ISC",
  "engines": {
    "node": ">= 0.14.9"
  },
  "dependencies": {
    "lighthouse-ci": "^1.13.1"
  },
  "scripts": {
    "lhci": "npx lighthouse-batch -h --seo 80 --best-practices 80 --accessibility 80 --fail-fast -f ./tests/lighthouse/lighthouserc.sites.txt"
  },
  "devDependencies": {
    "@lhci/cli": "^0.11.0"
  }
}

  5. #lighthouserc.sites.txt

    Next you package.json command points back to a txt file located at `./tests/lighthouse/lighthouserc.sites.txt` which has the list of urls to run against. You can see how we can use the code studio global variable $DYNAMIC_ENVIRONMENT_URL to pull the URL of the CDE the merge request build is connected to. 

    https://$DYNAMIC_ENVIRONMENT_URL/home-sitestudio
https://$DYNAMIC_ENVIRONMENT_URL/articles
https://$DYNAMIC_ENVIRONMENT_URL/articles/article-1
https://$DYNAMIC_ENVIRONMENT_URL/articles/article-2
https://$DYNAMIC_ENVIRONMENT_URL/events
https://$DYNAMIC_ENVIRONMENT_URL/events/event-1
https://$DYNAMIC_ENVIRONMENT_URL/events/event-2
https://$DYNAMIC_ENVIRONMENT_URL/people
https://$DYNAMIC_ENVIRONMENT_URL/places

Downloading the results

In the Code Studio UI, CI/CD > Pipelines page, you can review build jobs and download the artifacts that Code Studio saves. Using the steps above, we’ve added a new archive named `Create Lighthouse Test:archive` . Clicking on that link will download a zip with HTML and JSON files for each page in the list above. 

Code Studio Lighthouse Download

Example Lighthouse Test

Looking at one of the HTML files, you can see that lighthouse renders each page and runs the Performance, Accessibility, Best Practices, SEO and Progressive Web Apps tests. 

Example Lighthouse Report

Conclusion

The steps above can get you started on developing a clean DevOps process and ensuring you run the right tests at the right times in the right environment. 

Next Steps

  • Build a strategy around testing, where, when and what
  • Implement a POC and run process tests around that to determine if this fits into your existing workflow
  • Talk to an Acquia Professional Services Solutions member to hear more about how our PS team can help your team build a refined DevOps process using our best in class tools built for Drupal

Similar Tutorials