Dane Powell

Principal Software Engineer

Dane has been building enterprise Drupal websites since 2008. He has led and enabled technical teams across a variety of industries on projects ranging in scale from one-off decoupled sites for conferences to multi-thousand tenant multisite platforms for financial services agencies. He works with customers and partners to transform business requirements into technical solutions that exceed expectations, enjoys solving hard problems, and always aims to enable customers and partners to do the same. Dane is also an active member of and regular contributor to Drupal and the larger open-source community. He has contributed and maintained several enterprise modules, contributed over 1000 patches, commits, or pull requests, and is currently a maintainer of BLT, an automation framework for Drupal. He has also presented at several Drupal camps, with a focus on configuration management and automation. Outside of Drupal, Dane is a regular contributor to open-source projects (such as Serverless and Symfony) and enjoys developing for a number of other ecosystems including Amazon Alexa, Google Home, and Android. He is the creator and maintainer of several Alexa Skills and Android apps on the Google Play store. Several of these are open source and make use of server-side components such as Node.js and MongoDB. Through his work, he has also discovered and responsibly disclosed multiple security vulnerabilities in popular enterprise applications. Prior to joining the Drupal community, Dane was a NASA Space Technology Research Fellow at Rice University and Johnson Space Center, where he researched human-computer interaction and developed haptic teleoperation interfaces for NASA’s Robonaut program. His other interests include rock climbing, traveling, and spending time outdoors.

Working with BLT: An Automation Layer for Testing, Building, and Launching Drupal 8 Applications

January 10, 2018
9 minute read

Overview

Mike Madison, a Technical Architect in Acquia Professional Services, recently completed a Drupal site build for a major public transit agency in the United States. I spoke with him in December 2017 about his experiences using BLT on this project.

Mike said that BLT has been a critical component of the project’s success, and has especially helped in three primary ways: by accelerating project spinup, improving developer onboarding, and increasing development velocity and delivery consistency.

About the project

This transit agency set out to improve the efficiency of its customer experience, and create a cohesive user journey across all touchpoints and channels. To provide every commuter with the right experience, on the right device, at the right time, they needed to extend the information accessed on their website into train stations and platforms, as well as create an intranet.

This required the ability to push screen-based information, news, and alerts in real time to the thousand of countdown clocks that are installed across the transit’s lines. The central challenge was to successfully process real-time train data from the largest public transit system in the world, and display content rapidly enough so that it was relevant to passengers.

About BLT

BLT (Build and Launch Tool) is an open-source Acquia product that provides an automation layer for testing, building, and launching Drupal 8 applications. It accelerates the delivery of Drupal projects by assisting with project creation as well as day-to-day tasks such as automated testing, deployments, and developer onboarding. It is freely available on Github: https://github.com/acquia/blt

Now let’s dig in and look at...

The three ways that BLT contributed to this project’s success

1. Project Spin-Up: From bare repo to production in under two hours

Mike was able to spin up the project from scratch in under two hours before Sprint 0 even started. This meant that the entire project team could hit the ground running on day 1: developers pulled tickets, Mike merged their pull requests, and Acquia Pipelines immediately deployed their code to Cloud environments where the QA team could start testing.

"I didn't have to worry about Behat, CI, etc... it just worked," Mike said. “I spun up the entire project in under two hours, and I was watching TV while I did it.”

Specifically, this spin-up process included setting up:

A DrupalVM common local development environment
A Behat-based testing framework
Continuous integration and automated testing (Pipelines)
Automated Cloud Deployments (via Pipelines and Cloud Hooks)
Configuration management (Config Split)
A custom Lightning subprofile of the site in a production environment.

Prior to the availability of BLT, setting up all of these tools independently could easily take hundreds of hours. The goal of a Sprint 0 was often simply to set up such tooling.

In practice, it was rare that all of these tools were actually used on every project due to the overhead involved in setting up and maintaining them. Thus, BLT has not only saved time on project spin-up, but also provided a more comprehensive set of development processes and tools for each and every project, which in turns improves the consistency of delivery of projects such as this.

2. Accelerating developer onboarding

The use of a standardized tool and development process across all projects makes developer onboarding much faster.

Most developers on this transit project had prior experience with BLT-based projects, and were able to set up a local development environment and start opening pull requests on their first day on the project. BLT is especially good for onboarding because of its deep integration with DrupalVM.

Even for developers who have never heard of BLT, the onboarding process is fast and simple, thanks to the step-by-step setup documentation. However, Mike did say that developers with a strong grasp of modern Drupal development tools like Git, Composer, configuration management, and Drush are going to have a much better onboarding experience.

In fact, Mike said that a strong working knowledge of those underlying tools is the most significant factor in a developer’s success, and when things do go wrong, it’s most often with those tools.

Interestingly, most of the support requests that BLT receives are not due to bugs in BLT’s own code, but to problems with those underlying tools. Mike describes BLT itself as “foolishly simple” to use.

BLT already provides extensive documentation, trainings, and support (via the issue queue) for these tools, but clearly there is an opportunity for improvement here. It’s often not enough to tell people to “read the docs.” BLT needs to engage in more proactive outreach and training. It also needs to consider providing more tools to automate complex processes (such as configuration management) and provide real-time contextual guidance and support to users.

Finally, regardless of the amount of training and support provided by BLT, it’s important for any project using open-source tools to have motivated and self-starting developers who are capable of effectively troubleshooting, reading documentation, and knowing how to ask for support when necessary.

3. Improving development velocity and delivery consistency

Once developers began working in earnest on the transit project, BLT was most helpful in standardizing complex processes, which reduces the chance of user error and the length of feedback cycles for code reviews, QA, etc.

For instance, BLT provides a single command (`blt setup`) that can install PHP dependencies via composer install, install and build front-end components via Gulp or NPM, install Drupal, import configuration changes, and toggle environment-specific modules (such as devel).

Running this single command before starting work on a new ticket helps developers to ensure that they are working from a clean slate, and improves outcomes. Similarly, when committing changes, BLT performs quick PHP CodeSniffing in order to immediately catch any simple mistakes, reducing the cycle time for code reviews and feedback.

Challenges for BLT

Mike said that he sees one of the biggest challenges for BLT is expanding its user base by marketing to a wider and more diverse audience. It should be noted that BLT can be run in a huge variety of environments, and is fairly agnostic as to a project’s choice of hosting provider, source control system, or CI system. However, BLT has traditionally focused on supporting a core stack of tools (such as Github, Acquia Pipelines, and Acquia Cloud) that were also used internally by Acquia Professional Services.

Striking a balance between directly supporting a common stack of tools while also keeping a loose coupling with those tools will always be hard. However, there are some concrete ways that BLT could improve, such as by building integrations with more 3rd-party services. For instance, BLT has recently added support for additional CI tools such as Tugboat and Probo.

Finally, there are some common challenges in building enterprise applications that BLT may not be able to address directly, but may be able to coordinate with other projects to improve.

For example, corporate IT departments often impose unique constraints on projects that can hinder the use of open-source tools such as BLT, such as firewalls that block access to Github (where most Composer dependencies are stored), denying sudo/root access to developers (which is required to install virtual machines), only supporting Windows (which is not a preferred development environment), or using on-premises or otherwise restrictive source control systems that don’t work with Cloud-hosted CI tools.

In Conclusion

While the above challenges are real, so are the benefits to using BLT.

Based on Mike Madison’s recent experience, he recommends BLT to developers who are looking for ways to spin-up projects more quickly, improve onboarding, accelerate velocity, and upgrade delivery consistency.