The Story of Acquia DAM's Frontend Component Library

Co-authored by former Acquia Software Engineer, Mark Skelton.

Acquia DAM has always strived to create functional and useful user interfaces to allow content strategists, marketers, and others to manage their digital content. We have always kept a close eye on technology that would enable us to do that even better. This post will discuss the evolution of the technology behind our user interfaces including some of the good (and bad) choices we have made, as well as how we have arrived at what we think is the best way for us to effectively create consistent, functional, and useful interfaces for our users.

The Genesis of Patterns

In the beginning, the user interfaces for our SaaS products were written almost solely in Java. At the time, frontend development was a precarious place. Any sort of dynamic interaction on a webpage was essentially magic. It was a dangerous place too because we were still reeling from the effects of the browser wars and writing frontend code that would work on all browsers was very tricky even with a concentrated testing effort.

After a while, things started to settle down a bit. Browsers started to mature, and frontend code became a slightly less crazy place. This was around the time the concept of single-page applications (SPAs) started to arise. We began to dig into them with a major rewrite of a large chunk of our main SaaS product. We chose AngularJS at the time as the framework for our project. We liked Angular because – as Java developers – many of the concepts were similar. It was also backed by Google so surely it will be a stable, popular, and eventually mature framework (spoiler: this did not happen).

When Angular 2.0 dropped, we were really disappointed. The amount of breaking changes and work required to make our new SPA Angular 2 compatible was simply unjustifiable. Also, Angular was confusing. It introduced new words and concepts that just made things more complicated (“transclude the isolate scope”, anyone?). Surely, there had to be a better way…

… and thus began our foray into React.

We used React in a couple of greenfield projects that we planned to ship to customers, and developers quickly fell in love with it. Our first user-facing React application was up and running relatively quickly, and another soon followed. Our velocity was increasing, but we encountered some growing pains.

Despite teams working independently, we wanted our applications to have the same look and feel. Not only does a cohesive design look good, but it is less surprising for users and results in a better experience as they hop around different apps in our suite. Not only did we want our apps to be cohesive, but we also wanted our frontend code to be well-tested and to follow standards and best practices.

At first, we had teams developing their own individual components. So things like date-pickers, inputs and forms, and even common styling were all done independently. It seemed like a waste of time and effort to have different versions of what would become essentially the same component, and obviously copy-pasting between projects was not ideal. That’s when Patterns was created.

Image

Patterns was the name of a project that attempted to unify our frontend development practices with a shared component library. It was a home for shared styles, components, and even libraries and tools. Patterns also had what we called The Patterns Playground, where developers could easily integrate their components into a more simplified application. This was nothing revolutionary – many other organizations had shared component libraries – but it had many benefits: feedback loops were created so developers could more easily create, integrate, and test components; testers could test a component in isolation without having to setup a bunch of state in a “proper” app; and designers could more quickly see the results of their designs.

Image

These benefits did not come without some pitfalls, of course.

Challenges We Faced

Over time, Patterns grew from where it started as a side project to an integral part of front-end development at Acquia. Developers were able to build new features much quicker by using the components and styles provided by Patterns. However, these improvements came with some challenges.

First, at the time of its creation, there were not many great tools available for managing a monorepo of shared components. Lerna – the de-facto standard nowadays – was still in its infancy and had just been ported out of babel. Yarn and its wonderful workspace management tooling had not been released to the public yet. Fortunately, we were able to make it work well enough with just npm, a Makefile, and lots of glue code.

One of the foremost challenges we faced with Patterns was version management and publishing. When a package was published, other packages that depended on it would also need to be updated to the latest version. As the number of packages increased, so did the depth of dependent package hierarchy. For example, a simple color change in our style package would require a change to be made to our buttons which would require a change to our dialogs. Not only did this increase the difficulty of publishing packages, but it also led to dependency duplication when a dependent package was updated for some but not all packages. This could lead to the same component looking different because of mismatched versions.

Another challenge was incorrect or inconsistent package outputs. Publishing packages would be completed by developers from their local machine, which could lead to inconsistent results in the published package outputs if the developer failed to properly build the package locally before publishing. This was especially problematic when renaming files as old versions of transpiled files might exist and get unintentionally published to the registry.

In addition to these challenges, the project had fallen behind on some of our front-end standards and best practices, making development in Patterns less familiar to developers. Lack of support for TypeScript and unit testing were among these challenges in addition to inconsistency in package structure and component API approaches.

Improving the Architecture

After struggling with these problems for a while, we decided to overhaul some of the core architecture of Patterns. That said, we wanted to be very careful that changes we made didn’t decrease our velocity on the project or introduce bugs that would block other developers from completing work. So, we decided to create a new project where we could experiment with the proposed changes and use the new project as a template to bring these changes back into Patterns. The plan worked very well, and the new project we created became a utility library which now houses utility methods, build scripts, tooling configuration, and more.

The utility library (which we call Spanner), solved the version management problem by introducing Lerna to manage versioning and publishing of packages. Lerna tracks changes you make to packages and will automatically bump the versions of any change packages as well as packages that depend on them. This not only simplified our publishing process to a single step, but it also ensured that packages were always up to date and depending on the most recent version of other packages.

Lerna also helped solve our problem with inconsistent package outputs by allowing us to publish packages from CI. When a developer wanted to release a package, they would run a simple release script that would use Lerna to select the new version, bump all affected package versions, and push changes to the master branch. Our CI server would then recognize that packages needed to be published, and it would build and publish the packages to our npm registry. This ensured that every time we published a package, it was built from a fresh clone without any stale output.

As we made these changes, our velocity increased allowing us to make more improvements to Patterns including adopting TypeScript, increasing our unit test coverage, and adopting a more consistent package structure. Many of these improvements were also inspired by our learnings from Spanner which continues to guide the architecture of the Patterns project.

The Future Vision

Since embarking on the journey of re-architecting Patterns, we have experienced increased productivity, fewer bugs, more consistent components, and happy developers. The project has truly been a success, but there are still improvements that we plan to make.

First, we plan to improve changelog generation when publishing packages. Currently, a single changelog is maintained and developers contribute to it in their pull requests. This causes frequent merge conflicts and forces the developer who is releasing changes to manually determine the new version to be published. To mitigate these problems, we hope to transition Patterns to use Changesets which we have already implemented in Spanner with great success. In a nutshell, Changesets stores “intents to change” as individual files in the repository and merges them all together when releasing. This removes any change of conflicts and automatically enforces the same changelog structure for all releases.

Another continued area of improvement is TypeScript usage. While all newer packages in Patterns are written in TypeScript, many older packages are not, which can lead to more bugs and a less consistent developer experience. The same can be said for our unit tests which are a combination of Enzyme tests in our older packages and React Testing Library tests in our newer packages. Fortunately, these changes are not all or nothing and can be accomplished slowly over time.

There are certainly more areas we would like to improve such as visual testing and better documentation, but I’ll leave those for future blog posts.

Thanks for reading!