In this two-part series of blog posts, I'm going to show you how we built a Drupal 8 photo gallery site, integrated with Amazon S3, Rekognition, and Lambda to automatically detect faces (allowing us to automatically identify names!) and objects found in our photos.
Acquia Build Week
Every year, Acquia puts on Build Week, a week-long event where Acquia's engineering teams (along with other departments, like OCTO, Professional Services, and more!) come together to learn, collaborate, and inspire a little crazy. One of the most intense, fun, and productive parts of Build Week is the two-day Hackathon. Every year there are some new and interesting initiatives, spanning a very wide range of interests. Some projects, like the exploration and strengthening of Drupal 8's then-nascent Configuration Management tools, are geared towards building up Drupal 8. Other projects are more imaginative and exploratory, like augmented reality shopping experiences, which demonstrated how Acquia Lift can deliver a more personal and relevant shopping experience.
The dream - an open photo sharing experience
This year, I joined a team of Acquians who loved sharing photos, but wanted to have an open and powerful solution for building galleries and sharing images. Looking at our options, we determined that:
- Proprietary Cloud platforms (e.g. Google Photos, iCloud Photos, Facebook) take away some of your control over your online images, and some even require others to have an account to view shared photos.
- Traditional online galleries (e.g. Flickr) are severely lacking in features like facial recognition and categorization—and their futures are a little murky.
- Older Drupal-based (like Gallery) or open source photo gallery solutions (like Coppermine) are either outdated and no longer supported, or critical features like Amazon S3 support and privacy and permission options aren't supported.
Since we only had two days to solve all the world's photo gallery problems, we focused on the MVP features for a modern, open source photo gallery:
- Self-hosted, so you have total control over storage, organization, sharing, and privacy.
- Easy bulk upload and album-based organization of multiple photos.
- Use tried-and-true tech: Drupal 8 for content organization, Amazon S3 for file storage.
- Automatic face and object recognition, so we can do things like ask "show me all the pictures of Jeff Geerling", or "show me pictures of mountains".
Our primary goal was to get a demoable MVP product. But our stretch goal was to make this solution open source, so anyone else with the same requirements could benefit—or even go further and make Drupal an even better tool for photo sharing and organization!
The tools
We decided to use Drupal 8 as a CMS for all the gallery, user, and photo data, because it was easy to get it up and running on Acquia Cloud, and because modules for almost all the things we needed (bulk upload, S3 integration, etc.) already exist (saving us a lot of work!). Drupal also has a powerful and flexible user roles and permissions model, which means adding an important feature like gallery or photo-based access is easy. To power the marquee feature of the site—face and object recognition—we chose to use Amazon Rekognition. Rekognition uses deep learning to analyze billions of images, and then uses all that data to label objects and detect faces in individual photos. We needed a place to store all the photos (it would be inefficient to try to store many gigabytes of photos on our webservers!), so we chose to use Amazon S3 to store all the site's photos in one S3 bucket. Finally, we needed a way to get photos that were uploaded to S3 to be processed by Rekognition, then we needed to send that data back to Drupal so it could be stored in Taxonomies and displayed on the site. Lucky for us, AWS Lambda allows us to do just that, without having to write much code at all. We exposed a REST endpoint on our Drupal site (using Drupal 8's new REST API), and then had Lambda post data back to Drupal after Rekognition finished processing an image.
The team
(From left to right: Jeff, ASH, Glenn, Meagan, Matthew, and Rok.) We called our project S3 what I did there?, and our team had great diversity (helpful in making sure that not only is the tech good, but the presentation and organization were just as good!):
- Matthew Wagner (the Acquian who came up with this crazy idea).
- ASH Heath
- Meagan White
- Jeff Geerling (that's me!)
- Glenn Pratt
- Rok Zlender
Every member of the team shared a passion for photos and photography, and contributed a ton of work to get our project shipped, polished, and presented! Even though we didn't win the Hackathon, we had a great end result, and it was exciting to meet and work alongside these great people.
Setting the stage
Since we had only two days to architect and build the site, we needed a boost in terms of getting a Drupal 8 site up and running. Luckily, I already had built a very basic and simple photo gallery website that used Drupal 8 and the Media entity image module to upload and organize photos in albums. The site already had a functional bulk image uploader, and had responsive image styles configured so the photos looked nice and sharp on any device, retina display or no. It was also configured for running Drupal VM via Docker, and all the configuration was exported using core CMI, which made it easy to set up new local environments—just run docker-compose up, then use drush site-install with the --config-dir option, and the site would be identical to production, just without any of the content or pictures! Once I had gotten the 'bones' of the site transferred over from my existing installation, we set to work building out the functionality integration Drupal 8's Media modules with Amazon Web Services—and I'll dive into the details of how we did that in the next blog post—stay tuned!
