yeti logo icon
Close Icon
contact us
We'll reply within 24 hours.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Authorizing Google Cloud Platform Service Accounts from a Docker Container Running in Heroku

By
Will Harris
-
July 28, 2021

In this post I'm going to cover how we dealt with authorizing a Google Cloud Platform service account from a Docker image running in Heroku's cloud platform.

Recently we implemented a system using Google Cloud's Pub/Sub messaging service and My Business API to capture updates to Google Places businesses with a NodeJS server. Google Cloud requires applications to authenticate as service accounts in order to interact with their APIs. Service accounts allow a non-human user to authenticate and access Google's APIs.

The Google Pub/Sub client library loads service account credentials into a GOOGLE_APPLICATION_CREDENTIALS environment variable from an absolute path to a .json file in the filesystem. This meant that we needed access to a service-account.json file in our application's environment in order to authenticate and authorize our service account.

One of our other main requirements for this application was that we deploy the app to Heroku. We also deploy most of our projects in Docker containers, so we needed to figure out how to add the service account's credentials file to the Docker container's filesystem during Heroku's build process.

First (Unsuccessful) Attempts

The first solution we tried was adding the contents of the service-account.json file as a Heroku environment variable. Heroku accepted and saved the value, but it was not available within our Docker container. When we tried to read the value, we got a parsing error at the first " in the file. It seemed as though Heroku was unable to parse raw JSON stored in an environment variable.

Another solution we tried was using one of the "off-the-shelf" Heroku buildpacks that claim to help with this issue. They're both essentially just shell scripts that echo the value stored in a GOOGLE_CREDENTIALS environment variable into a file in the .profile.d/* directory. This directory is similar to /etc/profile.d in standard Unix filesystems, where shell startup files are typically stored.

Unfortunately, one of the current limitations of Heroku's "container" stack is that it does not run .profile or .profile.d/* scripts when booting a dyno. So neither of the Heroku buildpacks worked for us.

Solution

We're still dealing with a Linux filesystem in Docker though, and the filesystem has access to Heroku environment variables during the build process. So the question became: how can we output the contents of a Heroku environment variable to a .json file in the container's filesystem during the build process?

We ended up writing a small shell script that executes in our production Dockerfile to do just this.

First we added the contents of the service-account.json file to a GOOGLE_CREDENTIALS environment variable in Heroku.

We then added the following script ( add-google-credentials.sh ) to the root of our project:

#!/bin/sh

echo "Generating google-credentials.json from Heroku environment variable"

echo $GOOGLE_CREDENTIALS > google-credentials.json

exec "$@"

This script takes the value stored in the GOOGLE_CREDENTIALS environment variable and echoes it into a new .json file in the Docker container's filesystem, creating a fresh copy each time we build the container.

The exec "$@" line replaces the parent process with the current child process. This is important in Docker containers for signals to be proxied correctly, otherwise we might end up with data loss or orphan processes. More information about this can be found in this interesting Unix Stack Overflow discussion.

Finally we needed to actually call this script during the Docker build process, so we added the following two commands to our Dockerfile :

COPY add-google-credentials.sh /app/add-google-credentials.sh
ENTRYPOINT ["sh", "/app/add-google-credentials.sh"]

The first command copies the shell script from our project's root directory into the container's root directory.

The second command basically says "every time this container builds, run the add-google-credentials.sh script."

These changes allowed us to store a GOOGLE_APPLICATION_CREDENTIALS environment variable in Heroku with the value google-credentials.json, which is an absolute path to the service account credentials inside our container. After putting this together we were finally able to interact with the Google Cloud APIs that we needed for this project! 🎉

Will is a software developer at Yeti. In his free time you can find him riding bikes, hiking in the hills and searching  Bay Area restaurants for delicious vegetarian food.

You Might also like...

Busting 5 App Development Myths

The following are five pervasive myths about why app development should be a simple process — and the real reasons that a good product takes time.

Token-based Header Authentication for WebSockets behind Node.js

Using Node.JS to proxy requests to mutate them under the hood can be beneficial. In cases like these, it can also make your product more secure.

VIDEO: Building API's with Django and GraphQL

At our last Django Meetup Group event, Jayden Windle, the lead engineer at Jetpack, an on demand delivery company, talks building APIs with Django and GraphQL. Watch the video to learn more.

Browse all Blog Articles

Ready for your new product adventure?

Let's Get Started