The Missing Dev Guide for Firebase and MailChimp Integration

Moved from Medium. Original Post Date: August 27th 2022.

There is a Firebase extension available to create and update MailChimp contact data based on your Firebase Authentication and Firestore documents. The extension has some documentation, but it’s hard to follow the steps to configure it properly and troubleshoot issues that arise.

Configuring the extension

The extension leverages the following Firebase products:

• Authentication to manage (add/remove) users

• Cloud Firestore to create member tags, merge fields, and member events with Mailchimp.

Let’s focus first on the “required” fields to be configured in the extension. Luckily, those are pretty easy to configure and the extension has all the details for you to find those values.

• Cloud Functions location: Where do you want to deploy the functions created for this extension?

• Mailchimp OAuth Token: To obtain a Mailchimp OAuth Token, click here.

• Audience ID: What is the Mailchimp Audience ID to which you want to subscribe new users? To find your Audience ID: visit https://admin.mailchimp.com/lists, click on the desired audience or create a new audience, then select Settings. Look for Audience ID (for example, 27735fc60a).

• Contact status: When the extension adds a new user to the Mailchimp audience, what is their initial status? This value can be subscribed or pending. Subscribed means the user can receive campaigns; pending means the user still needs to opt-in to receive campaigns.

Required Configs in MailChimp Firebase Extension

Do not worry yet about configuring the rest of the extension settings. For the Firestore Cloud based capabilities, you can leave them as “unused”. This way, you can focus on verifying that “adding/removing users” works first, then add those more advanced capabilities later.

Advanced Configs in MailChimp Firebase Extension that can be left unused for now

NOTE: Always use “subscribed” rather than “pending” if you want the integration to just work. Read more if you want to learn why.

Once the extension is installed, it creates 5 Firebase functions: 2 that are authentication-based and 3 that are Firestore-based.

1. Authentication: (1) Create User, (2) Delete User

2. Cloud Firestore: (3) Create member tags, (4) merge fields, and (5) member events

Firebase functions created when extension is installed

Authentication Based Capabilities

Assuming you already use Firebase Authentication for new users that sign up to your web or mobile app, you could leverage the extension to add/remove those users in MailChimp. In other words, keep your list of users in MailChimp in sync with the one in Firebase Authentication.

Firebase Authentication

Adding Users to MailChimp

As mentioned earlier, when the extension is installed, it creates a Firebase Cloud function that listens for new user accounts (as managed by Firebase Authentication), and then automatically adds the new user to your specified MailChimp audience.

My first issue was managing to get a new user that signs up via Firebase Authentication to be created in MailChimp. I had the basic configs for Auth token and Audience Id, but for whatever reason, I couldn’t see the new contact being created in MailChimp.

Basic Testing for Adding User To MailChimp

In Firebase Authentication, click “Add User” to add a new user.

Adding a User Manually in Firebase Authentication

This creates this user with the following ID in Firebase Auth.

Added User in Firebase Authentication

In theory, I should be able to just go to MailChimp and see the newly created user. So I went to my MailChimp account at https://admin.mailchimp.com/lists/, clicked on the audience I connected earlier in the config, and found nothing.

Member Not Found in MailChimp

Troubleshooting

So why isn’t the user getting created in MailChimp?

I first went to the “user.create” Firebase function and opened the logs.

Firebase Function Logs Entry Point

In the logs, it looked like everything was working as expected.

Showing Success in Firebase Function Logs

It’s attempting to create in MailChimp the same user I added in Firebase Auth (same ID), and it’s showing that the addition was completed successfully.

Firebase ID: 9JjCwv9O2jPpELIN70KUsWmqpRr1

MailChimp ID: f8f1a4191871855a1d716485fac46eea

Yet, I do not see the user in MailChimp!

Looking at the source code of the function, it was basically making a POST call to /lists/<mailchimp_audienceId>/member passing to it email_address and status, but for whatever reason, the user was not visible in MailChimp.

exports.addUserToList = functions.handler.auth.user.onCreate(
  async (user) => {
    logs.start();

if (!mailchimp) {
      logs.mailchimpNotInitialized();
      return;
    }

const { email, uid } = user;
    if (!email) {
      logs.userNoEmail();
      return;
    }

try {
      logs.userAdding(uid, config.mailchimpAudienceId);
      const results = await mailchimp.post(
        `/lists/${config.mailchimpAudienceId}/members`,
        {
          email_address: email,
          status: config.mailchimpContactStatus,
        }
      );
      logs.userAdded(
        uid,
        config.mailchimpAudienceId,
        results.id,
        config.mailchimpContactStatus
      );
      logs.complete();
    } catch (err) {
      err.title === 'Member Exists' ? logs.userAlreadyInAudience( ) : logs.errorAddUser(err);
    }
  }
);

I then decided to check out the MailChimp API to see if the user is actually being created in MailChimp and maybe I’m not figuring where to go in the UI to find it.

Given that I already have the MailChimp user id from the logs, I decided to try and fetch it through the MailChimp API:

https://<dc>.api.mailchimp.com/3.0/lists/<mailchimp_audienceId>/members/<member_id>

To be able to make API calls, you need to create an API key. The <dc> for me was us19, this value can be pulled from the end of the API key.

In PostMan, I created the GET request, and there it was, the user is in the system!

GET Request in PostMan to retrieve a specific MailChimp member

It finally hit me. “Pending” status might be the problem. I went to MailChimp online documentation and learned this:

A pending contact is someone who has completed a signup form but has not confirmed their opt-in. The contact’s email address won’t be added to your audience until they confirm their opt-in, and you won’t be able to view them in your audience. After 60 days, if they have not confirmed their opt-in, the contact’s email address and any other information they provided will be removed and deleted.

I also saw this in my email. When I confirmed, the status moved from “pending” to “subscribed”, and then I was able to find this user in the MailChimp audience.

MailChimp Email to Confirm Subscription

I couldn’t find where this form could be configured (might relate to my Squarespace and MailChimp integration), and I also realized this might not be the best experience to welcome new users with a subscribe message. As a result, I opted to create new users in MailChimp with the “subscribed” status, so I can focus on building the right “Welcome” message for new users that sign up through my app.

Another thing to keep an eye on is using “valid” email addresses when testing. For example, I tried adding a user “mailchimptest@test.com”, but the user did not end up being added to MailChimp since it was considered as looking “fake” or “invalid”.

Error in the Logs for using Invalid Email Addresses

Deleting Users in MailChimp

For GDPR compliance, the extension also creates a Cloud Function that listens for existing user accounts to be deleted (as managed by Firebase Authentication), and then automatically removes them from your specified MailChimp audience.

Cloud Firestore Based Capabilities

The capabilities dependent on Cloud firestore have two things to configure:

1. Watch Path: the name or path of the Firebase collection to lookup data.

2. Config: the JSON mapping that will tell the extension what data to look for in the document to pass it to MailChimp

Each of those capabilities will get one Firebase Cloud function created, to listen to document changes in the specified Firestore collection and map the data based on the config defined in the extension into MailChimp.

1. Merge fields Cloud Function provides the ability to create new properties that can be associated with Mailchimp subscriber. The mergeFieldsHandler function listens for Firestore write events based on specified config path, then automatically populates the Mailchimp subscriber’s respective merge fields. In other words, all this does is get additional data about the user and map it into member (contact) fields in MailChimp.

2. Member Tags Cloud Function provides the ability to associate “metadata” or “labels” with a Mailchimp subscriber. The memberTagsHandler function listens for Firestore write events based on specified config path, then automatically classifies the document data as Mailchimp subscriber tags. So all this does is create tags in MailChimp and associate them with users (members).

3. Member events Cloud Function creates Mailchimp specific activity events that can be created and associated with a predefined action. The memberEventsHandler function listens for Firestore write events based on specified config path, then automatically uses the document data to create a Mailchimp event on the subscriber’s profile which can subsequently trigger automation workflows.

Updating User Data in MailChimp

This is a mechanism to pull in additional information about the user from a document in Firebase and funnel it as user properties for the contact (member) in MailChimp.

The two settings to configure:

• Firebase Merge Fields Watch Path: The Firestore collection to watch for merge field changes

• Firebase Merge Fields Config: Provide a configuration mapping in JSON format indicating which Firestore properties to listen for and associate as Mailchimp merge fields.

Firebase Merge Fields Config

The JSON format is simple, It has two required fields that need to be set:

• subscriberEmail — The Firestore document field capturing the user email as is recognized by Mailchimp. This is how MailChimp will know which member to update with the new field data.

• mergeFields — JSON mapping representing the Firestore document fields to associate with Mailchimp Merge Fields. Basically, what properties to take from the Firestore document and update in the MailChimp member.

Here is an example of what this configuration would look like in JSON:

{
  "mergeFields": {
    "firstName": "FNAME",
    "lastName": "LNAME",
    "phoneNumber": "PHONE"
  },
  "subscriberEmail": "emailAddress"
}

*|MERGE|* tags are MailChimp’s way to uniquely identify a field for a member. The format of the properties inside the “mergeFields” JSON property is as follows:

• “Firestore property name”: “MERGE field ID”

You can view and modify the Merge tags in MailChimp.

MERGE tags (Fields) for an Audience in MailChimp

The above sample configuration assumes your Firestore collection contains documents having the following properties:

{
  "firstName": "..", // The config property FNAME maps to this document field
  "lastName": "..", // The config property LNAME maps to this document field
  "phoneNumber": "..", // The config property PHONE maps to this document field
  "emailAddress": "..", // The config property "subscriberEmail" maps to this document field
}

Any data associated with the mapped fields will be considered Merge Fields and the Mailchimp user’s profile will be updated accordingly.

NOTE: To disable this cloud function listener, provide an empty JSON config {}.

Example: Pass Nested Fields in your Firestore doc to MailChimp

You can point to nested properties in the Firebase document. Here is an example:

{
  "mergeFields": { 
    "userProfile.displayName": "FULLNAME", 
    "userProfile.lastActiveDate": "LASTACTIVE", 
    "userProfile.memberSince": "SINCE",
    "userMetrics.daysActive": "DAYSACTIVE",   
    "userSettings.monthlyAccessPurchased": "HASMSUB",
    "userDiagnostics.appVersion": "APPVERSION",
    "userDiagnostics.supportOnDeviceSpeechRecognition": "DEVICESR"
  },
  "subscriberEmail": "userProfile.email"
}

The configuration should look like below, where “users” is the Firestore collection name and the JSON below contains the subscriberEmail and the field mappings between the Firestore document and the MailChimp member (mergeFields).

Firebase MailChimp Extension config for Merge Fields

Troubleshooting

One of the limitations of this extension is that it only passes to MailChimp the fields that have changed within the Firestore doc. So if there are properties for an existing user (ex: Member Since = June 2020) and you enable the extension today, it won’t detect this as a changed property and will not update the MailChimp member with the value. It will always be empty in MailChimp as the extension compares changes in the Firestore doc itself rather than changes between the Firestore document and MailChimp.

See bolded code in the function definition below. Ideally, it would be nice to just pass all the “latest” values for the mapped fields in “mergeFields”.

exports.mergeFieldsHandler = functions.handler.firestore.document
  .onWrite(async (event, context) => {
    // If an empty JSON configuration was provided then consider function as NO-OP
    if (_.isEmpty(config.mailchimpMergeField)) return null;

try {
      // Get the configuration settings for mailchimp merge fields as is defined in extension.yml
      const mergeFieldsConfig = config.mailchimpMergeField;

// Validate proper configuration settings were provided
      if (!mailchimp) {
        logs.mailchimpNotInitialized();
        return;
      }
      if (!mergeFieldsConfig.mergeFields || _.isEmpty(mergeFieldsConfig.mergeFields)) {
        functions.logger.log(`A property named 'mergeFields' is required`);
        return null;
      }
      if (!_.isObject(mergeFieldsConfig.mergeFields)) {
        functions.logger.log('Merge Fields config must be an object');
        return null;
      }

// Get snapshot of document before & after write event
      const prevDoc = event && event.before && event.before.data();
      const newDoc = event && event.after && event.after.data();

// Determine all the merge prior to write event
      const mergeFieldsToUpdate = Object.entries(mergeFieldsConfig.mergeFields).reduce((acc, [docFieldName, mergeFieldName]) => {
        const prevMergeFieldValue = _.get(prevDoc, docFieldName);
        // lookup the same field value in new snapshot
        const newMergeFieldValue = _.get(newDoc, docFieldName, '');
        // if delta exists, then update accumulator collection
        if (prevMergeFieldValue !== newMergeFieldValue) {
          acc[mergeFieldName] = newMergeFieldValue;
        }
        return acc;
      }, {});

// Compute the mailchimp subscriber email hash
      const subscriberHash = subscriberHasher(_.get(newDoc, mergeFieldsConfig.subscriberEmail));

const params = {
        status_if_new: config.mailchimpContactStatus,
        email_address: _.get(newDoc, mergeFieldsConfig.subscriberEmail),
        merge_fields: mergeFieldsToUpdate
      };

// Invoke mailchimp API with updated tags
      if (!_.isEmpty(mergeFieldsToUpdate)) {
        await mailchimp.put(`/lists/${config.mailchimpAudienceId}/members/${subscriberHash}`, params);
      }
    } catch (e) {
      functions.logger.log(e);
    }
  });

I would have loved to modify this code as follows to be able to simply pass the latest values, but unfortunately, Firebase does not currently support extending or modifying extension code.

const mergeFieldsToUpdate = Object.entries(mergeFieldsConfig.mergeFields).reduce((acc, [docFieldName, mergeFieldName]) => {
  const newMergeFieldValue = _.get(newDoc, docFieldName, '');
  acc[mergeFieldName] = newMergeFieldValue;
  return acc;
}, {});

I ended up adding a request on the MailChimp extension GitHub repo. Hopefully this gets actioned, but if it doesn’t, there is another option to manually import all the data into MailChimp. This way, you’ll have all the existing fields in MailChimp and the extension will handle changes in the future.

At a high level, here are the steps to I followed to pull my data from Firestore to BigQuery to MailChimp:

• Install the Stream Collections to BigQuery extension to export your Firestore collection into BigQuery

• Go to BigQuery and write a SQL query to pull the data you need. For example:

SELECT   REPLACE(JSON_EXTRACT(data, '$.userProfile.email'), '"', '') Email,
         REPLACE(JSON_EXTRACT(data, '$.userProfile.displayName'), '"', '') FullName,
         REPLACE(JSON_EXTRACT(data, '$.userProfile.memberSince'), '"', '') AS memberSince
FROM `handstand-quest.firestore_handstand_quest.users_raw_latest`

• In BigQuery, run the query and save results to CSV

• Go to MailChimp and import the CSV into your Audience. It’s actually a very well built capability that lets you visually map fields between your CSV and your MailChimp member fields, then import.

Adding User Tags in MailChimp

Those tags can be pushed from Firebase for the contact and then be used as tags in MailChimp to easily find contacts with specific tags.

You might wonder, what’s the point of tags if you can filter by fields. Well, you cannot filter by field values (like Excel) in MailChimp, that’s why tags will come in pretty handy.

The two settings to configure:

• Firebase Member Tags Watch Path: The Firestore collection to watch for member tag changes

• Firebase Member Tags Config: Provide a configuration mapping in JSON format indicating which Firestore event(s) to listen for and associate as Mailchimp member tags.

Firebase Member Tags Config

The JSON format is simple, It has two required fields that need to be set:

• subscriberEmail — The Firestore document field capturing the user email as is recognized by Mailchimp. This is how MailChimp will know which member to update the tag for.

• memberTags — The Firestore document fields(s) to retrieve data from and classify as subscriber tags in Mailchimp.

Here is an example of what this configuration would look like in JSON:

{
  "memberTags": ["domainKnowledge", "jobTitle"],
  "subscriberEmail": "emailAddress"
}

Assuming the Firestore document has the following fields:

{
  "emailAddress": "..", // The config property 'subscriberEmail' maps to this document field
  "jobTitle": "..", // The config property 'memberTags' maps to this document field
  "domainKnowledge": "..", // The config property 'memberTags' maps to this document field
}

NOTE: To disable this cloud function listener, provide an empty JSON config {}.

Example: Filter by Member Location

Here is a screenshot of the config in the extension.

Firebase MailChimp Extension config for Member Tags

With that up and running, I can now filter my audience in MailChimp by the different values of location. For example, filter all members with the tag “Seattle”.

Members filtered by the “Seattle” location tag

Troubleshooting

Make sure the values in the properties that you map as memberTags are things you would want to group by, since the point of tags is to easily filter members who have a certain quality. For example, “jobTitle” makes sense as a tag since it will make it easy to filter members by Job Title. But something like “firstName” might not make sense.

Also, if you try to use a Bool value as a tag, based on my testing, it does not work. Ideally, the config below would result in a tag titled “isCoach” when the property is true, but neither “isCoach” nor “true”/”false” tags get created.

{
  "memberTags": ["userProfile.isCoach"],
  "subscriberEmail": "userProfile.emailAddress"
}

If resolved, this request on the MailChimp extension Github repository could help.

NOTE: Just ignore data types that are not String, and tags should work just fine.

Sending Events for Automation in MailChimp

This makes it easy to leverage MailChimp actions and automations based on events that take place in your Firebase app. For example, on a new user signup, you could send them a welcome email.

The two settings to configure:

• Firebase Member Events Watch Path: The Firestore collection to watch for member event changes

• Firebase Member Events Config: Provide a configuration mapping in JSON format indicating which Firestore event(s) to listen for and associate as Mailchimp events.

Before leveraging this capability, it’s important to have an understanding of how events work in MailChimp. I recommend reading MailChimp’s Use Events for Behavioral Targeting and Track Outside Activity with Events to gain a perspective on all the awesome things you can do.

Firebase Member Events Config

The JSON format is simple, It has two required fields that need to be set:

• subscriberEmail — The Firestore document field capturing the user email as is recognized by Mailchimp.

• memberEvents — The Firestore document fields(s) to retrieve data from and classify as member events in Mailchimp.

Here is an example of what this configuration would look like in JSON:

{
  "memberEvents": [
    "activity"
  ],
  "subscriberEmail": "emailAddress"
}

Assuming the Firestore document has the following fields:

{
  "emailAddress": "..", // The config property "subscriberEmail" maps to this document field
  "activity": ["send_welcome_email"] // The config property "memberEvents" maps to this document field
}

NOTE: To disable this cloud function listener, provide an empty JSON config {}.

Example: Trigger Email to MailChimp

I use the Firebase Email Trigger extension to send emails based on interesting and relevant activity that happens in my app.

Given it is almost free (only cost is Firebase function running), I see myself sticking to it for email sending based on events. Clearly, there is a lot more power in leveraging MailChimp, but I’d rather stay on the MailChimp free plan (minimum paid plan is $59 per month). I’ll focus on the free capabilities of MailChimp in this section, but you should know that you would be able to do more if you have a paid MailChimp plan.

In my app, here are some of the types of events I listen to and send an email to a Google group. It helps me see on a daily basis interesting/important activities that are happening in my app.

• NewUser

• Purchase

• WorkoutCompleted

• DailyGoalAchieved

• Feedback

• etc…

This is my way to get better connected with usage in my app at a much more intimate level than metrics and Google Analytics could ever go.

With minor changes to my current process using the “Trigger Email” extension, I can route an event to MailChimp and leverage the free capability of being able to see in one place all the events that have happened relating to a specific member.

I basically write a message to the messages Firestore collection, and then the “Trigger Email” extension does the magic of sending the email and ensuring it gets delivered. The message basically looks like this:

{
  to: 'someone@example.com',
  message: {
    type: 'NewUser',
    subject: 'Hello from Firebase!',
    html: 'This is an <code>HTML</code> email body.',
  }
}

Here is a sample summarized email that I get at the end of each day.

Abridged Summary from Google Group, setup in “My membership settings” Subscription dropdown

I use the Abridged setting in Google Groups to get a single email summary in a day on my personal email that is part of the Google group, and on another email address I get every email in case I want to look at them “live” in the moment one by one.

Are are the different settings you could choose from in a Google Group:

• Each email — Messages sent individually as they’re posted to the group.

• Abridged — Up to 25 complete messages combined into single emails and sent daily.

• Digest — Summaries of up to 150 messages combined into single emails and sent daily.

• No email — Messages from the group are not sent.

To make it easy to connect it with the MailChimp extension, I added a new user email property that allows me to link my current event with the MailChimp extension:

{
  to: 'someone@example.com',
  message: {
    type: 'NewUser',
    user: {
      email: 'user@example.com',
      id: 'Firebase User ID'
    },
    subject: 'Hello from Firebase!',
    html: 'This is an <code>HTML</code> email body.',
  }
}

To map my Firestore document to MailChimp, I simply set this config:

{
  "memberEvents": ["message.type"],
  "subscriberEmail": "message.user.email"
}

Then, I simply configured the extension for events as follows:

Firebase MailChimp Extension config for Member Events

Here is the goodness I got in MailChimp (activity feed in Member profile). Of course, I can build up on this capability to trigger automations, customer journeys, etc…

Member Profile in MailChimp with example events

In your case, you could link MailChimp to any collection where you’re tracking events. You could even have a custom collection that simply routes Google Analytics events with an email, then get those events linked to the member in MailChimp.

Conclusion

The MailChimp extension for Firebase is very powerful and simple to use, but on the surface, it’s hard to see all the goodness that could be leveraged. I hope this article with detailed step-by-step instructions will help you make the most out of it… Enjoy!