Please guide how to send email using gmail api via gsuit service account

[meepeek]

My goal is for my nodejs app to send mail to notify me if there were unhandled code exception on the server.

The code below is what I use for testing, which I modified gmail api quickstart code to use keyFile instead of oauth. However, I stuck with the error "The API returned an error: Error: Invalid Credentials".

I use this auth code with spreadsheet api before and it was success. I also did enable Domain-wide Delegation and add the app to Google Admin access control with Trusted permission level.

Now I'm stuck and cannot find any nodejs example for gmail. Please help.

import googleapis from 'googleapis'

const {google} = googleapis

const auth = new google.auth.GoogleAuth({
    keyFile: './credentials/gmail-key.json',
    scopes: ['https://www.googleapis.com/auth/gmail.send,email,profile']
})

const gmail = google.gmail({version: 'v1', auth})

// console.log(gmail.users.messages.send)

listLabels(auth)

function listLabels(auth) {
    const gmail = google.gmail({version: 'v1', auth});
    gmail.users.labels.list({
// I use service account email
      userId: 'SERVICE_ACCOUNT_NAME@APP_NAME.iam.gserviceaccount.com',
    }, (err, res) => {
      if (err) return console.log('The API returned an error: ' + err);
      const labels = res.data.labels;
      if (labels.length) {
        console.log('Labels:');
        labels.forEach((label) => {
          console.log(`- ${label.name}`);
        });
      } else {
        console.log('No labels found.');
      }
    });
  }

[meepeek]

PS. If I use the same scope as in quickstart, 'https://www.googleapis.com/auth/gmail.readonly', it will print "The API returned an error: Error: Precondition check failed."

[JustinBeckwith]

@sqrrrl we get fairly frequent requests to help figure out GSuite + Service Account issues. Do you know if there's a canonical guide somewhere out there on the topic?

[sqrrrl]

There's not, but something our tech writers are looking at improving.

Most Workspace APIs expect to be called as an end-user, not a service account. Support for services accounts is the exception, not the rule, and there's only a handful of APIs where it's appropriate (e.g. Drive, though still discouraged.) Gmail and Calendar specifically do not allow it.

They all support domain-wide delegation where a service account is used to impersonate an end-user. But that's not what this code example is doing. It's using the service account identity itself which isn't allowed here. To do delegation, the credentials need to be scoped to the target user by setting the sub claim in the token. Looks like for this client that means creating the JWT client directly (https://github.com/googleapis/google-auth-library-nodejs/blob/master/src/auth/jwtclient.ts) and setting the subject arg to the user's email address. Some of the other libraries have convenience methods for doing this (similar to createScoped(scopes) -- createDelegated(user)) and that could be useful to add here.

The other change would be using the user email (or the 'me' alias which just means whoever the effective user is) for the userId parameter in the API request.

To summarize:

• Workspace APIs should (almost) always be called as an end-user, not a service account
• Service accounts are useful for admin-managed apps that need to impersonate users in a domain without their explicit consent (effective user for the credential is still an end-user though, consistent with the first point.)
• Node.js client could make that a little easier with convenience methods to get a delegated credential

[Akxe]

I too find the documentation around sending emails with the service account.

import { google } from 'googleapis';

class MailProvider {
	private gmail = google.gmail({
		version: 'v1',
		auth: new google.auth.GoogleAuth({
			keyFile: '../../assets/secrets/google.json',
			scopes: [
				'https://www.googleapis.com/auth/gmail.send',
				'https://www.googleapis.com/auth/gmail.readonly',
			],
		}),
	});

	async sendMail(sendAs: string, sendTo: string) {
		return this.gmail.users.messages.send({
			userId: sendAs,
			media: {
				mimeType: 'placeholder-value',
				body: 'placeholder-value',
			},
		});
	}
}

new MailProvider().sendMail('[email protected]', '[email protected]'); // [email protected] is email from domain

It is very hard to find what should I put to the media.mimeType property. The example has placeholder-value, not really useful. Also, where do I put the receiver of the message? Stack overflow only mentions the media.raw property, but everyone is composing the raw manually, I would like not to do that if possible...

[Tallyrald]

I found a way to make this work in a relatively painless way so I'll share in case someone needs it later. As you know, the docs are kind of useless for us nodejs folks so I'll put some explaining here too.

First of all, create the service account via Cloud Console or other methods & give it domain-wide delegation. No need to give any other roles or permissions as the GMail API is not exactly a GCloud app. Then head over to the Google Workspaces Admin website (https://admin.google.com) and under security -> API setting (or something similarly named) put the delegation ClientId into the allowed apps list. This is where you set the appropriate scopes too.

Now consider the following (tl;dr at bottom):

// Warning lots of comments ahead
import { gmail_v1, google } from 'googleapis';
import { encode } from 'js-base64'; // btoa can probably be used too

const { google } = googleapis;

/** Sends a test email (to & from) a test email account */
public static async SendTestEmail() {
    // Create an auth solution. This could be any authentication method that the googleapis package allows
    const authClient = new google.auth.JWT({
        keyFile: 'path/to/keyFile.json', // Service account key file
        scopes: ['https://mail.google.com/'], // Very important to use appropriate scopes. This one gives full access (only if you gave this to the service account too)
        subject: '[email protected]', // This is an email address that exists in your Google Workspaces account. The app will use this as 'me' during later execution.
    });

    // I'm not sure if this is necessary, but it visualizes that you are now logged in.
    await authClient.authorize();

    // Let's create a Gmail client via the googleapis package
    const gmail = new gmail_v1.Gmail({ auth: authClient });

    // The names are a bit confusing, but the important part is the users.messages.send is responsible for executing the send operation.
    // This means a single email, not batch sending
    const result = await gmail.users.messages.send({
        auth: authClient, // Pass the auth object created above
        requestBody: {
            // I'm fairly certain that the raw property is the most sure-fire way of telling the API what you want to send
            // This is actually a whole email, not just the body, see below
            raw: this.MakeEmail('[email protected]', '[email protected]', 'Subject string', 'Email body string'),
        },
        userId: 'me', // Using me will set the authenticated user (in the auth object) as the requester
    });

    return result;
}

/** Creates a very basic email structure
 * @param to Email address to send to
 * @param from Email address to put as sender
 * @param subject The subject of the email [warn: encoding, see comment]
 * @param message The body of the email
 */
private static MakeEmail(to: string, from: string, subject: string, message: string) {
    // OK, so here's the magic
    // The array is used only to make this easier to understand, everything is concatenated at the end
    // Set the headers first, then the recipient(s), sender & subject, and finally the message itself
    const str = [
        'Content-Type: text/plain; charset="UTF-8"\n', // Setting the content type as UTF-8 makes sure that the body is interpreted as such by Google
        'to: ', to,'\n',
        'from: ', from,'\n',
        // Here's the trick: by telling the interpreter that the string is base64 encoded UTF-8 string, you can send non-7bit-ASCII characters in the subject
        // I'm not sure why this is so not intuitive (probably historical/compatibility reasons),
        // but you need to make sure the encoding of the file, the server environment & everything else matches what you specify here
        'subject: =?utf-8?B?', encode(subject, true),'?=\n\n', // Encoding is base64 with URL safe settings - just in case you want a URL in the subject (pls no, doesn't make sense)
        message, // The message body can be whatever you want. Parse templates, write simple text, do HTML magic or whatever you like - just use the correct content type header
    ].join('');

    const encodedMail = encode(str, true); // Base64 encode using URL safe settings

    return encodedMail;
}

tl;dr:

• Authenticate an account email address to impersonate
• Compose email body however you want (and make sure the content type header reflects what you've put there)
• Concatenate email headers, recipient(s), sender, subject and body into a single string
• Base64 encode the whole thing
• Send it to the API

I haven't used attachments yet. If or when I do, I'll probably update this.

[radziszXTRF]

@Tallyrald - great hint thanks In admin panel (https://admin.google.com) one should go to "API controls" and then select "Domain wide delegation" (below) not "App access control" above, both support provisioning of clientId...

[andris9]

As people often end up in this issue regarding Gmail Service Accounts, then you can find a tutorial for generating the Service Account key file here (the tutorial is not for Nodemailer but EmailEngine, another project I maintain, but the end result is the same, you need that JSON formatted key file to extract client ID and service key values).

[gg-martins091]

The php sdk has a way to set the subject of a client used by a service pretty easliy.

It would be great to have something like this:

const jwtClient = new google.auth.JWT(...)
const service = google.calendar({auth: jwtClient, ...})

service.setSubject('[email protected]')
//or
service.getAuth().getClient().setSubject('[email protected]')

In my use case I'm always using a service account on the Workspace APIs and almost always impersonating a user through domain wide delegation, so it would be really helpful to have something like that.

[kmasterycsl]

Thanks @Tallyrald. It's working!

I also made an improved version with mail composer instead of build raw email

// Warning lots of comments ahead
import { gmail_v1, google } from "googleapis";
import MailComposer from "nodemailer/lib/mail-composer";

class Test {
  /** Sends a test email (to & from) a test email account */
  public static async SendTestEmail() {
    // Create an auth solution. This could be any authentication method that the googleapis package allows
    const authClient = new google.auth.JWT({
      keyFile: "your-service-account.json", // Service account key file
      scopes: ["https://mail.google.com/"], // Very important to use appropriate scopes. This one gives full access (only if you gave this to the service account too)
      subject: "[email protected]", // This is an email address that exists in your Google Workspaces account. The app will use this as 'me' during later execution.
    });

    // I'm not sure if this is necessary, but it visualizes that you are now logged in.
    await authClient.authorize();

    // Let's create a Gmail client via the googleapis package
    const gmail = new gmail_v1.Gmail({ auth: authClient });

    // The names are a bit confusing, but the important part is the users.messages.send is responsible for executing the send operation.
    // This means a single email, not batch sending

    const mailComposer = new MailComposer({
      from: "[email protected]",
      to: "[email protected]",
      subject: "Some subject here",
      text: "text content",
      html: "html content",
    });

    const rawMail = (await mailComposer.compile().build()).toString("base64");

    const result = await gmail.users.messages.send({
      auth: authClient, // Pass the auth object created above
      requestBody: {
        // I'm fairly certain that the raw property is the most sure-fire way of telling the API what you want to send
        // This is actually a whole email, not just the body, see below
        raw: rawMail,
      },
      userId: "me", // Using me will set the authenticated user (in the auth object) as the requester
    });

    return result;
  }
}

Test.SendTestEmail();