Shared/common dynamic template data + personalizations

[adamreisnz]

Issue Summary

As a developer, I want to be efficient with bandwidth and data that's being transmitted.

So when I need to send an email to 1,000 different people, and I use personalisations, it's great that I can send this in a single API request to Sendgrid.

However, there is still a lot of "waste" happening if I'm including dynamic template data which is the same for all the recipients

I am forced to include this in the dynamicTemplateData property of each individual personalisation.

It would be trivial to allow us to specify shared or common dynamicTemplateData in the root of the request data (as you would for a regular email without personalisations, along with possible specific dynamicTemplateData for each personalisation, and for Sendgrid to merge this on the server via a simple one level deep object merge, prior to sending emails.

This way, we won't have to repeat shared data 1000 times, and we'll be working together towards a faster and more efficient internet.

Example

The following example illustrates the problem. Now I know this is a trivial example, but imagine that the payload is significantly larger. For example, this email may be going out to 1000 attendees that signed up to an event, so we're passing in dynamic template data specific to each attendee (e.g. their name), but also shared dynamic template data pertaining to the information of the event they all signed up to, which could be a significant payload size.

Current API:

const data = {
  personalisations: [
    {
      to: '[email protected]',
      dynamicTemplateData: {
        personal: 'A personal message for Person 1',
        shared: 'Common data which is the same for all recipients', //Repeats
      }
    },
    {
      to: '[email protected]',
      dynamicTemplateData: {
        personal: 'A hidden message for Person 2',
        shared: 'Common data which is the same for all recipients', //Repeats
      }
    },
    {
      to: '[email protected]',
      dynamicTemplateData: {
        personal: 'A secret message for Person 3',
        shared: 'Common data which is the same for all recipients', //Repeats
      }
    },
    //...x1000
  ]
}

Proposed API:

const data = {
  //A top-level dynamic data placeholder for shared/common dynamic data
  dynamicTemplateData: {
    shared: 'Common data which is the same for all recipients', //Only specified once!
  },
  personalisations: [
    {
      to: '[email protected]',
      //Nested dynamic template data as is, for each individual personalisation
      dynamicTemplateData: {
        personal: 'A personal message for Person 1',
      }
    },
    {
      to: '[email protected]',
      dynamicTemplateData: {
        personal: 'A hidden message for Person 2',
      }
    },
    {
      to: '[email protected]',
      dynamicTemplateData: {
        personal: 'A secret message for Person 3',
      }
    },
    //...x1000
  ]
}

Later on the Sendgrid server (example code in JS):

//Merge common dynamic template data with personalisation specific ones
for (const item of data.personalisations) {
  item.dynamicTemplateData = Object.assign({}, data.dynamicTemplateData, item.dynamicTemplateData || {})
}

The proposed API could potentially save a lot of bandwidth and would be far more efficient, especially with large dynamic data payloads.

I am aware that this is not a NodeJS library specific issue, but as I work with this library exclusively I wanted to post it here and hope that this issue can be raised internally with the correct API team.

I also raised a similar issue last year (#1154) which doesn't seem to have progressed much. Although note that this issue is slightly different, and I think far easier to implement at Sendgrid's end.

[sandeep1995]

I do absolutely support this. I thought the API would behave like the proposed one but it doesn't.
It's a waste of bandwidth.

[JenniferMah]

Hi @adamreisnz! Can you send an email to [email protected] with this request? They will be able to generate a ticket for the correct internal Twilio Sendgrid team.

[adamreisnz]

I will try, but I have reported quite a few bugs (in the Sendgrid user interface) to Sendgrid support already, and that has all landed on deaf ears, with the same bugs still present years later.

[zeluspudding]

Interesting, my impression was that this was how the v3 api already worked :/ Source:

When sending an email with the v3 Mail Send endpoint, you define the various metadata about your message, such as the recipients, sender, subject, and send time, at the root level of a JSON request body. Personalizations allow you to override these various metadata for each email in an API request.

But I guess shared custom_args actually cannot be set at the root level... since I guess it is not considered metadata in the sentence above? Asking because I haven't yet tried it myself (and have been banking on this functionality).

**EDIT
I now see that custom_args does not appear to be used in handlebars templating, from the docs:

although it is not clear whether custom_args is literally supposed to be attached in the payload as key custom_args or if custom arguments can be any unreserved key... so I second @adamreisnz suggestion and an update to the docs to clarify!

[AntMaster7]

Interesting, my impression was that this was how the v3 api already worked :/ Source:

When sending an email with the v3 Mail Send endpoint, you define the various metadata about your message, such as the recipients, sender, subject, and send time, at the root level of a JSON request body. ...

If you set multiple recipients on the root node, they will all be listed in the e-mail. That's something you usually don't want in business scenarios. Otherwise you'd basically expose your customers.