Sendgrid Inbound Parse webhook and Java MimeMessage Compatibility

[SantoshDeepak1]

Issue Summary

I am trying to parse raw mime message which sengrid post to a URL by inbound parse web hook settings. Previously i was listening for incoming mails from Mailserver through Imap and from java MimeMessage i was able to convert it to the String and vice versa. Please see below code how i used to convert from MimeMessage to String and vice versa in java. I cannot convert the string raw mail message which sendgrid is posting through inbound parse webhook to javax.mail.internet.MimeMessage type. Is there anyway.

Code Snippet

private void convertMimeMessageToStringAndViceVersa(javax.mail.internet.MimeMessage message) {

        ByteArrayOutputStream bStream = new ByteArrayOutputStream();
        message.writeTo(bStream);
        String rawMimeMessageString = new String(bStream.toByteArray(), StandardCharsets.UTF_8.name());

        // Now from the above String to MimeMessage see below code
        
        Properties props = new Properties();
        Session session = Session.getDefaultInstance(props, null);
        ByteArrayInputStream bais = new ByteArrayInputStream(rawMimeMessageString.getBytes());
        javax.mail.internet.MimeMessage convertedMimeMessage = new MimeMessage(session, bais);
        
}

[shwetha-manvinkurke]

@SantoshDeepak1 Do you have an error log that you can share?

[SantoshDeepak1]

@shwetha-manvinkurke while constructing an object of MimeMessage it is not giving any error. But while i am trying to extract some information like from address/to address from javax.mail.internet.MimeMessage then its giving me null.

[shwetha-manvinkurke]

Have you debugged through to see the steps before you extract the info are doing what it's supposed to do? For example, does rawMimeMessageString has the right data?

[SantoshDeepak1]

rawMimeMessageString is sent by Sendgrid through inbound parse webhook. Sendgrid claims that the Raw Payload which we send to user is the original mime message. But in reality original mime message looks like attached file ('gmail_original_mime_message.txt'). Please compare two files attached. One is downloaded original mime message from gmail and another one is sendgrid mime message.

send_grid_payload_mime_message.txt
gmail_original_mime_message.txt

Note - If we change the extension of gmail_original_mime_message.txt to gmail_original_mime_message.eml and will try to open in microsoft outlook then you will able to see the original email. But incase of sendgrid it doesnot happen.

[SantoshDeepak1]

Hi @shwetha-manvinkurke
is there any update ?

[shwetha-manvinkurke]

Thanks for that info! Since in this case the data itself that's sent by sendgrid doesn't look right, I would recommend reaching out to support to see what they say.

[SantoshDeepak1]

@shwetha-manvinkurke before raising this ticket i had tried contacting support team. They said we have no idea about this please conatct with development team. So i asked in Github and stackoverflow. In stackoverflow no body answered the question yet :(.

https://stackoverflow.com/questions/67485686/sendgrid-inbound-parse-webhook-and-java-mimemessage-compatibility

[thinkingserious]

Hello @SantoshDeepak1,

Have you tried adjusting the "Send Raw" setting?

With best regards,

Elmer

[SantoshDeepak1]

Hi @thinkingserious I have tried both enabling/disabling POST the raw, full MIME message setting in Inbound parse webhook option. Still the issue is same.

[thinkingserious]

@SantoshDeepak1,

Thank you for the additional details!

I have revived this issue and that issue has been added to our internal backlog to be prioritized. Pull requests and +1s on the issue summary will help it move up the backlog.

Meanwhile, you may find this helper useful.

[SantoshDeepak1]

Thanks @thinkingserious . The raw text files that are in folder https://github.com/sendgrid/sendgrid-python/tree/main/sendgrid/helpers/inbound/sample_data.

I just looked at them. They are wrong Mime source files.
The Mime Source text files, if you save them with (.eml) extension they can easily opened in out look email client in windows.

For example take a look at mme_mail.txt.
Save it in your desktop. Change the extension of file from (.txt) to (.eml).
Open with microsoft outlook/or any email client. You can see the original email.

So if an user select the option (send raw mime message) though application UI, then sendgrid should sent that source file mme_mail.txt as an payload to enduser through inbound parse webhook.

I think i am clear now.

[thinkingserious]

Awesome! And thank you for following up!

[SantoshDeepak1]

Hi @thinkingserious. Is the issue will be solved ?

[thinkingserious]

Hello @SantoshDeepak1,

Please follow this issue to track the resolution of this issue. Currently, this helper library does not support inbound parsing.

[SantoshDeepak1]

Hi @thinkingserious ,

I think there is some confusion. The issue is not regarding to helper library. I have my own parser library. My concern is about inbound parsing webhook post url payload.

Compare these below raw payloads .

sendgrid_mime_raw_payload

actual_mime_raw_payload

My question is can i save the sendgrid_mime_raw_payload to .eml file with the help of your python library ?

[thinkingserious]

Hello @SantoshDeepak1,

The creation of the payload data is something our team can not help with. This repo pertains to the sendgrid-java helper library. However, if you provide us with your support ticket number, I can follow up internally to help guide our support team towards a possible solution.

With regards to saving the raw payload to a .eml file. That is not something the python library can do right now. Can you just change the file extension to .eml with your own parser library?