Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to verify signature for testing purposes #453

Open
gotchahn opened this issue Jan 6, 2021 · 7 comments
Open

Unable to verify signature for testing purposes #453

gotchahn opened this issue Jan 6, 2021 · 7 comments
Labels
status: waiting for feedback waiting for feedback from the submitter type: question question directed at the library

Comments

@gotchahn
Copy link

gotchahn commented Jan 6, 2021

Issue Summary

I'm trying to do some tests in rails to check if my sendgrid webhook endpoint is working. I'm in using the same values for PUBLIC KEY and SIGNATURE key shared on the fixtures page:

PUBLIC_KEY = 'MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE83T4O/n84iotIvIW4mdBgQ/7dAfSmpqIM8kF9mN1flpVKS3GRqe62gw+2fNNRaINXvVpiglSI8eNEc6wEA3F+g=='.freeze
FAILING_PUBLIC_KEY = 'MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEqTxd43gyp8IOEto2LdIfjRQrIbsd4SXZkLW6jDutdhXSJCWHw8REntlo7aNDthvj+y7GjUuFDb/R1NGe1OPzpA=='.freeze
SIGNATURE = 'MEUCIGHQVtGj+Y3LkG9fLcxf3qfI10QysgDWmMOVmxG0u6ZUAiEAyBiXDWzM+uOe5W0JuG+luQAbPIqHh89M15TluLtEZtM='.freeze
FAILING_SIGNATURE = 'MEUCIQCtIHJeH93Y+qpYeWrySphQgpNGNr/U+UyUlBkU6n7RAwIgJTz2C+8a8xonZGi6BpSzoQsbVRamr2nlxFDWYNH3j/0='.freeze
TIMESTAMP = '1600112502'.freeze
PAYLOAD = "#{[
{
email: '[email protected]',
event: 'dropped',
reason: 'Bounced Address',
sg_event_id: 'ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA',
sg_message_id: 'LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0',
'smtp-id': '<[email protected]>',
timestamp: 1_600_112_492
}
].to_json}\r\n".freeze # Be sure to include the trailing carriage return and newline!

But overall the verify method returns false to me.

Code Snippet

If I do this manually in the console, using the same values, the verify returns false:

irb(main):003:0> 
irb(main):004:0> ew = SendGrid::EventWebhook.new
=> #<SendGrid::EventWebhook:0x00007faa52aab758>
irb(main):005:0> 
irb(main):006:0> 
irb(main):007:0> public_key = 'MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE83T4O/n84iotIvIW4mdBgQ/7dAfSmpqIM8kF9mN1flpVKS3GRqe62gw+2fNNRaINXvVpiglSI8eNEc6wEA3F+g=='.freeze
=> "MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE83T4O/n84iotIvIW4mdBgQ/7dAfSmpqIM8kF9mN1flpVKS3GRqe62gw+2fNNRaINXvVpiglSI8eNEc6wEA3F+g=="
irb(main):008:0> 
irb(main):009:0> signature = 'MEUCIGHQVtGj+Y3LkG9fLcxf3qfI10QysgDWmMOVmxG0u6ZUAiEAyBiXDWzM+uOe5W0JuG+luQAbPIqHh89M15TluLtEZtM='.freeze
=> "MEUCIGHQVtGj+Y3LkG9fLcxf3qfI10QysgDWmMOVmxG0u6ZUAiEAyBiXDWzM+uOe5W0JuG+luQAbPIqHh89M15TluLtEZtM="
irb(main):010:0> 
irb(main):011:0> timestamp = '1600112502'.freeze
=> "1600112502"
irb(main):012:0> 
irb(main):013:0> payload = "#{[
irb(main):014:1>       {
irb(main):015:2>         email: '[email protected]',
irb(main):016:2>         event: 'dropped',
irb(main):017:2>         reason: 'Bounced Address',
irb(main):018:2>         sg_event_id: 'ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA',
irb(main):019:2>         sg_message_id: 'LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0',
irb(main):020:2>         'smtp-id': '<[email protected]>',
irb(main):021:2>         timestamp: 1_600_112_492
irb(main):022:2>       }
irb(main):023:1>     ].to_json}\r\n".freeze
=> "[{\"email\":\"[email protected]\",\"event\":\"dropped\",\"reason\":\"Bounced Address\",\"sg_event_id\":\"ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA\",\"sg_message_id\":\"LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0\",\"smtp-id\":\"\\[email protected]\\u003e\",\"timestamp\":1600112492}]\r\n"
irb(main):024:0> 
irb(main):025:0> ec_public_key = ew.convert_public_key_to_ecdsa(public_key)
=> #<OpenSSL::PKey::EC:0x00007faa52ab8160>
irb(main):026:0> 
irb(main):027:0> ew.verify_signature(ec_public_key, payload, signature, timestamp)
=> false
irb(main):028:0> 

Also this is my rspec that I was coding:

require "rails_helper"

ENV["SENDGRID_EVENT_HOOK_KEY"] = "MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEc1xJU2qhLFMxcOLdKWQIA2OZdmUTlNRT5xFEipLDnGkO0uhW8aJiIQxJGglBRiKWxNqm3jjRbUpaAaDH9WHkng==".freeze

describe Webhooks::SendgridController, type: "request" do
  describe "POST 'create'" do

    let(:example_sendrid_payload) do
      "#{[
        {
          email: '[email protected]',
          event: 'dropped',
          reason: 'Bounced Address',
          sg_event_id: 'ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA',
          sg_message_id: 'LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0',
          'smtp-id': '<[email protected]>',
          timestamp: 1_600_112_492
        }
      ].to_json}\r\n".freeze
    end

    let(:sendgrid_signed_header) do
      {
          "#{SendGrid::EventWebhookHeader::SIGNATURE}" => "MEYCIQCxfnpzX3CftRSaaA4wHWHOKEyHbCf5jbwL/z/QQpncqQIhANu8Ug7FTuQhgGzhSVQvDSIS64rp+fXz2AloRnV5qcTL".freeze,
          "#{SendGrid::EventWebhookHeader::TIMESTAMP}" => "1600112502".freeze
      }
    end

    it "responds with success when provided signature matches" do
      post webhooks_sendgrid_path, params: example_sendrid_payload, headers: sendgrid_signed_header

      expect(response).to have_http_status(200)
    end
end

Exception/Log

 1) Webhooks::SendgridController POST 'create' responds with success when provided signature matches
     Failure/Error: expect(response).to have_http_status(200)
       expected the response to have status code 200 but it was 401

Technical details:

  • sendgrid-ruby version: 6.3.8
  • ruby version: 2.6.5
@thinkingserious
Copy link
Contributor

Hello @gotchahn,

Thank you for reporting this issue to us. I have been unable to reproduce. Could you please provide some more detail about your setup? Here is the code I used to test:

require 'sendgrid-ruby'
include SendGrid

ew = SendGrid::EventWebhook.new
public_key = 'MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE83T4O/n84iotIvIW4mdBgQ/7dAfSmpqIM8kF9mN1flpVKS3GRqe62gw+2fNNRaINXvVpiglSI8eNEc6wEA3F+g=='.freeze
signature = 'MEUCIGHQVtGj+Y3LkG9fLcxf3qfI10QysgDWmMOVmxG0u6ZUAiEAyBiXDWzM+uOe5W0JuG+luQAbPIqHh89M15TluLtEZtM='.freeze
timestamp = '1600112502'.freeze
payload = "#{[
      {
        email: '[email protected]',
        event: 'dropped',
        reason: 'Bounced Address',
        sg_event_id: 'ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA',
        sg_message_id: 'LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0',
        'smtp-id': '<[email protected]>',
        timestamp: 1_600_112_492
      }
    ].to_json}\r\n".freeze
ec_public_key = ew.convert_public_key_to_ecdsa(public_key)
value = ew.verify_signature(ec_public_key, payload, signature, timestamp)
print value

This returned true.

With best regards,

Elmer

@thinkingserious thinkingserious added status: waiting for feedback waiting for feedback from the submitter type: question question directed at the library labels Jan 7, 2021
@gotchahn
Copy link
Author

gotchahn commented Jan 7, 2021

Hi @thinkingserious, I copy pasted your code in my rails console, and gave me false hmm, weird. I'm using Rails 6.0.3.2, Ruby 2.6.5 and Sendgrid-ruby 6.3.8

$ rails c
Running via Spring preloader in process 58835
Loading development environment (Rails 6.0.3.2)
irb(main):001:0> ew = SendGrid::EventWebhook.new
=> #<SendGrid::EventWebhook:0x00007faa4fe0c468>
irb(main):002:0> public_key = 'MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE83T4O/n84iotIvIW4mdBgQ/7dAfSmpqIM8kF9mN1flpVKS3GRqe62gw+2fNNRaINXvVpiglSI8eNEc6wEA3F+g=='.freeze
=> "MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE83T4O/n84iotIvIW4mdBgQ/7dAfSmpqIM8kF9mN1flpVKS3GRqe62gw+2fNNRaINXvVpiglSI8eNEc6wEA3F+g=="
irb(main):003:0> signature = 'MEUCIGHQVtGj+Y3LkG9fLcxf3qfI10QysgDWmMOVmxG0u6ZUAiEAyBiXDWzM+uOe5W0JuG+luQAbPIqHh89M15TluLtEZtM='.freeze
=> "MEUCIGHQVtGj+Y3LkG9fLcxf3qfI10QysgDWmMOVmxG0u6ZUAiEAyBiXDWzM+uOe5W0JuG+luQAbPIqHh89M15TluLtEZtM="
irb(main):004:0> timestamp = '1600112502'.freeze
=> "1600112502"
irb(main):005:0> payload = "#{[
irb(main):006:1>       {
irb(main):007:2>         email: '[email protected]',
irb(main):008:2>         event: 'dropped',
irb(main):009:2>         reason: 'Bounced Address',
irb(main):010:2>         sg_event_id: 'ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA',
irb(main):011:2>         sg_message_id: 'LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0',
irb(main):012:2>         'smtp-id': '<[email protected]>',
irb(main):013:2>         timestamp: 1_600_112_492
irb(main):014:2>       }
irb(main):015:1>     ].to_json}\r\n".freeze
=> "[{\"email\":\"[email protected]\",\"event\":\"dropped\",\"reason\":\"Bounced Address\",\"sg_event_id\":\"ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA\",\"sg_message_id\":\"LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0\",\"smtp-id\":\"\\[email protected]\\u003e\",\"timestamp\":1600112492}]\r\n"
irb(main):016:0> ec_public_key = ew.convert_public_key_to_ecdsa(public_key)
=> #<OpenSSL::PKey::EC:0x00007faa4ffe7f30>
irb(main):017:0> value = ew.verify_signature(ec_public_key, payload, signature, timestamp)
=> false
irb(main):018:0> print value
false=> nil
irb(main):019:0> 

@gotchahn
Copy link
Author

gotchahn commented Jan 7, 2021

BUT, noticed that if I do that code in a plain irb console, it works! but not on rails console or in my rails testing suite. Do you have any idea why it works only in a irb console??

$ irb
irb(main):001:0> require 'sendgrid-ruby'
=> true
irb(main):002:0> include SendGrid
=> Object
irb(main):003:0> 
irb(main):004:0> ew = SendGrid::EventWebhook.new
=> #<SendGrid::EventWebhook:0x00007fb08aa6cdd0>
irb(main):005:0> public_key = 'MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE83T4O/n84iotIvIW4mdBgQ/7dAfSmpqIM8kF9mN1flpVKS3GRqe62gw+2fNNRaINXvVpiglSI8eNEc6wEA3F+g=='.freeze
=> "MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE83T4O/n84iotIvIW4mdBgQ/7dAfSmpqIM8kF9mN1flpVKS3GRqe62gw+2fNNRaINXvVpiglSI8eNEc6wEA3F+g=="
irb(main):006:0> signature = 'MEUCIGHQVtGj+Y3LkG9fLcxf3qfI10QysgDWmMOVmxG0u6ZUAiEAyBiXDWzM+uOe5W0JuG+luQAbPIqHh89M15TluLtEZtM='.freeze
=> "MEUCIGHQVtGj+Y3LkG9fLcxf3qfI10QysgDWmMOVmxG0u6ZUAiEAyBiXDWzM+uOe5W0JuG+luQAbPIqHh89M15TluLtEZtM="
irb(main):007:0> timestamp = '1600112502'.freeze
=> "1600112502"
irb(main):008:0> payload = "#{[
irb(main):009:1>       {
irb(main):010:2>         email: '[email protected]',
irb(main):011:2>         event: 'dropped',
irb(main):012:2>         reason: 'Bounced Address',
irb(main):013:2>         sg_event_id: 'ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA',
irb(main):014:2>         sg_message_id: 'LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0',
irb(main):015:2>         'smtp-id': '<[email protected]>',
irb(main):016:2>         timestamp: 1_600_112_492
irb(main):017:2>       }
irb(main):018:1>     ].to_json}\r\n".freeze
=> "[{\"email\":\"[email protected]\",\"event\":\"dropped\",\"reason\":\"Bounced Address\",\"sg_event_id\":\"ZHJvcC0xMDk5NDkxOS1MUnpYbF9OSFN0T0doUTRrb2ZTbV9BLTA\",\"sg_message_id\":\"LRzXl_NHStOGhQ4kofSm_A.filterdrecv-p3mdw1-756b745b58-kmzbl-18-5F5FC76C-9.0\",\"smtp-id\":\"<[email protected]>\",\"timestamp\":1600112492}]\r\n"
irb(main):019:0> ec_public_key = ew.convert_public_key_to_ecdsa(public_key)
=> #<OpenSSL::PKey::EC:0x00007fb08aadf4e8>
irb(main):020:0> value = ew.verify_signature(ec_public_key, payload, signature, timestamp)
=> true
irb(main):021:0> print value
true=> nil

@thinkingserious
Copy link
Contributor

Hello @gotchahn,

Here are few things to check:

  1. What is the return of RUBY_VERSION while in the irb console vs rails console.
  2. Perhaps the rails console is using a different version of irb under the hood.
  3. What happens when you run outside of the console?

Thanks!

With best regards,

Elmer

@gotchahn
Copy link
Author

gotchahn commented Jan 8, 2021

Hi @thinkingserious,

  1. Both gave me 2.6.5 as the RUBY_VERSION.
  2. The ONLY difference I see is that the rails console is using a Spring preloader Running via Spring preloader in process.... dunno if that has something to do.
  3. Well, I tested it in my test specs, and the result is the same as the rails console.

I tested in production, and the endpoint works well with the validation of the REAL key and signature. Is just the made of ones for the tests that gives me false the verifier.

UPDATE:

I tested the rails console without the Spring preloader, and it returns false still.

@gotchahn
Copy link
Author

gotchahn commented Jan 12, 2021

Hi @thinkingserious,
I figured it out, it's the payload the cause of the issue, and mainly, the .to_json method with the 'smtp-id' section, look:

in irb console:

$ irb
irb(main):001:0> require 'sendgrid-ruby'
=> true
irb(main):002:0> include SendGrid
=> Object
irb(main):017:0> "'smtp-id': '<[email protected]>'".to_json
=> "\"'smtp-id': '<[email protected]>'\""

in rails console:

irb(main):001:0> "'smtp-id': '<[email protected]>'".to_json
=> "\"'smtp-id': '\\[email protected]\\u003e'\""
irb(main):002:0> 

The < and > are converted, so that's why the verifier doesn't returns true. Using JSON.generate worked:

irb(main):004:0> JSON.generate("'smtp-id': '<[email protected]>'")
=> "\"'smtp-id': '<[email protected]>'\""

But I have another question, If I slightly change the payload, the verifier returns false, any idea of how I can use a custom payload with a different type of event for example, and get the right signature for the public key??

@TastyPi
Copy link

TastyPi commented Jul 23, 2024

I realise this is an old issue, but I wanted to post a solution to the issue of how to generate signatures for custom payloads:

def sendgrid_key
  @sendgrid_key ||= OpenSSL::PKey::EC.generate("prime256v1")
end

def sendgrid_public_key
  Base64.strict_encode64(sendgrid_key.public_to_pem)
end

def sendgrid_signature(payload, timestamp)
  Base64.strict_encode64(sendgrid_key.dsa_sign_asn1(Digest::SHA256.digest("#{timestamp}#{payload}")))
end

def sendgrid_headers(body)
  timestamp = Time.now.to_i
  {
    "Content-Type" => "application/json;charset=utf-8",
    ::SendGrid::EventWebhookHeader::SIGNATURE => sendgrid_signature(body, timestamp),
    ::SendGrid::EventWebhookHeader::TIMESTAMP => timestamp
  }
end

Note that you have to generate a new key for your tests since SendGrid (correctly) does not provide the private key for generating signatures. We use an environment variable in our controller to get the public key, a gem that modifies ENV such as climate_control can be used to override it in tests, e.g.

ClimateControl.modify SENDGRID_PUBLIC_KEY: sendgrid_public_key do
  post "/sendgrid", params: payload, headers: sendgrid_headers(payload)
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: waiting for feedback waiting for feedback from the submitter type: question question directed at the library
Projects
None yet
Development

No branches or pull requests

3 participants