Mysql2::Error: Data too long for column 'user_agent'

[andrew-cybsafe]

Describe the bug

Occasional database error from HTTP requests that have a long user-agent header.

To Reproduce

curl 'https://domain/{server_id}/{email_tracking_id}' -A "Mozilla/5.0 (Linux; Android 13; Pixel 4a (5G) Build/TQ2A.230505.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/112.0.5615.136 Mobile Safari/537.36 GoogleApp/14.16.27.29.arm64 AppEngine-Google; (+http://code.google.com/appengine; appid: s~...................)"

an observe that a 500 is returned.

Expected behaviour

No errors to be raised, and long user agents either to be stored in the db correctly or truncated if they are excessively long.

Additional information/context

Stack trace:

Mysql2::Error: Data too long for column 'user_agent' at row 1 (Mysql2::Error)
  from mysql2 (0.5.6) lib/mysql2/client.rb:151:in `_query'
  from mysql2 (0.5.6) lib/mysql2/client.rb:151:in `block in query'
  from mysql2 (0.5.6) lib/mysql2/client.rb:150:in `handle_interrupt'
  from mysql2 (0.5.6) lib/mysql2/client.rb:150:in `query'
  from lib/postal/message_db/database.rb:326:in `query_on_connection'
  from lib/postal/message_db/database.rb:234:in `block in insert'
  from lib/postal/message_db/connection_pool.rb:20:in `use'
  from lib/postal/message_db/database.rb:345:in `with_mysql'
  from lib/postal/message_db/database.rb:233:in `insert'
  from lib/tracking_middleware.rb:89:in `dispatch_redirect_request'
  from lib/tracking_middleware.rb:26:in `call'
  from secure_headers (6.5.0) lib/secure_headers/middleware.rb:11:in `call'
  from railties (7.0.8.1) lib/rails/engine.rb:530:in `call'
  from puma (6.4.2) lib/puma/commonlogger.rb:47:in `call'
  from puma (6.4.2) lib/puma/configuration.rb:272:in `call'
  from puma (6.4.2) lib/puma/request.rb:100:in `block in handle_request'
  from puma (6.4.2) lib/puma/thread_pool.rb:378:in `with_force_shutdown'
  from puma (6.4.2) lib/puma/request.rb:99:in `handle_request'
  from puma (6.4.2) lib/puma/server.rb:464:in `process_client'
  from puma (6.4.2) lib/puma/server.rb:245:in `block in run'
  from puma (6.4.2) lib/puma/thread_pool.rb:155:in `block in spawn_thread'

[willpower232]

Definitely a rarity but worth fixing one way or the other

[andrew-cybsafe]

@willpower232 I don't mind attempting a PR for this. Would you support changing the field type to mediumtext or similar with a limit of 32k and truncating the value just in case?

[willpower232]

It isn't really my decision to make but I would probably say leave the database alone and limit it to 255 characters because otherwise it feels like a waste.