Refactor Redpanda Migrator components

[mihaitodor]

I hijacked this PR to address several issues:

Added

• New redpanda_migrator_offsets input.
• Fields offset_topic, offset_group, offset_partition, offset_commit_timestamp and offset_metadata added to the redpanda_migrator_offsets output.
• Fields kafka_key and max_in_flight for the redpanda_migrator_offsets output are now deprecated.
• Fields batching for the redpanda_migrator output is now deprecated.
• Field topic_lag_refresh_period added to the redpanda and redpanda_common inputs.
• Metric redpanda_lag now emitted by the redpanda and redpanda_common inputs.
• Metadata kafka_lag now emitted by the redpanda and redpanda_common inputs.

Fixed

• The redpanda_migrator_bundle output now skips schema ID translation when translate_schema_ids: false and schema_registry is configured.
• The redpanda_migrator output no longer rejects messages if it can't perform schema ID translation.
• The redpanda_migrator input no longer converts the kafka key to string.

Changed

• The kafka_key and max_in_flight fields of the redpanda_migrator_offsets output are now deprecated.
• Fields batch_size and multi_header for the redpanda_migrator input are now deprecated.

Redpanda Migrator offset metadata

One quick way to test this is via the following config. Note how I overwrite kafka_offset_metadata to foobar in a mapping processor.

input:
  redpanda_migrator_bundle:
    redpanda_migrator:
      seed_brokers: [ "localhost:9092" ]
      topics:
        - '^[^_]' # Skip internal topics which start with `_`
      regexp_topics: true
      consumer_group: migrator_bundle
      start_from_oldest: true
      replication_factor_override: true
      replication_factor: -1

    schema_registry:
      url: http://localhost:8081
      include_deleted: true
      subject_filter: ""

output:
  processors:
    - switch:
        - check: metadata("input_label") == "redpanda_migrator_offsets"
          processors:
            - mapping: |
                meta kafka_offset_metadata = "foobar"
  redpanda_migrator_bundle:
    redpanda_migrator:
      seed_brokers: [ "localhost:9093" ]
      max_in_flight: 1
      replication_factor_override: true
      replication_factor: -1

    schema_registry:
      url: http://localhost:8082

[mihaitodor]

When I added the redpanda_migrator input, I had both this gauge and the kafka_lag metadata field. I don't know if we want any of these available by default. Also, should this gauge name be somehow derived from the actual input type (redpanda, redpanda_common, redpanda_migrator, redpanda_migrator_offsets)?