From 3cef0e36cdb87f8122349375ea4863d364266830 Mon Sep 17 00:00:00 2001 From: donoghuc Date: Tue, 12 Nov 2024 11:53:06 -0800 Subject: [PATCH 1/2] Mark deprecated SSL settings as obsolete This commit updates SSL settings to be marked as obsolete: - Replace `ssl` with `ssl_enabled` - Replace `ca_file` with `ssl_certificate_authorities` - Replace `ssl_certificate_verification` with `ssl_verification_mode` `setup_ssl_params!` has been updated to only handle SSL inference when not explicitly configured. --- CHANGELOG.md | 9 +++- docs/index.asciidoc | 63 ++++++++-------------------- lib/logstash/inputs/elasticsearch.rb | 54 ++++-------------------- logstash-input-elasticsearch.gemspec | 2 +- spec/inputs/elasticsearch_spec.rb | 13 ++++++ 5 files changed, 48 insertions(+), 93 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c3d053f..f9a6514 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,10 @@ -## 4.20.5 - - Add `x-elastic-product-origin` header to Elasticsearch requests [#211](https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/211) +## 5.0.0 ## + - SSL settings that were marked deprecated in version `4.17.0` are now marked obsolete, and will prevent the plugin from starting. + - These settings are: + - `ssl`, which should bre replaced by `ssl_enabled` + - `ca_file`, which should bre replaced by `ssl_certificate_authorities` + - `ssl_certificate_verification`, which should bre replaced by `ssl_verification_mode` + - [#213](https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/213) ## 4.20.4 - Fix issue where the `index` parameter was being ignored when using `response_type => aggregations` [#209](https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/209) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index a317c52..04a50ea 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -96,7 +96,13 @@ TIP: Set the `target` option to avoid potential schema conflicts. [id="plugins-{type}s-{plugin}-options"] ==== Elasticsearch Input configuration options -This plugin supports the following configuration options plus the <> and the <> described later. +This plugin supports these configuration options plus the <> described later. + +NOTE: As of version `5.0.0` of this plugin, a number of previously deprecated settings related to SSL have been removed. +Please check out <> for details. + +NOTE: As of version `5.0.0` of this plugin, a number of previously deprecated settings related to SSL have been removed. +Please check out <> for details. [cols="<,<,<",options="header",] |======================================================================= @@ -478,6 +484,8 @@ Enable SSL/TLS secured communication to Elasticsearch cluster. Leaving this unspecified will use whatever scheme is specified in the URLs listed in <> or extracted from the <>. If no explicit protocol is specified plain HTTP will be used. +When not explicitly set, SSL will be automatically enabled if any of the specified hosts use HTTPS. + [id="plugins-{type}s-{plugin}-ssl_key"] ===== `ssl_key` * Value type is <> @@ -608,56 +616,21 @@ option when authenticating to the Elasticsearch server. If set to an empty string authentication will be disabled. -[id="plugins-{type}s-{plugin}-deprecated-options"] -==== Elasticsearch Input deprecated configuration options +[id="plugins-{type}s-{plugin}-obsolete-options"] +==== Elasticsearch Input Obsolete Configuration Options -This plugin supports the following deprecated configurations. +WARNING: As of version `5.0.0` of this plugin, some configuration options have been replaced. +The plugin will fail to start if it contains any of these obsolete options. -WARNING: Deprecated options are subject to removal in future releases. -[cols="<,<,<",options="header",] +[cols="<,<",options="header",] |======================================================================= -|Setting|Input type|Replaced by -| <> |a valid filesystem path|<> -| <> |<>|<> -| <> |<>|<> +|Setting|Replaced by +| ca_file | <> +| ssl | <> +| ssl_certificate_verification | <> |======================================================================= -[id="plugins-{type}s-{plugin}-ca_file"] -===== `ca_file` -deprecated[4.17.0, Replaced by <>] - -* Value type is <> -* There is no default value for this setting. - -SSL Certificate Authority file in PEM encoded format, must also include any chain certificates as necessary. - -[id="plugins-{type}s-{plugin}-ssl"] -===== `ssl` -deprecated[4.17.0, Replaced by <>] - -* Value type is <> -* Default value is `false` - -If enabled, SSL will be used when communicating with the Elasticsearch -server (i.e. HTTPS will be used instead of plain HTTP). - - -[id="plugins-{type}s-{plugin}-ssl_certificate_verification"] -===== `ssl_certificate_verification` -deprecated[4.17.0, Replaced by <>] - -* Value type is <> -* Default value is `true` - -Option to validate the server's certificate. Disabling this severely compromises security. -When certificate validation is disabled, this plugin implicitly trusts the machine -resolved at the given address without validating its proof-of-identity. -In this scenario, the plugin can transmit credentials to or process data from an untrustworthy -man-in-the-middle or other compromised infrastructure. -More information on the importance of certificate verification: -**https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf**. - [id="plugins-{type}s-{plugin}-common-options"] include::{include_path}/{type}.asciidoc[] diff --git a/lib/logstash/inputs/elasticsearch.rb b/lib/logstash/inputs/elasticsearch.rb index f0faa91..cbf9222 100644 --- a/lib/logstash/inputs/elasticsearch.rb +++ b/lib/logstash/inputs/elasticsearch.rb @@ -198,23 +198,12 @@ class LogStash::Inputs::Elasticsearch < LogStash::Inputs::Base # Set the address of a forward HTTP proxy. config :proxy, :validate => :uri_or_empty - # SSL - config :ssl, :validate => :boolean, :default => false, :deprecated => "Set 'ssl_enabled' instead." - - # SSL Certificate Authority file in PEM encoded format, must also include any chain certificates as necessary - config :ca_file, :validate => :path, :deprecated => "Set 'ssl_certificate_authorities' instead." - # OpenSSL-style X.509 certificate certificate to authenticate the client config :ssl_certificate, :validate => :path # SSL Certificate Authority files in PEM encoded format, must also include any chain certificates as necessary config :ssl_certificate_authorities, :validate => :path, :list => true - # Option to validate the server's certificate. Disabling this severely compromises security. - # For more information on the importance of certificate verification please read - # https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf - config :ssl_certificate_verification, :validate => :boolean, :default => true, :deprecated => "Set 'ssl_verification_mode' instead." - # The list of cipher suites to use, listed by priorities. # Supported cipher suites vary depending on which version of Java is used. config :ssl_cipher_suites, :validate => :string, :list => true @@ -242,7 +231,6 @@ class LogStash::Inputs::Elasticsearch < LogStash::Inputs::Base config :ssl_truststore_password, :validate => :password # The JKS truststore to validate the server's certificate. - # Use either `:ssl_truststore_path` or `:ssl_certificate_authorities` config :ssl_truststore_path, :validate => :path # The format of the truststore file. It must be either jks or pkcs12 @@ -264,6 +252,11 @@ class LogStash::Inputs::Elasticsearch < LogStash::Inputs::Base # If set, the _source of each hit will be added nested under the target instead of at the top-level config :target, :validate => :field_reference + # Obsolete Settings + config :ssl, :obsolete => "Set 'ssl_enabled' instead." + config :ca_file, :obsolete => "Set 'ssl_certificate_authorities' instead." + config :ssl_certificate_verification, :obsolete => "Set 'ssl_verification_mode' instead." + # config :ca_trusted_fingerprint, :validate => :sha_256_hex include LogStash::PluginMixins::CATrustedFingerprintSupport @@ -408,8 +401,6 @@ def setup_client_ssl ssl_options[:ssl] = true if @ssl_enabled unless @ssl_enabled - # Keep it backward compatible with the deprecated `ssl` option - ssl_options[:trust_strategy] = trust_strategy_for_ca_trusted_fingerprint if original_params.include?('ssl') return ssl_options end @@ -473,38 +464,11 @@ def setup_client_ssl end def setup_ssl_params! - @ssl_enabled = normalize_config(:ssl_enabled) do |normalize| - normalize.with_deprecated_alias(:ssl) - end - - # Infer the value if neither the deprecate `ssl` and `ssl_enabled` were set - infer_ssl_enabled_from_hosts - - @ssl_certificate_authorities = normalize_config(:ssl_certificate_authorities) do |normalize| - normalize.with_deprecated_mapping(:ca_file) do |ca_file| - [ca_file] - end + # Only infer ssl_enabled if it wasn't explicitly set + unless original_params.include?('ssl_enabled') + @ssl_enabled = effectively_ssl? + params['ssl_enabled'] = @ssl_enabled end - - @ssl_verification_mode = normalize_config(:ssl_verification_mode) do |normalize| - normalize.with_deprecated_mapping(:ssl_certificate_verification) do |ssl_certificate_verification| - if ssl_certificate_verification == true - "full" - else - "none" - end - end - end - - params['ssl_enabled'] = @ssl_enabled - params['ssl_certificate_authorities'] = @ssl_certificate_authorities unless @ssl_certificate_authorities.nil? - params['ssl_verification_mode'] = @ssl_verification_mode unless @ssl_verification_mode.nil? - end - - def infer_ssl_enabled_from_hosts - return if original_params.include?('ssl') || original_params.include?('ssl_enabled') - - @ssl_enabled = params['ssl_enabled'] = effectively_ssl? end def setup_hosts diff --git a/logstash-input-elasticsearch.gemspec b/logstash-input-elasticsearch.gemspec index d00115a..38f4c51 100644 --- a/logstash-input-elasticsearch.gemspec +++ b/logstash-input-elasticsearch.gemspec @@ -1,7 +1,7 @@ Gem::Specification.new do |s| s.name = 'logstash-input-elasticsearch' - s.version = '4.20.5' + s.version = '5.0.0' s.licenses = ['Apache License (2.0)'] s.summary = "Reads query results from an Elasticsearch cluster" s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program" diff --git a/spec/inputs/elasticsearch_spec.rb b/spec/inputs/elasticsearch_spec.rb index bddb6cf..8e24af5 100644 --- a/spec/inputs/elasticsearch_spec.rb +++ b/spec/inputs/elasticsearch_spec.rb @@ -58,6 +58,19 @@ end end + describe 'handling obsolete settings' do + [{:name => 'ssl', :replacement => 'ssl_enabled', :sample_value => true}, + {:name => 'ca_file', :replacement => 'ssl_certificate_authorities', :sample_value => 'spec/fixtures/test_certs/ca.crt'}, + {:name => 'ssl_certificate_verification', :replacement => 'ssl_verification_mode', :sample_value => false }].each do | obsolete_setting| + context "with obsolete #{obsolete_setting[:name]}" do + let (:config) { {obsolete_setting[:name] => obsolete_setting[:sample_value]} } + it "should raise a config error with the appropriate message" do + expect { plugin.register }.to raise_error LogStash::ConfigurationError, /The setting `#{obsolete_setting[:name]}` in plugin `elasticsearch` is obsolete and is no longer available. Set '#{obsolete_setting[:replacement]}' instead/i + end + end + end + end + context "against not authentic Elasticsearch" do before(:each) do Elasticsearch::Client.send(:define_method, :ping) { raise Elasticsearch::UnsupportedProductError.new("Fake error") } # define error ping method From cd2e2fd8bbc54dbf1f2a81109d74f0bb4422c41b Mon Sep 17 00:00:00 2001 From: donoghuc Date: Mon, 16 Dec 2024 11:25:47 -0800 Subject: [PATCH 2/2] Fix accidental removal of changelog entry During a rebase the latest 4 series entry was deleted. This restores it. --- CHANGELOG.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f9a6514..e70053a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,4 @@ -## 5.0.0 ## +## 5.0.0 - SSL settings that were marked deprecated in version `4.17.0` are now marked obsolete, and will prevent the plugin from starting. - These settings are: - `ssl`, which should bre replaced by `ssl_enabled` @@ -6,6 +6,9 @@ - `ssl_certificate_verification`, which should bre replaced by `ssl_verification_mode` - [#213](https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/213) +## 4.20.5 + - Add `x-elastic-product-origin` header to Elasticsearch requests [#211](https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/211) + ## 4.20.4 - Fix issue where the `index` parameter was being ignored when using `response_type => aggregations` [#209](https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/209)