Add elasticsearch-keystore with google cloud storage plugin credentials key file

[syst0m]

Hi,

trying to build an image, from your base image, that uses elasticsearch-keystore for storing sensitive secrets, that would be used to authenticate against a google cloud storage bucket for snapshotting the indices via a kubernetes curator cronjob.

My Dockerfile:

FROM quay.io/pires/docker-elasticsearch-kubernetes:6.2.2_1

# inital setup
COPY gcs.client.default.credentials_file /tmp/

ENV PATH /elasticsearch/bin:$PATH

WORKDIR /elasticsearch

# Copy configuration
COPY config /elasticsearch/config

# Copy run script
COPY run.sh /

# Set environment variables defaults
ENV ES_JAVA_OPTS "-Xms512m -Xmx512m"
ENV CLUSTER_NAME elasticsearch-default
ENV NODE_MASTER true
ENV NODE_DATA true
ENV NODE_INGEST true
ENV HTTP_ENABLE true
ENV NETWORK_HOST _site_
ENV HTTP_CORS_ENABLE true
ENV HTTP_CORS_ALLOW_ORIGIN *
ENV NUMBER_OF_MASTERS 1
ENV MAX_LOCAL_STORAGE_NODES 1
ENV SHARD_ALLOCATION_AWARENESS ""
ENV SHARD_ALLOCATION_AWARENESS_ATTR ""
ENV MEMORY_LOCK true
ENV REPO_LOCATIONS []

# Volume for Elasticsearch data
VOLUME ["/data"]

# fix "Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'NODE_NAME' when running elasticsearch-keystore
ENV NODE_NAME=""

# fix elasticsearch-keystore failing with mktemp: Invalid argument & create+populate keystore
RUN export ES_TMPDIR=`mktemp -d -t elasticsearch.XXXXXXXX` && /elasticsearch/bin/elasticsearch-keystore create
RUN export ES_TMPDIR=`mktemp -d -t elasticsearch.XXXXXXXX` && /elasticsearch/bin/elasticsearch-keystore add-file gcs.client.default.credentials_file /tmp/gcs.client.default.credentials_file
RUN chown elasticsearch:elasticsearch /elasticsearch/config/elasticsearch.keystore

USER elasticsearch

CMD ["/run.sh"]
 

run.sh is your script from this repo.
Here's what i get after deploying this as a new client pod:

kubectl logs es-client-canary-598c8f54f7-s47ms
sh: error setting limit: Operation not permitted
-> Downloading repository-gcs from elastic
[=================================================] 100%  
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin requires additional permissions @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

• java.lang.RuntimePermission accessDeclaredMembers
• java.lang.RuntimePermission setFactory
• java.lang.reflect.ReflectPermission suppressAccessChecks
• java.net.SocketPermission * connect,resolve
• java.net.URLPermission http://www.googleapis.com/* *:
• java.net.URLPermission https://www.googleapis.com/* *:
  See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
  for descriptions of what these permissions allow and the associated risks.
  -> Installed repository-gcs
  [2018-06-20T09:15:40,612][WARN ][o.e.b.JNANatives ] Unable to lock JVM Memory: error=12, reason=Out of memory
  [2018-06-20T09:15:40,615][WARN ][o.e.b.JNANatives ] This can result in part of the JVM being swapped out.
  [2018-06-20T09:15:40,615][WARN ][o.e.b.JNANatives ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
  [2018-06-20T09:15:40,616][WARN ][o.e.b.JNANatives ] These can be adjusted by modifying /etc/security/limits.conf, for example:

  allow user 'elasticsearch' mlockall

  elasticsearch soft memlock unlimited
  elasticsearch hard memlock unlimited
  [2018-06-20T09:15:40,616][WARN ][o.e.b.JNANatives ] If you are logged in interactively, you will have to re-login for the new limits to take effect.
  [2018-06-20T09:15:40,905][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [es-client-canary-598c8f54f7-s47ms] uncaught exception in thread [main]
  org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Unable to access 'path.repo' ([])
  at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.2.jar:6.2.2]
  at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.2.2.jar:6.2.2]
  Caused by: java.lang.IllegalStateException: Unable to access 'path.repo' ([])
  at org.elasticsearch.bootstrap.FilePermissionUtils.addDirectoryPath(FilePermissionUtils.java:70) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:323) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:262) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Security.configure(Security.java:123) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:208) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.2.jar:6.2.2]
  ... 6 more
  Caused by: java.nio.file.AccessDeniedException: /elasticsearch/[]
  at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:1.8.0_151]
  at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_151]
  at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_151]
  at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_151]
  at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_151]
  at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_151]
  at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_151]
  at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:421) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.FilePermissionUtils.addDirectoryPath(FilePermissionUtils.java:68) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:323) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:262) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Security.configure(Security.java:123) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:208) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.2.jar:6.2.2]
  at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.2.jar:6.2.2]
  ... 6 more

Any help would be greatly appreciated.
Thanks in advance!