Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a service account and token for use in ~/.kube/config #1458

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ae86c28
Create a service account and token during "pharos kubeconfig"
Aug 13, 2019
ab3a108
Revert kubeconfig and move to phase
Aug 14, 2019
e9fc165
Transport file fixes to support overwriting and path expansion
Aug 14, 2019
32d69bc
Trailing blank
Aug 14, 2019
074adb4
Skip installing a copy of /etc/kubernetes/admin.conf to home
Aug 14, 2019
3b37658
Add expand option to transport.file
Aug 14, 2019
c59ae39
Prefer ~/.kube/config in ConfigureClient
Aug 14, 2019
39b7d44
Simpler expand
Aug 14, 2019
5be5ced
Why it no work [cluster-e2e]
Aug 14, 2019
f70d81f
Cant modify frozen string [cluster-e2e]
Aug 14, 2019
dc6e523
Recomplexify [cluster-e2e]
Aug 14, 2019
c0b7278
Spec dont like expand [cluster-e2e]
Aug 14, 2019
e18ead5
More spec fix [cluster-e2e]
Aug 14, 2019
ddd82af
More fix [cluster-e2e]
Aug 14, 2019
890788f
Add clarifying comments [cluster-e2e]
Aug 14, 2019
b08de30
Update yardoc and retrigger e2e [cluster-e2e]
Aug 14, 2019
324c8ed
Use const [cluster-e2e]
Aug 14, 2019
888c5b9
Adding log messages to make sense of e2e failure [cluster-e2e]
Aug 14, 2019
6a7e4e5
And what does the config look like? [cluster-e2e]
Aug 14, 2019
5e4f466
bang [cluster-e2e]
Aug 14, 2019
986c2a4
Less noise, retrigger e2e [cluster-e2e]
Aug 14, 2019
c23f89c
Slight tweak. I dont know why this fails on drone. [cluster-e2e]
Aug 14, 2019
053ffab
Display host env [cluster-e2e]
Aug 15, 2019
2701644
Try with --kubeconfig [cluster-e2e]
Aug 15, 2019
1792003
Debug [cluster-e2e]
Aug 15, 2019
aea1b80
Merge branch 'master' into feature/sa_token
Aug 21, 2019
a91e213
Retrigger e2e [cluster-e2e]
Aug 21, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions lib/pharos/cluster_manager.rb
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ def apply_phases

apply_phase(Phases::PullMasterImages, master_hosts, parallel: true)
apply_phase(Phases::ConfigureMaster, master_hosts, parallel: false)
apply_phase(Phases::ConfigureServiceAccount, master_hosts, parallel: false)
apply_phase(Phases::ConfigureClient, master_only, parallel: false)
apply_phase(Phases::ReconfigureKubelet, config.hosts, parallel: true)

Expand Down
8 changes: 4 additions & 4 deletions lib/pharos/kube/config.rb
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,20 @@ def initialize(content = nil)
def config
@config ||= yaml_content || {
'apiVersion' => 'v1',
'kind' => 'Config',
'clusters' => [],
'contexts' => [],
'current-context' => nil,
'kind' => 'Config',
'users' => [],
'preferences' => {},
'users' => []
'current-context' => nil
}
end
alias to_h config

# Convert to YAML
# @return [String]
def dump
YAML.dump(config)
YAML.dump(JSON.parse(JSON.dump(config))) # dereference to get rid of *1 &1 etc in output
end
alias to_s dump

Expand Down
1 change: 0 additions & 1 deletion lib/pharos/phases/configure_master.rb
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ def install

def install_kubeconfig
transport.exec!('install -m 0700 -d ~/.kube')
transport.exec!('sudo install -o $USER -m 0600 /etc/kubernetes/admin.conf ~/.kube/config')
end

def reconfigure
Expand Down
102 changes: 102 additions & 0 deletions lib/pharos/phases/configure_service_account.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# frozen_string_literal: true

module Pharos
module Phases
class ConfigureServiceAccount < Pharos::Phase
title "Configure 'pharos-admin' service account"

ADMIN_USER = 'pharos-admin'
KUBECONFIG_PARAM = '--kubeconfig=/etc/kubernetes/admin.conf'

def call
create_service_account
create_cluster_role_binding

config = build_config

if config_file.exist?
existing_config = Pharos::Kube::Config.new(config_file.read)
config << existing_config
end

config_file.write(config.dump, overwrite: true)
config_file.chmod('0600')

validate
end

def validate
transport.exec!('kubectl get -n kube-system serviceaccount/pharos-admin')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using kubectl here because client is not yet configured?

Copy link
Contributor Author

@kke kke Aug 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added that just to validate that kubectl without sudo on master works without KUBECONFIG= or --kubeconfig=.

The next phase actually probably should be changed to use the file from home instead of /etc/kubernetes

end

def config_file
@config_file ||= transport.file(File.join(home_kube_dir.path, 'config'))
end

def home_kube_dir
transport.file(transport.file('~/.kube').readlink(escape: false, canonicalize: true)).tap do |dir|
transport.exec!("mkdir '#{dir}' && chmod 0700 '#{dir}") unless dir.exist?
end
end

def create_service_account
transport.exec!("sudo kubectl get #{KUBECONFIG_PARAM} -n kube-system serviceaccount/#{ADMIN_USER} || sudo kubectl #{KUBECONFIG_PARAM} -n kube-system create serviceaccount #{ADMIN_USER}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we don't need sudo for kubectl.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, maybe we need because this points to root readable kubeconfig?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error: Error loading config file "/etc/kubernetes/admin.conf": open /etc/kubernetes/admin.conf: permission denied

end

def create_cluster_role_binding
transport.exec!("sudo kubectl get #{KUBECONFIG_PARAM} clusterrolebinding pharos-cluster-admin || sudo kubectl create #{KUBECONFIG_PARAM} clusterrolebinding pharos-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:#{ADMIN_USER}")
end

# @return token_name [String]
def token_name
transport.exec!("sudo kubectl -n kube-system #{KUBECONFIG_PARAM} get serviceaccount/#{ADMIN_USER} -o jsonpath='{.secrets[0].name}'")
end

# @return token [String]
def token
@token ||= transport.exec!("sudo kubectl -n kube-system #{KUBECONFIG_PARAM} get secret #{token_name} -o jsonpath='{.data.token}' | base64 -d")
end

# @return [Pharos::Kube::Config]
def build_config
config = Pharos::Kube::Config.new
config.config['clusters'] << {
'cluster' => {
'certificate-authority-data' => certificate_authority_data,
'server' => "https://#{master_host.api_address}:6443"
},
'name' => @config.name
}

config.config['users'] << {
'user' => {
'token' => token
},
'name' => ADMIN_USER
}

config.config['contexts'] << {
'context' => {
'cluster' => @config.name,
'user' => ADMIN_USER
},
'name' => context_name
}

config.config['current-context'] = context_name

config
end

# @return [String]
def context_name
@context_name ||= "#{ADMIN_USER}@#{@config.name}"
end

# @return [String]
def certificate_authority_data
transport.exec!("sudo kubectl config view #{KUBECONFIG_PARAM} --raw --minify --flatten -o jsonpath='{.clusters[].cluster.certificate-authority-data}'")
end
end
end
end
16 changes: 10 additions & 6 deletions lib/pharos/transport/transport_file.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,13 @@ def dirname
end

# @param content [String]
# @param overwrite [Boolean] use force to overwrite target
# @return [Pharos::Transport::Command::Result]
# @raise [Pharos::ExecError]
def write(content)
def write(content, overwrite: false)
tmp = temp_file_path.shellescape
@client.exec!(
"cat > #{tmp} && (sudo mv #{tmp} #{escaped_path} || (rm #{tmp}; exit 1))",
"cat > #{tmp} && (sudo mv #{'-f ' if overwrite}#{tmp} #{escaped_path} || (rm -f #{tmp}; exit 1))",
stdin: content
)
end
Expand Down Expand Up @@ -76,10 +77,11 @@ def with_existing

# Moves the current file to target path
# @param target [String]
# @param overwrite [Boolean] use force to overwrite target
# @return [Pharos::Transport::Command::Result]
# @raise [Pharos::ExecError]
def move(target)
@client.exec!("sudo mv #{@path} #{target.shellescape}")
def move(target, overwrite: false)
@client.exec!("sudo mv #{'-f ' if overwrite}#{@path} #{target.shellescape}")
end
alias mv move

Expand All @@ -100,10 +102,12 @@ def link(target)
@client.exec!("sudo ln -s #{escaped_path} #{target.shellescape}")
end

# @param escape [Boolean] escape file path
# @param canonicalize [Boolean] canonicalize by following every symlink in every component of the file name recursively; all but the last component must exist
# @return [String, nil]
# @raise [Pharos::ExecError]
def readlink
target = @client.exec!("readlink #{escaped_path} || echo").strip
def readlink(escape: true, canonicalize: false)
target = @client.exec!("readlink #{'-f ' if canonicalize}#{escape ? escaped_path : @path} || echo").strip

return nil if target.empty?

Expand Down