Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debian installation on mco > 2.8 removes plugins path from libdir #326

Open
luksi1 opened this issue Sep 29, 2016 · 5 comments
Open

Debian installation on mco > 2.8 removes plugins path from libdir #326

luksi1 opened this issue Sep 29, 2016 · 5 comments

Comments

@luksi1
Copy link

luksi1 commented Sep 29, 2016

Mcollective 1.11.1
Puppet 3.8
OS: Ubuntu 16.04

According to the docs, the way I understand them, the plugins directory needs to be retained. The core lib needs can be taken away after 2.8. The problem with the following code is that you remove the plugins directory and only retain the core lib directory.

libdir = /usr/local/share/mcollective

when it should be

libdir = /usr/share/mcollective/plugins

  if versioncmp($::puppetversion, '4') < 0 {
    $confdir = '/etc/mcollective'
    $_core_libdir = $::osfamily ? {
      'Debian'  => '/usr/share/mcollective/plugins',
      'OpenBSD' => '/usr/local/libexec/mcollective',
      default   => '/usr/libexec/mcollective',
    }
    # Where this module will sync file-managed plugins to.
    # These paths may need revisiting by someone who understands FHS and
    # distribution standards for site-specific application-specific
    # library paths.
    $site_libdir = $::osfamily ? {
      'Debian'  => '/usr/local/share/mcollective',
      'OpenBSD' => regsubst($::rubyversion, '^(\d+)\.(\d+)\.(\d+)$', '/usr/local/lib/ruby/vendor_ruby/\1.\2/mcollective'),
      default   => '/usr/local/libexec/mcollective',
    }
  } else {
    $confdir     = '/etc/puppetlabs/mcollective'
    $_core_libdir = '/opt/puppetlabs/mcollective/plugins'
    $site_libdir = '/opt/puppetlabs/mcollective'
  }

  # Since mcollective version 2.8, there is no core libdir
  # https://docs.puppetlabs.com/mcollective/releasenotes.html#libdirloadpath-changes-and-core-plugins
  $mco_assumed_version = '2.8.5'

  $_mco_version = pick_default($::mco_version, $mco_assumed_version)
  if versioncmp($_mco_version, '2.8') >= 0 {
    $core_libdir = undef
  } else {
    $core_libdir = $_core_libdir
  }

I think the code should be something like this.

  $_mco_version = pick_default($::mco_version, $mco_assumed_version)
  if versioncmp($_mco_version, '2.8') >= 0 {
    $site_libdir = undef
  } 
@luksi1
Copy link
Author

luksi1 commented Sep 29, 2016

Here is how mcollective-puppet-agent looks on Ubuntu 16.04

# dpkg -L mcollective-puppet-agent
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/mcollective-puppet-agent
/usr/share/doc/mcollective-puppet-agent/copyright
/usr/share/doc/mcollective-puppet-agent/changelog.Debian.gz
/usr/share/mcollective
/usr/share/mcollective/plugins
/usr/share/mcollective/plugins/mcollective
/usr/share/mcollective/plugins/mcollective/agent
/usr/share/mcollective/plugins/mcollective/agent/puppet.rb

/usr/local/share/mcollective though is gone from mcollective-common

# dpkg -L mcollective-common
/.
/usr
/usr/lib
/usr/lib/ruby
/usr/lib/ruby/vendor_ruby
/usr/lib/ruby/vendor_ruby/mcollective
/usr/lib/ruby/vendor_ruby/mcollective/rpc.rb
/usr/lib/ruby/vendor_ruby/mcollective/monkey_patches.rb
/usr/lib/ruby/vendor_ruby/mcollective/connector.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginmanager.rb
/usr/lib/ruby/vendor_ruby/mcollective/client.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc
/usr/lib/ruby/vendor_ruby/mcollective/rpc/helpers.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/client.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/progress.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/reply.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/audit.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/result.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/agent.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/request.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/stats.rb
/usr/lib/ruby/vendor_ruby/mcollective/rpc/actionrunner.rb
/usr/lib/ruby/vendor_ruby/mcollective/facts
/usr/lib/ruby/vendor_ruby/mcollective/facts/yaml_facts.rb
/usr/lib/ruby/vendor_ruby/mcollective/facts/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/security.rb
/usr/lib/ruby/vendor_ruby/mcollective/log.rb
/usr/lib/ruby/vendor_ruby/mcollective/matcher.rb
/usr/lib/ruby/vendor_ruby/mcollective/unix_daemon.rb
/usr/lib/ruby/vendor_ruby/mcollective/application
/usr/lib/ruby/vendor_ruby/mcollective/application/rpc.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/plugin.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/ping.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/completion.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/describe_filter.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/facts.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/help.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/inventory.rb
/usr/lib/ruby/vendor_ruby/mcollective/application/find.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator
/usr/lib/ruby/vendor_ruby/mcollective/validator/regex_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/typecheck_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/shellsafe_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/ipv6address_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/length_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/shellsafe_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/ipv4address_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/ipv4address_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/array_validator.ddl
/usr/lib/ruby/vendor_ruby/mcollective/validator/length_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/regex_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/ipv6address_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/array_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator/typecheck_validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/windows_daemon.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/sum.ddl
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/average.ddl
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/summary.ddl
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result/collection_result.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result/numeric_result.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/sum.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/result.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/average.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/summary.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/runner.rb
/usr/lib/ruby/vendor_ruby/mcollective/optionparser.rb
/usr/lib/ruby/vendor_ruby/mcollective/ssl.rb
/usr/lib/ruby/vendor_ruby/mcollective/runnerstats.rb
/usr/lib/ruby/vendor_ruby/mcollective/shell.rb
/usr/lib/ruby/vendor_ruby/mcollective/validator.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor
/usr/lib/ruby/vendor_ruby/mcollective/vendor/require_vendored.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/lib
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/lib/systemu.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/b.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/a.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/d.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/e.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/f.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/samples/c.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/systemu.gemspec
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/Rakefile
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/README
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/LICENSE
/usr/lib/ruby/vendor_ruby/mcollective/vendor/systemu/README.erb
/usr/lib/ruby/vendor_ruby/mcollective/vendor/load_systemu.rb
/usr/lib/ruby/vendor_ruby/mcollective/matcher
/usr/lib/ruby/vendor_ruby/mcollective/matcher/parser.rb
/usr/lib/ruby/vendor_ruby/mcollective/matcher/scanner.rb
/usr/lib/ruby/vendor_ruby/mcollective/audit
/usr/lib/ruby/vendor_ruby/mcollective/audit/logfile.rb
/usr/lib/ruby/vendor_ruby/mcollective/config.rb
/usr/lib/ruby/vendor_ruby/mcollective/message.rb
/usr/lib/ruby/vendor_ruby/mcollective/application.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager.rb
/usr/lib/ruby/vendor_ruby/mcollective/facts.rb
/usr/lib/ruby/vendor_ruby/mcollective/aggregate.rb
/usr/lib/ruby/vendor_ruby/mcollective/cache.rb
/usr/lib/ruby/vendor_ruby/mcollective/exceptions.rb
/usr/lib/ruby/vendor_ruby/mcollective/registration
/usr/lib/ruby/vendor_ruby/mcollective/registration/agentlist.rb
/usr/lib/ruby/vendor_ruby/mcollective/registration/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/discovery
/usr/lib/ruby/vendor_ruby/mcollective/discovery/mc.rb
/usr/lib/ruby/vendor_ruby/mcollective/discovery/mc.ddl
/usr/lib/ruby/vendor_ruby/mcollective/discovery/flatfile.ddl
/usr/lib/ruby/vendor_ruby/mcollective/discovery/stdin.ddl
/usr/lib/ruby/vendor_ruby/mcollective/discovery/stdin.rb
/usr/lib/ruby/vendor_ruby/mcollective/discovery/flatfile.rb
/usr/lib/ruby/vendor_ruby/mcollective/applications.rb
/usr/lib/ruby/vendor_ruby/mcollective/generators.rb
/usr/lib/ruby/vendor_ruby/mcollective/data
/usr/lib/ruby/vendor_ruby/mcollective/data/fact_data.ddl
/usr/lib/ruby/vendor_ruby/mcollective/data/agent_data.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/collective_data.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/agent_data.ddl
/usr/lib/ruby/vendor_ruby/mcollective/data/result.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/collective_data.ddl
/usr/lib/ruby/vendor_ruby/mcollective/data/fact_data.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/fstat_data.ddl
/usr/lib/ruby/vendor_ruby/mcollective/data/fstat_data.rb
/usr/lib/ruby/vendor_ruby/mcollective/data/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl
/usr/lib/ruby/vendor_ruby/mcollective/ddl/dataddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl/discoveryddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl/validatorddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl/agentddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/ddl/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/data.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger
/usr/lib/ruby/vendor_ruby/mcollective/logger/file_logger.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger/console_logger.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger/syslog_logger.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/registration.rb
/usr/lib/ruby/vendor_ruby/mcollective/discovery.rb
/usr/lib/ruby/vendor_ruby/mcollective/util.rb
/usr/lib/ruby/vendor_ruby/mcollective/logger.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/agent_definition.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/compat.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/control.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/rules.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/changelog.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/Makefile.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/debian/copyright.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/module
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/module/README.md.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/module/_manifest.pp.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/module/Modulefile.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/redhat
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/templates/redhat/rpm_spec.erb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/standard_definition.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/rpmpackage_packager.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/debpackage_packager.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/ospackage_packager.rb
/usr/lib/ruby/vendor_ruby/mcollective/pluginpackager/modulepackage_packager.rb
/usr/lib/ruby/vendor_ruby/mcollective/agent.rb
/usr/lib/ruby/vendor_ruby/mcollective/agents.rb
/usr/lib/ruby/vendor_ruby/mcollective/agent
/usr/lib/ruby/vendor_ruby/mcollective/agent/discovery.rb
/usr/lib/ruby/vendor_ruby/mcollective/agent/rpcutil.rb
/usr/lib/ruby/vendor_ruby/mcollective/agent/rpcutil.ddl
/usr/lib/ruby/vendor_ruby/mcollective/ddl.rb
/usr/lib/ruby/vendor_ruby/mcollective/connector
/usr/lib/ruby/vendor_ruby/mcollective/connector/activemq.rb
/usr/lib/ruby/vendor_ruby/mcollective/connector/rabbitmq.rb
/usr/lib/ruby/vendor_ruby/mcollective/connector/rabbitmq.ddl
/usr/lib/ruby/vendor_ruby/mcollective/connector/activemq.ddl
/usr/lib/ruby/vendor_ruby/mcollective/connector/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/generators
/usr/lib/ruby/vendor_ruby/mcollective/generators/agent_generator.rb
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates/action_snippet.erb
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates/ddl.erb
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates/data_input_snippet.erb
/usr/lib/ruby/vendor_ruby/mcollective/generators/templates/plugin.erb
/usr/lib/ruby/vendor_ruby/mcollective/generators/data_generator.rb
/usr/lib/ruby/vendor_ruby/mcollective/generators/base.rb
/usr/lib/ruby/vendor_ruby/mcollective/vendor.rb
/usr/lib/ruby/vendor_ruby/mcollective/security
/usr/lib/ruby/vendor_ruby/mcollective/security/ssl.rb
/usr/lib/ruby/vendor_ruby/mcollective/security/aes_security.rb
/usr/lib/ruby/vendor_ruby/mcollective/security/psk.rb
/usr/lib/ruby/vendor_ruby/mcollective/security/base.rb
/usr/lib/ruby/vendor_ruby/mcollective.rb
/usr/share
/usr/share/doc
/usr/share/doc/mcollective-common
/usr/share/doc/mcollective-common/copyright
/usr/share/doc/mcollective-common/changelog.Debian.gz
/etc
/etc/mcollective
/etc/mcollective/metadata-help.erb
/etc/mcollective/data-help.erb
/etc/mcollective/rpc-help.erb
/etc/mcollective/discovery-help.erb

Not sure if you can simply switch core_libdir and site_libdir without affecting older versions, which is what I am testing our infrastructure (12.04, 14.04, 16.04). I can though say that the current configuration will not work on 16.04 unless you're possibly installing from the source packages or something as then the source would get copied to your libdir and all would be well.

Does a clean libdir and a fresh install of mcollective-puppet-agent really work on your test server with mco > 2.8?

@luksi1
Copy link
Author

luksi1 commented Sep 29, 2016

And I think this issue is along the same lines:

mcollective::plugin { 'puppet':
  package => true,
}

This will install mcollective-puppet-agent and then the next run will filebucket it! As the package will be installed under the site plugins directory.

@antaflos
Copy link
Contributor

As I've commented in issue #325 you should be using the Puppetlabs-provided MCollective packages on 12.04 and 14.04. If you then set core_libdir like this it should work nicely:

class { '::mcollective':
  ...
  core_libdir => '/usr/share/mcollective/plugins',
  ...
}

That's what we currently do on 12.04 and 14.04, using this module and MCollective 2.9 (from the Puppetlabs packages).

Unfortunately Puppetlabs do not provide MCollective packages on 16.04, so you can either try to make it work with whatever Ubuntu provides, or can you read through https://tickets.puppetlabs.com/browse/MCO-722 for other ideas.

As you can see the whole situation is a mess and I am not impressed by how Puppetlabs seem to treat MCollective more and more like a second-class citizen.

Semi-related question: how do you get Puppet 3.8 to work on 16.04? The default Ruby version on 16.04 is 2.3 which Puppet 3.8 does not support. Do you install an older Ruby version, perhaps from the Brightbox PPAs?

@luksi1
Copy link
Author

luksi1 commented Oct 2, 2016

This works nicely. I did not catch the need to set core_libdir as a necessity for Ubuntu in the docs; maybe this should be updated in the README file?

This is the output from a 16.04 LTS server we have.

myserver:~# ps -ef | grep puppet
root       442 32208  0 17:21 pts/0    00:00:00 grep --color=auto puppet
root      2916     1  0 aug24 ?        00:01:30 /usr/bin/ruby /usr/bin/puppet agent
myserver:~# ruby --version
ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]
myserver:~# dpkg -s puppet
Package: puppet
Status: install ok installed
Priority: optional
Section: admin
Installed-Size: 59
Maintainer: Ubuntu Developers <[email protected]>
Architecture: all
Version: 3.8.5-2
Depends: init-system-helpers (>= 1.18~), puppet-common (= 3.8.5-2), ruby | ruby-interpreter
Suggests: puppet-el, vim-puppet, etckeeper
Conffiles:
 /etc/init.d/puppet 9eb2754a651af8a0ad2091fbba775bcc
Description: configuration management system, agent
 Puppet is a server automation tool. It is composed of a declarative
 language for expressing system configuration, a client and server for
 distributing it, and a library for realizing the configuration.
 .
 The primary design goal of Puppet is to have an expressive enough
 language backed by a powerful enough library that you can write your
 own server automation applications in just a few lines of code.
 Puppet’s deep extensibility and open source license lets you add
 functionality as needed and share your innovations with others.
 .
 This package contains "puppet" service and init script used to run
 the puppet agent, and depend on the "puppet-common" package which
 contains the software.
Original-Maintainer: Puppet Package Maintainers <[email protected]>
Homepage: http://projects.puppetlabs.com/projects/puppet

@antaflos
Copy link
Contributor

antaflos commented Oct 2, 2016

I see, thank you. So you use the Puppet 3.8 packages provided by Ubuntu, not by Puppetlabs. I assume the Ubuntu package maintainers patched Puppet and its dependencies so that they work properly with Ruby 2.3.x. Which makes sense, since Puppetlabs don't even provide Puppet 3.8 packages for Ubuntu 16.04 any more (only PC1/AIO for Puppet 4). Good to know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants