Idea: Release the SecretManagement and SecretStore modules with PowerShell

[mgreenegit]

Summary of the new feature / enhancement

As a user, I have a consistent need to manage secure strings without exposing them in scripts and other automation artifacts. To rely on this capability in PowerShell, I need it to always be available without the need to add the installation to every new project, or finding workarounds to add it in the tools of my choice that run PowerShell.

Proposed technical implementation details (optional)

This experience would align with other modules that are released with the "default" installation footprint of PowerShell. There is no need to merge the source repo's or manage the projects as a single entity. When the PowerShell packages are built, the latest Microsoft.PowerShell.SecretManagement and Microsoft.PowerShell.SecretStore modules would be included.

[kilasuit]

Yes please

[fMichaleczek]

Agree

[Amir-Mir]

This would be great. Please do add these module with PS7.

[tristanbarcelon]

Would they be installed in the "built-in" Modules folder, e.g., %PROGRAMFILES%\PowerShell\7\Modules in Windows to avoid being uninstalled from the cmdlet?

[guidooliveira]

That would be super useful

[JustinGrote]

As long as there's still pursuit of breaking out all the subsystems and having a "slim" powershell option, I'm fine with a "batteries included" version that includes SecretManagement, it follows with the philosophies of .NET and the BCL.

[kilasuit]

@JustinGrote there is under this issue which is still in the consider bucket

[JustinGrote]

@JustinGrote there is under this issue which is still in the consider bucket

There's lots of subsystems (remoting, etc.) that can be stripped out too and is a work item, that's more what I want slim to be rather than a simple repackaging.

[LeonarddeR]

I'd really appreciate this!

[jaamarcelis]

Wow, great idea!

[halape]

Yes, please. That'd be really neat. Not to mention an extra push to migrate to PS7. :)

[cblackuk]

Absolutely yes!!! Then yes, after that yes, and then yes please :-) Maybe not in that order, but that would be an awesome addition. What a fantastic idea!

[namangoel19]

This will be awesome. This is general problem and having a separate package is pain.

[hongbiao1]

Awesome!!!

[jborean93]

I'm going to have to go against the curve here and say -1 (the following is a cleaned up version of my Twitter posts https://twitter.com/BoreanJordan/status/1550006497752788992)

This seems to be going against the general way of working in PowerShell where new contributions outside of the existing set of builtin cmdlets and engine details are being redirected to 3rd party modules and the PSGallery. One of the harder aspects of this is getting buy in to people willing to use modules and it seems like MS is coming across a similar problem with SecretManagement and SecretStore. To me it's a bit annoying that the pwsh team now is trying to just bypass these challenges by just bundling the modules with the default install making it even less likely to look at modules and the extras it provides.

If the problem is people are not allowed or don't know how to install modules the answer isn't to bundle it with pwsh but to figure out what those blockers are and come up with solutions to make it easier for pepole. You help make this less of a problem and it will result in more people willing to use modules rather than rely on what comes out of the box. This means more people will want to try and contribute code to pwsh itself rather than go their own way and result in a poorer ecosystem overall.

Another counterpoint is that when you bundle things then people become reliant on the version it came with. Now my code will need to continue to support older versions of these modules because of the bulk of the users will be relying on what comes in box and will never upgrade things. On the extreme end of the scale we saw this with Pester where WinPS shipped with v3. If these stay truly independent then the bulk of the users will more than likely follow along with the latest version out there. Pwsh will also be constrained with whatever version it can bundle as people will be expecting no breaking changes in minor releases which means either;

• SecretManagement will have long lived minor released 1.2.x will live on for the life of the pwsh release it was bundled with, or
• The version included in a pwsh release will stagnate and people will be stuck on that old version unless the next 7.x release of pwsh is out

In addition to that it might make the SecretManagement team less likely to take on larger or potentially breaking changes as they are now expected to be backwards compatible just like people expect pwsh 7.x to mostly be backwards compatible with 6 and earlier.

TLDR: If you want to encourage a vibrant ecosystem you need to be part of it and drive the adoption of things not in the box.

[SteveL-MSFT]

@jborean93 agree that Pester is a good example of both success and failure. It helped drive adoption of testing with PowerShell, but now we have a hard time removing the old version from Windows while Pester itself has moved on due to compatibility concerns.

I think for SecretManagement, the question is whether secrets management is a core platform need or not. If not, then this is an easy discussion and it should just be on the gallery.

[daxian-dbw]

I tend to agree with @jborean93. One of the longstanding problems for PSReadLine is that many users stay on old PS versions and thus keep running into issues that were already fixed in new versions of the PSReadLine module (we get tons of issues opened for bugs that were already fixed).

Also, for anyone hosting PowerShell, they would still need to install the SecreteManagement module separately because the SDK nuget packages won't be able to bundle the SecreteManagement module.

I think another question to ask is: how much efforts are we really saving if we bundle the SecreteManagement module in PowerShell package? For users that rely on SecreteManagement in devops, they likely also depend on other modules in the gallery, and maybe installing SecreteManagement along with others doesn't really cost additional efforts.

[SteveL-MSFT]

We appreciate all of the feedback. The @PowerShell/powershell-committee discussed this and agree with the concerns. At this time we will not consider shipping SecretMgmt and SecretStore in PS7, but looking at a different approach to make it easier for users to discover sets of modules.

[kilasuit]

@SteveL-MSFT re this
I think for SecretManagement, the question is whether secrets management is a core platform need or not. If not, then this is an easy discussion and it should just be on the gallery.
Managing credentials is a core need in secure automation, which is why I think both modules should be included in future releases

[iSazonov]

Since different communities have different needs we might look at Windows Server - today it is nano, core and full. We could do the same. It could be some distributives or one with options.

[SteveL-MSFT]

@kilasuit I agree that secure secrets management SHOULD be part of the platform. However, I think we can have a solution where we have a (open to better names) devops module in PowerShellGallery that depends on SecretManagement, SecretStore, and other modules that make sense for that domain so users can install just that "one" module to get everything they probably need.

[Kyahx]

Two ideas:

1. Stub cmdlets or basic implementations that prompt the user to install the "full featured" modules from the Gallery -- similar to how running Get-Help for the first time prompts to install the help files.
2. Make the error from running a #requires -Module script more helpful, optionally prompting the user to install those modules from the gallery or any other source if they are available.

[JustinGrote]

I like idea 2 a lot, but the tricky thing in implementation will be tying the engine to a packagemanagement system and how that would work.

[ThomasNieto]

There was a suggested feature on PowerShellGet v3 RFC #237 which would find the command and suggest to the user which command to run to install the module.

Support for automatic searches from Find-PSResource -Type Command when a command in not found in module discovery

PS> Get-Satisfaction
Get-Satisfaction : The term 'Get-Satisfaction' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-Satisfaction
+ ~~~
+ CategoryInfo          : ObjectNotFound: (Get-Satisfaction:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
Suggestion: You can obtain `Get-Satisfaction` by running `Install-PSResource HappyDays`.
PS> Install-PSResource HappyDays

[kilasuit]

On @daxian-dbw point here

One of the longstanding problems for PSReadLine is that many users stay on old PS versions

That's a historic issue with how things work with PowerShell & Windows Updates, which realistically I think modules like PSReadLine, or Pester, that were added to Windows, should be updated alongside any updates made to inboxed versions of PowerShell, but that's an argument to have with the Windows team, because the right way would be that an inboxed module was updated when it was updated elsewhere. Though we had absolute hell with PowerShellGet 1.0.0.1 back in the Windows 10 insider days where each new version of Windows had a new 1.0.0.1 version of PowerShellGet which was because of some flawed packaging and release process into the Windows Builds. That was back in 2016 mind, so I hope that things have changed since then.

Only thing that I know is that if I was building Windows, and I had a package like PowerShellGet that I'd taken a dependency on, I'd make sure to inbox the latest stable version on each build as opposed to what seemed to be repackaged rebuilt versions of the module that were in some cases functionally nothing alike. Perhaps that was a learning curve for the then team that authored PowerShellGet, but it was one of the big reasons why I pushed hard for the PowerShell team to rip out the dependency on PackageManagement in PowerShellGet.

Also on @daxian-dbw point here

I think another question to ask is: how much efforts are we really saving if we bundle the SecreteManagement module in PowerShell package? For users that rely on SecreteManagement in devops, they likely also depend on other modules in the gallery, and maybe installing SecreteManagement along with others doesn't really cost additional efforts.

Not all users of PowerShell are working to DevOps Methodologies or even in some cases secure or repeatable IT practices, and in my years of consulting with organisations, many of those that are holding their businesses up in the IT department are still yet to learn how to automate anything, let alone automate it in a secure fashion.
There is a significant amount of IT out there that has been installed manually and managed manually via GUI's because that's how it always was for a significant number of Windows Server Administrators for a number of decades now.

To @jborean93 point

If the problem is people are not allowed or don't know how to install modules the answer isn't to bundle it with pwsh but to figure out what those blockers are and come up with solutions to make it easier for pepole.

The problem in many cases is that there are a number of organisations who run and manage critical IT infrastructure for others, who, quite frankly, should not be allowed to do so. Some of the blockers in those that run their own infrastructure or are more well established organisations is that there is often a need to have a internal review of any module that is published to the gallery, as an independent review process, whereas if we bundled the SecretManagement & SecretStore Modules in with PowerShell by default or into a PowerShell + Recommended Modules for Automation type release then you'd gain more adoption and traction & use of them,
Most of the time it is either a security reason, politics or in some cases some organisations only want support from Microsoft and no other 3rd parties. The latter is the one that we can't really help with I don't think. A huge part is more about understanding who is allowed to use PowerShell at all, let alone use things from the community. It's in my 10 years across IT most of it in the PowerShell community one of those problems that unfortunately I can only see changing if significant changes happens across all sectors that rely on IT & that they end up doing it properly, which realistically, I'm afraid is very unlikely to happen. So we have to meet in the middle as best as we can and bolster the community to help people do things in as much the right way as is entirely possible. It is a super hard problem & one that I don't think is likely to be solved in it's entirety, no matter how much I would like to see it be & boy have I tried to do so, as have many others too.
That's on the windows side, it may be very different on linux/unix side, but I can only tell you what I've seen over the years.

[SteveL-MSFT]

PowerShellGet v3 will (eventually) have improved ability to install a large set of modules and their dependencies concurrently. Along with that, the new PSFeedbackProvider experimental feature in 7.4 will allow PowerShellGet v3 integration where if you try to use a cmdlet or module that is not installed, it can tell the user how to install the module to get that cmdlet. Shipping modules with PS7 is way different than shipping modules with Windows. Windows has a much stricter requirement for backwards compat (which is why we can't ship PS7 in Windows and just keep updating it to newest .NET). Pester is a prime example of this problem. We can't update it nor remove it right now because folks are dependent on that version.

In the long run, we want to have a smaller PS7 package and have folks be comfortable updating modules from PowerShellGallery rather than always needing to get latest PS7 to get latest version of some module.

[JustinGrote]

I 100% agree with the vision being PowerShell being a lean core runtime with a lot of the existing subsystems broken out into modules, but this will require package management to be as mature as pnpm or dotnet restore to quickly fetch and restore dependencies in both imperative and declarative ways in order to be effective and not vastly annoy the user with "friction" that they don't bother to use anything not in-box.

[AspenForester]

I'd like to add that one thing that I think would improve the adoption of secrets management would be promotion of quality SecretManagement extensions. While it's entirely possible that I've hitched my horse to the wrong post, I happen to like KeePass, and the only extension for KeePass is the beta experimentation that @JustinGrote published and hasn't come back to in over a year.

Side Bar: That's not meant to be a jab at you, Justin, you have other priorities, and that's ok. Clearly this project didn't provide enough value to warrant much more of your limited time.

Without the (availability of) other extensions SecretManagement is of limited value.
From my point of view, I would love for SecretManagement to really valuable and really easy to use, so that it becomes a no brainer to just use it. Having it "just be there" is a step in that direction, in my opinion.