@MaikuMori What is the use case for piping? Why not just write the output to a file and then run sops -e -i <filename>?

I have a script which decodes content using sops -d enc_file and reads stdout. Then it changes the content dynamically and needs to update the original encoded file. Currently, I work around this by creating a temp file, writing the content there then running sops on it and then overwrite the original file, then delete the temp file. It would be much nicer if sops could just read stdin.

Even doing something like cat asdf | base64 | sops --output asdf.enc -e - in terminal would be nice. (Alternatively cat asdf | base64 | sops -e - > asdf.enc)

Here are other issues which mention this as preferred way:

• Make sops use config file when using STDIN #510 (mentions -)
• Editing file in-place with data from stdin #696 (could be solved with -)
• Document how to operate on stdin #330 (suggested alternative solution, but you can clearly see that /dev/stdin is not expected behavior. People expect piping to just work or work with -)

To make this even more useful sops should use config file when input is passed via stdin (tracked by #510). To decide which config to use we could use CWD or the directory of output file if specified.

Maybe also defaulting input/output type to binary unless specifically specified.

Additionally on windows, running in cygwin.

sops --decrypt --input-type yaml --output-type yaml --output /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml.dec /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml
#   Error: cannot operate on non-existent file

While

sops --decrypt --input-type yaml --output-type yaml - < /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml > /cygdrive/c/Users/RUNNER~1/AppData/Local/Temp/tmp.6bAckDpOVi/assets/values/sops/secrets.yaml.dec 

Would not.

I'd like to add support for this feature. It prevents the necessity of unencrypted intermediate files that might potentially linger/be exposed on Windows. We are forced to work in a Windows environment, while the app is deployed to a Linux environment via CI/CD (also Linux based). We cannot use /dev/stdin and this request would provide a clean platform-independent way of reading via stdin.

Just a quick comment for those who look a workaround, I used a temporary file with nanoseconds in the time to include the yaml content and remove it right after the sops execution.

if [[ $OSTYPE == 'darwin'* ]]; then
curTime=$(gdate +%s.%N)
else
curTime=$(date +%s)
fi

tempFile=./temp/${curTime}

ENCRYPTED_CONTENT=$(echo "$CONTENT" > $tempFile | sops -e --encrypted-regex "$ENCRYPTED_REGEX" --input-type yaml --output-type yaml $tempFile)

rm -f $tempFile

If sops --encrypt by default outputs to stdout, why doesn't sops --decrypt read from stdin by default without a filename argument?

While the filename convention of using "-" to denote stdin is helpful, why not just read from stdin if no filename is given at all and stdin is not a tty?

What is the use case for piping? Why not just write the output to a file and then run sops -e -i <filename>?

One use case is never writing the unencrypted content to disk. Using this as a practise when working with secret files could reduce the risk of committing the secrets unencrypted as you're (supposedly) never having unencrypted secrets in your workspace to begin with.

I really, really wish secret-generating-command | sops secret.enc.yaml would just do what I want, which is to read the secret from stdin and write it out encrypted to secret.enc.yaml (overwriting if it exists) using the rules from .sops.yaml.

Getting this fixed is still on the roadmap; the original plan was to resurrect #1104 and fix the remaining issues with it, but so far nobody got around to actually do it. (I've added this issue to the roadmap, since having the closed PR on it isn't very helpful as a reminder...)

I think we need to discuss how the interface should look like:

1. Use a magic filename -;
2. Add an option --use-stdin (or something like that) to sops encrypt and sops decrypt (which reduces the number of positional arguments expected to zero, and for sops encrypt requires the use of --filename-override).
3. Allow to provide no positional argument (instead of one), and if that's the case, read from stdin (and require the use of --filename-override when encrypting).

I'm currently tending to 2. Other opinions?

CC @getsops/maintainers

Is there any reason why gen-secret | sops encrypt secret.enc.yaml is not feasible? The majority of the time I encrypt data with sops I get it from a command and need sops to encrypt and write it to a file using the rules configured in .sops.yaml. It sounds to me like option two requires me to write gen-secret | sops encrypt --use-stdin --filename-override secret.enc.yaml which is tedious.

Is there any reason why gen-secret | sops encrypt secret.enc.yaml is not feasible?

Yes: sops encrypt secret.enc.yaml already does something else right now (it encrypts secret.enc.yaml). Changing it behavior would be a breaking change.

Variant 3 would be the least amount of typing: gen-secret | sops encrypt --filename-override secret.enc.yaml (Unless you have a rule matching -, then you can do gen-secret | sops encrypt - with variant 1.)

I would also much prefer option 3. It is the default behaviour that I would expect for tools on the shell command line.

Using a magic filename - can lead to new complaints: FiloSottile/age#494
For that reason I would prefer to avoid variant 1.

I've implemented variant 3 in #1690.