How do you generate a YAML file with comments?

[octogonz]

We have a tool that generates YAML config files for CI pipelines.

The files contain informative code comments. Here's a representative example (with a bunch of fields deleted for brevity):

# DO NOT EDIT THIS FILE!!!
# To modify this file, make changes to the associated config files
# and then invoke the "generate-pipelines" tool.
variables:
  - name: NodeVersion
    value: 12
jobs:
  - job: PRBuild
    condition: succeeded()
    steps:
      - checkout: self
        fetchDepth: 1
        lfs: true
        
      - script: 'node common/scripts/install-run-rush.js install'
        displayName: 'Install Rush'
        
      - script: 'node common/scripts/install-run-rush.js rebuild'
        displayName: 'AWS Pipelines Check'
        
      # Generated from 'apps/example-app/infra/production/aws-stacks.json'
      - task: CloudFormationCreateOrUpdateStack@1 
        inputs:
          templateFile: 'common/config/aws/templates/bucket.cf.json'

          # Below parameter(s) generated from 1 file(s)
          # apps/example-app/infra/production/bucket-provision.cf-params.json
          templateParameters: | 
            -
              ParameterKey: 'BucketName'
              ParameterValue: 'example-app.example.com'
            -
              ParameterKey: 'CostOwner'
              ParameterValue: 'example-app'
            -
              ParameterKey: 'CostService'
              ParameterValue: 'example-app.example.com'
          capabilityAutoExpand: true
          onFailure: 'DELETE'
          
      # Generated from 'apps/example-app/infra/production/aws-stacks.json'
      - task: CloudFormationCreateOrUpdateStack@1 
        inputs:
          awsCredentials: 'AWS Production'
          regionName: 'us-east-1'

As you can see, the comments aren't merely pasted at the top of the file, they need to appear inline in the file as well.

We started using a simple string template, but it is tricky to escape parameters correctly. The js-yaml project seems like a better solution.

But does the yaml.dump() API provide some way to inject comment strings?

Comments are a core feature of YAML, so it seems that there should be some way to do that. Thanks!

CC @hbo-iecheruo

[Sirros]

I need this function too

[puzrin]

This parser implementation does NOT have AST and can't manage metadata like comments and so on. This question was already asked and answered in previous issues.

[octogonz]

@puzrin You don't have any interest in fixing that? For example maybe dump() could accept a map or callback that associates comment strings with JSON nodes. The problem does not seem insurmountable.

[puzrin]

@octogonz i guess, you don't understand the cost of such "fix". That's really full rewrite. So, no, i'm not interested to spend a lot of time for that. Because, i have other projects to do.

But, if you know how to solve problem better than me, you can suggest PR.

[perlpunk]

I am an author of a YAML library and I know a list of YAML libraries. Very few support creating comments. It's hard, I concur with @puzrin .
You might try out this newer JS library: https://github.com/eemeli/yaml
It was written using the (relatively) new YAML Test Suite, and supports roundtripping comments.

[octogonz]

@perlpunk Thank you! We'll check it out.

[lttlrck]

@perlpunk that works great, thank you.

[olee]

I implemented the following utility which allows using yaml-js to generate output with comments in it:

const YAML_COMMENT_PREFIX = '___YAML_COMMENT_';
const YamlCommentRegex = /( *)___YAML_COMMENT_\d+: *([^ |].*)/g;
const MultilineYamlCommentRegex = /( *)___YAML_COMMENT_\d+: *\|-\n((\1 +)[^ ].*(?:\n\3[^ ].+)*)/g;

let yamlCommentIndex = 0;

/**
 * Creates a key to be used as a YAML comment
 */
export function yamlComment() {
    return YAML_COMMENT_PREFIX + yamlCommentIndex++;
}

/**
 * Replaces YAML comment keys created with {@link yamlComment} with proper YAML comment syntax
 */
export function processYamlComments(baseYaml: string) {
    return baseYaml
        .replace(YamlCommentRegex, (_match, leadingSpace: string, comment: string) => leadingSpace + '# ' + comment.trim())
        .replace(
            MultilineYamlCommentRegex,
            (_match, leadingSpace: string, comment: string) => comment
                .split('\n')
                .map(x => `${leadingSpace}# ${x.trim()}`)
                .join('\n')
        );
}

Example:

const sampleData = {
    [yamlComment()]: 'Comment on root key / start of file',
    ROOT: {
        [yamlComment()]: 'Comment on child key',
        CHILD_1: 'test',
        [yamlComment()]: 'Multi-line comment\non child key',
        CHILD_2: "'this' is quoted",
        [yamlComment()]: "This comment's text has special characters!",
        CHILD_3: 'key 3',
    },
};

test('test', () => {
    const yaml = processYamlComments(dump(sampleData));
    expect(yaml).toMatchInlineSnapshot(`
        "# Comment on root key / start of file
        ROOT:
          # Comment on child key
          CHILD_1: test
          # Multi-line comment
          # on child key
          CHILD_2: '''this'' is quoted'
          # This comment's text has special characters!
          CHILD_3: key 3
        "
    `);
});

Relates to #680