Integrating collection renderer from ActionView (#43)

# Goal
We should be able to effectively render collection of data and be able to cache it.

## Problem statement
Previously, we have used following code to render collection:
```ruby
pb.cache!(@account.offers_cache_key, expires_in: 12.hours) do
  pb.featured_offers @featured_offers, partial: "offer", as: :offer

  pb.normal_offers @normal_offers, partial: "offer", as: :offer
end
```

But we need to introduce fragment caching to offers, since list cache has high miss rate. This is how implementing fragment caching in pbbuilder should look like:

```ruby
pb.cache!(@account.offers_cache_key, expires_in: 12.hours) do
  pb.featured_offers partial: "offer", as: offer, collection: @featured_offers, cached: true
  pb.normal_offers partial: "offer", as: :offer, collection: @normal_offers, cached: true
end
```

This syntax is heavily inspired by jbuilder. Here is list of examples from that gem that uses cached:  and collection: attributes.
```ruby
json.partial! partial: 'posts/post', collection: @posts, as: :post
```
```ruby
json.partial! 'posts/post', collection: @posts, as: :post
```
```ruby
json.partial! "post", collection: @posts, cached: true, as: :post
```
```ruby
json.array! @posts, partial: "post", as: :post, cached: true
```
```ruby
json.array! @posts, partial: "posts/post", as: :post, cached: true
```
```ruby
json.comments @post.comments, partial: "comments/comment", as: :comment, cached:true
```

## Proposed solution

In this PR, we're implementing more effective  rendering of collection with a help of `ActiveView::CollectionRenderer`. Support for caching of partial is currently not in the scope, but this will help implement caching and caching digest later.

Following syntax should be supported:
```ruby
pb.friends partial: "racers/racer", as: :racer, collection: @Racers
```

```ruby
pb.friends "racers/racer", as: :racer, collection: @Racers
```

Previous syntax works as it used to before. These will remain unchanged and will not use collection renderer (at least for now)
```ruby
pb.partial! @racer, racer: Racer.new(123, "Chris Harris", friends)
```
```ruby
pb.friends @friends, partial: "racers/racer", as: :racer
```

## TODO
- [x] Add documentation for this change
- [ ] Check performance difference between Collection Renderer and our approach. (without cache for now)
- [x] It's using PartialRenderer with rails 7 now - this seems like a bug, it should use CollectionRenderer.

## Prior work
Some inspiration for this PR could be found here:
* rails/jbuilder#501
* https://github.com/rails/rails/blob/main/actionview/lib/action_view/renderer/partial_renderer/collection_caching.rb#L20
* https://github.com/rails/rails/tree/main/actionview collection_renderer
* https://github.com/rails/rails/blob/main/actionview/lib/action_view/renderer/partial_renderer/collection_caching.rb#L20

Author: Stanislav (Stas) Katkov

.github/workflows/test.yml

@@ -13,7 +13,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        ruby: ["2.7", "3.0", "3.1", "3.2"]
+        ruby: ["3.0", "3.1", "3.2", "3.3"]
 
         gemfile: [ "rails_6_1", "rails_7_0"]
         experimental: [false]

.ruby-version

@@ -6,6 +6,6 @@ appraise "rails-7-0" do
   gem "rails", "~> 7.0.0"
 end
 
-appraise "rails-head" do
-  gem "rails", github: "rails/rails", branch: "main"
-end
+appraise "rails-7-1" do
+  gem "rails", "~> 7.1.0"
+end

Appraisals

@@ -7,3 +7,5 @@ gemspec
 
 gem "rake"
 gem "appraisal"
+
+gem "ruby-lsp"

Gemfile

@@ -1,21 +1,26 @@
 # Pbbuilder
 PBBuilder generates [Protobuf](https://developers.google.com/protocol-buffers) Messages with a simple DSL similar to [JBuilder](https://rubygems.org/gems/jbuilder) gem.
 
-## Compatibility
-We don't aim to have 100% compatibility with jbuilder gem, but we closely follow jbuilder's API design. 
+
+At least Rails 6.1 is required.
+
+## Compatibility with jBuilder
+We don't aim to have 100% compitability and coverage with jbuilder gem, but we closely follow jbuilder's API design to maintain familiarity.
 
 | | Jbuilder | Pbbuilder |
 |---|---|---|
 |  set! | ✅ | ✅ |
 |  cache! | ✅ | ✅ |
 |  cache_if! | ✅ | ✅ |
 | cache_root! | ✅|  |
+| collection cache | ✅|  |
 | extract! | ✅ | ✅ |
 | merge! | ✅ | ✅ |
-| deep_format_keys! | ✅ |  |
 | child! | ✅ |  |
 | array! | ✅ |  |
-| ignore_nil! | ✅ |  |
+| .call | ✅ |  |
+
+Due to protobuf message implementation, there is absolutely no need to implement support for `deep_format_keys!`, `key_format!`, `key_format`, `deep_format_keys`, `ignore_nil!`, `ignore_nil!`, `nil`. So those would never be added.
 
 ## Usage
 The main difference is that it can use introspection to figure out what kind of protobuf message it needs to create.
@@ -76,8 +81,26 @@ Here is way to use partials with collection while passing a variable to it
 pb.accounts @accounts, partial: "account", as: account
 ```
 
+## Collections (or Arrays)
+There are two different methods to render a collection. One that uses ActiveView::CollectionRenderer
+```ruby
+pb.friends partial: "racers/racer", as: :racer, collection: @racers
+```
+
+```ruby
+pb.friends "racers/racer", as: :racer, collection: @racers
+```
+
+And there are other ways, that don't use Collection Renderer (not very effective probably)
+```ruby
+pb.partial! @racer, racer: Racer.new(123, "Chris Harris", friends)
+```
+```ruby
+pb.friends @friends, partial: "racers/racer", as: :racer
+```
+
 ### Caching
-Fragment caching is supported, it uses Rails.cache and works like caching in HTML templates:
+it uses Rails.cache and works like caching in HTML templates:
 
 ```
 pb.cache! "cache-key", expires_in: 10.minutes do
@@ -93,6 +116,8 @@ pb.cache_if! !admin?, "cache-key", expires_in: 10.minutes do
 end
 ```
 
+Fragment caching support is currently in the works.
+
 ## Installation
 Add this line to your application's Gemfile:
 

README.md

@@ -0,0 +1,9 @@
+# This file was generated by Appraisal
+
+source "https://rubygems.org"
+
+gem "rake"
+gem "appraisal"
+gem "rails", "~> 6.0"
+
+gemspec path: "../"

gemfiles/rails_6.gemfile

@@ -4,6 +4,7 @@ source "https://rubygems.org"
 
 gem "rake"
 gem "appraisal"
+gem "ruby-lsp"
 gem "rails", "~> 6.1.0"
 
 gemspec path: "../"

gemfiles/rails_6_1.gemfile

@@ -0,0 +1,9 @@
+# This file was generated by Appraisal
+
+source "https://rubygems.org"
+
+gem "rake"
+gem "appraisal"
+gem "rails", "~> 7.0"
+
+gemspec path: "../"

gemfiles/rails_7.gemfile

@@ -4,6 +4,7 @@ source "https://rubygems.org"
 
 gem "rake"
 gem "appraisal"
+gem "ruby-lsp"
 gem "rails", "~> 7.0.0"
 
 gemspec path: "../"

gemfiles/rails_7_0.gemfile

@@ -0,0 +1,10 @@
+# This file was generated by Appraisal
+
+source "https://rubygems.org"
+
+gem "rake"
+gem "appraisal"
+gem "ruby-lsp"
+gem "rails", "~> 7.1.0"
+
+gemspec path: "../"

gemfiles/rails_7_1.gemfile

@@ -40,42 +40,49 @@ def method_missing(...)
     set!(...)
   end
 
+  def attributes!
+    @message.to_h
+  end
+
   def respond_to_missing?(field)
-    !!@message.class.descriptor.lookup(field.to_s)
+    !!_descriptor_for_field(field)
   end
 
   def set!(field, *args, &block)
     name = field.to_s
-    descriptor = @message.class.descriptor.lookup(name)
+    descriptor = _descriptor_for_field(name)
     ::Kernel.raise ::ArgumentError, "Unknown field #{name}" if descriptor.nil?
 
-    if block
+    if ::Kernel.block_given?
       ::Kernel.raise ::ArgumentError, "can't pass block to non-message field" unless descriptor.type == :message
 
       if descriptor.label == :repeated
         # pb.field @array { |element| pb.name element.name }
         ::Kernel.raise ::ArgumentError, "wrong number of arguments #{args.length} (expected 1)" unless args.length == 1
         collection = args.first
         _append_repeated(name, descriptor, collection, &block)
-        return
+      else
+        # pb.field { pb.name "hello" }
+        ::Kernel.raise ::ArgumentError, "wrong number of arguments (expected 0)" unless args.empty?
+        message = (@message[name] ||= _new_message_from_descriptor(descriptor))
+        _scope(message, &block)
       end
-
-      ::Kernel.raise ::ArgumentError, "wrong number of arguments (expected 0)" unless args.empty?
-      # pb.field { pb.name "hello" }
-      message = (@message[name] ||= _new_message_from_descriptor(descriptor))
-      _scope(message, &block)
     elsif args.length == 1
       arg = args.first
       if descriptor.label == :repeated
         if arg.respond_to?(:to_hash)
           # pb.fields {"one" => "two"}
-          arg.to_hash.each do |k, v|
-            @message[name][k] = v
-          end
-        elsif arg.respond_to?(:to_ary)
+          arg.to_hash.each { |k, v| @message[name][k] = v }
+        elsif arg.respond_to?(:to_ary) && !descriptor.type.eql?(:message)
           # pb.fields ["one", "two"]
           # Using concat so it behaves the same as _append_repeated
+
           @message[name].concat arg.to_ary
+        elsif arg.respond_to?(:to_ary) && descriptor.type.eql?(:message)
+          # pb.friends [Person.new(name: "Johnny Test"), Person.new(name: "Max Verstappen")]
+          # Concat another Protobuf message into parent Protobuf message (not ary of strings)
+
+          args.flatten.each {|obj| @message[name].push descriptor.subtype.msgclass.new(obj)}
         else
           # pb.fields "one"
           @message[name].push arg
@@ -103,10 +110,7 @@ def set!(field, *args, &block)
   end
 
   def extract!(element, *args)
-    args.each do |arg|
-      value = element.send(arg)
-      @message[arg.to_s] = value
-    end
+    args.each { |arg| @message[arg.to_s] = element.send(arg) }
   end
 
   # Merges object into a protobuf message, mainly used for caching.
@@ -115,11 +119,11 @@ def extract!(element, *args)
   def merge!(object)
     ::Kernel.raise Pbbuilder::MergeError.build(target!, object) unless object.class == ::Hash
 
-    object.each_key do |key|
-      next if object[key].respond_to?(:empty?) && object[key].empty?
+    object.each do |key, value|
+      next if value.respond_to?(:empty?) && value.empty?
 
-      descriptor = @message.class.descriptor.lookup(key.to_s)
-      ::Kernel.raise ::ArgumentError, "Unknown field #{name}" if descriptor.nil?
+      descriptor = _descriptor_for_field(key)
+      ::Kernel.raise ::ArgumentError, "Unknown field #{key}" if descriptor.nil?
 
       if descriptor.label == :repeated
         # optional empty fields don't show up in @message object,
@@ -128,50 +132,26 @@ def merge!(object)
           @message[key.to_s] = _new_message_from_descriptor(descriptor)
         end
 
-        if object[key].respond_to?(:to_hash)
-          object[key].to_hash.each {|k, v| @message[key.to_s][k] = v}
-        elsif object[key].respond_to?(:to_ary)
-          elements = object[key].map do |obj|
+        if value.respond_to?(:to_hash)
+          value.to_hash.each {|k, v| @message[key.to_s][k] = v}
+        elsif value.respond_to?(:to_ary)
+          elements = value.map do |obj|
             descriptor.subtype ? descriptor.subtype.msgclass.new(obj) : obj
           end
 
           @message[key.to_s].replace(elements)
         end
       else
-        if object[key].class == ::String
+        if descriptor.type == :message
+          @message[key.to_s] = descriptor.subtype.msgclass.new(value)
+        else
           # pb.fields {"one" => "two"}
-          @message[key.to_s] = object[key]
-        elsif object[key].class == ::TrueClass || object[key].class == ::FalseClass
           # pb.boolean true || false
-          @message[key.to_s] = object[key]
-        elsif object[key].class == ::Array
           # pb.field_name do
           #    pb.tags ["ok", "cool"]
           # end
 
-          @message[key.to_s] = object[key]
-        elsif object[key].class == ::Hash
-          if @message[key.to_s].nil?
-            @message[key.to_s] = _new_message_from_descriptor(descriptor)
-          end
-
-          object[key].each do |k, v|
-            if object[key][k].respond_to?(:to_hash)
-              if @message[key.to_s][k.to_s].nil?
-                descriptor = @message[key.to_s].class.descriptor.lookup(k.to_s)
-                @message[key.to_s][k.to_s] = _new_message_from_descriptor(descriptor)
-              end
-
-              _scope(@message[key.to_s][k.to_s]) { self.merge!(object[key][k]) }
-            elsif object[key][k].respond_to?(:to_ary)
-              @message[key.to_s][k.to_s].replace object[key][k]
-            else
-              # Throws an error, if we try to merge nil object into empty value.
-              next if object[key][k].nil? && @message[key.to_s][k.to_s].nil?
-
-              @message[key.to_s][k.to_s] = object[key][k]
-            end
-          end
+          @message[key.to_s] = value
         end
       end
     end
@@ -184,6 +164,10 @@ def target!
 
   private
 
+  def _descriptor_for_field(field)
+    @message.class.descriptor.lookup(field.to_s)
+  end
+
   # Appends protobuf message with existing @message object
   #
   # @param name string

lib/pbbuilder.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require 'action_view/renderer/collection_renderer'
+
+class Pbbuilder
+  class CollectionRenderer < ::ActionView::CollectionRenderer # :nodoc:
+    def initialize(lookup_context, options, &scope)
+      super(lookup_context, options)
+      @scope = scope
+    end
+
+    private
+
+    def build_rendered_template(content, template, layout = nil)
+      super(content || pb.attributes!, template)
+    end
+
+    def build_rendered_collection(templates, _spacer)
+      pb.set!(field, templates.map(&:body))
+    end
+
+    def pb
+      @options[:locals].fetch(:pb)
+    end
+
+    def field
+      @options[:locals].fetch(:field).to_s
+    end
+  end
+end

lib/pbbuilder/collection_renderer.rb

@@ -1,9 +1,9 @@
 # frozen_string_literal: true
 
 Pbbuilder = Class.new(begin
-  require 'active_support/proxy_object'
-  ActiveSupport::ProxyObject
+    require 'active_support/proxy_object'
+    ActiveSupport::ProxyObject
 rescue LoadError
-  require 'active_support/basic_object'
-  ActiveSupport::BasicObject
+    require 'active_support/basic_object'
+    ActiveSupport::BasicObject
 end)

lib/pbbuilder/pbbuilder.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require 'pbbuilder/collection_renderer'
+
 # PbbuilderTemplate is an extension of Pbbuilder to be used as a Rails template
 # It adds support for partials.
 class PbbuilderTemplate < Pbbuilder
@@ -37,6 +39,30 @@ def set!(field, *args, **kwargs, &block)
         super(field, *args) do |element|
           _set_inline_partial(element, kwargs)
         end
+      elsif kwargs.has_key?(:collection) && kwargs.has_key?(:as)
+        # pb.friends partial: "racers/racer", as: :racer, collection: [Racer.new(1, "Johnny Test", []), Racer.new(2, "Max Verstappen", [])]
+        # collection renderer
+        options = kwargs.deep_dup
+
+        options.reverse_merge! locals: options.except(:partial, :as, :collection, :cached)
+        options.reverse_merge! ::PbbuilderTemplate.template_lookup_options
+
+        collection = options[:collection] || []
+        partial = options[:partial]
+        options[:locals].merge!(pb: self)
+        options[:locals].merge!(field: field)
+
+        if options.has_key?(:layout)
+          raise ::NotImplementedError, "The `:layout' option is not supported in collection rendering."
+        end
+  
+        if options.has_key?(:spacer_template)
+          raise ::NotImplementedError, "The `:spacer_template' option is not supported in collection rendering."
+        end
+
+        CollectionRenderer
+          .new(@context.lookup_context, options) { |&block| _scope(message[field.to_s],&block) }
+          .render_collection_with_partial(collection, partial, @context, nil)
       else
         # pb.best_friend partial: "person", person: @best_friend
         # Call set! as a submessage, passing in the kwargs as partial options