Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(URGENT) XML not being read in from custom model #117

Open
opoudjis opened this issue Oct 24, 2024 · 6 comments
Open

(URGENT) XML not being read in from custom model #117

opoudjis opened this issue Oct 24, 2024 · 6 comments
Assignees
Labels
bug Something isn't working

Comments

@opoudjis
Copy link
Contributor

opoudjis commented Oct 24, 2024

Another blocker.

Consider the following code, which uses an externally defined class for content:

require "lutaml/model"
require "lutaml/model/xml_adapter/nokogiri_adapter"
require "relaton"
require "relaton/cli"

Lutaml::Model::Config.configure do |config|
  config.xml_adapter = Lutaml::Model::XmlAdapter::NokogiriAdapter
  config.yaml_adapter = Lutaml::Model::YamlAdapter::StandardYamlAdapter
  config.json_adapter = Lutaml::Model::JsonAdapter::StandardJsonAdapter
end

module SimpleModel
  class Bibdata < Lutaml::Model::Serializable
    # attribute :code, :string
    model ::RelatonBib::BibliographicItem
  end

  class Address < Lutaml::Model::Serializable
    attribute :street, :string, raw: true
    attribute :city, :string, raw: true
    attribute :bibdata, Bibdata

    yaml do
      map "street", to: :street
      map "city", to: :city
      map "bibdata", to: :bibdata, with: { from: :bibdata_from_yaml,
                                           to: :bibdata_to_yaml }
    end

    xml do
      root "address"
      map_element "street", to: :street # ,  with: { from: :street_from_xml, to: :street_to_xml }
      map_element "city", to: :city # , with: { from: :city_from_xml, to: :city_to_xml }
      map_element "bibdata", to: :bibdata, with: { from: :bibdata_from_xml,
                                                   to: :bibdata_to_xml }
    end

    def bibdata_from_yaml(model, value)
      value and !value.empty? or return
      model.bibdata = ::Relaton::Cli::YAMLConvertor.convert_single_file(value)
    end 

    def bibdata_to_yaml(model, doc)
      doc["bibdata"] = model.bibdata&.to_hash
    end 

    def bibdata_from_xml(model, node)
      model.bibdata = node # Relaton::Cli.parse_xml(node.content)
    end 

    def bibdata_to_xml(model, parent, doc)
      b = model.bibdata or return
      elem = b.to_xml(bibdata: true, date_format: :full)
      doc.add_element(parent, elem)
    end 

    def city_from_xml(model, node)
      model.city = node
    end

    def city_to_xml(model, parent, doc)
      doc.add_element(parent, "<city>#{model.city}</city>")
    end

    def street_from_xml(model, node)
      model.street = node
    end

    def street_to_xml(model, parent, doc)
      doc.add_element(parent, "<street>#{model.street}</street>")
    end
  end


  xml = <<~XML
      <address>
      <street>
          A <p>b</p> B <p>c</p> C
      </street>
      <bibdata type="collection" schema-version="v1.2.8">  <title type="title-main" format="text/plain" language="en">JCGM Collection 1</title>
      <docidentifier type="iso">JCGM 12345</docidentifier>
      <date type="created">
        <on>2020-01-01</on>
      </date>
      <date type="issued">
        <on>2020-01-01</on>
      </date>
      <edition>1</edition>
      <copyright>
        <from>2020</from>
        <owner>
          <organization>
            <name>International Organization for Standardization</name>
            <abbreviation>JCMG</abbreviation>
          </organization>
        </owner>
      </copyright>
    </bibdata>
      </address>
  XML

  yaml = <<~YAML
    street: A <a>N</a> B
    city: B <a>N</a> B
    bibdata:
      title:
        type: title-main
        language: en
        content: JCGM Collection 1
      type: collection
      docid:
        type: iso
        id: JCGM 12345
      edition: 1
      date:
        - type: created
          value: "2020"
        - type: issued
          value: "2020"
      copyright:
        owner:
          name: International Organization for Standardization
          abbreviation: JCMG
        from: "2020"
  YAML

  puts "YAML > YAML"
  sample = SimpleModel::Address.from_yaml(yaml).to_yaml
  puts sample
  puts "XML > YAML"
  sample = SimpleModel::Address.from_xml(xml).to_yaml
  puts sample
  puts "YAML > XML"
  sample = SimpleModel::Address.from_yaml(yaml).to_xml
  puts sample
  puts "XML > XML"
  sample = SimpleModel::Address.from_xml(xml).to_xml
  puts sample

And I am not going to come up with a simple test class instead of ::RelatonBib::BibliographicItem, because after three months of this, I'm not going to spend an extra couple of hours doing mockups.

The custom model is working fine for reading YAML, via ::Relaton::Cli::YAMLConvertor.convert_single_file(value). Both the YAML and the XML outputs work.

The custom model fails to read the XML, other than the schema-version attribute: all it outputs is:

<bibdata schema-version="v1.2.9"/>
bibdata:
  schema-version: v1.2.9

In Lutaml::Model::Serialize::ClassMethods#apply_mappings, doc.to_xml has bibdata in it intact:

"<metanorma-collection>\n  <bibdata type=\"collection\">\n    <title type=\"title-main\" format=\"text/plain\" language=\"en\">ISO Collection 1</title>\n    <docidentifier type=\"iso\">ISO 12345</docidentifier>\n    <date type=\"created\">\n      <on>2020-01-01</on>\n    </date>\n    <date type=\"issued\">\n      <on>2020-01-01</on>\n    </date>\n    <edition>1</edition>\n    <copyright>\n      <from>2020</from>\n      <owner>\n        <organization>\n          <name>International Organization for Standardization</name>\n          <abbreviation>ISO</abbreviation>\n        </organization>\n      </owner>\n    </copyright>\n  </bibdata>\n  <directive>\n    <key>documents-external</key>\n....

And the conversion to mapping_hash in

doc_hash = doc.parse_element(doc.root, self, :xml)

preserves the content:

{"text"=>["\n  ", "\n  ", "\n  "],
 "street"=>"\n      A <p>b</p> B <p>c</p> C\n  ",
 "bibdata"=>
  {"text"=>["  ", "\n  ", "\n  ", "\n  ", "\n  ", "\n  ", "\n"],
   "title"=>{"text"=>"JCGM Collection 1", "type"=>"title-main", "format"=>"text/plain", "language"=>"en"},
   "docidentifier"=>{"text"=>"JCGM 12345", "type"=>"iso"},
   "date"=>
    [{"text"=>["\n    ", "\n  "], "on"=>{"text"=>"2020-01-01"}, "type"=>"created"},
     {"text"=>["\n    ", "\n  "], "on"=>{"text"=>"2020-01-01"}, "type"=>"issued"}],
   "edition"=>{"text"=>"1"},
   "copyright"=>
    {"text"=>["\n    ", "\n    ", "\n  "],
     "from"=>{"text"=>"2020"},
     "owner"=>
      {"text"=>["\n      ", "\n    "],
       "organization"=>
        {"text"=>["\n        ", "\n        ", "\n      "],
         "name"=>{"text"=>"International Organization for Standardization"},
         "abbreviation"=>{"text"=>"JCMG"}}}},
   "type"=>"collection",
   "schema-version"=>"v1.2.8"}}

I sure this is going wrong because of the foundational problem with mapping_hash which I have pointed out, that it does not cope with mixed content, or with leading whitespace. The complication here is that this is not happening when writing XML, but when reading XML: mapping_hash preserves the information ostensibly from the foregoing, but just doesn't pass it on to serialisation though apply_mappings(doc_hash, format).

In fact, when I make

    xml do
      root "address", mixed: true

which should be coping with mixed content, not only does XML > YAML fail to output the bibdata, but XML > XML crashes:

/Users/nickn/.rbenv/versions/3.3.4/lib/ruby/gems/3.3.0/gems/lutaml-model-0.3.14/lib/lutaml/model/xml_adapter/xml_document.rb:142:in `add_to_xml': undefined method `new' for nil (NoMethodError)

            options[:mapper_class].new.send(rule.custom_methods[:to], element, xml.parent, xml)
                                  ^^^^
	from /Users/nickn/.rbenv/versions/3.3.4/lib/ruby/gems/3.3.0/gems/lutaml-model-0.3.14/lib/lutaml/model/xml_adapter/nokogiri_adapter.rb:83:in `block (2 levels) in build_ordered_element'
@opoudjis opoudjis added the bug Something isn't working label Oct 24, 2024
@ronaldtse ronaldtse changed the title XML not being read in from custom model (URGENT) XML not being read in from custom model Oct 25, 2024
@ronaldtse
Copy link
Contributor

@HassanAkbar @suleman-uzair has this PR fixed this issue?

#119

@HassanAkbar
Copy link
Member

@ronaldtse

@HassanAkbar @suleman-uzair has this PR fixed this issue?

#119

#119 will not fix this issue; it was for a separate issue that occurred when using custom models with the render_default check.

@ronaldtse
Copy link
Contributor

Got it.

@ronaldtse
Copy link
Contributor

@HassanAkbar so #126 fixes part of this problem but not fully?

@HassanAkbar
Copy link
Member

@ronaldtse It fixes the crash and adds an internal representation of the node to the mapping hash. So I think this problem is fully solved by #126.

@opoudjis Is there something else needed for this ticket?

@ronaldtse
Copy link
Contributor

The latest gem is released for this.

@opoudjis could you please help try if this works? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants