Standard metadata for AI services

[loleg]

The transition from API to MCP services in LLM-based infrastructure presents us, in my opinion, with a big challenge to redefine the value and authority of metadata. The same issues that we have with CSV rows and columns are now faced by anyone trying to process & extract structured data. Except that where we had the odd misformatted NULL value, today we are dealing with the perfidy of stochastic correlation and other hallucinations. The development of Reasoning models (RLMs) is very exciting, and I hope that in this area (not to mention RAGs, MCPs etc.) the idea of a Data Package is still relevant and practical.

Could the Open Data Editor and similar projects drive future versions of Frictionless Data in this direction?

(This point was raised during a Q&A with the ODE team)

[loleg]

A couple of viewpoints from the www (NB: I am not affiliated with either project)

"Structured outputs guarantee a model returns a response that matches your exact schema" -- Hugging Face guide

"LLMs don’t have to speak in riddles. With the right tools, they can speak structured data fluently." -- Jennifer D'Souza, Ph.D.

"The debate about tokenization flaws is increasingly outdated. What really matters is how LLMs use structured data to improve accuracy, reduce hallucinations, and enhance decision-making." -- Schema App blog