Feature/schema references

[mattwahner]

Hello!

I am looking to add in functionality for schema references in this pull request. Specifically, I noticed a PR has already been created for this, PR#92, but seems to have lost traction for getting merged in. Although there are still a few changes I plan to make to this pull request before considering it ready to be pulled in, I wanted to get a pulse check into the likelihood of this PR getting merged, especially as compared to PR#92. Things I still am looking to change include:

• Caching some of the API calls I have added into the register function.
• Write integration tests.
• Overall code cleanup.

This PR is intended to only add schema references to AVRO. Protobuf namely seems to be giving me issues while encoding nested structures, and seems to not be in a state of feature parity with AVRO. Likely, more fixes will be need to be addressed before including schema reference functionality to other schema types.

Here is an example of how client code would look like while interfacing with these changes:

const { SchemaRegistry, SchemaType } = require('@kafkajs/confluent-schema-registry');
const dotenv = require('dotenv');
dotenv.config();

const schemaRegistry = new SchemaRegistry({ host: process.env.REGISTRY_HOST });

const cookieSchema = JSON.stringify({
    type: "record",
    name: "Cookie",
    namespace: "com.food",
    fields: [
        { name: "name", type: "string" },
        { name: "calories", type: "int" }
    ]
});

const cakeSchema = JSON.stringify({
    type: "record",
    name: "Cake",
    namespace: "com.food",
    fields: [
        { name: "name", type: "string" },
        { name: "size", type: "int" }
    ]
});

const genericFoodSchema = JSON.stringify({
    type: 'record',
    name: 'GenericFood',
    namespace: 'com.food',
    fields: [
        { name: 'food', type: ['Cookie', 'Cake'] }
    ]
});

const main = async () => {
    const { id: cookieId } = await schemaRegistry.register({
        type: SchemaType.AVRO,
        schema: cookieSchema
    });

    const { id: cakeId } = await schemaRegistry.register({
        type: SchemaType.AVRO,
        schema: cakeSchema
    });

    const { id: genericFoodId } = await schemaRegistry.register({
        type: SchemaType.AVRO,
        schema: genericFoodSchema
    }, {
        referenceSchemaIds: [
            cookieId,
            cakeId
        ]
    });

    const encodedPayload = await schemaRegistry.encode(genericFoodId, {
        food: {
            'com.food.Cookie': {
                name :'BigOlCookie',
                calories: 1000
            }
        }
    });

    const decodedPayload = await schemaRegistry.decode(encodedPayload);
    console.log('calories', decodedPayload.food['com.food.Cookie'].calories);
};

main();

[brinchj]

I've started a discussion around references in the related issue here: #82

I think we are a bunch who are interested in this and I think we could team up and produce the changes needed together in uniform 👍