Support for Namespaces

[paule96]

What would you like to be added:

It would be nice to organize components/schema keys in namesspaces if the name allows is.
So if you have a open api schema like this:

openapi: "3.0.0"
info:
  version: 1.0.0
  title: Swagger Petstore
  license:
    name: MIT
servers:
  - url: http://petstore.swagger.io/v1
paths:
  /pets:
    get:
      summary: List all pets
      operationId: listPets
      tags:
        - pets
      parameters:
        - name: limit
          in: query
          description: How many items to return at one time (max 100)
          required: false
          schema:
            type: integer
            maximum: 100
            format: int32
      responses:
        '200':
          description: A paged array of pets
          headers:
            x-next:
              description: A link to the next page of responses
              schema:
                type: string
          content:
            application/json:    
              schema:
                $ref: "#/components/schemas/Pets"
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
    post:
      summary: Create a pet
      operationId: createPets
      tags:
        - pets
      responses:
        '201':
          description: Null response
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
  /pets/{petId}:
    get:
      summary: Info for a specific pet
      operationId: showPetById
      tags:
        - pets
      parameters:
        - name: petId
          in: path
          required: true
          description: The id of the pet to retrieve
          schema:
            type: string
      responses:
        '200':
          description: Expected response to a valid request
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/PetStore.Pet"
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
components:
  schemas:
    PetStore.Pet:
      type: object
      required:
        - id
        - name
      properties:
        id:
          type: integer
          format: int64
        name:
          type: string
        tag:
          type: string
    Pets:
      type: array
      maxItems: 100
      items:
        $ref: "#/components/schemas/PetStore.Pet"
    Error:
      type: object
      required:
        - code
        - message
      properties:
        code:
          type: integer
          format: int32
        message:
          type: string

You can see the Open API name of the object Pet is PetStore.Pet.
Because this alone is not a valid C# type the generator currently removes the . so the C# model right now will get the following name: PetStorePet.

This feature request should change the behavior in this case to produce the following:

• create the namespace PetStore
• create in the namespace PetStore the class Pet

This is currently a feature that would make a cleaner support for OpenApi documents that were produced from languages that use like C# a . as a namespace divider.
The problem with this approach is, that namespaces are not supported by the open API specification, so this would be some magic this library does extra.

Why is this needed:

Currently in AutoSkd you can define models to exclude. (this switch is currently not working, I already started to implement a fix)
The problem is, if you exclude a model, a user of the AutoSdk needs to provide this model. For example over global usings. So the generated code can compile.
But if you have for example something like: System.Text.Json.Nodes.JsonObject as a model type to exclude you have no way to compile the generated result. Because the type used in the generated models is:

SystemTextJsonNodesJsonObject

The dots are removed

Implementation idea

Currently in the code all symbols like that:

['_', '-', '.', ' ', '\\', '/', '[', ']']

are removed from any csharp name that will be generated.
I would remove on any of these replacement methods the ..

Currently also in the object SchemaContext the id property get's multiple times overridden. I would suggest to use here the unique marker from the open api doc instead of something that has only to do with the csharp name. (better separation of concerns)

Next, I would like to implement ClassData on the SchemaContext more precisely so it can fulfill the following needs:

• generate the logical namespace of the class it is representing (this means without the namespace that is maybe defined in the settings of the user)
• generate the class name based on the following code: Sanetize(context.Reference.Id.Split('.').Last())

Then the TypeData can use this information to successfully build a type with the full namespace (settings.Namespace + '.' +classData.Namespace).
If the model is on the list of the exclusion, then the namespace in the ClassData should be used.

@HavenDV what is your opinion about it?

[HavenDV]

Right now everything is in one namespace.
I was planning to do automatic splitting into namespaces based on tags a little later, where common models will be in .Shared namespace or just in global

I didn't fully understand your idea, but I fully support it if it will be implemented as a separate explicitly enabled option both through the console and through Source Generators

Right now model exclusion should work, there are tests here that check Exclude/Include, but maybe it doesn't cover all options

[paule96]

@HavenDV to the exclude topic:

you can write a test like that:

[TestMethod]
public async Task ExcludeModelsAsync()
{
    var compilation = CSharpCompilation.Create(
        "compilation",
        [
                SyntaxFactory.ParseSyntaxTree("[assembly: System.CLSCompliantAttribute(true)]"),
            ],
            await H.Generators.Tests.Extensions.LatestReferenceAssemblies.Net90.ResolveAsync(null, CancellationToken.None),
            new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));

    var openApiSpec = new CustomAdditionalText(
    "petstore.yaml",
    new H.Resource("petstore.yaml").AsString());
    openApiSpec.Options.Add("OpenApiSpecification", "true");
    var additionalFiles = new[] { openApiSpec }.ToImmutableArray<AdditionalText>();
    var optionsProvider = new AdditionalTextOptionsProvider(new()
    {
        [nameof(Settings.ExcludeModels)] = "Pet;",
        [nameof(Settings.Namespace)] = "TestNamespace",
        [nameof(Settings.ClassName)] = "PetClient",
    });

    var driver = CSharpGeneratorDriver.Create(
         new IIncrementalGenerator[] { new SdkGenerator() }
            .Select(GeneratorExtensions.AsSourceGenerator)
            .ToArray())
        .AddAdditionalTexts(additionalFiles);
    driver = driver.WithUpdatedAnalyzerConfigOptions(optionsProvider);

    driver.RunGeneratorsAndUpdateCompilation(compilation, out var outputCompilation, out var diagnostics);
    var generatedClasses = outputCompilation.SyntaxTrees
        .SelectMany(x => x.GetRoot().DescendantNodes())
        .OfType<NamespaceDeclarationSyntax>()
        .SelectMany(x => x.Members.OfType<ClassDeclarationSyntax>())
        .ToArray();
    generatedClasses.Where(c => c.Identifier.Text == "Pet").Should().BeEmpty();
}

it will fail.
Maybe I understand something wrong with the exclude or the implementation is broken.
The broken line for that was:
https://github.com/tryAGI/AutoSDK/blob/185dd5e2579b5bfe2a474038b0b4672fee96bff5/src/libs/AutoSDK/Sources/Data.cs#L115C8-L131C23

The problem is that the exclude of models should happen after the childs are loaded.
But as described above, it will not really help anybody to have the exclude feature alone, because the full type name of the reference to that model need to match :)

[HavenDV]

Yes, I checked, there is only one test about it using github OpenAPI spec and called "filtering.yaml" in tests:

"filtering.yaml" => new Dictionary<string, string>
            {
                // migrations/list-for-org
                ["build_property.AutoSDK_IncludeTags"] = "migrations",
                //["build_property.AutoSDK_IncludeTags"] = "actions;activity;apps;billing;checks;code-scanning;codes-of-conduct;emojis;dependabot;dependency-graph;gists;git;gitignore;issues;licenses;markdown;merge-queue;meta;migrations;oidc;orgs;packages;projects;pulls;rate-limit;reactions;repos;search;secret-scanning;teams;users;codespaces;copilot;security-advisories;interactions;classroom;desktop;enterprise-teams",
                //["build_property.AutoSDK_IncludeModels"] = "CreateEmbeddingRequest;CreateModerationResponse;Error;ErrorResponse;ListModelsResponse;Model;DeleteModelResponse;CreateCompletionRequest",
            },

it works well but it's about including

also possible you need to remove ';' symvol

[paule96]

@HavenDV for now I will stop investigating this issue. I made not really a working result over a week now.
But my work-in-progress branch can be find here.

https://github.com/paule96/AutoSDK/tree/exclude_models_fix

Sadly after over a week it's still not working.
But I guess I will not continue with this branch. Maybe I will continue at some day with a new idea.

[HavenDV]

I created a Draft PR based on your thread, not sure when I will do it but I think your work is useful. But ultimately I am not sure only about support for automatic namespace detection via dot in the model name, it sounds like some extension of the OpenAPI standard, maybe it could also be implemented with some duplication via some x-namespace properties of models? maybe with a preliminary pass through OpenAPI some CLI helper to create these x-namespace based on names with dot

[paule96]

@HavenDV yes ofc that's definitely a non open api standard implementation.
Because simply their are no namespaces in open api.
Maybe it would be helpful to define a namespace delimiter or something like that. (maybe even as an array)