Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom logical types get omitted from the classpath #564

Open
chadparry opened this issue Nov 20, 2024 · 1 comment · May be fixed by #565
Open

Custom logical types get omitted from the classpath #564

chadparry opened this issue Nov 20, 2024 · 1 comment · May be fixed by #565

Comments

@chadparry
Copy link

chadparry commented Nov 20, 2024 •
edited
Loading

The SchemaAssistant tracks all the jars that the generated code will need on the classpath, including the classes for all logical types. However, it neglects to track the classes that represent logical types derived from fixed types. Whenever a logical type is represented by a class not in the Java SDK and not in the same jar as its Conversion class, compilation will fail because the class can't be found.

This is an example of a runtime error in the logs.

/tmp/generated1750127073899966702/com/linkedin/avro/fastserde/generated/serialization/AVRO_1_11/ExerciseLogicalTypes_GenericSerializer_1478679821.java:33: error: cannot access UID
        convertedValue0 = Conversions.convertToRawType(convertedValue0, this.logicalTypeSchema__692588075, this.logicalTypeSchema__692588075 .getLogicalType(), this.conversion_uid);
                                     ^
  class file for org.example.logging.UID not found
Note: /tmp/generated1750127073899966702/com/linkedin/avro/fastserde/generated/serialization/AVRO_1_11/ExerciseLogicalTypes_GenericSerializer_1478679821.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
1 error
2024-11-19 17:48:56.752 [ERROR] 1885108 --- [ompile-thread-1] m.linkedin.avro.fastserde.FastSerdeCache : Serializer class instantiation exception
@chadparry chadparry linked a pull request Nov 20, 2024 that will close this issue
Open
@chadparry
Copy link
Author

As I look at this more, I suspect the failure is not confined to fixed types. My use case just happened to be a fixed type, which follows a different code path through SchemaAssistant.classFromSchema. But I believe this error could be reproduced for any logical type where the Java converted type and the Conversion class are in separate jars from each other. If they happen to be in the same jar, which is frequently the case, then the scan for import statements will find the Conversion class and pull in both classes. The attached fix will still resolve the problem for all logical types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support custom logical types derived from fixed types chadparry/avro-util
1 participant
@chadparry