Add Annotator Trainability #244
Conversation
|
@tschaffter I can't request a review. Also I'm not sure why the |
| description: Whether the tool implements trainability | ||
| type: boolean | ||
| default: false | ||
| trainedAnnotatorStatus: |
There was a problem hiding this comment.
The term "annotator" refers to a specific type of tool. Prefer the term "tool" here.
There was a problem hiding this comment.
It's not efficient to query the entire Tool object just to know this status. I'd keep Tool for static information, and create a specific endpoint to get training status information with a light response.
| loaded: | ||
| description: Whether the annotator is using a trained model | ||
| type: boolean | ||
| dataset: |
There was a problem hiding this comment.
Where does this value come from?
| description: Information about whether the annotator is using a trained model, and which one it is | ||
| properties: | ||
| loaded: | ||
| description: Whether the annotator is using a trained model |
There was a problem hiding this comment.
I would be more descriptive here. For example, whether the trained model is ready to be used for inference. Also the "loading" status may be more related to inference that "training status". Consider moving this property to an "inference status" schema? What about introducing a ToolStatus - or ToolInferenceStatus - schemas? This "loaded" property could become ToolStatus.initialized = true or ToolStatus.ready = true. Here initialized or ready would be that in general the tool is ready to do what it's meant to do, i.e. generating predictions.
| @@ -0,0 +1,17 @@ | |||
| get: | |||
There was a problem hiding this comment.
Rename toolTrainingStatus or simply trainingStatus since we won't train anything else than tool.
There was a problem hiding this comment.
If the endpoint of this path will be /tool/trainingStatus so no need to include tool again in this schema name.
There was a problem hiding this comment.
What about /tool/training/status ?
There was a problem hiding this comment.
The other one will be /tool/initialization/ and /tool/initialization/status?
| @@ -0,0 +1,18 @@ | |||
| post: | |||
There was a problem hiding this comment.
Loading a trained model is a step of an initialization process. What about having a path initializeTraining and initializeInference?
| @@ -0,0 +1,16 @@ | |||
| post: | |||
There was a problem hiding this comment.
The endpoint will be /tool/train
|
@cascadianblue Please also describe
|
|
@cascadianblue Regarding the error in the workflow job |
|
Note to self: add a walkthrough of which endpoints to call and in which order |
`TrainedAnnotatorStatus` -> `TrainedToolStatus`
@cascadianblue Do we have this somewhere? I'll work on the training specification this week. |
My first attempt at creating endpoints / schemas for:
as well as reporting on the status of these two actions.