Schedule job to update claimed artifacts

[adpi2]

Motivation

Some artifacts are published without the scmInfo field in their pom files which we use to map artifacts to github repositories.

To handle these we have a scaladex-contrib project in which library authors can claim that they own some artifacts.

We need to run a scheduled job to take this information into account and update artifacts that are claimed.

Identified Obstacles

Currently artifacts with no project (github repository) are not indexed. We now need to index them so that the scheduler can find them in the database.

Implementation guidelines

1. Index artifacts with no associated projects
2. Add a method in LocalStorage that can load the claims.json file from the scaladex-contrib folder.
3. Add methods in SchedulerDatabase to load all the claimed artifacts and update their corresponding projects.
4. Schedule a job in the SchedulerService to update the claimed artifacts.

As a bonus the scheduler can also fetch the scaladex-contrib main branch from Github.

Expectations

1. The new methods in LocalStorage and SchedulerDatabase should be tested.

Tasks

• Index artifacts with no associated projects
• Add a method in LocalStorage that can load the claims.json file from the scaladex-contrib folder.
• Add methods in SchedulerDatabase to load all the claimed artifacts and update their corresponding projects.
• Schedule a job in the SchedulerService to update the claimed artifacts.

[jyoo980]

I can work on indexing artifacts with no associated projects

[jyoo980]

@adpi2

Quick question for this, it looks like a number of methods in Artifact.scala (and beyond!) operate on the assumption that projectRef is always defined, e.g.,

scaladex/modules/core/shared/src/main/scala/scaladex/core/model/Artifact.scala

Lines 38 to 43 in 22f1672

	private def artifactHttpPath: String = s"/${projectRef.organization}/${projectRef.repository}/$artifactName"
	val mavenReference: Artifact.MavenReference = Artifact.MavenReference(groupId.value, artifactId, version.encode)
	def release: Release =
	Release(projectRef.organization, projectRef.repository, platform, language, version, releaseDate)

But we know this is no longer going to be the case. Do you have any suggestions on how this should be handled? Maybe for the url-specific ones, we can point to a 404 page (if one exists)

[adpi2]

Those methods should probably return Option. They are called in the twirl files (the front-end generator) to create links between pages. So then we need to deal with the None case. One way is to not display the <a> element if the url is None:

@for(url <- artifact.fullHttpUrl) { <a href="@url"> ... </a>}

By the way, I think many of those methods will soon disappear because of the new project page we are currently working on with @mlachkar. I am going to work on it today. It should be merged later this week or maybe next week.

[jyoo980]

Sounds good, I'll wait :)