type index uses the wrong namespace

[melvincarvalho]

As reported here #995 (comment)

@prefix solid: <https://www.w3.org/ns/solid/terms#>.
<> a solid:TypeIndex

Is being produced in some type indexes.

My own is fine : https://melvin.solid.community/settings/publicTypeIndex.ttl

@prefix solid: <http://www.w3.org/ns/solid/terms#>.

But I can confirm that more recent accounts have the incorrect https namespace. This is a major bug, and we need to urgently find the source.

[melvincarvalho]

Unfortunately this also means that we are possibly going to have to go into the server and change users' files in order to make the system work.

This is the right thing to do on solid.community, imho. Inrupt is going to have to decide whether they wish to make this decision. I would recommend yes! There may be some workarounds, but I cant think of one at the moment.

cc : @timbl @johnbizguy

Hopefully not too many accounts are affected. First step tho is to work out what is causing this, and how much of an issue it is.

[csarven]

I'd agree that it is quite likely a typo. Perhaps @dmitrizagidulin can quickly confirm that - perhaps more importantly, if there was a reason to be different than the http use of the ns.

https is also used in privateTypeIndex.ttl

re:

git log --follow -p default-templates/new-account/settings/publicTypeIndex.ttl
git log --follow -p default-templates/new-account/settings/privateTypeIndex.ttl

they were always using https. So, @melvincarvalho , if your publicTypeIndex.ttl used http, it is unlikely that it came from the templates. A simple explanation may be that you've changed it manually at some point?

[melvincarvalho]

Looks like the files are here :

https://github.com/solid/node-solid-server/blob/master/default-templates/new-account/settings/privateTypeIndex.ttl

https://github.com/solid/node-solid-server/blob/master/default-templates/new-account/settings/publicTypeIndex.ttl

[melvincarvalho]

A simple explanation may be that you've changed it manually at some point?

Could well be! I can do some further analysis to confirm. That might actually make life a bit easier.

Do note : the canonical prefixes are clearly articulated in the vocab README

https://github.com/solid/vocab#created-for-solid

[melvincarvalho]

So the steps needed :

• Fix the templates
• Fix the tests
• Priority push to master
• Update servers code
• Update server user accounts, as appropriate

Anything else?

I'd like to get this fixed on solid.community asap or else apps will not work or work inconsistently.

[melvincarvalho]

As a stop gap I think I can manually edit the template files to fix the typo, and write a script to substitute the the right namespace in all ~/settings/ directories that I find.

Thoughts welcome!

[csarven]

Acknowledged. I don't doubt the http use at all - as it is by the far the majority of the uses in code/data out there. As I've mentioned elsewhere, it was always http to the best of knowledge. I recall discussing at some point (somewhere) but we decided to stick to http. The README, the base in the ns, type registries... came much later.

[melvincarvalho]

Quick 5 minute script to fix this issue :

https://gist.github.com/melvincarvalho/b6c7121f2c239c1878a26e9465fa1166

Im sure it can be improved upon.

Disclaimer : for guidance only, do not consider this to be production quality

[melvincarvalho]

I am now able to fix solid.community but have not done so yet, I'll coordinate this with @ericprud who I've run the plan by.

Only concern raised is that we dont want to be out of sync with whatever is on github.

@kjetilk is there a way we can get a patch into the master of 4.4.0? -- we're currently running 4.2.0 which is a side issue, but would like to upgrade.

[melvincarvalho]

So, I think this is quite a tricky one to fix. @ericprud has sensibly suggested to have any changes in github. What I'd also like to do is keep an audit trail of the steps. So I've done a few things.

• I've started an org for solid.community. Anyone that would like to help out with the pod please ask here to be added to the org.

• This repo will contain any changes we make. In this case we will add a patch to the templates so that solid terms goes from https -> http for new users.

• I've created a very rough scripts repo that will contain scripts that perform fixes and help maintain a pod. The first one is fixsolidterms.sh. It's very raw at the moment, but I've tried it on my personal pod and it works.

• What I'm currently worried about is whether or not special characters in user names could be an issue. I'll do some more testing before running it.

• I've created a wiki that we can hopefully update with useful information about the pod.

• One thing I would like to do is that every time we are forced to alter users' files, we can add an entry to the wiki saying what we did, and why we did it. I think audit and transparency are important to be aligned with solid's principles.

• Ideally, would like to align this with inrupt, because at the moment type indexes and apps that use them are likely to be broken. So it's going to be advantageous if we get a consistent app experience across pods, wrt finding data. Fortunately not too many apps (that I know of) do this right now, but in future they are likely to.

I hope this addresses as many concerns on this issue as I can, feedback welcome!

[kjetilk]

@melvincarvalho Yes, I think it this is something we can do a release off of master of, e.g. 4.4.1 once we have a PR for it.

Meeting week now, but I'll try to catch up on the bugs when I can.

[melvincarvalho]

OK, then let's try and align around 4.4.1. I'll put in a PR for the change. Should be quite small.

It can wait a few days unless there are some apps are breaking due to this.

More concern is changing the data on the server, which is kind of manual.

[kjetilk]

I'll put in a PR for the change.

Any chance you could do submit it soon, @melvincarvalho ? I'd like to get it into 5.0.0 too as soon as possible.

[melvincarvalho]

@kjetilk should be addressed in #1027

LGTM

[kjetilk]

I'll close this now, since it is fixed in the 5.0.0 beta, which is what I'm pursuing. If someone wants to backport it to 4.x, please do so, we can certainly make a release, but there is little focus on that now.

[melvincarvalho]

Thanks again for working on this PR.

I believe, there is a consensus to move solid.community to 4.4, in order to benefit from the new quota system, hopefully I can free up some time to help with that.

@kjetilk I thought the plan was to add this the 4.4 branch? Fixing type indexes is beneficial in order for apps to do the right thing.

If someone wants to backport it to 4.x, please do so, we can certainly make a release, but there is little focus on that now

OK, I'll give it a try! If you could let me know the release process, Ill see if im able to create a 4.4.1 release, then. Or just walk me through the steps and I'll go through them, as best I can.

Do you think it would also be nice for the community pod to coordinate with inrupt.net so that there is a consistent user and developer experience? I think this has generally been considered desirable. If so, do you have any info on what version inrupt.net is running (or how I can find out -- ie 4.3 or 4.4) and upgrade plans. Thanks in advance!

[kjetilk]

@melvincarvalho I'm just afraid to spread myself to thinly, as I tend to do that, so that's why I focused on 5.0.0, as I think everyone should go there ASAP, the "S" in ASAP isn't right now, as we are already spread to thinly :-) . I can certainly merge the PR and do the release, that's just a few buttons to press, though. But it is really great if you could make the PR, @melvincarvalho against master and make sure it passes Travis.

[melvincarvalho]

@kjetilk got it! Thanks, I'll give it a try.