Basic prep #1792
Basic prep #1792
Conversation
Implements Per Resource Events notifications in Node Solid Server. Implementation Notes: + Uses `--experimental-require-module` to load esm packages natively. Requires node > 22.0.0. Start scripts and test invocations have been appropriately modified. + NSS converts strings into an older streaming format which was not being detected by Express-PREP. Express-PREP was modified to ask the user if the body provided is a stream, thus circumventing this issue. + Notifications are triggered from a common middleware, which is invoked after the response has been succesfully sent. + Uses Express PREP supplied default template for notifications.
Add Solid/Activity Streams format notifications: + Provides notifictions in JSON-LD and Turtle. + Extends notifications to PUT and POST methods. + Add Event-ID header field to the response of a write method.
|
I do not understand why nyc is failing with EEXIST. Will need some help here as I am blind testing on Windows. |
nyc does not work with ESM called with require().
|
If I disable nyc (now replaced with c8), I get just one other error https://github.com/CxRes/node-solid-server/actions/runs/10639483327/job/29497511727#step:12:702 I am not sure if this error is an artifact of running tests in my own repo or actually a bug! |
Parent path is correctly determined only for non root resources. Parent notifications are only generated when resources have a parent.
Node 20.17.0 LTS supports require(ESM). Therefore, allowing current Node LTS as well.
When checking response `Content-Type` during integration tests, relaxed all media-type values to regex.
CxRes
force-pushed
the
basic-prep
branch
2 times, most recently
from
5c7f336 to
6355d9e
Compare
Picks the repository from where the PR branch exists.
|
Tests pass. Phew! I would need guidance for writing tests specific to notifications. Also need to check with reviewers if they are happy migrating to Node 20 (which is the active LTS). |
Well done.
I don't know what to say.
|
Fixes the following issues: + Linebreaks are not sent after boundaries in the same notification, instead they are sent at the start of notification. + DELETE triggers lastEvent only if the path for notification is the same as the one where events occured.
|
I added tests as well (coverage, though not 100%, covers all relevant functionality), which means that this PR should have been ready to merge. However, one surface test, which has absolutely nothing to do with this PR, fails! The failure only happens with the commit that adds tests. |
| @@ -1,3 +1,3 @@ | |||
| #!/usr/bin/env node | |||
| #!/usr/bin/env -S node --experimental-require-module | |||
There was a problem hiding this comment.
This is needed because the middlewares for PREP use ESM syntax and NSS is CJS. This magic flag allows importing ESM code in CJS code.
This will also prove generally useful, since it creates a path to migrate NSS to ESM one file at a time.
There was a problem hiding this comment.
There was a problem hiding this comment.
I understand the need to support your ESM middleware, but adding --experimental-require-module project-wide isn’t the best approach. Since it’s still experimental, it could introduce unpredictable behavior in different environments or future Node.js versions.
To ensure long-term stability and compatibility with the existing CJS modules, it’s better to bundle your libraries with a transpiler. This will let you use ESM without requiring the entire project to rely on an experimental flag
|
Hi @CxRes, during the call today you mentioned wanting a review. I'd be happy to try and help. Just a quick question—do you have an issue or a high-level overview tied to this, or a bit of context? |
|
Could this be a switch for NSS. For example Websocket uses |
|
@melvincarvalho Thanks for the offer! The high level overview are the specifications: https://cxres.github.io/prep/draft-gupta-httpbis-per-resource-events.html Re Switch: There is no switch right now. I don't think one is even needed, because everything is HTTP and unless the request has the specific header ( The other enhancement that I would like to investigate (not for this PR) is a way for the user to specify how to turn on and off PREP for a resource and globally for a POD or specify which media-types to use for notifications, maybe using description resource or ACLs. I don't want to rush you, but I would really like to merge the code soon, though, because I want to make the case for it working at the next IETF and those discussions have already started (and maybe even coax one or two developers to implement clients for it, which IMHO is very easy with the libraries I provide). This was meant to be ready at the last IETF, and as you are already aware, been held up for many months. If the code is not breaking anything, I am happy to make improvements over the winter. |
Thanks! Here's how it's handled for websockets—might be a similar approach for this. node-solid-server/bin/lib/options.js Line 141 in f5b2b5c |
|
This also needs a version bump, consistent with semver |
PREP can be disabled when the server is started with the `--no-prep` flag.
Bumping is the job of the maintainers ;-) ! |
|
Yay! Tests are passing again. Thanks for the flag tip @melvincarvalho! |
I agree. Since it is not a breaking change and impacts no one except those who want to experiment with it, we should implement it ASAP. Talk of future possibilities like HTTP/2 is a total red-herring and not relevant to adoption of the hard work that exists now. |
There was a problem hiding this comment.
I don't have a strong opinion on what features should be optional or available by default (for NSS). So, this is more of a reflection on the PR and current thread. Approving it strictly because it is based on a Solid CG work item; developed by that item's primary author; tests passing; there is no strong objection to not have this feature.
If more testing is necessary, it should be qualified. More concrete todos. For the time being, I'm not aware of other "seasonal" testing, maturing this (or any) feature over a full version, or even having it deployed widely before it can be part of NSS core / default. If there is a criteria along those, I'd appreciate pointers.
I'd just suggest that if this PR does not pose new privacy and/or security concerns whether outlined in NSS or in relevant specifications, then it is probably acceptable, and improvements can come in over time.
I suggest a quick middle ground. Merge the PR, and unless any major issues come up until the next NSS release, let it be there. So, that means in the mean time, folks can do tests against "live" servers, irrespective of the testing method (automated, semi-automated, or manual), and report back with success/fail. If there is a clear process already on releases, then best to ignore what I wrote and refer to that instead =)
What stops someone from running NSS behind a reverse proxy that handles HTTP/2? If the problem is with having PREP on by default, I suggest merging this PR without the default and creating a separate PR that makes it on by default, dealing with that detail with input from people who will update their deployments. |
|
@csarven , @bourgeoa Before approving, please take into account this limitation from the spec:
I strongly agree with this point. The assumption that the best place to get updates is directly from the resource is, in practice, problematic. It can lead to an excessive number of open connections without efficient multiplexing. A dedicated updater service is a far more scalable and efficient approach, as it avoids the overhead and connection limits that PREP introduces. I also recommend thoroughly reviewing the code and demonstrating a working example before moving forward with approval, as this hasn't been done yet. |
|
I'd just like to mention that one of Solid CG's positions on https://github.com/solid/solid-spec/blob/master/api-websockets.md can be found in https://solidproject.org/ED/protocol#notifications-protocol :
In fact, the Solid CG kicked off the Solid Notifications Panel to address the shortcomings with SOLID-WEBSOCKETS-API. One of the outputs was the Solid Notifications Protocol, and a number of channel types. (Aside: The Solid Notifications Protocol was listed as one of the input documents for the LWS WG charter.) With respect to WebSockets, the work on WebSocketChannel channel type that works part of Solid Notifications Protocol is what's deemed to be the way to approach WebSockets in Solid Protocol. And just to clarify, I mention all of this not because NSS (or its maintainers, we, anyone) "must" follow the above but that we/they should be mindful of the path that's been in the works for a number of years now by the CG. So, all things considered, there should be more energy put into adopting the most recent work while working on deprecating the older work. This shouldn't be a debate at this point. We're past that. If there is conflicting or stronger data, that can be shared (publicly linkable) and reviewed. That aside, Solid-PREP is part of the incubation work in the CG: https://solidproject.org/TR/ and it is not proposed to replace WebSockets in the Solid Protocol (AFAIK) but that it can be adopted by implementers (servers, applications..) that wish to support it in their products. Regarding:
As I understand it, minor clarification: the browser limits apply to per host, not domain.
Can we study those problems based on practice? As I understand it, Server-sent Events is one example where it uses the resource directly to get updates. But if some problematic practices are not known to WHATWG, perhaps it'd be best to bring up to them. |
|
@csarven Thanks for bringing your wealth of knowledge about Solid to provide this historic retrospective. I have also updated the topmost comment to explain the reason we need another notifications mechanism in NSS, along with a table comparing the various notification mechanisms available in Solid.
Thanks for the correction, I
That is why this PR is so important to merge. Having working servers imho is the best way to compare different approaches in the real world. |
PREP dependencies have been updated to the latest versions that use the newest HTTP Structured Fields specification RFC9651.
Just a quick note on the comparison table. It’s worth pointing out that WebSockets inherits its AuthN/AuthZ from HTTPS, since WSS is an HTTP upgrade protocol. So, the suggestion that it lacks these by design isn’t quite accurate. It’s more a matter of aligning it properly or fixing the bugs to ensure everything works as expected, security-wise. Given that we’re quite thinly spread at the moment, I think it’s important we focus on getting the current system fully functional and bug-free. |
|
Please add to the comparison table which versions of HTTP each column is suitable for. This limitation from the spec should also be included in the comparison, especially since NSS runs on HTTP/1.1:
|
@melvincarvalho, I see you keep claiming that NSS runs on HTTP/1.1 could you please clarify what stops one from deploying it behind a reverse proxy Nginx/HAproxy/Traefik/ you name it, which will handle the HTTP/2? |
Notifications on the parent container are now sent when the parent is root.
Notification emitted when a resource is created as a result of POST on a container has activity type set to `as:Add`.
Notification emitted when a resource is created as a result of POST on a container sets the `target` property to the `Location` header, consistent with the requirements of `as:Add`.
Notification emitted on the container when a resource in it is removed, sets the `origin` property to the location of the removed resource, consistent with the requirements of `as:Remove`.
Added a test to check that a PREP notification is correctly emitted when a container is created with POST.
This fixes "undefined" notifications being emitted when the requested notification format is not Solid PREP RDF.
`req.url` is not overwritten anymore when PUT is used to create a container. This allows subsequent middlewares to have access to the original request URL.
Added tests to check that PREP notifications are correctly emitted on a container, when a container inside it is created or deleted.
Added debugging statements for failed notifications. Also clarified that notification errors should not be passed on to Express for error handling, since the mutation that triggered the notification was already successful.
When the negotiated media-type for notifications is not RDF, the `location` property is correctly set to the `Location` header in the generated notifications. Express-PREP now handles this by default.
|
@elf-pavlik, just to clarify, NSS does run on HTTP/1.1. This is a fact, not a claim. Regarding reverse proxying for HTTP/2:
In short, reverse proxies don’t solve the issue. There will always be users who need NSS to remain lightweight and simple, running on HTTP/1.1 without the overhead of added layers, more complex installations, or potential issues. |
Just a quick note about versioning. 😊 While maintainers typically handle the final bump, semantic versioning does ask that a PR proposes the right version change, especially if there are any breaking changes or big shifts involved. In this case, if there’s any shift towards HTTP/2 or other significant updates, it’s probably worth suggesting a major version bump (moving to 6.x) to reflect that. API changes or anything that might affect existing users would definitely fall under that category. For stability, I’d suggest keeping the current 5.x branch free of these changes. Maybe experimental work could go in a new branch or 6.x? That way, we keep things smooth for users on 5.x who depend on everything working just as it does today. |
This doesn't matter for the Notification Receiver in a web browser since they will connect over HTTP/2
This needs more details, including the auto-renewal of certificates. Modern reverse proxies handle that automatically. Two complete setups would need to be compared side by side.
They don't need to enable PERP. If the argument is about having it on by default, I think a reasonable step, which I already suggested, would be to merge it with PREP disabled by default. At the same time, someone could provide deployment instructions, playbook, flake, compose file (you name it), which would make it easy to deploy it behind a reverse proxy and use config that enables PREP. Looking at https://github.com/nodeSolidServer/node-solid-server/graphs/contributors |
There was a problem hiding this comment.
This is a breaking change which makes the conformance tests to break
bourgeoa@DESKTOP-PIRJOCS:/mnt/d/github/specification-tests$ ./run.sh nss
Running tests on nss and reporting to /mnt/d/github/specification-tests/reports/nss
Using default tag: latest
latest: Pulling from solidproject/conformance-test-harness
Digest: sha256:7d07eab722244e416fd01b4af058f373988067c7612d6dd661f27bb5a0211f88
Status: Image is up to date for solidproject/conformance-test-harness:latest
docker.io/solidproject/conformance-test-harness:latest
RUNNING: docker run -i --rm -v /mnt/d/github/specification-tests/reports/nss:/reports --env-file=nss.env solidproject/conformance-test-harness --output=/reports --target=https://github.com/solid/conformance-test-harness/nss
Oct 26, 2024 4:40:27 PM org.jboss.logmanager.JBossLoggerFinder getLogger
ERROR: The LogManager accessed before the "java.util.logging.manager" system property was set to "org.jboss.logmanager.LogManager". Results may be unexpected.
Oct 26, 2024 4:40:28 PM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator 1.1.19-SNAPSHOT
Oct 26, 2024 4:40:28 PM io.quarkus.bootstrap.runner.Timing printStartupTime
INFO: testharness 1.1.19-SNAPSHOT on JVM (powered by Quarkus 3.8.2) started in 1.029s.
Oct 26, 2024 4:40:28 PM io.quarkus.bootstrap.runner.Timing printStartupTime
INFO: Profile prod activated.
Oct 26, 2024 4:40:28 PM io.quarkus.bootstrap.runner.Timing printStartupTime
INFO: Installed features: [cdi, config-yaml, hibernate-validator, logging-json, qute, smallrye-context-propagation]
__ ____ __ _____ ___ __ ____ ______
--/ __ \/ / / / _ | / _ \/ //_/ / / / __/
-/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2024-10-26 16:40:28,677 INFO [org.sol.tes.con.Config] (main) Sources: [https://solidproject.org/TR/protocol, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl, https://github.com/solid-contrib/specification-tests/blob/main/protocol/requirement-comments.ttl, https://solidproject.org/TR/wac, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/requirement-comments.ttl, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/wac-spec-additions.ttl]
2024-10-26 16:40:28,683 INFO [org.sol.tes.con.Config] (main) Path mappings: [https://github.com/solid-contrib/specification-tests/blob/main => file:///data]
2024-10-26 16:40:28,684 INFO [org.sol.tes.con.Config] (main) Output directory: /reports
2024-10-26 16:40:28,684 INFO [org.sol.tes.con.Config] (main) Subjects URL: file:/data/test-subjects.ttl
2024-10-26 16:40:28,684 INFO [org.sol.tes.con.Config] (main) Target server: https://github.com/solid/conformance-test-harness/nss
2024-10-26 16:40:28,684 INFO [org.sol.tes.con.Config] (main) Connect timeout: 5000
2024-10-26 16:40:28,685 INFO [org.sol.tes.con.Config] (main) Read timeout: 5000
2024-10-26 16:40:28,685 INFO [org.sol.tes.con.Config] (main) Max threads: 8
2024-10-26 16:40:28,685 INFO [org.sol.tes.con.Config] (main) Alice WebID: https://alice.solidcommunity.net:8443/profile/card#me
2024-10-26 16:40:28,685 INFO [org.sol.tes.con.Config] (main) Alice IDP: https://solidcommunity.net:8443/
2024-10-26 16:40:28,686 INFO [org.sol.tes.con.Config] (main) Bob WebID: https://bob.solidcommunity.net:8443/profile/card#me
2024-10-26 16:40:28,686 INFO [org.sol.tes.con.Config] (main) Bob IDP: https://solidcommunity.net:8443/
2024-10-26 16:40:28,687 INFO [org.sol.tes.con.Config] (main) Solid IdP: null
2024-10-26 16:40:28,687 INFO [org.sol.tes.con.Config] (main) Server root: null
2024-10-26 16:40:28,688 INFO [org.sol.tes.con.Config] (main) Test container: null
2024-10-26 16:40:28,688 INFO [org.sol.tes.con.Config] (main) Tolerable failures: null
2024-10-26 16:40:28,688 INFO [org.sol.tes.con.Config] (main) Allow self-signed: false
2024-10-26 16:40:28,769 INFO [org.sol.tes.ConformanceTestHarness] (main) Solid Specification Conformance Test Harness: 1.1.19-SNAPSHOT
2024-10-26 16:40:28,808 INFO [org.sol.tes.ConformanceTestHarness] (main) ===================== DISCOVER TESTS ========================
2024-10-26 16:40:28,808 INFO [org.sol.tes.uti.DataRepository] (main) Loading https://solidproject.org/TR/protocol with base https://solidproject.org/TR/protocol
2024-10-26 16:40:29,994 INFO [org.sol.tes.uti.DataRepository] (main) Loading file:/data/protocol/solid-protocol-test-manifest.ttl with base https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl
2024-10-26 16:40:30,039 INFO [org.sol.tes.uti.DataRepository] (main) Loading file:/data/protocol/requirement-comments.ttl with base https://github.com/solid-contrib/specification-tests/blob/main/protocol/requirement-comments.ttl
2024-10-26 16:40:30,045 INFO [org.sol.tes.uti.DataRepository] (main) Loading https://solidproject.org/TR/wac with base https://solidproject.org/TR/wac
2024-10-26 16:40:30,280 INFO [org.sol.tes.uti.DataRepository] (main) Loading file:/data/web-access-control/web-access-control-test-manifest.ttl with base https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl
2024-10-26 16:40:30,287 INFO [org.sol.tes.uti.DataRepository] (main) Loading file:/data/web-access-control/requirement-comments.ttl with base https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/requirement-comments.ttl
2024-10-26 16:40:30,289 INFO [org.sol.tes.uti.DataRepository] (main) Loading file:/data/web-access-control/wac-spec-additions.ttl with base https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/wac-spec-additions.ttl
2024-10-26 16:40:30,293 INFO [org.sol.tes.dis.TestSuiteDescription] (main) Test suite version: 0.0.19 2024-03-21
2024-10-26 16:40:30,300 INFO [org.sol.tes.ConformanceTestHarness] (main) ==== TEST CASES FOUND: 41 - [https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#if-none-match-asterisk, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#content-type-reject, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#slash-semantics-exclude, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#authentication-header, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#content-negotiation-turtle, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#content-negotiation-jsonld, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#content-negotiation-named-graphs, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#uri-assignment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#writing-resource-containment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#describedby-unique, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#method-not-allowed, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#post-uri-assignment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#slug-uri-assignment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#read-method-allow, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#delete-protect-nonempty-container, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#delete-remove-containment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#post-target-not-found, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-simple-requests, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-preflight-requests, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-access-control-headers, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-acao-vary, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-options, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-enumerate, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-accept-acah, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#read-method-support, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#acl-object-none, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#acl-object-access-to, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#acl-object-default, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#acl-object-access-to-default, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#protected-operation-acl-propagation, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-header-exists, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-user-access-direct, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-user-access-indirect, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-public-access-direct, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-public-access-indirect, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#read-access-bob, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#read-access-agent, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#read-access-public, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#write-access-agent, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#write-access-bob, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#write-access-public]
2024-10-26 16:40:30,316 INFO [org.sol.tes.con.TestSubject] (main) TestSubject https://github.com/solid/conformance-test-harness/nss
2024-10-26 16:40:30,317 INFO [org.sol.tes.ConformanceTestHarness] (main) ==== SKIP TAGS: [acp]
2024-10-26 16:40:30,317 INFO [org.sol.tes.ConformanceTestHarness] (main) ==== APPLY NAME FILTERS: null
2024-10-26 16:40:30,317 INFO [org.sol.tes.ConformanceTestHarness] (main) ==== APPLY STATUS FILTERS: null
2024-10-26 16:40:30,319 INFO [org.sol.tes.ConformanceTestHarness] (main) ==== FILTERED TEST CASES (41): [https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#if-none-match-asterisk, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#content-type-reject, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#slash-semantics-exclude, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#authentication-header, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#content-negotiation-turtle, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#content-negotiation-jsonld, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#content-negotiation-named-graphs, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#uri-assignment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#writing-resource-containment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#describedby-unique, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#method-not-allowed, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#post-uri-assignment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#slug-uri-assignment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#read-method-allow, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#delete-protect-nonempty-container, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#delete-remove-containment, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#post-target-not-found, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-simple-requests, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-preflight-requests, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-access-control-headers, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-acao-vary, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-options, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-enumerate, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#cors-accept-acah, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl#read-method-support, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#acl-object-none, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#acl-object-access-to, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#acl-object-default, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#acl-object-access-to-default, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#protected-operation-acl-propagation, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-header-exists, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-user-access-direct, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-user-access-indirect, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-public-access-direct, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#server-wac-allow-public-access-indirect, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#read-access-bob, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#read-access-agent, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#read-access-public, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#write-access-agent, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#write-access-bob, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl#write-access-public]
2024-10-26 16:40:30,328 INFO [org.sol.tes.ConformanceTestHarness] (main) ==== RUNNING TEST CASES (41): [file:///data/protocol/writing-resource/if-none-match-asterisk.feature, file:///data/protocol/writing-resource/content-type-reject.feature, file:///data/protocol/writing-resource/slash-semantics-exclude.feature, file:///data/protocol/authentication/header.feature, file:///data/protocol/content-negotiation/content-negotiation-turtle.feature, file:///data/protocol/content-negotiation/content-negotiation-jsonld.feature, file:///data/protocol/content-negotiation/content-negotiation-named-graphs.feature, file:///data/protocol/writing-resource/uri-assignment.feature, file:///data/protocol/writing-resource/containment.feature, file:///data/protocol/resources/describedby-unique.feature, file:///data/protocol/read-write-resource/method-not-allowed.feature, file:///data/protocol/read-write-resource/post-uri-assignment.feature, file:///data/protocol/read-write-resource/post-uri-assignment-slug.feature, file:///data/protocol/read-write-resource/read-method-allow.feature, file:///data/protocol/writing-resource/delete-protect-nonempty-container.feature, file:///data/protocol/writing-resource/delete-remove-containment.feature, file:///data/protocol/writing-resource/post-target-not-found.feature, file:///data/protocol/cors/simple-requests.feature, file:///data/protocol/cors/preflight-requests.feature, file:///data/protocol/cors/access-control-headers.feature, file:///data/protocol/cors/acao-vary.feature, file:///data/protocol/cors/preflight.feature, file:///data/protocol/cors/enumerate-headers.feature, file:///data/protocol/cors/accept-acah.feature, file:///data/protocol/read-write-resource/read-method-support.feature, file:///data/web-access-control/acl-object/container-none.feature, file:///data/web-access-control/acl-object/container-access-to.feature, file:///data/web-access-control/acl-object/container-default.feature, file:///data/web-access-control/acl-object/container-access-to-default.feature, file:///data/web-access-control/protected-operation/acl-propagation.feature, file:///data/web-access-control/wac-allow/header-exists.feature, file:///data/web-access-control/wac-allow/user-access-direct.feature, file:///data/web-access-control/wac-allow/user-access-indirect.feature, file:///data/web-access-control/wac-allow/public-access-direct.feature, file:///data/web-access-control/wac-allow/public-access-indirect.feature, file:///data/web-access-control/protected-operation/read-access-bob.feature, file:///data/web-access-control/protected-operation/read-access-agent.feature, file:///data/web-access-control/protected-operation/read-access-public.feature, file:///data/web-access-control/protected-operation/write-access-agent.feature, file:///data/web-access-control/protected-operation/write-access-bob.feature, file:///data/web-access-control/protected-operation/write-access-public.feature]
2024-10-26 16:40:30,328 INFO [org.sol.tes.ConformanceTestHarness] (main) ===================== REGISTER CLIENTS ========================
2024-10-26 16:40:30,923 INFO [org.sol.tes.htt.AuthManager] (main) Loaded WebID Document for [https://alice.solidcommunity.net:8443/profile/card#me]
2024-10-26 16:40:31,375 INFO [org.sol.tes.htt.AuthManager] (main) Login and get access token for alice: [https://alice.solidcommunity.net:8443/profile/card#me]
2024-10-26 16:40:31,935 DEBUG [org.sol.tes.htt.Client] (main) Request access token for alice using grant type authorization_code
2024-10-26 16:40:32,004 DEBUG [org.sol.tes.htt.Client] (main) request:
> POST https://solidcommunity.net:8443/token
> Accept: application/json
> Authorization: Basic ***NjM2E=
> Content-Type: application/x-www-form-urlencoded
> DPoP: ***6KQOaw
> User-Agent: Solid-Conformance-Test-Suite
2024-10-26 16:40:32,072 DEBUG [org.sol.tes.htt.Client] (main) response:
< HTTP_1_1 200
< accept-events:
< access-control-allow-credentials: true
< access-control-expose-headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Accept-Put, Updates-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate, MS-Author-Via, X-Powered-By
< allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
< connection: keep-alive
< content-length: 1744
< content-type: application/json; charset=utf-8
< date: Sat, 26 Oct 2024 16:40:32 GMT
< etag: W/"6d0-FofbfBQl1MWGri3bW+Wvqi/GXjc"
< keep-alive: timeout=5
< vary: Accept, Authorization, Origin
< x-powered-by: solid-server/5.7.11
{"access_token":"***T7XUbQ","token_type":"Bearer","expires_in":1209600,"refresh_token":"***addaba","id_token":"***2KrnOw"}
2024-10-26 16:40:32,103 INFO [org.sol.tes.htt.Client] (main) Access token for alice - expiration: NumericDate{1731170432 -> Nov 9, 2024, 4:40:32 PM UTC}
2024-10-26 16:40:32,495 INFO [org.sol.tes.htt.AuthManager] (main) Loaded WebID Document for [https://bob.solidcommunity.net:8443/profile/card#me]
2024-10-26 16:40:32,836 INFO [org.sol.tes.htt.AuthManager] (main) Login and get access token for bob: [https://bob.solidcommunity.net:8443/profile/card#me]
2024-10-26 16:40:33,345 DEBUG [org.sol.tes.htt.Client] (main) Request access token for bob using grant type authorization_code
2024-10-26 16:40:33,358 DEBUG [org.sol.tes.htt.Client] (main) request:
> POST https://solidcommunity.net:8443/token
> Accept: application/json
> Authorization: Basic ***NkYWI=
> Content-Type: application/x-www-form-urlencoded
> DPoP: ***qd0CxQ
> User-Agent: Solid-Conformance-Test-Suite
2024-10-26 16:40:33,428 DEBUG [org.sol.tes.htt.Client] (main) response:
< HTTP_1_1 200
< accept-events:
< access-control-allow-credentials: true
< access-control-expose-headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Accept-Put, Updates-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate, MS-Author-Via, X-Powered-By
< allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
< connection: keep-alive
< content-length: 1737
< content-type: application/json; charset=utf-8
< date: Sat, 26 Oct 2024 16:40:34 GMT
< etag: W/"6c9-nEbfxvewOnLE91AKYNo8XiF0hoQ"
< keep-alive: timeout=5
< vary: Accept, Authorization, Origin
< x-powered-by: solid-server/5.7.11
{"access_token":"***bzQBJw","token_type":"Bearer","expires_in":1209600,"refresh_token":"***0d5b50","id_token":"***o53CGw"}
2024-10-26 16:40:33,431 INFO [org.sol.tes.htt.Client] (main) Access token for bob - expiration: NumericDate{1731170434 -> Nov 9, 2024, 4:40:34 PM UTC}
2024-10-26 16:40:33,432 INFO [org.sol.tes.ConformanceTestHarness] (main) ===================== PREPARE SERVER ========================
2024-10-26 16:40:33,595 INFO [org.sol.tes.con.TestSubject] (main) Read WebID Document for [https://alice.solidcommunity.net:8443/profile/card#me]
2024-10-26 16:40:33,599 INFO [org.sol.tes.con.TestSubject] (main) Found [1] Pods, checking them...
2024-10-26 16:40:33,601 INFO [org.sol.tes.con.TestSubject] (main) Checking Pod [https://alice.solidcommunity.net:8443/]
2024-10-26 16:40:33,614 DEBUG [org.sol.tes.htt.Client] (main) request:
> HEAD https://alice.solidcommunity.net:8443/
> Authorization: DPoP ***T7XUbQ
> DPoP: ***5cPUtg
> User-Agent: Solid-Conformance-Test-Suite
2024-10-26 16:40:33,894 DEBUG [org.sol.tes.htt.Client] (main) response:
< HTTP_1_1 200
< accept-patch: text/n3, application/sparql-update, application/sparql-update-single-match
< accept-post: */*
< access-control-allow-credentials: true
< access-control-expose-headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Accept-Put, Updates-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate, MS-Author-Via, X-Powered-By
< allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
< connection: keep-alive
< content-length: 2
< content-type: text/turtle; charset=utf-8
< date: Sat, 26 Oct 2024 16:40:34 GMT
< etag: W/"2-nOO9QiTIwXgNtWtBJezz8kv3SLc"
< keep-alive: timeout=5
< link: <.acl>; rel="acl", <.meta>; rel="describedBy", <http://www.w3.org/ns/ldp#Container>; rel="type", <http://www.w3.org/ns/ldp#BasicContainer>; rel="type", <http://www.w3.org/ns/pim/space#Storage>; rel="type"
< ms-author-via: SPARQL
< set-cookie: nssidp.sid=s%3AG4iG0JcRPsrm0iExWW1LVaHYFXI-HkZX.4nnZ2o6ttAJAHZl%2FJSlZwL9dNwU4Kf3PXx3dER1ff9A; Domain=.solidcommunity.net; Path=/; Expires=Sun, 27 Oct 2024 16:40:34 GMT; HttpOnly; Secure
< updates-via: wss://alice.solidcommunity.net:8443
< vary: Accept, Authorization, Origin
< wac-allow: user="read write append control",public="read"
< x-powered-by: solid-server/5.7.11
2024-10-26 16:40:33,942 INFO [org.sol.tes.con.TestSubject] (main) Pod found: [https://alice.solidcommunity.net:8443/]
2024-10-26 16:40:33,942 INFO [org.sol.tes.con.TestSubject] (main) Test subject test container: https://alice.solidcommunity.net:8443/
2024-10-26 16:40:33,945 DEBUG [org.sol.tes.htt.Client] (main) request:
> HEAD https://alice.solidcommunity.net:8443/
> Authorization: DPoP ***T7XUbQ
> DPoP: ***aD7Xrg
> User-Agent: Solid-Conformance-Test-Suite
2024-10-26 16:40:34,092 DEBUG [org.sol.tes.htt.Client] (main) response:
< HTTP_1_1 200
< accept-patch: text/n3, application/sparql-update, application/sparql-update-single-match
< accept-post: */*
< access-control-allow-credentials: true
< access-control-expose-headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Accept-Put, Updates-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate, MS-Author-Via, X-Powered-By
< allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
< connection: keep-alive
< content-length: 2
< content-type: text/turtle; charset=utf-8
< date: Sat, 26 Oct 2024 16:40:34 GMT
< etag: W/"2-nOO9QiTIwXgNtWtBJezz8kv3SLc"
< keep-alive: timeout=5
< link: <.acl>; rel="acl", <.meta>; rel="describedBy", <http://www.w3.org/ns/ldp#Container>; rel="type", <http://www.w3.org/ns/ldp#BasicContainer>; rel="type", <http://www.w3.org/ns/pim/space#Storage>; rel="type"
< ms-author-via: SPARQL
< set-cookie: nssidp.sid=s%3Ahy-0xV3YDDswsdVbLKgD_ddZ3QBjJraC.yL3yRO4s8j8baz0QhzjYTIZSB4SwWW7oW6%2FYux0pj3I; Domain=.solidcommunity.net; Path=/; Expires=Sun, 27 Oct 2024 16:40:34 GMT; HttpOnly; Secure
< updates-via: wss://alice.solidcommunity.net:8443
< vary: Accept, Authorization, Origin
< wac-allow: user="read write append control",public="read"
< x-powered-by: solid-server/5.7.11
2024-10-26 16:40:34,105 DEBUG [org.sol.tes.htt.Client] (main) request:
> HEAD https://alice.solidcommunity.net:8443/.acl
> Authorization: DPoP ***T7XUbQ
> DPoP: ***iY7xJA
> User-Agent: Solid-Conformance-Test-Suite
2024-10-26 16:40:34,289 DEBUG [org.sol.tes.htt.Client] (main) response:
< HTTP_1_1 200
< accept-patch: text/n3, application/sparql-update, application/sparql-update-single-match
< accept-post: */*
< accept-put: */*
< access-control-allow-credentials: true
< access-control-expose-headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Accept-Put, Updates-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate, MS-Author-Via, X-Powered-By
< allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
< connection: keep-alive
< content-length: 2
< content-type: text/turtle; charset=utf-8
< date: Sat, 26 Oct 2024 16:40:35 GMT
< etag: W/"2-nOO9QiTIwXgNtWtBJezz8kv3SLc"
< keep-alive: timeout=5
< link: <.acl.acl>; rel="acl", <.acl.meta>; rel="describedBy", <http://www.w3.org/ns/ldp#Resource>; rel="type"
< ms-author-via: SPARQL
< set-cookie: nssidp.sid=s%3AeKZkawXzBEeeK-5kFavaYRCWWaDMJvVh.SE0MiU%2F6QIh3TqkpY0e1cJY1nmqNXScJG29g6p810%2BI; Domain=.solidcommunity.net; Path=/; Expires=Sun, 27 Oct 2024 16:40:35 GMT; HttpOnly; Secure
< updates-via: wss://alice.solidcommunity.net:8443
< vary: Accept, Authorization, Origin
< wac-allow: user="read write append control",public=""
< x-powered-by: solid-server/5.7.11
2024-10-26 16:40:34,291 INFO [org.sol.tes.con.TestSubject] (main) The Pod is using [WAC] for access control
2024-10-26 16:40:34,695 DEBUG [org.sol.tes.htt.Client] (main) request:
> PUT https://alice.solidcommunity.net:8443/W9ld9r/
> Authorization: DPoP ***T7XUbQ
> Content-Type: text/turtle
> DPoP: ***ZWiKPA
> Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"
> User-Agent: Solid-Conformance-Test-Suite
2024-10-26 16:40:34,776 DEBUG [org.sol.tes.htt.Client] (main) response:
< HTTP_1_1 201
< access-control-allow-credentials: true
< access-control-expose-headers: Authorization, User, Location, Link, Vary, Last-Modified, ETag, Accept-Patch, Accept-Post, Accept-Put, Updates-Via, Allow, WAC-Allow, Content-Length, WWW-Authenticate, MS-Author-Via, X-Powered-By
< allow: OPTIONS, HEAD, GET, PATCH, POST, PUT, DELETE
< connection: keep-alive
< content-length: 7
< content-type: text/plain; charset=utf-8
< date: Sat, 26 Oct 2024 16:40:35 GMT
< etag: W/"7-rM9AyJuqT6iOan/xHh+AW+7K/T8"
< event-id: f6YYiD
< keep-alive: timeout=5
< ms-author-via: SPARQL
< set-cookie: nssidp.sid=s%3AbHcO3oJxAfTWghBCP_0b-72KgFEZAi_l.dWkNuwsJWICkmo7waIN%2B5ld1pvnux3DGZvwYlu8F%2BL8; Domain=.solidcommunity.net; Path=/; Expires=Sun, 27 Oct 2024 16:40:35 GMT; HttpOnly; Secure
< vary: Accept, Authorization, Origin
< x-powered-by: solid-server/5.7.11
Created
2024-10-26 16:40:34,939 ERROR [org.sol.tes.Application] (main) Application failed: org.solid.testharness.utils.TestHarnessInitializationException: Failed to prepare server Caused by: org.solid.testharness.utils.TestHarnessException: Error response=404 trying to get content for https://alice.solidcommunity.net:8443/W9ld9r/
at org.solid.testharness.config.TestSubject.prepareServer(TestSubject.java:162)
at org.solid.testharness.config.TestSubject_ClientProxy.prepareServer(Unknown Source)
at org.solid.testharness.ConformanceTestHarness.setupTestHarness(ConformanceTestHarness.java:202)
at org.solid.testharness.ConformanceTestHarness.runTestSuites(ConformanceTestHarness.java:152)
at org.solid.testharness.ConformanceTestHarness_ClientProxy.runTestSuites(Unknown Source)
at org.solid.testharness.Application.run(Application.java:95)
at org.solid.testharness.Application_ClientProxy.run(Unknown Source)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:132)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
at org.solid.testharness.Application.main(Application.java:79)
Caused by: org.solid.testharness.utils.TestHarnessException: Error response=404 trying to get content for https://alice.solidcommunity.net:8443/W9ld9r/
at org.solid.testharness.http.SolidClientProvider.getContentAsTurtle(SolidClientProvider.java:162)
at org.solid.testharness.utils.SolidResourceProvider.getContentAsTurtle(SolidResourceProvider.java:82)
at org.solid.testharness.config.TestSubject.prepareServer(TestSubject.java:157)
... 10 more
Oct 26, 2024 4:40:34 PM io.quarkus.bootstrap.runner.Timing printStopTime
INFO: testharness stopped in 0.007s
Exit code: 1
|
Can you please try the conformance test at the 33997ae commit. Given this is a container 404, the only change in the code that even could cause this failure is the commit after that. Also, is the the only test that is failing, or are there others? Can you direct me to the repo where the test is set up? |
I think this may have been merged prematurely. This part also still needs attention. Perhaps a revert would be best. |
This PR implements PREP/Solid-PREP as an alternative to the existing notifications mechanism in NSS which was deprecated for security reasons.
PREP/Solid-PREP bring many improvements to the existing notifications mechanism in NSS, as illustrated in this table:
Link Headers
The last published version of Solid Protocol and the ED mentions that WebSockets-API that NSS uses does not include an authentication mechanism. That was the main reasons why Solid-CG's notifications panel deprecated the WebSockets-API and transitioned Solid to the Solid Notifications Protocol. Solid-PREP is part of this incubation work in the CG.
Footnotes
PREP can fetch notifications along with the representation body in the response. ↩
Solid-PREP serves notifications that are identical to Solid Notifications Protocol, with suitable content negotiation requesting an RDF media-type. ↩