Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic prep #1792

Merged
merged 23 commits into from
Nov 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
21b15be
feat: PREP Notifications
CxRes Aug 30, 2024
f4fb5a0
feat: Add Solid-PREP Notifications
CxRes Aug 30, 2024
109ce09
Bump CI and Docker to node v22
CxRes Aug 30, 2024
7b23e57
Replace nyc with c8
CxRes Aug 30, 2024
3d93e13
fix: Correct Parenting for Solid-PREP Notifications
CxRes Sep 8, 2024
fd99081
Step Down to Node 20
CxRes Sep 8, 2024
8021633
fix: Relax Content-Type Checks in Integration Tests
CxRes Sep 8, 2024
e0ebc31
fix: Repository for Surface Tests
CxRes Sep 9, 2024
beaaa03
chore: Bump Express PREP
CxRes Oct 7, 2024
bda034a
feat: Add Tests for PREP
CxRes Oct 8, 2024
b0e7eb0
feat: Add Flag to disable PREP
CxRes Oct 10, 2024
6a287e4
refactor: Remove Commented Code in `get.js`
CxRes Oct 13, 2024
a6067e3
chore: Bump PREP Dependencies
CxRes Oct 19, 2024
568b58a
fix: Send Parent Notifications on Root
CxRes Oct 23, 2024
3fa2be9
fix: Activity is `as:Add` on Container POST
CxRes Oct 23, 2024
6f4b22b
fix: Set `target` on Container POST
CxRes Oct 23, 2024
2a277ff
fix: Set `origin` when Contained Resource is Removed
CxRes Oct 23, 2024
d08a88f
test: Notification when Creating a Container with POST
CxRes Oct 23, 2024
33997ae
chore: Bump Express PREP
CxRes Oct 23, 2024
1f6c16f
fix: Do Not Overwrite `req.url` on Container PUT
CxRes Oct 23, 2024
c933a74
test: Notification when Creating/Deleting Container in Container
CxRes Oct 23, 2024
2da77b0
refactor: Add Debugging for Failed Notifications
CxRes Oct 23, 2024
44dc0bc
fix: Correctly Set Location on non-RDF Notifications
CxRes Oct 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 18 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,30 @@ jobs:

strategy:
matrix:
node-version: [18.x]
node-version: [ '>=20.17.0' ]
os: [ubuntu-latest]

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

# extract repository name
- if: github.event_name == 'pull_request'
run: echo "REPO_NAME=${{ github.event.pull_request.head.repo.full_name }}" >> $GITHUB_ENV

- if: github.event_name != 'pull_request'
run: echo "REPO_NAME=${GITHUB_REPOSITORY}" >> $GITHUB_ENV

# extract branch name
- if: github.event_name == 'pull_request'
run: echo "BRANCH_NAME=${GITHUB_HEAD_REF}" >> $GITHUB_ENV

- if: github.event_name != 'pull_request'
run: echo "BRANCH_NAME=${GITHUB_REF#refs/heads/}" >> $GITHUB_ENV

# print repository name
- name: Get repository name
run: echo 'The repository name is' $REPO_NAME

# print branch name
- name: Get branch name
run: echo 'The branch name is' $BRANCH_NAME
Expand All @@ -40,12 +53,12 @@ jobs:
# test code
- run: npm run standard
- run: npm run validate
- run: npm run nyc
- run: npm run c8
# Test global install of the package
- run: npm pack .
- run: npm install -g solid-server-*.tgz
# Run the Solid test-suite
- run: bash test/surface/run-solid-test-suite.sh $BRANCH_NAME
- run: bash test/surface/run-solid-test-suite.sh $BRANCH_NAME $REPO_NAME

# TODO: The pipeline should automate publication to npm, so that the docker build gets the correct version
# This job will only dockerize solid-server@latest / solid-server@<tag-name> from npmjs.com!
Expand All @@ -56,7 +69,7 @@ jobs:
runs-on: ubuntu-latest
steps:

- uses: actions/checkout@v2
- uses: actions/checkout@v4

- uses: olegtarasov/[email protected]
id: tagName
Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ Your users will have a dedicated folder under `./data` at `./data/<username>.<yo
> To use Gmail you may need to configure ["Allow Less Secure Apps"](https://www.google.com/settings/security/lesssecureapps) in your Gmail account unless you are using 2FA in which case you would have to create an [Application Specific](https://security.google.com/settings/security/apppasswords) password. You also may need to unlock your account with ["Allow access to your Google account"](https://accounts.google.com/DisplayUnlockCaptcha) to use SMTP.

also add to `config.json`
```
```
"useEmail": true,
"emailHost": "smtp.gmail.com",
"emailPort": "465",
Expand Down Expand Up @@ -206,6 +206,7 @@ $ solid start --help
--multiuser Enable multi-user mode
--idp [value] Obsolete; use --multiuser
--no-live Disable live support through WebSockets
--no-prep Disable Per Resource Events
--proxy [value] Obsolete; use --corsProxy
--cors-proxy [value] Serve the CORS proxy on this path
--suppress-data-browser Suppress provision of a data browser
Expand Down Expand Up @@ -271,7 +272,7 @@ docker run -p 8443:8443 --name solid node-solid-server


This will enable you to login to solid on https://localhost:8443 and then create a new account
but not yet use that account. After a new account is made you will need to create an entry for
but not yet use that account. After a new account is made you will need to create an entry for
it in your local (/etc/)hosts file in line with the account and subdomain, i.e. --

```pre
Expand All @@ -280,16 +281,16 @@ it in your local (/etc/)hosts file in line with the account and subdomain, i.e.

You can modify the config within the docker container as follows:

- Copy the `config.json` to the current directory with:
- Copy the `config.json` to the current directory with:
```bash
docker cp solid:/usr/src/app/config.json .
```
- Edit the `config.json` file
- Copy the file back with
- Copy the file back with
```bash
docker cp config.json solid:/usr/src/app/
```
- Restart the server with
- Restart the server with
```bash
docker restart solid
```
Expand Down
6 changes: 6 additions & 0 deletions bin/lib/options.js
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,12 @@ module.exports = [
flag: true,
default: false
},
{
name: 'no-prep',
help: 'Disable Per Resource Events',
flag: true,
default: false
},
// {
// full: 'default-app',
// help: 'URI to use as a default app for resources (default: https://linkeddata.github.io/warp/#/list/)'
Expand Down
2 changes: 1 addition & 1 deletion bin/solid
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#!/usr/bin/env node
#!/usr/bin/env -S node --experimental-require-module
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed because the middlewares for PREP use ESM syntax and NSS is CJS. This magic flag allows importing ESM code in CJS code.

This will also prove generally useful, since it creates a path to migrate NSS to ESM one file at a time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the need to support your ESM middleware, but adding --experimental-require-module project-wide isn’t the best approach. Since it’s still experimental, it could introduce unpredictable behavior in different environments or future Node.js versions.

To ensure long-term stability and compatibility with the existing CJS modules, it’s better to bundle your libraries with a transpiler. This will let you use ESM without requiring the entire project to rely on an experimental flag

const startCli = require('./lib/cli')
startCli()
2 changes: 1 addition & 1 deletion bin/solid.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#!/usr/bin/env node
#!/usr/bin/env -S node --experimental-require-module
const startCli = require('./lib/cli')
startCli()
2 changes: 1 addition & 1 deletion docker-image/src/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:16-alpine
FROM node:20-alpine

# hadolint ignore=DL3018
RUN apk add --no-cache openssl
Expand Down
16 changes: 14 additions & 2 deletions lib/create-app.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@ const ResourceMapper = require('./resource-mapper')
const aclCheck = require('@solid/acl-check')
const { version } = require('../package.json')

const acceptEvents = require('express-accept-events').default
const events = require('express-negotiate-events').default
const eventID = require('express-prep/event-id').default
const prep = require('express-prep').default

const corsSettings = cors({
methods: [
'OPTIONS', 'HEAD', 'GET', 'PATCH', 'POST', 'PUT', 'DELETE'
Expand Down Expand Up @@ -61,6 +66,12 @@ function createApp (argv = {}) {

const app = express()

// Add PREP support
if (argv.prep) {
app.use(eventID)
app.use(acceptEvents, events, prep)
}

initAppLocals(app, argv, ldp)
initHeaders(app)
initViews(app, configPath)
Expand Down Expand Up @@ -115,7 +126,7 @@ function createApp (argv = {}) {
}

// Attach the LDP middleware
app.use('/', LdpMiddleware(corsSettings))
app.use('/', LdpMiddleware(corsSettings, argv.prep))

// https://stackoverflow.com/questions/51741383/nodejs-express-return-405-for-un-supported-method
app.use(function (req, res, next) {
Expand Down Expand Up @@ -168,6 +179,7 @@ function initAppLocals (app, argv, ldp) {
app.locals.enforceToc = argv.enforceToc
app.locals.tocUri = argv.tocUri
app.locals.disablePasswordChecks = argv.disablePasswordChecks
app.locals.prep = argv.prep

if (argv.email && argv.email.host) {
app.locals.emailService = new EmailService(argv.templates.email, argv.email)
Expand Down Expand Up @@ -287,7 +299,7 @@ function initWebId (argv, app, ldp) {
initAuthentication(app, argv)

if (argv.multiuser) {
app.use(vhost('*', LdpMiddleware(corsSettings)))
app.use(vhost('*', LdpMiddleware(corsSettings, argv.prep)))
}
}

Expand Down
1 change: 1 addition & 0 deletions lib/debug.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ exports.accounts = debug('solid:accounts')
exports.email = debug('solid:email')
exports.ldp = debug('solid:ldp')
exports.fs = debug('solid:fs')
exports.prep = debug('solid:prep')
3 changes: 3 additions & 0 deletions lib/handlers/delete.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@ async function handler (req, res, next) {
debug('DELETE -- Request on' + req.originalUrl)

const ldp = req.app.locals.ldp
const prep = req.app.locals.prep
try {
await ldp.delete(req)
debug('DELETE -- Ok.')
// Add event-id for notifications
prep && res.setHeader('Event-ID', res.setEventID())
res.sendStatus(200)
next()
} catch (err) {
Expand Down
39 changes: 32 additions & 7 deletions lib/handlers/get.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,13 @@ const translate = require('../utils.js').translate
const error = require('../http-error')

const RDFs = require('../ldp').mimeTypesAsArray()
const isRdf = require('../ldp').mimeTypeIsRdf

const prepConfig = 'accept=("message/rfc822" "application/ld+json" "text/turtle")'

async function handler (req, res, next) {
const ldp = req.app.locals.ldp
const prep = req.app.locals.prep
const includeBody = req.method === 'GET'
const negotiator = new Negotiator(req)
const baseUri = ldp.resourceMapper.resolveUrl(req.hostname, req.path)
Expand Down Expand Up @@ -103,22 +107,35 @@ async function handler (req, res, next) {
debug(' sending data browser file: ' + dataBrowserPath)
res.sendFile(dataBrowserPath)
return
} else if (stream) {
} else if (stream) { // EXIT text/html
res.setHeader('Content-Type', contentType)
return stream.pipe(res)
}
}

// If request accepts the content-type we found
if (stream && negotiator.mediaType([contentType])) {
res.setHeader('Content-Type', contentType)
let headers = {
'Content-Type': contentType
}
if (contentRange) {
const headers = { 'Content-Range': contentRange, 'Accept-Ranges': 'bytes', 'Content-Length': chunksize }
res.writeHead(206, headers)
return stream.pipe(res)
} else {
return stream.pipe(res)
headers = {
...headers,
'Content-Range': contentRange,
'Accept-Ranges': 'bytes',
'Content-Length': chunksize
}
res.statusCode = 206
}

if (prep & isRdf(contentType) && !res.sendEvents({
config: { prep: prepConfig },
body: stream,
isBodyStream: true,
headers
})) return
res.set(headers)
return stream.pipe(res)
}

// If it is not in our RDFs we can't even translate,
Expand All @@ -130,6 +147,14 @@ async function handler (req, res, next) {
// Translate from the contentType found to the possibleRDFType desired
const data = await translate(stream, baseUri, contentType, possibleRDFType)
debug(req.originalUrl + ' translating ' + contentType + ' -> ' + possibleRDFType)
const headers = {
'Content-Type': possibleRDFType
}
if (prep && isRdf(contentType) && !res.sendEvents({
config: { prep: prepConfig },
body: data,
headers
})) return
res.setHeader('Content-Type', possibleRDFType)
res.send(data)
return next()
Expand Down
Loading
Loading