Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Become Ractor-safe to allow usage in non-main Ractor. #277

Open
okeeblow opened this issue Jul 26, 2021 · 1 comment
Open

Become Ractor-safe to allow usage in non-main Ractor. #277

okeeblow opened this issue Jul 26, 2021 · 1 comment

Comments

@okeeblow
Copy link
Contributor

Hello again :)

I've been experimenting with Ractors since upgrading to Ruby 3.0 for #275 but quickly ran into Ractor::UnsafeError when trying to call Ox::sax_parse in anything except the main Ractor.

Per Ruby's Ractor C extension documentation (link below):
"By default, all C extensions are recognized as Ractor-unsafe. If C extension becomes Ractor-safe, the extension should call rb_ext_ractor_safe(true) at the Init_ function and all defined method marked as Ractor-safe. Ractor-unsafe C-methods only been called from main-ractor. If non-main ractor calls it, then Ractor::UnsafeError is raised."

I don't like to open seemingly-large feature requests like this without making some attempt at it myself first,
and luckily it seems like Ox::sax_parse Just Works™ since I marked it Ractor-safe, even with the class_cache. Confirming this safety, making any remaining changes to Ox::Sax, and expanding this to the non-Sax parts of Ox are all unfortunately out of my depth as a n00b C coder, so I would appreciate if you could take this over if it interests you. I am happy with just Sax support since I have no current need for marshalling, but I imagine other Ox users wouldn't be satisfied if stratified. I have a PR incoming for this change.

Official Ractor info:

Blogs:
okeeblow added a commit to okeeblow/ox that referenced this issue Jul 26, 2021
@okeeblow
Mark Ox::sax_parse as Ractor-safe and add a Ractor-based `Ox::S…
b784352 
…ax` example.

For ohler55#277

I've been experimenting with `Ractor`s since upgrading to Ruby 3.0 for ohler55#275
but quickly ran into `Ractor::UnsafeError` when trying to call `Ox::sax_parse` in anything except the main `Ractor`.

Per Ruby's `Ractor` C extension documentation (link below):
"By default, all C extensions are recognized as `Ractor`-unsafe. If C extension becomes `Ractor`-safe,
the extension should call `rb_ext_ractor_safe(true)` at the `Init_` function and all defined method marked as `Ractor`-safe.
`Ractor`-unsafe C-methods only been called from main-ractor. If non-main ractor calls it, then `Ractor::UnsafeError` is raised."

I don't like to open seemingly-large feature requests like this without making some attempt at it myself first,
and luckily it seems like `Ox::sax_parse` Just Works™ since I marked it `Ractor`-safe, even with the `class_cache`.
Confirming this safety, making any remaining changes to `Ox::Sax`, and expanding this to the non-`Sax` parts of `Ox`
are all unfortunately out of my depth as a n00b C coder, so I would appreciate if you could take this over if it interests you.
I am happy with just `Sax` support since I have no current need for marshalling,
but I imagine other `Ox` users wouldn't be satisfied if stratified.

In this commit:
- Enable `rb_ext_ractor_safe` preprocessor macro via `have_func` in `extconf.rb`.
- Mark `Init_Ox` and `ox_sax_parse` as `Ractor` -safe.
- Add a new `Ractor`-based `Ox::Sax` example exercising both parallel and serial `Ox::Sax` handler `Ractor`s
  to parse data from `shared-mime-info` XML files many users likely already have on their systems.

Official `Ractor` info:

- "Ractor: a proposal for a new concurrent abstraction without thread-safety issues": https://bugs.ruby-lang.org/issues/17100 (ruby/ruby#3365)
- Ruby's official `Ractor` documentation: https://docs.ruby-lang.org/en/master/doc/ractor_md.html
- "A way to mark C extensions as thread-safe, Ractor-safe, or unsafe": https://bugs.ruby-lang.org/issues/17307 (ruby/ruby#3824)
- Ruby's C Extension `Ractor` documention covering `rb_ext_ractor_safe`: https://docs.ruby-lang.org/en/master/doc/extension_rdoc.html#label-Appendix+F.+Ractor+support
- A `Ractor` C Extension from the creator of `Ractor` that might serve as a useful example: https://github.com/ko1/ractor-tvar

Blogs:

- "Ractors: Multi-Core Parallel Processing Comes to Ruby 3": https://www.ruby3.dev/ruby-3-fundamentals/2021/01/27/ractors-multi-core-parallel-processing-in-ruby-3/
- "Ruby Ractor Experiments: Safe async communication" :https://ivoanjo.me/blog/2021/02/14/ractor-experiments-safe-async/
- "Playing with Ruby Ractors": https://billy-ruffian.co.uk/playing-with-ruby-ractors/
- "How Fast are Ractors?": https://www.fastruby.io/blog/ruby/performance/how-fast-are-ractors.html (https://github.com/noahgibbs/ractor_basic_benchmarks/tree/main/benchmarks)

Before this change:

```
[okeeblow@emi#CHECKING-YOU-OUT] time ./bin/checking-you-out ~/224031-dot-jpg
Received /home/okeeblow/.local/share/mime/packages/user-extension-rsrc.xml
/home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:375:in `sax_parse': ractor unsafe method called from not main ractor (Ractor::UnsafeError)
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:375:in `block in open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:331:in `open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:331:in `open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:24:in `block (2 levels) in remember_me'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:19:in `loop'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:19:in `block in remember_me'
<internal:ractor>:583:in `send': The incoming-port is already closed (Ractor::ClosedError)
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:86:in `block in extended'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `each'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `each_with_object'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `extended'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/inner_spirit.rb:216:in `extend'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/inner_spirit.rb:216:in `<top (required)>'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out.rb:10:in `require_relative'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out.rb:10:in `<top (required)>'
        from ./bin/checking-you-out:3:in `require_relative'
        from ./bin/checking-you-out:3:in `<main>'
./bin/checking-you-out ~/224031-dot-jpg  0.12s user 0.05s system 100% cpu 0.168 total
```

My new `Ox::Sax` Ractor example script's usage:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb
Please provide the path to a `shared-mime-info` XML package and some media-type query arguments (e.g. 'image/jpeg')
```

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml
Please provide some media-type query arguments (e.g. 'image/jpeg')
```

Finding all-extant types:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml image/jpeg font/ttf application/xhtml+xml image/x-pict
Parallel Ractors
["Worker 0 gave us JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]",
 "Worker 1 gave us TrueType 字型 (font/ttf) [.ttf]",
 "Worker 2 gave us XHTML 網頁 (application/xhtml+xml) [.xhtml,.xht]",
 "Worker 3 gave us Macintosh Quickdraw/PICT 繪圖 (image/x-pict) [.pct,.pict,.pict1,.pict2]"]

Serial Ractor
"ONLY ONE OX gave us [#<CYO JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]>, #<CYO TrueType 字型 (font/ttf) [.ttf]>, #<CYO XHTML 網頁 (application/xhtml+xml) [.xhtml,.xht]>, #<CYO Macintosh Quickdraw/PICT 繪圖 (image/x-pict) [.pct,.pict,.pict1,.pict2]>]"
```

…and not finding invalid ones:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml lol/rofl fart/butt image/jpeg
Parallel Ractors
["Worker 0 gave us nothing",
 "Worker 1 gave us nothing",
 "Worker 2 gave us JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]"]

Serial Ractor
"ONLY ONE OX gave us [nil, nil, #<CYO JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]>]"
```

Unit tests pass.
@okeeblow okeeblow mentioned this issue Jul 26, 2021
Merged
@ohler55
Copy link
Owner

ohler55 commented Jul 26, 2021

I'll need to do some reading on Ractor to make sure Ox is indeed actor safe then the addition of the call is easy. Just one place.

ohler55 pushed a commit that referenced this issue Jul 27, 2021
@okeeblow
Mark Ox::sax_parse as Ractor-safe and add a Ractor-based `Ox::S…
5cc4bff 
…ax` example. (#278)

For #277

I've been experimenting with `Ractor`s since upgrading to Ruby 3.0 for #275
but quickly ran into `Ractor::UnsafeError` when trying to call `Ox::sax_parse` in anything except the main `Ractor`.

Per Ruby's `Ractor` C extension documentation (link below):
"By default, all C extensions are recognized as `Ractor`-unsafe. If C extension becomes `Ractor`-safe,
the extension should call `rb_ext_ractor_safe(true)` at the `Init_` function and all defined method marked as `Ractor`-safe.
`Ractor`-unsafe C-methods only been called from main-ractor. If non-main ractor calls it, then `Ractor::UnsafeError` is raised."

I don't like to open seemingly-large feature requests like this without making some attempt at it myself first,
and luckily it seems like `Ox::sax_parse` Just Works™ since I marked it `Ractor`-safe, even with the `class_cache`.
Confirming this safety, making any remaining changes to `Ox::Sax`, and expanding this to the non-`Sax` parts of `Ox`
are all unfortunately out of my depth as a n00b C coder, so I would appreciate if you could take this over if it interests you.
I am happy with just `Sax` support since I have no current need for marshalling,
but I imagine other `Ox` users wouldn't be satisfied if stratified.

In this commit:
- Enable `rb_ext_ractor_safe` preprocessor macro via `have_func` in `extconf.rb`.
- Mark `Init_Ox` and `ox_sax_parse` as `Ractor` -safe.
- Add a new `Ractor`-based `Ox::Sax` example exercising both parallel and serial `Ox::Sax` handler `Ractor`s
  to parse data from `shared-mime-info` XML files many users likely already have on their systems.

Official `Ractor` info:

- "Ractor: a proposal for a new concurrent abstraction without thread-safety issues": https://bugs.ruby-lang.org/issues/17100 (ruby/ruby#3365)
- Ruby's official `Ractor` documentation: https://docs.ruby-lang.org/en/master/doc/ractor_md.html
- "A way to mark C extensions as thread-safe, Ractor-safe, or unsafe": https://bugs.ruby-lang.org/issues/17307 (ruby/ruby#3824)
- Ruby's C Extension `Ractor` documention covering `rb_ext_ractor_safe`: https://docs.ruby-lang.org/en/master/doc/extension_rdoc.html#label-Appendix+F.+Ractor+support
- A `Ractor` C Extension from the creator of `Ractor` that might serve as a useful example: https://github.com/ko1/ractor-tvar

Blogs:

- "Ractors: Multi-Core Parallel Processing Comes to Ruby 3": https://www.ruby3.dev/ruby-3-fundamentals/2021/01/27/ractors-multi-core-parallel-processing-in-ruby-3/
- "Ruby Ractor Experiments: Safe async communication" :https://ivoanjo.me/blog/2021/02/14/ractor-experiments-safe-async/
- "Playing with Ruby Ractors": https://billy-ruffian.co.uk/playing-with-ruby-ractors/
- "How Fast are Ractors?": https://www.fastruby.io/blog/ruby/performance/how-fast-are-ractors.html (https://github.com/noahgibbs/ractor_basic_benchmarks/tree/main/benchmarks)

Before this change:

```
[okeeblow@emi#CHECKING-YOU-OUT] time ./bin/checking-you-out ~/224031-dot-jpg
Received /home/okeeblow/.local/share/mime/packages/user-extension-rsrc.xml
/home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:375:in `sax_parse': ractor unsafe method called from not main ractor (Ractor::UnsafeError)
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:375:in `block in open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:331:in `open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:331:in `open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:24:in `block (2 levels) in remember_me'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:19:in `loop'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:19:in `block in remember_me'
<internal:ractor>:583:in `send': The incoming-port is already closed (Ractor::ClosedError)
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:86:in `block in extended'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `each'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `each_with_object'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `extended'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/inner_spirit.rb:216:in `extend'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/inner_spirit.rb:216:in `<top (required)>'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out.rb:10:in `require_relative'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out.rb:10:in `<top (required)>'
        from ./bin/checking-you-out:3:in `require_relative'
        from ./bin/checking-you-out:3:in `<main>'
./bin/checking-you-out ~/224031-dot-jpg  0.12s user 0.05s system 100% cpu 0.168 total
```

My new `Ox::Sax` Ractor example script's usage:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb
Please provide the path to a `shared-mime-info` XML package and some media-type query arguments (e.g. 'image/jpeg')
```

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml
Please provide some media-type query arguments (e.g. 'image/jpeg')
```

Finding all-extant types:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml image/jpeg font/ttf application/xhtml+xml image/x-pict
Parallel Ractors
["Worker 0 gave us JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]",
 "Worker 1 gave us TrueType 字型 (font/ttf) [.ttf]",
 "Worker 2 gave us XHTML 網頁 (application/xhtml+xml) [.xhtml,.xht]",
 "Worker 3 gave us Macintosh Quickdraw/PICT 繪圖 (image/x-pict) [.pct,.pict,.pict1,.pict2]"]

Serial Ractor
"ONLY ONE OX gave us [#<CYO JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]>, #<CYO TrueType 字型 (font/ttf) [.ttf]>, #<CYO XHTML 網頁 (application/xhtml+xml) [.xhtml,.xht]>, #<CYO Macintosh Quickdraw/PICT 繪圖 (image/x-pict) [.pct,.pict,.pict1,.pict2]>]"
```

…and not finding invalid ones:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml lol/rofl fart/butt image/jpeg
Parallel Ractors
["Worker 0 gave us nothing",
 "Worker 1 gave us nothing",
 "Worker 2 gave us JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]"]

Serial Ractor
"ONLY ONE OX gave us [nil, nil, #<CYO JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]>]"
```

Unit tests pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ohler55 @okeeblow