[BLOG] the story about building scipy with flang on windows #795

h-vetinari · 2023-10-29T22:04:38Z

Text styling

The blog is written with plain language (where relevant).
If there are headers, they use the proper header tags in order to do so (with only one level-one header).
All links describe where they link to (for example, check the Quansight labs website).
Any kind of styling that the author uses (for example, bold for emphasis) is consistent throughout the blog.

Non-text contents

Blog post featured image is in PNG or JPEG format, not SVG.
All content is represented as text (for example, images need alt text and videos need captions or descriptive transcripts).
If there are emojis, there are not more than three in a row.
Don't use flashing gifs or videos.
If it were to be read as plain text, the blog still makes sense and no information is missing.

For hero.png, I manually modified an existing svg using https://svgeditoronline.com/editor/, and added the following logos:

vercel · 2023-10-29T22:04:42Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
labs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 8, 2023 1:14pm

h-vetinari · 2023-10-29T22:53:45Z

For context: I had drafted this in google docs and already incorporated some feedback from Ralf. The textual content of the blog should hopefully be relatively final already.

What remains is getting things to build & render here correctly, and complete the checklist. W.r.t. to the alt text, I tried looking around how it should be added, but the example does not explain it, and I couldn't find other blog posts using it (except in the initial table with alt:, but I'm talking about images that occur in the running text). AFAICT:

![example image](example-image.jpg 'An exemplary image')
  ^^^^^^^^^^^^^                     ^^^^^^^^^^^^^^^^^^
  never shows up anywhere?          hopefully the alt text?

Also, the error messages are pretty inscrutable to me. The only relevant-seeming error I see is:

Error: Export encountered errors on following paths:
	/blog/[slug]: /blog/building-scipy-with-flang
    at /vercel/path0/node_modules/next/dist/export/index.js:395:19
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Span.traceAsyncFn (/vercel/path0/node_modules/next/dist/trace/trace.js:79:20)
    at async /vercel/path0/node_modules/next/dist/build/index.js:1118:21
    at async Span.traceAsyncFn (/vercel/path0/node_modules/next/dist/trace/trace.js:79:20)
    at async /vercel/path0/node_modules/next/dist/build/index.js:995:17
    at async Span.traceAsyncFn (/vercel/path0/node_modules/next/dist/trace/trace.js:79:20)
    at async build (/vercel/path0/node_modules/next/dist/build/index.js:65:29)

One reason might be the raw HTML table, with which I hope to achieve the following effect:

👉 OS 👇CPU Arch.	Linux	MacOS	Windows
x86_32	GCC	Apple Clang	MSVC (arm64 added recently)
x86_64
arm64
aarch64		➖	➖
ppc64le		➖	➖
...		➖	➖

it works in Github, but not sure if it works in the markdown flavour in use here?

Finally, not sure what a good headline image for the blog would be. I've been looking at pictures of oncoming freight trains (a metaphor that appears in the blog), but I'm also fine to use one of the stock images.

rgommers · 2023-11-01T07:11:43Z

@bskinn or @gabalafou would you be able to help with the alt text, table and image questions here?

@h-vetinari thanks! I'll review again once there's a preview. One thing you may want to do is reduce the size of ABI_meme.png, 1.7 MB is a little excessive.

h-vetinari · 2023-11-01T07:33:23Z

One thing you may want to do is reduce the size of ABI_meme.png, 1.7 MB is a little excessive.

Didn't notice, that wasn't intentional. What's a good size to aim for (in pixels or file size)?

bskinn · 2023-11-01T16:23:05Z

W.r.t. to the alt text, I tried looking around how it should be added

@h-vetinari, the syntax for in-flow Markdown images is:

![alt text](image src URL)

Your images are in the right place, and from what I can tell you've constructed the src URLs correctly (starting with /posts/{...}.

Other items:

The date needs to be a parse-able date; please replace the XX with an estimated pub date, say 7 Nov
You need to add the featuredImage and hero properties into the metadata, per the instructions in the README
- Please note the constraints for these images as stated in the instructions
- The hero image will be rendered over a large area, so it needs to be high res. SVG is best, if possible. If you don't have a hero in mind, feel free to browse the other Labs posts and duplicate a hero image from there. While per [BUG] - Copy-pasted featured images #668 we plan to consolidate and de-duplicate these re-used images, for now we're creating per-post copies of them

I created an author entry for you in the Labs Storyblok space, so that should work cleanly with the build now. I created it with an anonymous avatar. If you would like your picture to show on the post, please point me at a suitable one for me to use.

We'll see if/how the post builds with these fixes. A plain HTML table like that shouldn't break the build, though it might not format very nicely. We may have to do some custom component work to render that as an actual table. (Or, include it as an image.)

bskinn · 2023-11-01T16:26:16Z

Finally, not sure what a good headline image for the blog would be. I've been looking at pictures of oncoming freight trains (a metaphor that appears in the blog), but I'm also fine to use one of the stock images.

Just saw this. If you want, we can request something from design. That would likely push back the earliest feasible go-live date, though.

bskinn · 2023-11-01T16:30:07Z

One thing you may want to do is reduce the size of ABI_meme.png, 1.7 MB is a little excessive.

Didn't notice, that wasn't intentional. What's a good size to aim for (in pixels or file size)?

Let's convert to JPG (without resizing). It's already got some loss/blur to it, so a JPG conversion likely won't degrade much. By my GIMPing, it drops it to under 400kB.

h-vetinari · 2023-11-01T22:41:40Z

Thanks for the review!

The hero image will be rendered over a large area, so it needs to be high res. SVG is best, if possible

I played around with https://svgeditoronline.com/editor/, but the resulting file was not idempotent upon re-import (some things got corrupted, not sure if the error would be in the SVG itself or just the software), and it also got ~2MB large (after including the logos in question). That said, I'm happy to try to fix the SVG and use that (EDIT: it's been a pain, I tried getting a higher quality extract; it's ~3600x1700px, so hopefully good enough)

If you want, we can request something from design. That would likely push back the earliest feasible go-live date, though.

I'm probably dating myself quite badly with the Indiana Jones meme, but I came up with a simple visual for the story. If that's not serious enough, I'm happy to take a stock image (unless someone like Ralf or Tania would like to wait for a design, I'd rather get this published).

bskinn · 2023-11-01T23:51:55Z

Hold any further commits for a bit, @h-vetinari -- I've fixed the build and need to push my changes to your fork.

bskinn · 2023-11-02T00:10:12Z

Ok, @h-vetinari -- commit pushed that fixed the build locally for me:

MDX apparently insists on HTML element attributes being quoted
For whatever reason, we've had much better luck using the > syntax (see, e.g., this bit) for metadata strings containing quotes.

bskinn · 2023-11-02T00:13:40Z

Oh, and I fixed a couple of things that looked like typos.

And, the VS Code environment is set up to autoformat Markdown, so it did its thing there. We can revert whatever needed bits.

It may be dating me, too, but FWIW I really like your Indy featured image, I suspect it accurately conveys the sense of rapidly impending kerblooie of the approaching deadline of the 3.12.0 release. I'll leave it to the Labs folks to actually decide, though.

h-vetinari

Thanks so much for your help @bskinn! :)

Just one question

apps/labs/posts/building-scipy-with-flang.md

bskinn · 2023-11-02T01:06:28Z

The table rendering we have configured does not look much like what you provided above. This has always been something of a struggle with this tech stack.

I think the options here are either to (i) render the table elsewhere, screencap (or otherwise render to an image), and embed; or (ii) to build a React component that makes it look the way you want and include that. (The MDX will let us do that.) There could be other options, too... thoughts, @gabalafou?

h-vetinari · 2023-11-02T01:14:42Z

Depending on how much HTML the stack here supports, we could probably try to set bgcolor (or similar) per table/column/cell? Github doesn't suppor that, which is why I didn't bother trying.

Though I don't particularly mind if we pre-render the table... 🤷
Does someone know a nice way to do that?

apps/labs/posts/building-scipy-with-flang.md

h-vetinari · 2023-11-02T01:22:44Z

If you would like your picture to show on the post, please point me at a suitable one for me to use.

Can I send one to your email address in the git log?

bskinn · 2023-11-02T01:26:46Z

Can I send one to your email address in the git log?

Sure!

bskinn · 2023-11-02T14:19:46Z

@h-vetinari Photo now set in Storyblok. Kicked off a redeploy of the preview build to see how it looks.

also put all footnotes at the bottom, to make it easier to find them (matches rendered result), and how many of them there are already.

rgommers

This looks close to ready - the story reads well, and it's also in good shape visually. I did a last full review, all but two or three comments are minor textual ones. I think we can get this published once these are addressed.

apps/labs/posts/building-scipy-with-flang.md

rgommers · 2023-11-07T14:25:10Z

apps/labs/posts/building-scipy-with-flang.md

+Despite being the product of heroic efforts by many very bright people,
+`numpy.distutils` is generally regarded as a pile of hacks, and has been in
+minimal maintenance mode for years.
+In many ways, this is what led to the creation of conda.


Something in this section, in particular related to this sentence and the wheels <-> distutils connection, is a little off. It's hard to propose a direct rephrase, but the basic issue is that distutils/setuptools are build systems, while wheels (and eggs) are package formats; distribution of those formats on PyPI was what fell short and led to Conda (and this predated wheels).

but the basic issue is that distutils/setuptools are build systems, while wheels (and eggs) are package formats

I'm probably taking a few too many short-cuts here, but doing justice to all this without walls of text is really hard. In my defense, I did caveat in the beginning that the recap would be "incomplete and opinionated". ;-)

I can maybe highlight Dustin's talk more, which gives a very nice historical walkthrough (minus conda). In any case, I'll attempt a clarification.

distribution of those formats on PyPI was what fell short and led to Conda (and this predated wheels).

I wasn't around at that time, but from what I can tell, the timing coincided pretty closely. Continuum Analytics was started in 2012, and the wheel spec got defined around that time too.

My understanding is that the impetus for conda was by far not only about those package formats and their distribution on PyPI. Again from that blog post:

In fact, numpy.distutils replaces most of the innards of distutils but is still shackled by the architecture and imperative approach to what should fundamentally be a declarative problem.

Hence why conda was moving things from setup.py (imperative; specs resolved at execution time) to recipes (declarative dependencies in advance).

Realised/remembered a couple things while working on this:

I had wanted to setup some background for the removal of distutils later

The whole wheel thing came in because I wanted to also call out some successes and not just appear to have an entire section dedicated to how python packaging sucks.

The flow of the story further up (+ the meme) made it a no-brainer to talk about wheels, as that's what was necessary to get rid of users needing a compiler.

In any case, I took a stab at rewriting that section and I think it's now better. It goes more to the heart of the issue, it works in some allusions to the imperative/declarative divide that spurred the creation of conda, and it relates much more directly to the embedded image.

Would be happy to hear your thoughts.

Agreed, it is better now.

Hence why conda was moving things from setup.py (imperative; specs resolved at execution time) to recipes (declarative dependencies in advance).

I now understand what you meant. It's about install_requires and build_requires. To me, that's dependency specification and is only 1-2% of what's in setup.py, and the other 98-99% is building the package - that remained unchanged. The metadata -> conda recipe move was basically pyproject.toml rather than the wheel format, so a lot more recent.

Anyway, let's call this good, thanks for iterating on it.

apps/labs/posts/building-scipy-with-flang.md

h-vetinari · 2023-11-08T06:22:40Z

@rgommers, here's a combined view of the changes since your last review. Aside from the overhaul of ## Python packaging, most of the changes are just small readability improvements (remove parentheses, break up sentences, etc.).

Co-authored-by: h-vetinari <[email protected]>

rgommers

LGTM now, let's get this published. Thanks a lot @h-vetinari! And thanks @bskinn for help with the table & blog format.

h-vetinari · 2023-11-08T22:56:49Z

Thanks for the review here @rgommers, and the help @bskinn! 🙏

h-vetinari requested review from pavithraes, trallard and rgommers as code owners October 29, 2023 22:04

vercel bot had a problem deploying to Preview October 29, 2023 22:07 Failure

vercel bot had a problem deploying to Preview October 29, 2023 22:27 Failure

vercel bot had a problem deploying to Preview October 29, 2023 22:46 Failure

vercel bot had a problem deploying to Preview November 1, 2023 22:35 Failure

h-vetinari force-pushed the scipy branch from e46aa01 to 888f061 Compare November 1, 2023 22:35

vercel bot had a problem deploying to Preview November 1, 2023 22:39 Failure

vercel bot had a problem deploying to Preview November 1, 2023 23:49 Failure

vercel bot deployed to Preview November 2, 2023 00:11 View deployment

h-vetinari commented Nov 2, 2023

View reviewed changes

apps/labs/posts/building-scipy-with-flang.md Outdated Show resolved Hide resolved

bskinn reviewed Nov 2, 2023

View reviewed changes

apps/labs/posts/building-scipy-with-flang.md Outdated Show resolved Hide resolved

h-vetinari commented Nov 2, 2023

View reviewed changes

apps/labs/posts/building-scipy-with-flang.md Outdated Show resolved Hide resolved

vercel bot deployed to Preview November 2, 2023 01:22 View deployment

vercel bot deployed to Preview November 2, 2023 12:36 View deployment

provide a bit more colour on preparations that happened in advance

4513c4b

also put all footnotes at the bottom, to make it easier to find them (matches rendered result), and how many of them there are already.

vercel bot deployed to Preview November 6, 2023 06:53 View deployment

rgommers added labs 🔭 Items related to the Labs website type: content 📝 labels Nov 7, 2023

rgommers reviewed Nov 7, 2023

View reviewed changes

collected minor changes based on feedback

fd1a137

vercel bot deployed to Preview November 7, 2023 22:57 View deployment

try replacing CI screenshot with code section

8c36529

vercel bot deployed to Preview November 7, 2023 23:16 View deployment

rewrite "Python packaging" section

182ef66

vercel bot deployed to Preview November 8, 2023 01:54 View deployment

minor edits upon reread

d83e535

h-vetinari commented Nov 8, 2023

View reviewed changes

apps/labs/posts/building-scipy-with-flang.md Outdated Show resolved Hide resolved

vercel bot deployed to Preview November 8, 2023 02:15 View deployment

combat parenthesitis, especially in first few paragraphs

f5fb93d

vercel bot deployed to Preview November 8, 2023 02:46 View deployment

h-vetinari added 2 commits November 8, 2023 16:52

more fine-tuning

4fe1a1c

renumber footnotes sequentially

67a5ba6

vercel bot deployed to Preview November 8, 2023 05:59 View deployment

Update apps/labs/posts/building-scipy-with-flang.md

dfac7da

Co-authored-by: h-vetinari <[email protected]>

vercel bot deployed to Preview November 8, 2023 13:14 View deployment

rgommers approved these changes Nov 8, 2023

View reviewed changes

rgommers merged commit c6e7e52 into Quansight:develop Nov 8, 2023
1 check passed

bskinn mentioned this pull request Nov 8, 2023

Release 2023-11-08 #804

Merged

h-vetinari deleted the scipy branch November 8, 2023 18:44

gabalafou pushed a commit that referenced this pull request Nov 28, 2023

[BLOG] the story about building scipy with flang on windows (#795)

5fb94e2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BLOG] the story about building scipy with flang on windows #795

[BLOG] the story about building scipy with flang on windows #795

h-vetinari commented Oct 29, 2023 •

edited

Loading

vercel bot commented Oct 29, 2023 •

edited

Loading

h-vetinari commented Oct 29, 2023 •

edited

Loading

rgommers commented Nov 1, 2023

h-vetinari commented Nov 1, 2023

bskinn commented Nov 1, 2023 •

edited

Loading

bskinn commented Nov 1, 2023

bskinn commented Nov 1, 2023 •

edited

Loading

h-vetinari commented Nov 1, 2023 •

edited

Loading

bskinn commented Nov 1, 2023

bskinn commented Nov 2, 2023

bskinn commented Nov 2, 2023

h-vetinari left a comment

bskinn commented Nov 2, 2023

h-vetinari commented Nov 2, 2023

h-vetinari commented Nov 2, 2023

bskinn commented Nov 2, 2023

bskinn commented Nov 2, 2023

rgommers left a comment

rgommers Nov 7, 2023

h-vetinari Nov 7, 2023

h-vetinari Nov 8, 2023

rgommers Nov 8, 2023

h-vetinari commented Nov 8, 2023 •

edited

Loading

rgommers left a comment

h-vetinari commented Nov 8, 2023

[BLOG] the story about building scipy with flang on windows #795

[BLOG] the story about building scipy with flang on windows #795

Conversation

h-vetinari commented Oct 29, 2023 • edited Loading

Text styling

Non-text contents

vercel bot commented Oct 29, 2023 • edited Loading

h-vetinari commented Oct 29, 2023 • edited Loading

rgommers commented Nov 1, 2023

h-vetinari commented Nov 1, 2023

bskinn commented Nov 1, 2023 • edited Loading

bskinn commented Nov 1, 2023

bskinn commented Nov 1, 2023 • edited Loading

h-vetinari commented Nov 1, 2023 • edited Loading

bskinn commented Nov 1, 2023

bskinn commented Nov 2, 2023

bskinn commented Nov 2, 2023

h-vetinari left a comment

Choose a reason for hiding this comment

bskinn commented Nov 2, 2023

h-vetinari commented Nov 2, 2023

h-vetinari commented Nov 2, 2023

bskinn commented Nov 2, 2023

bskinn commented Nov 2, 2023

rgommers left a comment

Choose a reason for hiding this comment

rgommers Nov 7, 2023

Choose a reason for hiding this comment

h-vetinari Nov 7, 2023

Choose a reason for hiding this comment

h-vetinari Nov 8, 2023

Choose a reason for hiding this comment

rgommers Nov 8, 2023

Choose a reason for hiding this comment

h-vetinari commented Nov 8, 2023 • edited Loading

rgommers left a comment

Choose a reason for hiding this comment

h-vetinari commented Nov 8, 2023

h-vetinari commented Oct 29, 2023 •

edited

Loading

vercel bot commented Oct 29, 2023 •

edited

Loading

h-vetinari commented Oct 29, 2023 •

edited

Loading

bskinn commented Nov 1, 2023 •

edited

Loading

bskinn commented Nov 1, 2023 •

edited

Loading

h-vetinari commented Nov 1, 2023 •

edited

Loading

h-vetinari commented Nov 8, 2023 •

edited

Loading