
User-level threads....... with threads. - Paul Turner - Google
About this video
Paul Turner of Google presents kernel interfaces for user-level scheduling of real threads, aiming at a uniform synchronous programming model across C++, Java, Go and Python. It introduces the switchto-style system calls behind Google's fibers.
Why it appears in the archive
HN cites it as the key explanation of cooperative, user-scheduled kernel threads and the prior art behind later async and coroutine debates.
AI-generated summary and labels from the source dataset; verify important details against the video. Source & credits.
From the Hacker News discussion
Paul Turner explained this really well at this year's Linux Plumbers Conference. The whole talk is fantastic, but the explanation of what pron is describing in particular (and how it could be improved) starts around 8:39: [VIDEO]
Read original comment ↗If you're interested in this area you might like this talk from the Linux plumbers' conference: "Over the past year we've been looking at how a uniform, performant, synchronous programming model can be implemented across all of Google's core languages (C++, Java, Go, Python), while maintaining compatibility with existing code." [VIDEO]
Read original comment ↗Oh OK, I definitely could have written that better. From what I can tell, M:N threading has been thoroughly abandoned by Linux, but it might be worth revisiting given the horrendous I/O scaling of Linux kernel threads. I would imagine scheduling user threads cooperatively simplifies things a lot. Google has a very interesting talk about cooperative native thread scheduling, but they haven't upstreamed their code: [VIDEO] And here's what linux devs are actually doing (it's underwhelming, something like 50% of what the hardware is capable of): [LINK] In the meantime, people who actually need the performance (HPC and even some enterprise servers) have to bypass the kernel entirely. It looks like that's going to be possible even in virtualized environments, thanks to hardware support: [LINK]
Read original comment ↗Related to this discussion, this talk on user-scheduled OS threads is really interesting. It's a shame no proposed implementation or experimental patches have appeared [VIDEO]
Read original comment ↗Allow me to third that. It's one of dangers assuming that what you learned at university 15-20 years ago still applies. Another interesting talk: [0] (and even that is 5 years old) [0] [VIDEO]
Read original comment ↗Google needs to get off its ass and open source Fibers (not to be confused with Google Fiber). Fibers sparked plenty of joy in me back when I was at Google. It's just a more dignified way of doing things with threads, IMO. Here's the talk which, unfortunately, does not show the higher level APIs I'm talking about: [VIDEO] . If this were released today everyone would slap their foreheads and say "of course that's how it should be done". Google folks have already removed the obstacle in that direction by adding userspace switching to the kernel in recent kernel releases. Just go all the way now, and release the kraken. Last update I've seen on this is this article from June of this year: [LINK] Imagine how excruciating it must have been to all the Google people on the C++ Standard committee to work on this stuff that comes nowhere close to what they've been using internally for years, and not be able to talk about it in any kind of detail.
Read original comment ↗It's hinted at in the video linked in the email[1]: > Over the past year we've been looking at how a uniform, performant, synchronous programming model can be implemented across all of Google's core languages (C++, Java, Go, Python), while maintaining compatibility with existing code. [VIDEO]
Read original comment ↗There is no User-Mode scheduling in the Linux Kernel to my knowledge, but the video[1] mentioned the post describes Linux patches that add support for it. [1] [VIDEO]
Read original comment ↗