异步编程：协作式多任务处理¶

这是关于异步编程的系列文章中的第二篇。整个系列试图回答一个简单的问题：“什么是异步？”。起初我开始研究这个问题时，我以为我知道它是什么。后来才发现我对它一无所知。那么让我们来了解一下吧！

This is the second post in a series on asynchronous programming. The whole series tries to answer a simple question: "What is asynchrony?". In the beginning, when I first started digging into the question, I thought I knew what it is. It turned out that I didn't know the slightest thing about asynchrony. So let's find out!

整个系列：

• 异步编程：阻塞 I/O 和非阻塞 I/O
• 异步编程：协作式多任务处理
• 异步编程：等待 Future
• 异步编程：Python3.5+

Whole series:

• Asynchronous programming. Blocking I/O and non-blocking I/O
• Asynchronous programming. Cooperative multitasking
• Asynchronous programming. Await the Future
• Asynchronous programming. Python3.5+

在上一篇文章中，我们讨论了多请求下的并发处理。最后发现我们可以在线程或进程的帮助下实现。除此之外还有一个选择——协作式多任务处理（又名非抢占式多任务处理）。

In the previous post, we talked about how to provide concurrent processing of several requests. Hence we suggested that it could be implemented with the help of threads or processes. However, there is one more option — cooperative multitasking(aka non-preemptive multitasking).

操作系统的调度程序或是计划程序，可以处理进程、线程，并在它们之间切换。坏消息是，它并不能知道我们的应用程序是如何工作的，所以它将在随机的时刻暂停线程/进程（可能不是在最好的时刻），保存上下文，并切换到下一个线程/进程（抢占式多任务处理）。但是我们开发者知道我们的应用程序是如何工作的。例如，程序在很短的时间内在 CPU 上执行一些计算操作，另外的大多数时间等待网络 I/O，因此我们更清楚何时处理请求间的切换。

What we are saying here is that the operating system is definitely amazing as there are schedulers/planners, it can handle processes, threads, switch between them, etc. Unfortunately it still doesn't know how our application works so it will pause the thread/process at a random moment(probably not the best one), save the context, and switch to the next thread/process(aka preemptive multitasking). But we, developers, know how our application works. We know that we have short periods when some computing operations are performed on the CPU, but most of the time we wait for network I/O, and we know better when to switch between processing individual requests.

从操作系统的角度来看，协作式多任务处理只是一个执行线程，但是在它内部，应用程序可以决定何时在处理单个请求/命令之间切换。一旦数据到达，应用程序就开始读取它并解析请求，例如将数据发送到数据库，这是一个阻塞操作，但应用程序可以开始处理另一个请求，而不是等待数据库的响应。这个过程被称为“协作”，因为所有任务/命令都必须协作才能使整个过程按照规划工作。虽然它们彼此间穿插，但仍在单个控制线程中，该线程被称为协作调度程序，其作用是启动进程，并让进程有机会主动收回控制权。

From an OS perspective, cooperative multitasking is just one execution thread, but inside it, the application has the power to decide when to switch between processing individual requests/commands. Once some data arrives, the application reads it, parses the request, sends the data to the database for example, and this is a blocking operation, but instead of waiting for a response from the database, it can start processing another request. This is called "cooperation" because all tasks/commands have to cooperate to make the entire planning scheme work. They intersperse with each other but in a single control thread, known as a cooperative scheduler, whose role is to start processes and give them the opportunity to voluntarily take control back.

这比抢占式多任务处理更容易，因为开发人员总是知道当一个任务正在执行时，另一个任务没有执行。尽管在单处理器系统中多线程应用程序也会以交错的方式执行，但是使用线程的程序员仍然必须考虑错误，以便在切换到多核系统时应用程序不会出错。然而即使在多核系统上，单线程异步系统也总是以交错方式执行。

This is easier than preemptive multitasking because the developer always knows that when one task Is being preformed, the other is not. Although in a single-processor system a multithreaded application will be executed in an interleaved manner as well, a programmer using threads must still think about errors so that the application doesn't work incorrectly when switching to a multi-core system. However, a single-threaded asynchronous system will always be executed in interleaved fashion even on a multi-core system.

编写此类程序的复杂性在于：切换、维护上下文、将每个任务组织成包含中断的小步骤序列的过程落在开发人员的肩上。从另一个角度看，我们在效率上更胜一筹，因为没有无必要的上下文切换，比如在线程和进程之间切换时的处理器上下文。

The complexity of writing such programs lies in the fact that this process of switching, maintaining the context as such, organizing each task as a sequence of smaller steps performed with interruptions, falls on the shoulders of developers. On the other hand, we win in efficiency because there are no unnecessary context switches, like a processor context when switching between threads and processes.

实现协作式多任务处理有两种方式——回调和协作线程。

There are two ways to implement cooperative multitasking — callbacks and cooperative threads.

回调¶

Callbacks

所有的合作操作都会导致真正的操作在未来的某个时间才运行，我们的执行线程应该在操作完成时返回最终结果。所以为了得到结果，我们必须给这些操作注册回调函数——如果请求或者操作成功，它将调用某个函数，如果请求或者操作不成功，它就调用另一个函数。回调是一种显式方式，即开发人员应该编写程序，仿佛他真的不知道回调函数在什么时候被调用。

All cooperative operations cause the action to occur sometime in the future and our execution thread should return the result when it is ready. So in order to get the result, we have to register a callback — if the request/operation is successful, it will call one function, if request/operation is not successful, it will call another. Callback is an explicit approach, i.e the developer should write programs as if he really doesn't know when the callback function will be called.

因为回调是显式的且大多数现代语言都支持它，所以应用得最为广泛。此外，还有 Future 和 Promise——它们拥有更明确的 API，但与回调没有本质区别。

It is the most widely used option because it is explicit and supported by most modern languages. Also, there are Futures or Promises — it's kind of the same thing internally but with more clear API.

回调的利弊:

• 它不同于线程程序，因而也没有线程的问题;
• 回调函数会吞噬异常;
• 回调可能变得混乱并带来调试上的困难；

The pros and cons:

• It differs from threaded programs and doesn't have their problems;
• Callbacks swallow exceptions;
• The callback becomes confusing and difficult to debug.

协作线程¶

Cooperative Threads

不同于回调，协助线程是隐式的。开发人员以一种似乎没有协作式多任务处理的形式编写程序。这种方法有不同的层次：用户线程（又名绿色线程）或协程。

The second way is implicit when developers write a program in such a way that there seems to be no cooperative multitasking. There are different shades of this approach: user-threads(aka green threads) or coroutines.

通过绿色线程，我们可以像原来那样执行阻塞操作，还能像非阻塞的一样期望立即得到结果。因为在“底层”会有一个黑魔法——某个框架或编程语言使阻塞操作变得非阻塞，并将控制权转移到其他执行线程，但这不是建立在操作系统线程上的，而是在逻辑线程（用户级线程）上的。这些线程由“普通”用户进程执行，而不是由操作系统执行。

Using green threads, we can perform a blocking operation as we have done before and expect the result right away as if it is non-blocking. But there is black magic "under the hood" — there is a framework or programming language that makes the blocking operation non-blocking and transfers control to some other execution thread but not in the sense of the operating system thread but in the sense of a logical thread (user-level thread). These threads are executed by a "normal" user process and not by the OS.

在协程中应该编写添加一些“检查点”，在那里函数可以暂停和恢复。退出可以通过调用其他协程来完成，这些协程稍后可能返回到调用原始协程的位置。协程与线程非常相似。然而，协程是协作多任务的，而线程往往是优先多任务的。不需要互斥锁、信号量等同步原语，也不需要操作系统的支持。

In coroutines, you should write programs that add some "checkpoints" where your function can be paused and resumed. Exit can be done by calling other coroutines, which may later return to the point where they are calling the original coroutine. Coroutines are very similar to threads. However, coroutines are cooperatively multitasked, while threads tend to be preemptively multitasked. There is no need for synchronization primitives such as mutexes, semaphores, etc. and no need for support from the operating system.

协作线程的利弊:

• 它们是在用户空间级别控制的，而不是操作系统；
• 给人的感觉像是同步编程；
• 包括所有正常线程编程的问题，除了切换 CPU 上下文；

The pros and cons:

• They are controlled at the user-space level, not the OS;
• They feel like synchronous programming;
• Include all the problems of normal threading programming except switching the context of the CPU.

Reactor/Proactor 模式¶

Reactor/Proactor patterns

在协作多任务处理中，总是有一个处理引擎负责所有 I/O 处理。它在设计模板名称之后被称为 Reactor。反应堆接口说，“给我一些套接字和回调函数，当套接字准备好用于I/O时，我将调用你的回调函数。反应器作业是通过将所有处理委托给适当的处理程序(工作者)来对I/O事件作出反应。处理程序执行处理，因此不需要阻塞I/O，只要注册事件的处理程序或回调函数来处理它们。

Within cooperative multitasking, there is always a processing engine that is responsible for all I/O processing. It is called the Reactor after the design template name. The reactor interface says, "Give me a bunch of your sockets and your callbacks, and when that socket is ready for I/O, I will call your callback functions. A reactor job is to react to I/O events by delegating all the processing to the appropriate handler(worker). The handlers perform processing, so there is no need to block I/O, as long as handlers or callbacks for events are registered to take care of them.

反应堆设计模板的目的是避免为每个消息、请求和连接创建线程的常见问题。它从多个处理程序接收事件，并依次将它们分发给相应的事件处理程序。原则上，标准Reactor允许应用程序同时运行事件，同时保持单线程处理的简单性。它通常使用非阻塞同步I/O(查看I/O模型中的多路复用)。更有趣的是Proactor模式。它是Reactor模式的异步版本。它通常使用操作系统提供的真正的异步I/O操作(查看I/O模型中的AIO)。

The purpose of the reactor design template is to avoid the common problem of creating a thread for each message, request, and connection. It receives events from multiple handlers and sequentially distributes them to the corresponding event handlers. In principle, the standard Reactor allows the application to be run with simultaneous events while maintaining the simplicity of single-thread processing. It would usually use non-blocking synchronous I/O(check out multiplexing in the I/O models). What is more interesting is the Proactor pattern. It's an asynchronous version of the Reactor pattern. It usually uses true asynchronous I/O operations provided by the OS(check out AIO in the I/O models).

但这种方法也有局限性。

But there are limitations to such an approach.

首先，通过使用这种模式，它限制了在任何受支持的平台上可以执行的操作类型。另一方面，反应器可以处理任何事件类型。其次，缓冲区空间有限。缓冲区必须为I/O期间的每个异步操作准备，而I/O基本上可以一直运行下去。这两个范例是nginx HTTP服务器的基础，Node.js via libuv, Twisted Python，以及Python中新的asyncio库。

Firstly by using this pattern, it limits the types of operations that you can perform on any supported platform. Reactors, on the other hand, can handle any event types. Secondly, there are buffer space limitations. A buffer has to be for each asynchronous operation for the duration of the I/O which can run basically forever. Those two paradigms underlie the nginx HTTP server, Node.js via libuv, Twisted Python, and the new asyncio libraries in Python.

最佳实践¶

Best approach

没有哪个选择是完美无缺的。这种组合效果最好，因为协同多任务处理通常会胜出，特别是在连接挂掉很长时间的情况下。例如，web套接字是一个长期连接。如果您分配一个进程或一个线程来处理一个web套接字，那么您就显著地限制了一次到一个后端服务器的连接数量。因为连接将持续很长时间，所以保持多个同时连接非常重要，而每个连接需要做的工作很少。

But none of the options is really perfect. The combination works best because cooperative multitasking usually wins, especially if your connections hang up for a long time. For example, a web socket is a long-lasting connection. If you allocate a single process or a single thread to handle a single web socket, you significantly limit the number of connections to one backend server at a time. And because the connection will last a long time, it's important to keep many simultaneous connections, while each connection will have little work to do.

多任务处理的问题是它只使用一个处理器核心。虽然可以在一台机器上运行一个应用程序的多个实例，但这并不总是方便的，也有它的缺点）。所以一个比较好的方案是：使用一个 reactor/proactor 模式运行多个进程，然后在每个进程中使用协作式多任务处理。

The problem with multitasking is that it can only use one processor core. Clearly, you can run multiple instances of an application on the same machine, although this is not always convenient and has its drawbacks). Therefore it is a good idea to run multiple processes using a reactor/proactor and use cooperative multitasking within each process.

一方面这种组合可以使用系统中所有可用的处理器核心，另一方面，它可以在每个核心内部高效地工作，而无需为每个单独的连接分配大量的资源。

This combination allows, on one hand, to use all available processor cores in our system and, on the other hand, it works efficiently inside each core without allocating a lot of resources to handle each individual connection.

结语¶

Conclusion

编写协作式多任务处理程序的困难在于维护上下文、切换上下文的过程落在了开发者的肩上。但也是通过这种方法，我们才能避免不必要的切换来获得效率上的提升。

The difficulty in writing applications that use cooperative multitasking is that this switching process while maintaining the context as such, falls on the shoulders of poor developers. On the other hand, by using this approach, we attain efficiency by avoiding unnecessary switches.

一个更有趣的解决方案是将协作式多任务处理与 Reactor/Proactor 模式相结合。

A more interesting solution comes from combining cooperative multitasking with Reactor/Proactor patterns.

下一篇文章将讨论异步编程以及它与同步编程的区别，我们将在新的层次上使用新的术语来讨论原有的概念。

In the next post, we will talk about asynchronous programming itself and how it differs from synchronous programming, about old concepts but considered on a new level and using new terms.