How does event-driven programming even work?
I’ve always wondered how event-driven programming worked – it is very different from the programming paradigms I was taught in school. I was confused by the asynchronous nature of callbacks and promises. It was also interesting to me how something like setTimeout or setInterval was implemented! It seemed non-trivial for this to be implemented in another language like C/C++ without constantly checking a timer in several areas of your code.
A synchronous version of this written in C/C++ would have a guaranteed output order of:
But in Node.js, the output would likely be something closer to:
It almost looks like the Node.js runtime was smart enough to do other work on the CPU while an I/O operation was happening! Under the hood, Node.js is adding hello_world(1) to the task queue. While executing hello_world(1), it notices that some I/O needs to be done so it *does some magic to be discussed later* and executes the next item on the task queue which is hello_world(2). Eventually, the Node.js runtime will get an event added to its task queue notifying it that writing to 1.txt file has completed and it will finish up the method call hello_world(1).
The most interesting part here is the mechanism in which Node.js skips blocking on I/O and executes a different event instead of completing the first hello_world(1). And then, *somehow* the runtime gets a notification that the file has been written to and executes the callback in fs.writeFile. To do all this and more, Node.js uses an asynchronous I/O library called libuv.
Node.js uses libuv as a wrapper to do I/O that would otherwise block the CPU for several cycles. When fs.writeFile is called, a request is sent to libuv telling it to write some content to a file. Eventually, once the content is written, libuv will send a notification back to Node.js telling it the write operation has been completed and it should run the callback for fs.writeFile. Here is an example of how libuv works when handling file I/O:
In this example, we have added two events to our event loop and uv_run begins running the events. In a traditional C/C++ synchronous style program, we’d expect these to execute sequentially and take a long time because each I/O operation takes a long time. However, using libuv as an async I/O library with an event loop, I/O blocking becomes less of an issue because we are able to execute other pending events while another event is blocked on I/O. To prove that, a possible output of running the above program is:
As you can see, the program doesn’t open, write, and then close each file sequentially. Instead, it opens each file, then writes to them and closes them in batches. This is because while the program is waiting for the file to do I/O, it executes the operations for another event. For example, while it is waiting to open file #1, it sends syscalls to open files #2 and #3.
But...how does it work under the hood?
An initial guess as to how this is implemented in libuv is to spawn a separate thread for every I/O operation and block on it. Once the I/O operation has completed, the thread exits and returns to the main libuv thread. The main libuv thread then notifies Node.js that the I/O operation has completed. However, this is likely very slow. Spawning a new thread for every I/O request is a lot of additional CPU overhead! Can we do better?
Another idea I have is to constantly run the poll syscall on all the file descriptors of interest, waiting for the event of interest to occur. In this design, we would only need one libuv thread and that thread would have a loop constantly polling all the file descriptors of interest to check if it is ready. This method would scale linearly O(n) with the number of file descriptors. Unfortunately, this method also isn’t fast enough. You can imagine a Node.js webserver running and having to loop through 5000 file descriptors on every iteration to check for a read or write event.
After a bit more digging and understanding how high-performance web servers like NGINX handle this problem (C10K problem), I came across epoll. The benefit of epoll vs. poll is that epoll only returns file descriptors which have some data update, so there’s no need to scan all of the watched file descriptors. This seems much better than poll and is indeed how libuv implements its async I/O on Linux.
On Linux, epoll works by having the kernel update the epoll per process data structure for every event on a monitored file descriptor. When a user space program requests all the file descriptors that have updates, the kernel already has this list of updated file descriptors and simply has to transfer it to user space. This contrasts from poll because in poll, the kernel needs to query all the file descriptors by iterating through them.
What about setTimer and setInterval, how are those implemented?
Now that we have a rough understanding of how I/O is implemented in single-threaded Node.js, how do features like setTimer and setInterval work? These don’t use libuv but it is pretty easy to guess how it might work. Because we now know that Node.js is an event-driven language and constantly pulls events off a task queue, it is easy to fathom that the runtime checks every timer or interval to see if it has expired on every event loop iteration. If it has, then it runs the callback for the timer or interval. If not, it skips to the next phase in the event loop. It is important to note that not all expired timers and intervals will be processed in one loop, the runtime often has a maximum number of events that it will process in each phase.