Deep Dive Into Node.js

JavaScript has risen to prominence over the past decade especially, as the features and tools it offers are ideally designed for modern web development (among many other use cases). JavaScript’s combination of speed, accessibility, and a massive developer base worldwide has led to a rising number of tools based in the language. And perhaps the most popular tool in that set is Node.js.

Why all the hype around Node.js? A package of tools written primarily in JavaScript, including Google’s V8 JavaScript engine, the libuv platform abstraction layer, and a core library, Node.js was specifically designed for developers as a tool for working in the non-blocking, event-driven I/O paradigm. In simpler terms, Node.js was designed to shine specifically in real-time web applications which employ push technology. 

As apps become more and more real-time (with information refreshing in margins of seconds, as opposed to the minutes and longer of the Internet of the past), solutions for building these types of apps become equally more and more important. That’s why Node.js continues to draw loyal developers, and offer a robust solution for modern programming challenges and needs. But to know why it’s such a useful asset for your programming knowledge, you need to know what Node.js includes, and what it can do. That’s where this guide can help.

Why Was Node.js Initially Developed? 

After years and years of stateless-web (founded on the stateless request-response paradigm), modern web applications are now more and more real-time, two-way connections. In simpler terms, these are cases where both the server and client can initiate communication, allowing for a more free data exchange. This is very different to the historical stateless web response paradigm, where the client always initiates communication. Additionally, it’s all based on the open web stack.

Many people point to tools like Java Applets and Flash as examples of how this type of functionality has existed for years. However, those tools were actually just sandboxed environments using the web as a transport protocol to be delivered to the client. Additionally, they were run in isolation and often operated over non-standard ports, which may have required extra permissions and other annoying hurdles to jump through.

As a result of its aforementioned advantages (and this new web paradigm), Node.js now plays a critical role in the technology stack of many high-profile companies who depend on its unique benefits. 

How Does Node.js Work? 

Simply put, Node.js fills a very specific and essential need in modern web development. For example, Node.js is not ideal for CPU-intensive operations, which actually removes most of the advantages it offers programmers. Instead, Node.js is ideal for building fast and scalable network applications (scalability, which allows software to function optimally on a variety of systems, is also an essential element of modern web development). This is due to its ability to handle a huge amount of simultaneous connections via high throughput. Put another way, Node.js can help you manage intense surges of real-time traffic and server requests.

So what does the actual nuts and bolts of this functionality look like? Well, compared to traditional web-serving techniques where every connection (or request) generates a new thread, which thus chews up system RAM and eventually maxes out at the amount of RAM available, Node.js operates on a single-thread, using non-blocking I/O calls. This allows Node.js to support thousands upon thousands of concurrent connections held within the event loop.

One potential downside of writing Node.js applications is the potential of sharing a single thread between all clients requests. This is problematic because heavy computation could clog up Node’s single thread and cause problems for all of the clients, as incoming requests would be blocked until this computation was completed. Fortunately, the rapidly growing Node.js developer community has already generated a variety of useful tools to solve this issue (as well as others that have emerged).

What Should You Know About The Node Package Manager (NPM)?

One of the most popular and useful elements of Node.js is the standard built-in support for package management using NPM, which is included by default with any Node.js installation. Very similar to the idea of Ruby Gems, NPM offers programmers a set of publicly available and  reusable components. These are all available through easy installation via an online repository, and come along with version and dependency management.

Some of the most useful npm modules today are:

• express – otherwise known as Express.js, this tool is a Sinatra-inspired web development framework for Node.js, and the go-to standard for the majority of Node.js applications in use on the modern web (including by some of the biggest names in software)
• socket.io and sockjs – Server-side component of the two most common websockets components in use today
• lodash (underscore, lazy.js) – This is your standard JavaScript utility belt (and there are several to choose from)
• hapi – hapi is a configuration-centric framework for building web and services applications, which is gaining popularity because of its ease-of-use
• moment – A JavaScript date library for parsing, validating, manipulating, and formatting dates
• connect – as an HTTP server framework, this offers a collection of high performance “plugins” known as middleware, which functions as a base for Express
• pug (formerly Jade) – One of the more popular templating engines, inspired by HAML, as well as a default in Express
• mongodb and mongojs – MongoDB wrappers to provide the API for MongoDB object databases in Node.js.
• redis – Redis client library.
• forever – Probably the most common utility for ensuring that a given node script runs continuously. This is essential for keeping a Node.js production upright during specific bugs and failures
• bluebird – A full featured Promises/A+ implementation with exceptionally good performance

There are tons and tons more tools available, and more being added by the global JavaScript community every day. Like any good and popular language, the more bugs and issues that are found, the more solutions are developed to solve them for you.

What Are Some Popular Use Cases For Node.js?

Queued Inputs 

In the case where you may be intaking a high amount of concurrent data, your database can become a bottleneck. Fortunately, Node.js can easily handle concurrent connections themselves. However, since database access can be a blocking operation, this can cause trouble on the back-end. The solution is to acknowledge the client’s behavior before the data is truly written to the database.

With that approach, the system maintains its responsiveness even when being taxed by a heavy load. This is particularly useful when the client doesn’t need firm confirmation of a successful data write. This might include situations like the logging or writing of user-tracking data, processed in batches and not used until a later time; as well as operations that don’t need to be reflected instantly (like updating a ‘Likes’ count on Facebook) where eventual consistency is acceptable.

Data gets queued through a cacheing or message queuing infrastructure and subsequently digested by a separate database batch-write process, or computation intensive processing backend services, written in a better performing platform for such tasks. While other languages and frameworks can perform similar behavior, the hardware is not the same, and Node.js does it more efficiently and reliably.

Chat 

Chat is the most common real-time, multi-user application where Node.js is often employed. The chat application is really the ideal example for the benefits Node.js offers, since it’s a lightweight, high traffic, data-intensive (but low processing/computation) application that runs across distributed devices. For those of you who are just beginning to learn Node.js, this is also a great place to learn the ins and outs, since chat is simple but also covers most of the paradigms you’d ever use in any typical Node.js application.

Data Streaming 

In more traditional web platforms, HTTP requests and responses are often handled like isolated events, while in actuality they’re streams. This observation can be utilized in Node.js to build some cool features, such as processing files while they’re still being uploaded (since the data comes in through a stream and thus can be processed in an online fashion). This is particularly useful in the case of real-time audio or video encoding, and proxying between different data sources.

Proxy 

Speaking of proxying, Node.js is easily employed as a server-side proxy, where it can tackle a significant amount of simultaneous connections in a non-blocking manner. It’s particularly useful for proxying different services with different response times, or for collecting data from multiple source points.