Digital Ocean, the cloud provider, has been garnering more attention of late. I'm the latest of late adopters for all things, as that practice has saved me a great deal of wasted time and frustration over the years. So I still run most of my own web and mail servers on AWS, largely to keep my hand in and stay up to date. For my level of personal use the high cost to processing power ratio obtained from AWS instances isn't much of an issue. Times change, however, and I decided to move a couple of Ubuntu servers from AWS to Digital Ocean to test the waters. A Brief Note on Cost Differences The cost of running a roughly equivalent web server on Digital Ocean is between a third and two-thirds of that on AWS: lower if you don't much care ...
Back before Nginx introduced support for websockets in version 1.3 I wrote a couple of posts to outline how to serve both SSL-encrypted websocket and web traffic on the same port and same server with Node.js in the back end and either an HAProxy or Varnish and Stunnel front end. The typical scenario here is that you are setting up servers for a single page web application that uses Express.js to serve content and Socket.IO to manage websocket connections. This might be a chatroom application, for example, the ubiquitous project that everyone hacks together to illustrate how to use Socket.IO. It is often the case that your single page application is a part of a larger site, and that site may or may not be served by Express.js. So it's ...
For the purposes of this post let us say that you are building a single page web application that uses Socket.IO as a way to push messages from the server to connected clients via WebSockets or one of a range of fallback approaches for browsers lacking a WebSocket implementation. The opening stages of loading the application thus run as follows: An HTTP/S request to load the application page. HTTP/S requests to load assets - e.g. Javascript. The client establishes a WebSocket or other Socket.IO connection. Since the Socket.IO connection follows the page load, by that point a session has already been established on the web server. Most applications are stateful to the point of using sessions, and is usually the case that you want the ...
In Node.js it is a fairly common practice to fork one or more companion child processes to a main process, such as when you need precise timing over short intervals, or when you need to run something computationally intensive and don't want to tie up the main thread of operation. Node.js provides the core child_process and cluster modules to manage forking, sharing of ports, and communication between processes to support various different types of multi-process application. A topic that arises fairly frequently is how to ensure that child processes end when the main process ends. This is actually not as easy to arrange as you might think, as there are a potentially wide variety of circumstances to accommodate. For example, to pick a ...
The Redis datastore is a great tool when managing state in a backend cluster of simple processes that all perform the same function. For the purposes of this post we'll say that the cluster consists of Node.js processes. In Node.js, it is is essential to cluster multiple processes even when building comparatively small services. On the one hand there is the need for redundancy, such that the frontend can continue to provide service if (more usually a matter of when) a single backend process dies, but it is also the case that Node.js is effectively a single thread of operation for your code. You need to run multiple Node.js processes in order to take advantage of the fact that your server has more than one processor. So let us say that ...
I was recently working on a short Node.js project that involved CSV parsing and automation of access to Google Spreadsheets. This post consists of a few notes on what was learned - not all of the tools in the ecosystem are all that they claim to be, sadly. Miso.Dataset Google Spreadsheet Importers Don't Work Miso.Dataset is a toolkit for manipulating tabular data. Insofar as the actual manipulation goes, it's pretty useful and can save you a lot of time. The Node.js distribution is a port of a client distribution, however, which means that whether or not non-core functionality actually works in Node.js is a coin-toss. The Google Spreadsheet importer is one such non-functional item; it works by calling one of the JSONP/XML format views ...
A Node.js process is, as I'm sure you're aware, essentially a single thread of execution for your Javascript code. It keeps things running along smartly by making use of the downtime spent waiting for I/O operations to complete in order to execute more Javascript code. That it is a single thread means that you don't have wrestle with fun things like concurrency and locking, which are thorny enough topics to have merited the invention of numerous architectures that really only exist so as to allow developers to avoid encountering concurrency and locking in day to day coding. The flip side of this coin is that a busy Node.js process isn't all that good at doing things to a schedule when the schedule intervals are on the order of 100ms or ...
One of the more entertaining aspects of Javascript is that - barring good encapsulation - you can reach in and tinker with everything. This also means that you, the developer, can dig yourself a far deeper hole than would otherwise be the case. So let us open this post by noting that public APIs exist for a reason, and that if you base critical code on manipulating the internals of someone else's package then you're creating a bunch of work for someone down the line. Internals change far more readily than APIs, and today's hack is tomorrow's technical debt - or more likely tomorrow's cascade of thrown errors. So that said, onwards. At a high level the internals of the Express web application framework for Node.js are fairly ...
A business based on recurring payments is usually a good business to be in: the mathematics of the thing are fairly simple, in that you are trying to keep a customer present, happy, and paying for long enough to make more than the combined cost of acquiring and then serving them. For many online businesses, the acquisition cost is the expensive part and the rest is a marginal expense per customer - or at least that seems to be the trend in these days of customer service done the Google way. In any case, the point is that a simple spreadsheet can show how well your business is doing, what needs to be adjusted, and produce projections for the future. You are trying to reduce the costs of acquisition and serving customers well while ...
Forever is a useful tool for running a Node.js process with monitoring; it can be set to restart a failed process, and has a few other helpful features along the same lines. In this post you'll find a couple of exceedingly simple scripts for running a Node.js process as a service on Ubuntu using Forever. One thing I've found in the course of building and tinkering with Node.js is that there's a lot to be said for explicitly setting all the necessary environment variables in your service script. That way the same basic script will serve just fine for locally built versus package manager installations of Node.js, and for code that moves from being a local development copy to a package installed via NPM. Just change the relevant ...