Build Your Own Video Streaming Server with Node.js and Koa (2023)

• Download source code - 2.6 MB
• Download source code from GitHub

Table of Contents

1. Introduction
2. Setting up the Project
3. Video Streaming Server
   1. The Server
   2. Video Player
   3. Video Streaming
4. Video Caching
5. Bringing the Pieces Together
6. References
7. History

Introduction

Widely used on the Internet and on mobile networks, video streaming allows to play video streams without having to download them. When working with large amount of data, streams allow to send data chunk by chunk from a source to a destination in a particular order.

The purpose of this article is to set up our own video streaming server that serves videos hosted on a server without having to download the entire video file. The end user can move on different positions of the video without having to download all the files, which would be heavy.

If you host videos on a server, give access to video files, and use <video> tag with the video file url in src, there will be two major problems:

1. The user will have to wait for the browser to download the entire video file to play it and this could be annoying if the video file is large.
2. You cannot prevent the end user from downloading the video file.

In that case, the <video> tag would look like this:

<video src="videos/video.mp4"></video>

So, we are going to set up a video streaming server on port 3000 with this api /api/video/:name which will allow us to stream the video.

Doing so, the <video> tag will look like this:

<video src="http://localhost:3000/api/video/video.mp4"></video>

The src attribute will point to the video streaming server that doesn't exist for the moment.

Setting up the Project

The first step will be to initialize our Node.js project:

npm init

Then, we will have to update package.json in order to add support of ES6 by setting module as type:

{ "name": "streaming", "type": "module", "version": "1.0.0", "description": "Video Streaming Server", "author": "Akram El Assas",}

Regarding the library to use for streaming, we can use Express or any other library you want but in this project we will use Koa because its middleware system is very interesting.

You can use Express if you want to, you'll have to make a little modifications to make it work.

Now, we are going to install the dependencies:

npm i koa koa-router koa-sendfile dotenv

• koa: Koa is a web framework designed by the team behind Express, which aims to be a smaller, more expressive, and more robust foundation for web applications and APIs. By leveraging async functions, Koa allows you to ditch callbacks and greatly increase error-handling. Koa does not bundle any middleware within its core, and it provides an elegant suite of methods that make writing servers fast and enjoyable.
• koa-router: Router middleware for Koa. Koa does not support routes in the core module. We need to use the koa-router module to easily create routes in Koa.
• koa-sendfile: Basic file-sending utility for koa.
• dotenv: Dotenv is a zero-dependency module that loads environment variables from a .env file into process.env. Storing configuration in the environment separate from code is based on The Twelve-Factor App methodology.

Then, we will install development dependencies:

npm i -D @types/node @types/koa @types/koa-router nodemon

• @types/node: To have autocomplete while using Node.js in Visual Studio Code.
• @types/koa: To have autocomplete while using Koa in Visual Studio Code.
• @types/koa-router: To have autocomplete while using Koa Router in Visual Studio Code.
• nodemon: To automatically restart the video streaming server each time changes are detected.

Video Streaming Server

The Server

First and foremost, we'll start by importing Koa, creating a new server and start listening on port 3000:

import Koa from 'koa'const PORT = parseInt(process.env.PORT, 10) || 3000const app = new Koa()//// Start the server on the specified PORT//app.listen(PORT)console.log('Video Streaming Server is running on Port', PORT)

If we run our application using the following command:

npm run dev

And try to access http://localhost:3000, we'll see that our server is running on port 3000.

Video Player

Then, we will server an HTML page (public/index.html) that will contain a video player.

The source code of the HTML page is as follows:

<!DOCTYPE html><html><head> <link rel="icon" href="data:,"> <meta name="viewport" content="width=device-width, initial-scale=1" /> <title>Streaming</title> <style> body { background-color: #000000; } video { position: absolute; top: 0; right: 0; bottom: 0; left: 0; max-height: 100%; max-width: 100%; margin: auto; object-fit: contain; } </style></head><body> <video src="http://localhost:3000/api/video/video.mp4" playsInline muted autoplay controls controlsList="nodownload" > </video></body></html>

Since we are using ES6, __dirname variable is not available. So we have to build it.

To serve the HTML page, we will be using sendFile utility. sendfile returns a promise that resolves to the fs.stat() result of the filename. If sendfile() resolves, that doesn't mean that a response is set - the filename could be a directory. Instead, check if (context.status).

In the source code below, we simply create a route to server the HTML page containing the video player. Then, we add Koa Router middleware and create the server on the specified port.

import Koa from 'koa'import KoaRouter from 'koa-router'import sendFile from 'koa-sendfile'import url from 'url'import path from 'path'import fs from 'fs'import util from 'util'const __filename = url.fileURLToPath(import.meta.url)const __dirname = path.dirname(__filename)const PORT = parseInt(process.env.PORT, 10) || 3000const app = new Koa()const router = new KoaRouter()//// Serve HTML page containing the video player//router.get('/', async (ctx) => { await sendFile(ctx, path.resolve(__dirname, 'public', 'index.html')) if (!ctx.status) { ctx.throw(404) }})//// Add Koa Router middleware//app.use(router.routes())app.use(router.allowedMethods())//// Start the server on the specified PORT//app.listen(PORT)console.log('Video Streaming Server is running on Port', PORT)

router.routes() returns router middleware which dispatches a route matching the request and router.allowedMethods() returns separate middleware for responding to OPTIONS requests with an Allow header containing the allowed methods, as well as responding with 405 Method Not Allowed and 501 Not Implemented as appropriate.

A Koa application is an object containing an array of middleware functions which are composed and executed in a stack-like manner upon request. Koa is similar to many other middleware systems that you may have encountered such as Ruby's Rack, Connect, and so on - however a key design decision was made to provide high level "sugar" at the otherwise low-level middleware layer. This improves interoperability, robustness, and makes writing middleware much more enjoyable. This includes methods for common tasks like content-negotiation, cache freshness, proxy support, and redirection among others. Despite supplying a reasonably large number of helpful methods, Koa maintains a small footprint, as no middleware are bundled.

Philosophically, Koa aims to fix and replace Node, whereas Express augments Node. Koa uses promises and async functions to rid apps of callback hell and simplify error handling. It exposes its own ctx.request and ctx.response objects instead of Node's req and res objects. Koa uses its own custom objects: ctx, ctx.request, and ctx.response. These objects abstract Node's req and res objects with convenience methods and getters/setters.

Express, on the other hand, augments Node's req and res objects with additional properties and methods and includes many other framework features, such as routing and templating, which Koa does not.

Thus, Koa can be viewed as an abstraction of Node.js's http modules, whereas Express is an application framework for Node.js.

Thus, if you'd like to be closer to Node.js and traditional Node.js-style coding, you probably want to stick to Connect/Express or similar frameworks. If you want to get rid of callbacks, use Koa.

As result of this different philosophy is that traditional Node.js middleware, i.e., functions of the form (req, res, next), are incompatible with Koa. Your application will essentially have to be rewritten from the ground, up.

Now, if we try to access http://localhost:3000, we'll get the video player without any content for the moment because we haven't implemented video streaming yet.

Video Streaming

We will be streaming videos from the following url: http://localhost:3000/api/video/:name

We will add another middleware, and in this one, we will begin by checking if the url of the request is correct:

//// Serve video streaming//router.get('/api/video/:name', async (ctx, next) => { // // Check video file name // const { name } = ctx.params if ( !/^[a-z0-9-_ ]+\.mp4$/i.test(name) ) { return next() }})

The url of the request must be in the following format /api/video/:name, must have name in its url parameters and the name must be a case insensitive alphanumeric string that can contain dashes, underscores, spaces and finishes with the extension .mp4. You can specify any regex format you want to but for the sake of simplicity, we've chosen that one.

While reading the video on the client side, the browser will send HTTP requests to the server in order to fetch chucks of the video and the server will respond by sending the specified chuncks one by one. This allows us to avoid fetching the entire video file and prevents the end user from downloading the video file.

In video streaming, the key point is the Range HTTP request header. The Range HTTP request header indicates the part of a document that the server should return. Several parts can be requested with one Range header at once, and the server may send back these ranges in a multipart document. If the server sends back ranges, it uses the 206 Partial Content for the response. If the ranges are invalid, the server returns the 416 Range Not Satisfiable error. The server can also ignore the Range header and return the whole document with a 200 status code.

Here is a sample of the Range HTTP request header:

Range: bytes=983040-Range: <unit>=<start>-<end>

In the case of a video, Range will contain the starting point in bytes of the video file from where we will start reading the video file. The server will send to the client a stream starting from start and ending at end. start and end ranges start from 0.

If the request headers do not contain Range, we'll return a bad request:

//// Check Range HTTP request header//const { request, response } = ctxconst { range } = request.headersif (!range) { ctx.throw(400, 'Range not provided')}

Then, we will check the video file:

//// Check video file//const videoPath = path.resolve(__dirname, 'videos', name)try { await util.promisify(fs.access)(videoPath)} catch (err) { if (err.code === 'ENOENT') { ctx.throw(404) } else { ctx.throw(err.toString()) }}

If the file does not exist, we'll return the status 404 and if there is any other error while trying to access the file, we'll return the status 500. And finally, the error will be logged on the server side.

Now, we'll need to set up the response headers:

//// Calculate start Content-Range//const parts = range.replace('bytes=', '').split('-')const rangeStart = parts[0] && parts[0].trim()const start = rangeStart ? parseInt(rangeStart, 10) : 0//// Calculate video size and chunk size//const videoStat = await util.promisify(fs.stat)(videoPath)const videoSize = videoStat.sizeconst chunkSize = 10 ** 6 // 1mb//// Calculate end Content-Range//// Safari/iOS first sends a request with bytes=0-1 range HTTP header// probably to find out if the server supports byte ranges//const rangeEnd = parts[1] && parts[1].trim()const __rangeEnd = rangeEnd ? parseInt(rangeEnd, 10) : undefinedconst end = __rangeEnd === 1 ? __rangeEnd : (Math.min(start + chunkSize, videoSize) - 1) // We remove 1 byte because start and end start from 0const contentLength = end - start + 1 // We add 1 byte because start and end start from 0//// Set HTTP response headers//response.set('Content-Range', `bytes ${start}-${end}/${videoSize}`)response.set('Accept-Ranges', 'bytes')response.set('Content-Length', contentLength)

First of all, we retrieve the value of the Range HTTP request header, then we extract from it start in bytes. After that, we retrieve the size of the video file in bytes by using stat utility function. Then, we calculate end in bytes using the chunk size and rangeEnd. Safari/iOS first sends a request with bytes=0-1 range HTTP header probably to find out if the server supports byte ranges. In case of Safari/iOS, if rangeEnd is equal to 1, we respond back with end to 1. If rangeEnd is not provided or if rangeEnd is greater than 1, we use the chunk size to calculate end. We used 1 MB as chuck size but you can use any size you want to. Then, we calculate contentLength using end and start. And finally, we add the response HTTP headers using response.set function.

The Content-Range response HTTP header indicates where in a full body message a partial message belongs.

The Accept-Ranges HTTP response header is a marker used by the server to advertise its support for partial requests from the client for file downloads. The value of this field indicates the unit that can be used to define a range.

The Content-Length header indicates the size of the message body, in bytes, sent to the recipient.

And finally, we set the response status to 206 which indicates that the request has succeeded and the body contains the requested ranges of data. We set the response type and send the video stream from start to end. We only send a single chuck of the video file on each request:

//// Send video file stream from start to end//const stream = fs.createReadStream(videoPath, { start, end })stream.on('error', (err) => { console.log(err.toString())})response.status = 206response.type = path.extname(name)response.body = stream

The response.type is used to indicate the original media type of the resource. In our case, we are using video/mp4 media type.

Now, if we run our server and try to play the video, we'll see in the Network tab of Chrome Dev Tools that only the requested parts are sent to the browser and not the whole file:

If we click on a request, we can see the request and response headers:

And below is the timing of the above request:

Below is a screenshot of Network tab of Safari Dev Tools on macOS:

If we see the console on the server side, we'll notice ECONNRESET, ECANCELED and ECONNABORTED errors. Koa returns these errors because when the browser closes the connection, the server tries to read the stream. So the server says that it cannot read a closed stream. These errors can be simply ignored:

//// We ignore ECONNRESET, ECANCELED and ECONNABORTED errors// because when the browser closes the connection, the server// tries to read the stream. So, the server says that it cannot// read a closed stream.//app.on('error', (err) => { if (!['ECONNRESET', 'ECANCELED', 'ECONNABORTED'].includes(err.code)) { console.log(err.toString()) }})

Video Caching

If you play the video with Google Chrome and move backward, you'll notice that the same video chunks are fetched again. You can clearly see that in the Network tab of Chrome Dev Tools. In other words, the video chunks are not cached. There are some reasons why video caching is not available on Google Chrome and here are few of them:

• Caching could require a significant amount of memory and disk storage.
• If caching is enabled, the video player will play the video and at the same time, it will save the fetched video chunks in cache. The player would be much performant if it only plays the video.
• A very large number of videos are played only once. Another really large fraction of videos are skipped within a few seconds of being loaded up.

Thus, between the high complexity and the low return on investment for just caching video files for one user, it is generally a losing proposition to cache video files.

Bringing the Pieces Together

We set up our own video streaming server with Node.js and Koa. Now, the whole source code of our server looks like follows:

import Koa from 'koa'import KoaRouter from 'koa-router'import sendFile from 'koa-sendfile'import url from 'url'import path from 'path'import fs from 'fs'import util from 'util'const __filename = url.fileURLToPath(import.meta.url)const __dirname = path.dirname(__filename)const PORT = parseInt(process.env.PORT, 10) || 3000const app = new Koa()const router = new KoaRouter()//// Serve HTML page containing the video player//router.get('/', async (ctx) => { await sendFile(ctx, path.resolve(__dirname, 'public', 'index.html')) if (!ctx.status) { ctx.throw(404) }})//// Serve video streaming//router.get('/api/video/:name', async (ctx, next) => { // // Check video file name // const { name } = ctx.params if ( !/^[a-z0-9-_ ]+\.mp4$/i.test(name) ) { return next() } // // Check Range HTTP request header // const { request, response } = ctx const { range } = request.headers if (!range) { ctx.throw(400, 'Range not provided') } // // Check video file // const videoPath = path.resolve(__dirname, 'videos', name) try { await util.promisify(fs.access)(videoPath) } catch (err) { if (err.code === 'ENOENT') { ctx.throw(404) } else { ctx.throw(err.toString()) } } // // Calculate start Content-Range // const parts = range.replace('bytes=', '').split('-') const rangeStart = parts[0] && parts[0].trim() const start = rangeStart ? parseInt(rangeStart, 10) : 0 // // Calculate video size and chunk size // const videoStat = await util.promisify(fs.stat)(videoPath) const videoSize = videoStat.size const chunkSize = 10 ** 6 // 1mb // // Calculate end Content-Range // // Safari/iOS first sends a request with bytes=0-1 range HTTP header // probably to find out if the server supports byte ranges // const rangeEnd = parts[1] && parts[1].trim() const __rangeEnd = rangeEnd ? parseInt(rangeEnd, 10) : undefined const end = __rangeEnd === 1 ? __rangeEnd : (Math.min(start + chunkSize, videoSize) - 1) const contentLength = end - start + 1 // // Set HTTP response headers // response.set('Content-Range', `bytes ${start}-${end}/${videoSize}`) response.set('Accept-Ranges', 'bytes') response.set('Content-Length', contentLength) // // Send video file stream from start to end // const stream = fs.createReadStream(videoPath, { start, end }) stream.on('error', (err) => { console.log(err.toString()) }) response.status = 206 response.type = path.extname(name) response.body = stream})//// We ignore ECONNRESET, ECANCELED and ECONNABORTED errors// because when the browser closes the connection, the server// tries to read the stream. So, the server says that it cannot// read a closed stream.//app.on('error', (err) => { if (!['ECONNRESET', 'ECANCELED', 'ECONNABORTED'].includes(err.code)) { console.log(err.toString()) }})//// Add Koa Router middleware//app.use(router.routes())app.use(router.allowedMethods())//// Start the server on the specified PORT//app.listen(PORT)console.log('Video Streaming Server is running on Port', PORT)

References

• Koa - Expressive middleware for Node.js using async functions
• Koa Router - Router middleware for Koa
• Koa sendfile - basic file-sending utility for Koa
• MDN - Range HTTP request header
• MDN - Accept-Ranges HTTP response header
• MDN - Content-Range response HTTP header
• MDN - Content-Length header
• MDN - 206 Partial Content

History

• 21^st December 2022
  • Initial release
• 29^th December 2022
  • Added Koa Router
  • Added some styling to the video player
  • Added Video Caching section
  • Updated video filename regex
  • Handled ECONNABORTED error
  • Enhanced error handling
  • Fixed macOS and iOS issues
• 06^th January 2023
  • Added some clarifying comments to source code