My Quest To Find The Best NodeJS Http Client With HTTPS and Basic Auth

This may be a completely useless post, but it took me a bit to figure out some of the pieces until I got here, so, I thought maybe it’s worth sharing!

What’s this & how did I come by it?

Instapaper is a nice service that provides you with “read it later” bookmarklet. You use this bookmarklet to add any web page to their log and you can read that page later via their site, a mobile application (from them or 3rd party), or via Kindle (or other book readers)!

Apart from their bookmarklet, they provide a simple API to add URLs programmatically. This is useful if you for example want to extract URLs from other services, say twitter favourites. They also provide a Full API (OAuth API) for doing everything their site and apps do, except some things in this so called Full API are (at least to be) only for paid subscribers.


Instapaper’s simple API is quite interesting though, according to the docs, you can use HTTP and provide only username (email) -not even password- to add a URL, and you can use POST or GET. That’s scary for anyone who thinks about security, but -maybe- OK for what the API does. They also provide the API in proper HTTPS with basic HTTP authentication (and if you want more, you can always use the Full API’s OAuth).

I wanted to use their API in HTTPS using POST and Basic Authentication. In my tests I was happy to try HTTP and/or GET and/or sending username in clear-text. I also thought I’d write this in Node because I thought it should be easy. Node people often serve REST/HTTP APIs, so, they must be good at consuming them too (even though in reality most consumers are client JS not node), right? Well, maybe yes.

The Code

I can explain the challenges, how I overcame them, and then show the final code, but I have spoken too much already, let’s begin with the code.

var basicAuthenticationParams = {
username: ‘MY_EMAIL’,
password: ‘MY_PASSWORD’

var requestParams = {
url: ‘’
};“”, {
form: requestParams,
auth: basicAuthenticationParams
function (error, response, body) {
// This is only returned for network errors, not server errors
if (!!error) {
console.log(“Network Error:”);
throw error; // it’s actual JS Error object not string

var statusCode = !!response ?
parseInt(response.statusCode, 10) || null
: null;
// I could have checked for 200,
// but technically 200-299 are success codes,
// and more importantly, Instapaper sends 201 for success
if (!!statusCode && statusCode >= 200 && statusCode < 300) {
console.log(); // Extra line separator

console.log(“HTTP Status:”);
console.log(); // Extra line separator

// In instapaper’s case, by default,
// they just send status code in body too


See? It was all very straight forward. Well, it wasn’t for me. I tried several NPM packages to get this effect, and none was going well. The “request” package was the one that did the trick. this is generally accepted in Node, the built-in API is very low level and there is some NPM package that gives you the nicer one.

For all the other packages I tried, they didn’t have any dependencies on other packages. This “request” package gave me the usual shock seeing so many NPM packages downloaded and installed, but this is the philosophy of NPM, build small reusable pieces built on top of other pieces.

Getting the request headers right (Oh my!)

While the packages were not working, interchanging HTTP status codes between 403 (invalid credentials) and 400 (missing parameters), it was very hard to tell what I needed. Even associating to proxy to get the output to Fiddler didn’t go well with for example the “easyHttp” package (there is a similarly-named NuGet package for .NET BTW, but it’s not related at all I think).

I went to my favourite Chrome Postman extension, which works nicely like Fiddler for the “compose request” part (actually, arguably is even way easier). The request parameters (was using username as another request parameter, not using Basic Authentication, just to test) didn’t seem to work until I changed the default “form-data” to “x-www-form-urlencoded”


Which added the following header to the request: “Content-Type: application/x-www-form-urlencoded”. This kind of makes sense. This overly permissive API feels more optimized for sending from web-pages, where forms are a front concept.

I tried to add the header when I was using other packages but that didn’t seem to tick it. I tried this one, and as you can see didn’t even need to set that. I just used the “form” property of the options though instead of “body” or “”json” properties.

This is why I’m posting this, to hopefully help someone googling get up and running quicker than me. This is not just Instapaper, this can be very helpful in so many situations.

For the rest of you (.NET / non NodeJS devs)…

If you are not a Node person but still want to have a fiddle, after you install Node (which includes NPM package manager by default), then you save the code to a JS file “some-file.js”, preferably in a new folder.

Assuming Node and NPM are in the PATH (installer default), you can open Cmd or PowerShell console in that folder and run npm install request then node some-file.js (no quotes in both), and see the results in front of you.

Share With Friends:

How did I learn that?

As a bonus for coming here, I'm giving away a free newsletter for web developers that you can sign up for from here.

It's not an anything-and-everything link list. It's thoughtfully collected picks of articles and tools, that focus on Angular 2+, ASP.NET (4.x/MVC5 and Core), and other fullstack developer goodies.

Take it for a test ride, and you may unsubscribe any time.

You might also want to support me by checking these out [Thanks]:

  • Paul Ocalian

    try requester:

    i use it for basic auth and more complex things as well.

    you just add the following into your options:

    auth:{username: ‘un’, password: ‘pass’}