October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsClean PCRecommendedOne scan can reveal what keeps slowing WindowsLook for cleanup and repair opportunities.Run ScanOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk7 min

Build a Lightweight BuiltWith Alternative with Node.js

Build a modest Node.js technology detector with bounded HTTP fetching, data-driven fingerprints, and inspectable evidence—without mistaking it for a commercial technographic database.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

You can build a small Node.js website technology detector by fetching one public page, checking a short catalog of visible fingerprints, and reporting each match with the evidence that triggered it. The result is useful for learning, internal checks, or a narrowly defined workflow—not a replacement for the breadth, historical data, live analysis, or enrichment workflows of commercial technographic services.

What a lightweight detector can—and cannot—tell you

Website technology detection is fingerprint matching: a scanner looks for signals exposed by a page or its response. The Wappalyzer project documentation says it “inspects HTML code, as well as JavaScript variables, response headers and more.” Its example fingerprint specification also includes fields such as cookies, DNS records, DOM features, and script URLs. Wappalyzer project repository

As an Amazon Associate I earn from qualifying purchases.

A first version should support only a few technologies with clear, observable signals. A missing match means the scanner did not find one of its selected signals under the current conditions; it does not prove that the site does not use that technology. Configuration, proxies, page variants, and the scanner’s limited catalog can all affect what is visible.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Keep the stages separate: input URL → validation and safety checks → HTTP(S) fetch → evidence extraction → fingerprint matching → structured result. Separating transport, extraction, and matching makes it easier to change one part without embedding technology-specific logic throughout the program.

Build a single-page Node.js scanner

The example below uses Node.js’s built-in HTTP and HTTPS modules, so it has no third-party package dependency. It fetches only the submitted URL, follows a small number of redirects, applies a timeout and response-size limit, and returns evidence for a handful of illustrative fingerprints. Node.js documents its outbound request and secure connection APIs in the HTTP and HTTPS references.

Save as detector.mjs and run with node detector.mjs https://example.com/. The catalog entries are demonstrations, not a comprehensive or validated technology database.

import http from 'node:http';
import https from 'node:https';
import { isIP } from 'node:net';

const MAX_BYTES = 1_000_000;
const TIMEOUT_MS = 8_000;
const MAX_REDIRECTS = 3;

// Illustrative rules only: verify signals and add tests before relying on them.
const fingerprints = [
  {
    name: 'WordPress',
    category: 'CMS',
    checks: [
      { type: 'html', pattern: /<metab[^>]*bname=["']generator["'][^>]*bcontent=["'][^"']*WordPress/i },
      { type: 'script URL', pattern: //wp-content/|/wp-includes//i },
    ],
  },
  {
    name: 'Shopify',
    category: 'E-commerce',
    checks: [
      { type: 'script URL', pattern: /cdn.shopify.com/i },
      { type: 'HTML', pattern: /Shopify.theme/i },
    ],
  },
  {
    name: 'Express',
    category: 'Web framework',
    checks: [
      { type: 'response header', header: 'x-powered-by', pattern: /^express$/i },
    ],
  },
];

function isPublicHostname(hostname) {
  const host = hostname.toLowerCase().replace(/^[|]$/g, '').replace(/.$/, '');
  if (host === 'localhost' || host.endsWith('.localhost') || host.endsWith('.local')) return false;
  if (!isIP(host)) return true; // Production code must resolve DNS and check every address.
  if (host.includes(':')) {
    // Reject IPv6 loopback, unique-local, link-local, and IPv4-mapped addresses.
    return !(host === '::1' || host.startsWith('fc') || host.startsWith('fd') ||
      host.startsWith('fe8') || host.startsWith('fe9') || host.startsWith('fea') ||
      host.startsWith('feb') || host.startsWith('::ffff:'));
  }
  const octets = host.split('.').map(Number);
  const [a, b] = octets;
  return !(a === 0 || a === 10 || a === 127 ||
    (a === 169 && b === 254) || (a === 172 && b >= 16 && b <= 31) ||
    (a === 192 && b === 168) || a >= 224);
}

function requestPage(url, redirects = 0) {
  return new Promise((resolve, reject) => {
    if (!['http:', 'https:'].includes(url.protocol)) return reject(new Error('Only HTTP and HTTPS URLs are allowed.'));
    if (!isPublicHostname(url.hostname)) return reject(new Error('The destination is not allowed.'));

    const transport = url.protocol === 'https:' ? https : http;
    const req = transport.get(url, { headers: { 'User-Agent': 'SmallTechDetector/1.0' } }, (res) => {
      const status = res.statusCode || 0;
      if ([301, 302, 303, 307, 308].includes(status) && res.headers.location) {
        res.resume();
        if (redirects >= MAX_REDIRECTS) return reject(new Error('Too many redirects.'));
        let next;
        try { next = new URL(res.headers.location, url); }
        catch { return reject(new Error('Invalid redirect URL.')); }
        // requestPage validates the next URL as well, including its scheme and hostname.
        return requestPage(next, redirects + 1).then(resolve, reject);
      }

      const chunks = [];
      let size = 0;
      res.on('data', (chunk) => {
        size += chunk.length;
        if (size > MAX_BYTES) {
          req.destroy(new Error('Response exceeded the 1 MB limit.'));
          return;
        }
        chunks.push(chunk);
      });
      res.on('end', () => resolve({
        status,
        headers: res.headers,
        body: Buffer.concat(chunks).toString('utf8'),
        finalUrl: url.href,
      }));
      res.on('error', reject);
    });

    req.setTimeout(TIMEOUT_MS, () => req.destroy(new Error('Request timed out.')));
    req.on('error', reject);
  });
}

function detect(page) {
  const evidenceText = page.body;
  const matches = [];
  for (const fingerprint of fingerprints) {
    const evidence = [];
    for (const check of fingerprint.checks) {
      if (check.type === 'response header') {
        const value = page.headers[check.header];
        if (value && check.pattern.test(String(value))) {
          evidence.push({ type: check.type, value: `${check.header}: ${value}` });
        }
      } else {
        const match = check.pattern.exec(evidenceText);
        if (match) evidence.push({ type: check.type, value: match[0].slice(0, 180) });
      }
    }
    if (evidence.length) matches.push({ name: fingerprint.name, category: fingerprint.category, evidence });
  }
  return matches;
}

const input = process.argv[2];
if (!input) {
  console.error('Usage: node detector.mjs https://example.com/');
  process.exit(1);
}

try {
  const url = new URL(input);
  const page = await requestPage(url);
  const result = {
    requestedUrl: url.href,
    finalUrl: page.finalUrl,
    httpStatus: page.status,
    matches: detect(page),
  };
  console.log(JSON.stringify(result, null, 2));
} catch (error) {
  console.error(`Scan failed: ${error.message}`);
  process.exitCode = 1;
}

Because HTML code examples must remain valid, the regular expressions above escape angle brackets in their HTML checks. In a JavaScript source file, use literal < and > characters in those regular expressions; HTML-escape them only when embedding source in an HTML article.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Harden URL fetching before accepting untrusted input

A scanner that accepts a user-supplied destination can be abused as a way to reach internal services. The example rejects common local and private literal addresses, but it is not a complete production-grade defense: domain names can resolve to private addresses, and DNS answers can change between validation and connection. A public service should resolve hostnames, reject loopback, private, link-local, and cloud metadata IP ranges, and ensure the connection uses an address that passed validation. Apply the same checks to every redirect destination.

  • Allow only http: and https:; reject other URL schemes.
  • Use a short request timeout, a small byte limit, and a fixed redirect limit.
  • Do not crawl links or forward arbitrary request headers, cookies, or credentials.
  • Return useful errors for DNS, TLS, timeout, redirect, and non-success HTTP status cases. A response with a non-success status is not itself a technology match.
  • If this becomes a service, add rate limits and logging, and avoid exposing the fetcher as an unrestricted proxy.

The code is an instructional starting point rather than a hardened public URL-fetching service. In particular, a deployed scanner needs robust DNS/IP validation at connection time and careful handling of unusual response encodings and compression.

Keep fingerprints data-driven and results inspectable

A catalog lets you add signals without growing a tangle of technology-specific conditionals. The Wappalyzer specification is a useful model for a structured record: it can describe evidence from headers, HTML, scripts, cookies, DNS, and dependencies between technologies. Start with fields you actually use, then extend the schema when a concrete rule needs another evidence type.

Return the evidence that caused each match. For example, a result might say that the X-Powered-By response header matched a rule, or that a script URL contained a recognizable path. A practical result record can include:

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Name and category: the technology label and broad type.
  • Evidence: signal type and the observed value or a safely shortened excerpt.
  • Version: include only when a specific signal supports a version claim; otherwise omit it.
  • Confidence label: optional, but define what each label means rather than inventing percentages.

One distinctive vendor-specific header may be stronger evidence than a generic word in page markup. Multiple independent signals can make a rule more useful, but no match should be presented as a complete inventory of a site’s stack.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Test each rule, including cases that should not match

For each fingerprint, keep a saved page or response fixture that contains the expected signal and at least one negative case that includes a similar but unrelated string. This helps catch generic substring rules that fire too easily. Test header matching separately from HTML and URL matching, and check that version extraction is not inferred from a presence-only marker.

Do not claim an accuracy rate without evaluating a defined set of sites and documenting how it was selected and labeled. A few successful fixtures show that rules behave as intended for those examples; they do not establish broad coverage or real-world precision.

When a small detector is enough—and when to use a lookup API

Decision axis Small Node.js detector Existing lookup API
Scope A limited, reader-maintained fingerprint catalog. Broader technology lookup and vendor-maintained data, depending on provider and plan. See BuiltWith API documentation and Wappalyzer API overview.
Freshness Depends on your fetch behavior and how you maintain rules. Wappalyzer documents cached and live analysis options in its technology lookup documentation.
Workflow A local command-line tool or custom endpoint you control. Wappalyzer positions its API for automation, enrichment, and embedded workflows; this is the provider’s own description, not an independent comparison. Wappalyzer FAQ
Cost and limits You maintain the infrastructure and rules. Check current plans, credits, rate limits, and terms with the provider. BuiltWith documents API-key authentication and lookup options in its Domain API documentation.
Data rights Your own rules still require responsible collection and use of public pages. BuiltWith states restrictions on reselling its data as-is and providing duplicate functionality; review the provider’s current terms before building around its data.

For one-off manual checks, Wappalyzer’s FAQ points readers to its website lookup or browser extension; for automated lookups or embedding in a workflow, it points to its API. That is Wappalyzer’s own positioning. BuiltWith documents root-domain lookups, API-key authentication, several response formats, multiple-domain lookup, and bulk jobs in its API documentation; the details and limits are product specifications that can change, so check the provider’s current documentation before depending on them.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Use third-party data and credentials carefully

If you choose a commercial API, keep its API key on the server or in a local environment variable—not in browser code or a public repository. Read the current terms before storing, republishing, or using returned data to build a competing dataset. A local detector with rules you wrote is a different project from copying another provider’s technology records.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
PC Slower Than It Used to Be?Free scan - under a minute
Crashes, No Sound, or Screen Glitches?Free driver scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.