Introduction to HashiCorp Nomad: Alternative Orchestrator

You've probably heard Kubernetes is the industry standard for container orchestration. It powers everything from small startups to Fortune 500 companies. But Kubernetes has a learning curve that can make even experienced DevOps engineers sweat. If you're looking for something simpler, more flexible, or just different, HashiCorp Nomad might be worth your attention.

Nomad is a flexible workload orchestrator that can schedule containers, VMs, and bare metal workloads. It's designed to be simple to operate while still offering powerful features. This guide will walk you through what Nomad is, how it compares to Kubernetes, and when you might want to use it instead of the Kubernetes ecosystem.

What is HashiCorp Nomad?

HashiCorp Nomad is a cluster manager and workload orchestrator developed by HashiCorp. Unlike Kubernetes, which focuses almost exclusively on containers, Nomad can schedule a wide variety of workloads:

• Containers: Docker and containerd images
• Virtual Machines: Launch and manage VMs on your infrastructure
• Bare Metal: Schedule physical servers and bare metal workloads
• Batch Jobs: Run one-off or periodic batch processing tasks
• Service Discovery: Built-in service discovery for your applications

This flexibility makes Nomad appealing for teams that need to manage diverse workloads without maintaining multiple orchestration tools. You can run your web applications in containers, your batch jobs as VMs, and your legacy services on bare metal—all from a single cluster manager.

Nomad Architecture: How It Works

Understanding Nomad's architecture helps explain why it's simpler than Kubernetes. Nomad has three main components:

1. Nomad Server

The Nomad server is responsible for maintaining the cluster state and making scheduling decisions. It runs in a quorum configuration (typically 3 or 5 nodes) to ensure high availability. The servers handle:

• Resource allocation and scheduling
• Client registration and health checks
• Job submission and status updates
• Consensus through the Raft consensus algorithm

2. Nomad Client

Clients are the agents that run on your worker nodes. They register with the servers and report their available resources (CPU, memory, disk, network). Clients execute jobs assigned to them by the servers. Clients can run:

• Containers using Docker or containerd
• VMs using your preferred hypervisor
• Bare metal processes directly

3. Consul

Consul is HashiCorp's service discovery and configuration management tool. Nomad integrates tightly with Consul for:

• Service registration and discovery
• Health checking of services
• Distributed configuration management
• Key-value storage for configuration

This integration means you get service discovery and health checking without needing a separate tool like CoreDNS or etcd.

Nomad vs Kubernetes: A Comparison

Choosing between Nomad and Kubernetes is a significant decision. Here's how they compare across key dimensions:

When to Choose Nomad

Nomad shines in scenarios where you need simplicity and flexibility:

• Mixed workload environments: You need to run containers, VMs, and bare metal workloads together
• Simple deployments: Your applications don't require complex orchestration features
• Limited DevOps resources: You want to manage fewer moving parts
• Quick setup: You need to get up and running quickly without months of learning

When to Choose Kubernetes

Kubernetes is the better choice when:

• Container-only workloads: You're running only containers
• Complex networking requirements: You need advanced networking features
• Large, mature ecosystems: You want access to thousands of add-ons and tools
• Team expertise: Your team already knows Kubernetes
• Enterprise requirements: You need advanced features like Pod Security Policies, Network Policies, etc.

Installing Nomad: A Quick Start

Let's get Nomad running locally to see how it works. This example uses Docker Compose for simplicity.

First, create a docker-compose.yml file:

version: '3.8'
 
services:
  nomad:
    image: hashicorp/nomad:latest
    container_name: nomad
    ports:
      - "4646:4646"  # HTTP API
      - "4647:4647"  # RPC
      - "4648:4648"  # Serf LAN
    volumes:
      - ./nomad:/etc/nomad.d
    command: agent -dev
    networks:
      - nomad-network
 
  consul:
    image: hashicorp/consul:latest
    container_name: consul
    ports:
      - "8500:8500"  # HTTP UI
      - "8600:8600/udp"  # DNS
    command: agent -dev
    networks:
      - nomad-network
 
networks:
  nomad-network:
    driver: bridge

Create a Nomad configuration file at nomad/nomad.hcl:

datacenter = "dc1"
data_dir = "/opt/nomad/data"
 
client {
  enabled = true
  server = false
  options = {
    "driver.allowlist" = "docker"
  }
}
 
consul {
  address = "127.0.0.1:8500"
  address = "127.0.0.1:4647"
}

Start the services:

docker-compose up -d

Verify Nomad is running:

curl http://localhost:4646/v1/status/leader

You should see the Nomad server address in the response.

Scheduling Your First Job

Now let's schedule a simple web application. Create a file named web-app.nomad:

job "web-app" {
  datacenters = ["dc1"]
  type = "service"
 
  group "web" {
    count = 2
 
    network {
      port "http" {
        to = 8080
      }
    }
 
    task "app" {
      driver = "docker"
 
      config {
        image = "nginx:alpine"
        ports = ["http"]
      }
 
      resources {
        cpu    = 100
        memory = 128
      }
    }
  }
}

Submit this job to Nomad:

nomad job run web-app.nomad

Check the job status:

nomad job status web-app

You should see two instances of the web app running, each with its own IP address and port. Nomad automatically handles load balancing and health checking.

Advanced Features: VM Scheduling

One of Nomad's unique capabilities is scheduling VMs. This is useful for running legacy applications or workloads that require specific OS configurations.

Create a VM job file vm-job.nomad:

job "legacy-app" {
  datacenters = ["dc1"]
  type = "service"
 
  group "legacy" {
    count = 1
 
    network {
      port "http" {
        to = 8080
      }
    }
 
    task "app" {
      driver = "qemu"
 
      config {
        image = "https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img"
        memory = 1024
        cpu = 1000
        ports = ["http"]
      }
 
      config {
        args = [
          "-m", "1024M",
          "-smp", "1",
          "-drive", "file=ubuntu-22.04.qcow2,if=virtio",
          "-netdev", "user,id=net0,hostfwd=tcp::8080-:80",
          "-device", "virtio-net-pci,netdev=net0"
        ]
      }
 
      resources {
        cpu    = 1000
        memory = 1024
      }
    }
  }
}

Submit and monitor the job:

nomad job run vm-job.nomad
nomad job status legacy-app

Nomad will launch a VM, configure it, and run your application. This is a powerful feature for hybrid environments that need to run both modern containers and legacy VMs.

Service Discovery with Consul

Nomad integrates with Consul for service discovery. When your application starts, it registers with Consul. Other services can discover it automatically.

Create a simple web service that registers with Consul:

// app.js
const http = require('http');
 
const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.end('Hello from Nomad!');
});
 
server.listen(8080, () => {
  console.log('Server running on port 8080');
});

Create a package.json:

{
  "name": "consul-demo",
  "version": "1.0.0",
  "main": "app.js",
  "scripts": {
    "start": "node app.js"
  }
}

Create a Nomad job file consul-demo.nomad:

job "consul-demo" {
  datacenters = ["dc1"]
  type = "service"
 
  group "web" {
    count = 2
 
    network {
      port "http" {
        to = 8080
      }
    }
 
    task "app" {
      driver = "docker"
 
      config {
        image = "node:18-alpine"
        args = ["node", "app.js"]
        ports = ["http"]
      }
 
      config {
        args = [
          "-e", "CONSUL_HTTP_ADDR=127.0.0.1:8500"
        ]
      }
 
      resources {
        cpu    = 100
        memory = 128
      }
    }
  }
}

Submit the job and check Consul:

nomad job run consul-demo.nomad
nomad job status consul-demo
curl http://localhost:8500/v1/catalog/service/consul-demo

The Consul API will return information about your running instances, including their IP addresses and health status.

Monitoring and Operations

Nomad provides built-in metrics and logging, but for production use, you'll want additional monitoring.

Nomad Metrics

Nomad exposes metrics on port 4646:

curl http://localhost:4646/v1/metrics

You can scrape these metrics with Prometheus:

# prometheus.yml
scrape_configs:
  - job_name: 'nomad'
    static_configs:
      - targets: ['localhost:4646']

Logging

Nomad logs to stdout/stderr by default. For centralized logging, you can use:

• Fluentd
• Logstash
• Loki
• ELK stack

Health Checks

Nomad supports multiple health check types:

• script: Execute a script and check the exit code
• http: HTTP GET request to a URL
• tcp: TCP connection check
• grpc: gRPC health check

Example health check configuration:

task "app" {
  driver = "docker"
 
  config {
    image = "nginx:alpine"
  }
 
  check {
    name     = "http-check"
    type     = "http"
    path     = "/"
    interval = "10s"
    timeout  = "2s"
  }
 
  resources {
    cpu    = 100
    memory = 128
  }
}

Production Considerations

Running Nomad in production requires some additional setup:

High Availability

Deploy Nomad servers in a quorum configuration (3 or 5 nodes) across different availability zones. Use the Raft consensus algorithm for fault tolerance.

Security

Enable TLS for all communication:

tls {
  http = true
  rpc  = true
  ca_file   = "/etc/nomad.d/tls/ca.pem"
  cert_file = "/etc/nomad.d/tls/nomad.pem"
  key_file  = "/etc/nomad.d/tls/nomad-key.pem"
}

Backup and Recovery

Regularly backup the Nomad state directory:

tar -czf nomad-backup-$(date +%Y%m%d).tar.gz /opt/nomad/data

Resource Quotas

Set resource quotas to prevent runaway jobs:

namespace "production" {
  quota {
    cpu    = 10000
    memory = 40960
  }
}

Conclusion

HashiCorp Nomad offers a compelling alternative to Kubernetes for teams that want simplicity and flexibility. Its ability to schedule containers, VMs, and bare metal workloads from a single cluster manager makes it unique in the orchestration space.

The key takeaways are:

• Nomad is simpler to learn and operate than Kubernetes
• It supports diverse workloads beyond containers
• Built-in service discovery via Consul reduces tooling complexity
• It's ideal for mixed environments and teams with limited DevOps resources

If you're running only containers and need Kubernetes's massive ecosystem, stick with Kubernetes. But if you want a simpler, more flexible orchestrator that can handle multiple workload types, Nomad is worth exploring.

Platforms like ServerlessBase can help you deploy and manage Nomad clusters alongside your other applications, providing a unified interface for your entire infrastructure.

Next Steps

Ready to try Nomad yourself? Start with the official HashiCorp documentation and the quick start guide. Install Nomad locally using Docker, schedule a few jobs, and explore its features. You might find that Nomad's simplicity makes it the perfect fit for your infrastructure needs.