How to Set Up a Linux Server from Scratch

You've probably deployed applications on shared hosting or managed platforms. But there comes a point when you need full control over your infrastructure. Setting up a Linux server from scratch gives you that control, but it also means you're responsible for everything—security, updates, networking, and performance tuning.

This guide walks you through deploying a production-ready Linux server from bare metal or a cloud provider. We'll use Ubuntu Server 22.04 LTS as our reference distribution, but the principles apply to any Linux distribution.

Choosing the Right Linux Distribution

Before you even boot the installer, you need to pick a distribution. The choice affects your tooling, documentation, and community support.

Ubuntu LTS is the safest choice for most users. It has the largest community, the most tutorials, and enterprise backing from Canonical. Debian is excellent if you want a pure, community-driven distribution. Rocky Linux and AlmaLinux are Red Hat alternatives if you're migrating from RHEL.

For this guide, we'll use Ubuntu Server 22.04 LTS.

Step 1: Provision the Server

If you're using a cloud provider, create a new instance with these minimum specifications:

• OS: Ubuntu Server 22.04 LTS 64-bit
• RAM: 2GB minimum (4GB recommended)
• CPU: 1 vCPU minimum (2 vCPUs recommended)
• Storage: 40GB SSD minimum
• Network: Public IP address required
• Security Group: Allow SSH (22), HTTP (80), HTTPS (443)

If you're using bare metal, boot from the Ubuntu Server installation media and follow the on-screen prompts. Select "Install Ubuntu Server" and choose your language.

Important: Always use a strong password during installation. The default password is never secure enough for production.

Step 2: Initial Security Configuration

The moment your server gets an IP address, it becomes a target. Attackers scan the internet for open ports and vulnerable services. You need to harden your server before installing anything else.

Change the Default SSH Port

The default SSH port (22) is scanned constantly by bots. Changing it reduces automated attacks significantly.

# Edit the SSH configuration
sudo nano /etc/ssh/sshd_config
 
# Find and modify these lines
Port 2222
PermitRootLogin no
PasswordAuthentication no
 
# Restart SSH to apply changes
sudo systemctl restart sshd

Now connect using ssh -p 2222 user@your-server-ip. Update your firewall to allow port 2222 instead of 22.

Configure the Firewall

Ubuntu uses ufw (Uncomplicated Firewall) by default. Enable it and allow only necessary ports:

# Enable firewall
sudo ufw enable
 
# Allow SSH (replace 2222 with your custom port)
sudo ufw allow 2222/tcp
 
# Allow HTTP and HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
 
# Deny all other incoming traffic by default
sudo ufw default deny incoming
 
# Allow all outgoing traffic
sudo ufw default allow outgoing
 
# Check status
sudo ufw status verbose

Your firewall should show only SSH, HTTP, and HTTPS as allowed incoming connections.

Create a Non-Root User

Never log in as root directly. Create a regular user with sudo privileges instead:

# Create a new user
sudo adduser yourusername
 
# Add user to sudo group
sudo usermod -aG sudo yourusername
 
# Switch to the new user
su - yourusername
 
# Test sudo access
sudo whoami

You should see root output, confirming sudo works.

Configure SSH Key Authentication

Password authentication is convenient but insecure. SSH keys are the standard for secure remote access.

# Generate SSH key pair on your local machine
ssh-keygen -t ed25519 -C "your_email@example.com"
 
# Copy the public key to the server
ssh-copy-id -p 2222 yourusername@your-server-ip
 
# Test key-based login
ssh -p 2222 yourusername@your-server-ip

Now disable password authentication entirely in /etc/ssh/sshd_config:

PasswordAuthentication no
PubkeyAuthentication yes

Restart SSH and verify you can still log in with your key.

Step 3: System Updates and Basic Tools

Your server comes with outdated packages. Update everything before installing additional software:

# Update package lists
sudo apt update
 
# Upgrade all installed packages
sudo apt upgrade -y
 
# Install essential tools
sudo apt install -y curl wget git vim htop ufw fail2ban unattended-upgrades

Why these tools?

• curl and wget: HTTP client tools for testing and downloads
• git: Version control
• vim: Text editor (you'll need it for configuration files)
• htop: Interactive process viewer (better than top)
• ufw: Firewall management
• fail2ban: Brute-force attack protection
• unattended-upgrades: Automatic security updates

Step 4: Install and Configure Fail2Ban

Fail2Ban monitors log files and bans IP addresses that repeatedly fail authentication attempts. It's your first line of defense against brute-force attacks.

# Enable fail2ban service
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
 
# Check status
sudo fail2ban-client status
 
# Create a custom jail for SSH
sudo nano /etc/fail2ban/jail.local

Add this configuration to jail.local:

[sshd]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600

Restart fail2ban to apply changes:

sudo systemctl restart fail2ban

Now test it by intentionally failing SSH authentication three times. You'll see the IP banned in the fail2ban status.

Step 5: Set Up Automatic Security Updates

Unpatched vulnerabilities are the #1 cause of server compromises. Configure unattended-upgrades to install security patches automatically.

# Edit the configuration
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades
 
# Enable automatic updates
sudo nano /etc/apt/apt.conf.d/10periodic
 
# Add these lines to 10periodic
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "1";
APT::Periodic::AutocleanInterval "7";
APT::Periodic::MinAge "2";

This configuration updates packages weekly and applies security upgrades automatically. You can also configure email notifications for failed updates.

Step 6: Configure Time Synchronization

Accurate time is critical for logging, SSL certificates, and distributed systems. Use systemd-timesyncd, which is included by default:

# Check the current time
timedatectl
 
# Enable NTP synchronization
sudo timedatectl set-ntp true
 
# Verify it's working
timedatectl status

You should see NTP service: active and System clock synchronized: yes.

Step 7: Set Up a Swap File

If your server runs out of RAM, the kernel starts swapping to disk, which dramatically slows performance. A swap file provides a safety net for memory spikes.

# Check available RAM
free -h
 
# Create a 2GB swap file (adjust size as needed)
sudo fallocate -l 2G /swapfile
 
# Set correct permissions
sudo chmod 600 /swapfile
 
# Mark the file as swap space
sudo mkswap /swapfile
 
# Enable the swap file
sudo swapon /swapfile
 
# Make the swap file permanent
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
 
# Adjust swappiness (0-100, higher = more aggressive swapping)
sudo sysctl vm.swappiness=10
 
# Make swappiness persistent
echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf

Check swap status with free -h. You should now see the swap file listed.

Step 8: Install and Configure a Web Server

Now that your server is secure, install a web server. We'll use Nginx because it's lightweight, fast, and widely used.

# Install Nginx
sudo apt install -y nginx
 
# Start and enable Nginx
sudo systemctl enable nginx
sudo systemctl start nginx
 
# Check the status
sudo systemctl status nginx

Visit your server's IP address in a browser. You should see the default Nginx welcome page.

Configure Nginx

Edit the default site configuration:

sudo nano /etc/nginx/sites-available/default

Update the server block:

server {
    listen 80;
    listen [::]:80;
 
    server_name your-domain.com;  # Replace with your domain or IP
 
    root /var/www/html;
    index index.html index.htm index.nginx-debian.html;
 
    location / {
        try_files $uri $uri/ =404;
    }
 
    # Security headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;
}

Test the configuration:

sudo nginx -t

If the test passes, reload Nginx:

sudo systemctl reload nginx

Step 9: Set Up a Database

Most applications need a database. We'll install PostgreSQL, a robust relational database.

# Install PostgreSQL
sudo apt install -y postgresql postgresql-contrib
 
# Start and enable PostgreSQL
sudo systemctl enable postgresql
sudo systemctl start postgresql
 
# Check status
sudo systemctl status postgresql

PostgreSQL creates a postgres user by default. Switch to this user and create a new database:

# Switch to postgres user
sudo -u postgres psql
 
# Create a new database and user
CREATE DATABASE myapp;
CREATE USER myappuser WITH PASSWORD 'secure_password_here';
GRANT ALL PRIVILEGES ON DATABASE myapp TO myappuser;
\q

Exit the PostgreSQL prompt and test the connection:

sudo -u postgres psql -d myapp -U myappuser -c "SELECT version();"

You should see the PostgreSQL version output.

Step 10: Configure SSL/TLS with Let's Encrypt

HTTPS is no longer optional. Let's Encrypt provides free SSL certificates that automatically renew.

Install Certbot

# Install Certbot and the Nginx plugin
sudo apt install -y certbot python3-certbot-nginx

Obtain and Configure the Certificate

# Get a certificate and configure Nginx automatically
sudo certbot --nginx -d your-domain.com

Certbot will ask for your email and agree to the terms. It will then configure SSL and set up automatic renewal.

Verify the renewal works:

sudo certbot renew --dry-run

If you see "Simulation successful," you're all set. Certbot will automatically renew certificates before they expire.

Step 11: Set Up Monitoring

You can't manage what you don't measure. Install basic monitoring tools to track CPU, memory, and disk usage.

# Install htop (already installed in Step 3)
htop
 
# Install monitoring tools
sudo apt install -y sysstat iotop nethogs

Create a Basic Monitoring Script

Create a script to check server health:

nano ~/monitor.sh

Add this content:

#!/bin/bash
 
echo "=== Server Health Check $(date) ==="
echo ""
 
echo "CPU Usage:"
top -bn1 | grep "Cpu(s)"
echo ""
 
echo "Memory Usage:"
free -h
echo ""
 
echo "Disk Usage:"
df -h
echo ""
 
echo "Top 5 Processes by Memory:"
ps aux --sort=-%mem | head -n 6
echo ""
 
echo "Top 5 Processes by CPU:"
ps aux --sort=-%cpu | head -n 6

Make it executable:

chmod +x ~/monitor.sh

Run it to see your server's current state:

./monitor.sh

Step 12: Deploy Your First Application

Now deploy a simple application to test everything. We'll use a Node.js application.

Install Node.js

# Install Node.js and npm
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt install -y nodejs
 
# Verify installation
node --version
npm --version

Create a Simple Application

# Create a project directory
mkdir ~/myapp
cd ~/myapp
 
# Initialize a Node.js project
npm init -y
 
# Install Express
npm install express
 
# Create the application
nano app.js

Add this code to app.js:

const express = require('express');
const app = express();
const port = 3000;
 
app.get('/', (req, res) => {
  res.send('Hello from your Linux server!');
});
 
app.listen(port, () => {
  console.log(`Server running at http://localhost:${port}`);
});

Run the Application

# Start the application
node app.js

Visit http://your-server-ip:3000 in your browser. You should see the "Hello from your Linux server!" message.

Configure Nginx as a Reverse Proxy

Update your Nginx configuration to proxy traffic to the Node.js application:

sudo nano /etc/nginx/sites-available/default

Replace the location / block with:

location / {
    proxy_pass http://localhost:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
}

Test and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Now visit http://your-domain.com (or your server's IP). The request is proxied to Node.js, and you see the application response.

Step 13: Set Up Automated Backups

Data loss is inevitable. You need automated backups with a retention policy.

Install Backup Tools

sudo apt install -y rsync tar

Create a Backup Script

nano ~/backup.sh

Add this content:

#!/bin/bash
 
# Configuration
BACKUP_DIR="/var/backups/myapp"
SOURCE_DIR="/home/myapp"
DATE=$(date +%Y%m%d_%H%M%S)
RETENTION_DAYS=7
 
# Create backup directory
mkdir -p $BACKUP_DIR
 
# Create compressed archive
tar -czf $BACKUP_DIR/myapp_backup_$DATE.tar.gz -C $SOURCE_DIR .
 
# Remove old backups
find $BACKUP_DIR -name "myapp_backup_*.tar.gz" -mtime +$RETENTION_DAYS -delete
 
echo "Backup completed: $BACKUP_DIR/myapp_backup_$DATE.tar.gz"

Make it executable:

chmod +x ~/backup.sh

Test the Backup

./backup.sh

Check the backup directory:

ls -lh /var/backups/myapp/

You should see a timestamped archive file.

Schedule Automatic Backups

# Edit crontab
crontab -e

Add this line to run backups daily at 2 AM:

0 2 * * * /home/yourusername/backup.sh >> /home/yourusername/backup.log 2>&1

Verify Cron is Running

sudo systemctl status cron

Your backups will now run automatically every day.

Step 14: Configure Log Rotation

Logs grow indefinitely and can fill your disk. Configure log rotation to manage log file sizes.

sudo nano /etc/logrotate.d/nginx

Add this configuration:

/var/log/nginx/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        [ -f /var/run/nginx.pid ] && kill -USR1 `cat /var/run/nginx.pid`
    endscript
}

This configuration:

• Rotates logs daily
• Keeps 14 days of logs
• Compresses old logs
• Doesn't rotate if the log is empty
• Creates new logs with proper permissions

Test the configuration:

sudo logrotate -f /etc/logrotate.d/nginx

Check if the log files were rotated and compressed.

Step 15: Document Your Server

Documentation is often overlooked until you need it. Create a server documentation file with all your configurations.

nano ~/server-docs.md

Add this template:

# Server Documentation
 
## Server Information
- **Hostname**: your-server-name
- **IP Address**: your-server-ip
- **OS**: Ubuntu Server 22.04 LTS
- **Provisioned**: YYYY-MM-DD
 
## Access Information
- **SSH Port**: 2222
- **SSH User**: yourusername
- **SSH Key**: ~/.ssh/id_ed25519.pub
- **Web Server**: Nginx
- **Database**: PostgreSQL
 
## Services Running
- Nginx (Web Server)
- PostgreSQL (Database)
- Node.js Application (Port 3000)
 
## Security Configuration
- Firewall: ufw (allow 2222, 80, 443)
- Fail2Ban: Enabled
- Automatic Updates: Enabled
- SSL/TLS: Let's Encrypt
 
## Backup Information
- Backup Location: /var/backups/myapp/
- Retention: 7 days
- Schedule: Daily at 2 AM
 
## Monitoring
- Health Check Script: ~/monitor.sh
- Log Location: /var/log/nginx/, /var/log/syslog
 
## Important Commands
- SSH: ssh -p 2222 yourusername@your-server-ip
- Restart Nginx: sudo systemctl restart nginx
- Restart PostgreSQL: sudo systemctl restart postgresql
- Check Logs: sudo tail -f /var/log/nginx/access.log

Keep this file updated whenever you make changes to your server.

Conclusion

Setting up a Linux server from scratch is a multi-step process that requires attention to security, automation, and documentation. You've now configured:

• A hardened server with SSH key authentication and a firewall
• Automatic security updates and fail2ban protection
• A web server (Nginx) with SSL/TLS certificates
• A database (PostgreSQL) for your applications
• Monitoring tools to track server health
• Automated backups with a retention policy
• Log rotation to manage disk space

The next step is to deploy your actual application. Platforms like ServerlessBase can simplify the deployment process by handling reverse proxy configuration, SSL certificates, and monitoring automatically, so you can focus on building your application rather than managing infrastructure.

Remember that server administration is an ongoing process. Keep your system updated, monitor your logs regularly, and document any changes you make. A well-maintained server is secure, reliable, and easy to manage.

Next Steps

1. Deploy your application: Follow your application's deployment guide, using the Nginx reverse proxy configuration you set up.
2. Set up monitoring alerts: Configure alerts for high CPU usage, disk space, or failed logins.
3. Implement a backup strategy: Test your backups regularly and verify they can be restored.
4. Learn more about Linux administration: Explore topics like systemd services, containerization, and Kubernetes.
5. Consider a management platform: Tools like ServerlessBase can automate many of these tasks and provide a unified interface for managing multiple servers.

Your server is now production-ready. Treat it with care, and it will serve your applications reliably for years.