Skip to content

Troubleshooting

Common issues and solutions for MESH deployments.

Connection issues

Can't Connect to Control plane

Symptoms:

  • meshcli up fails with connection error
  • "Unable to reach control plane" message
  • Timeout errors

Solutions:

  1. Verify control plane URL
curl https://mesh.yourdomain.com/health
# Should return: {"status":"ok"}
  1. Check DNS resolution
nslookup mesh.yourdomain.com
dig mesh.yourdomain.com
  1. Check firewall
# Test HTTPS connectivity
telnet mesh.yourdomain.com 443

# Check if port is open
nc -zv mesh.yourdomain.com 443
  1. Verify control plane is running
docker ps | grep headscale
docker logs headscale
  1. Check reverse proxy
# Nginx
sudo nginx -t
sudo systemctl status nginx

# Caddy
sudo systemctl status caddy
  1. Try different network
  2. Switch from WiFi to mobile data (or vice versa)
  3. Some networks block VPN traffic

Daemon Won't Start

Symptoms:

  • tailscaled-amnezia fails to start
  • Socket errors
  • Permission denied errors

Solutions:

  1. Check if already running
ps aux | grep tailscaled
sudo pkill tailscaled-amnezia
  1. Check socket permissions
sudo rm -rf /var/run/mesh/tailscaled.sock
sudo mkdir -p /var/run/mesh
sudo chmod 755 /var/run/mesh
  1. Check state directory
sudo mkdir -p /var/lib/mesh
sudo chmod 755 /var/lib/mesh
  1. Run with verbose logging
sudo tailscaled-amnezia \
  --socket=/var/run/mesh/tailscaled.sock \
  --state=/var/lib/mesh/tailscaled.state \
  --statedir=/var/lib/mesh \
  --verbose=2
  1. Check systemd service (if using)
sudo systemctl status mesh
sudo journalctl -u mesh -f

Authentication Fails

Symptoms:

  • "Invalid auth key" error
  • "Auth key expired" error
  • Login prompt doesn't appear

Solutions:

  1. Verify pre-auth key
# On control plane
docker exec headscale headscale preauthkeys list
  1. Generate new key
docker exec headscale headscale preauthkeys create --expiration 24h --reusable
  1. Check key expiration
  2. Pre-auth keys expire after the specified time

  3. Generate a new key if expired

  4. Try interactive authentication

# Without --authkey
sudo meshcli up --login-server=https://mesh.yourdomain.com
# Follow the URL to authenticate

No Mesh IP Assigned

Symptoms:

  • meshcli status doesn't show a mesh IP address
  • Node appears offline
  • Can't communicate with other nodes

Solutions:

  1. Verify pre-auth key is valid
# On control plane, check if key is valid and not expired
docker exec headscale headscale preauthkeys list
  1. Check control plane logs
docker logs headscale
# Look for registration errors
  1. Ensure user exists
# List users
docker exec headscale headscale users list

# Create user if missing
docker exec headscale headscale users create default
  1. Try reconnecting
sudo ./meshcli down
sudo ./meshcli up --login-server=https://mesh.yourdomain.com --authkey=YOUR_KEY
  1. Check node registration
# On control plane
docker exec headscale headscale nodes list
# Verify your node appears in the list

Connectivity issues

No Peer-to-Peer Connection (Using DERP Relay)

Symptoms:

  • meshcli status shows DERP address in CurAddr
  • Slow connection speeds
  • High latency

Solutions:

  1. Check NAT type
sudo meshcli netcheck
  1. Check UDP connectivity
# Test if UDP is blocked
nc -u -v mesh.yourdomain.com 3478
  1. Check firewall rules
# Linux
sudo iptables -L -n -v
sudo ufw status

# Allow WireGuard port (usually random high port)
sudo ufw allow 41641/udp
  1. Enable UPnP/NAT-PMP on router
  2. Check router settings

  3. Enable UPnP or NAT-PMP for automatic port forwarding

  4. Try different network

  5. Some networks (corporate, public WiFi) block UDP

  6. Try mobile data or different WiFi network

  7. Force direct connection

export TS_DEBUG_ALWAYS_USE_DERP=false
sudo -E meshcli up --login-server=https://mesh.yourdomain.com

Peers Not Visible

Symptoms:

  • meshcli status --peers shows no peers
  • Devices don't appear in mesh
  • Can't ping peers

Solutions:

  1. Verify both devices are connected
sudo meshcli status
# Should show "Running"
  1. Check ACLs
# On control plane
docker exec headscale headscale nodes list
cat config/acl.yaml
  1. Verify same control plane

  2. Ensure all devices connect to the same control plane URL

  3. Check user assignment

docker exec headscale headscale nodes list
# Verify nodes are in correct user/group
  1. Restart daemon
sudo systemctl restart mesh
# or
sudo pkill tailscaled-amnezia
sudo tailscaled-amnezia ...

Connection Drops Frequently

Symptoms:

  • Intermittent connectivity
  • Frequent reconnections
  • "Peer offline" messages

Solutions:

  1. Check network stability
ping -c 100 8.8.8.8
# Look for packet loss
  1. Disable power saving (mobile)
  2. Android: Settings > Battery > Optimize battery usage > MESH > Don't optimize

  3. Disable "Adaptive battery" for MESH app

  4. Increase keepalive

# Edit WireGuard config to add:
PersistentKeepalive = 25
  1. Check DERP relay stability
sudo meshcli netcheck
# Check DERP latencies
  1. Monitor logs
sudo journalctl -u mesh -f
# Look for errors or warnings

ADB Issues

ADB Connection Fails

Symptoms:

  • adb connect fails
  • "Connection refused" error
  • "Unable to connect" error

Solutions:

  1. Verify device is online
sudo meshcli status --peers
ping 100.64.X.X
  1. Check ADB port
# On Android device (via USB)
adb shell getprop service.adb.tcp.port
# Should return 5555
  1. Enable ADB-over-WiFi
# Via USB
adb tcpip 5555
adb connect 100.64.X.X:5555
  1. Restart ADB
adb kill-server
adb start-server
adb connect 100.64.X.X:5555
  1. Check firewall on Android
  2. Some security apps block ADB

  3. Temporarily disable and test

  4. Verify ADB is enabled in MESH app

  5. Open MESH app
  6. Settings > ADB-over-WiFi > Enabled

ADB Disconnects Frequently

Symptoms:

  • ADB connection drops during operations
  • "device offline" errors
  • Need to reconnect frequently

Solutions:

  1. Disable power saving

  2. Settings > Battery > MESH > Don't optimize

  3. Keep MESH app in foreground

  4. Don't minimize the app during ADB operations

  5. Use persistent connection

# Keep pinging to maintain connection
ping 100.64.X.X &
adb connect 100.64.X.X:5555
  1. Check network stability
  2. Ensure stable WiFi/mobile connection
  3. Avoid switching networks during operation

Performance issues

Slow Transfer Speeds

Symptoms:

  • Slow file transfers
  • High latency
  • Timeouts during large operations

Solutions:

  1. Check connection type
sudo meshcli status --json | jq '.Peer[] | {name: .HostName, addr: .CurAddr}'
# Direct connection is faster than DERP relay
  1. Disable AmneziaWG obfuscation
# If not needed for censorship resistance
sudo rm /etc/mesh/amneziawg.conf
sudo systemctl restart mesh
  1. Use lighter obfuscation
  2. Switch from Heavy to Balanced or Light preset

  3. See AmneziaWG Configuration

  4. Check network bandwidth

# Test bandwidth
iperf3 -s  # On one peer
iperf3 -c 100.64.X.X  # On other peer
  1. Compress data
# Compress before transfer
adb shell tar -czf /sdcard/data.tar.gz /data/local/tmp/
adb pull /sdcard/data.tar.gz

High CPU Usage

Symptoms:

  • High CPU usage by tailscaled-amnezia
  • Device heating up
  • Battery drain (mobile)

Solutions:

  1. Disable obfuscation
  2. AmneziaWG adds CPU overhead

  3. Disable if not needed

  4. Reduce connection checks

# Increase check interval
sudo meshcli set --netcheck-interval=10m
  1. Limit peer count
  2. Use ACLs to limit visible peers

  3. Only connect to necessary devices

  4. Check for loops

# Look for routing loops
sudo meshcli routes

Control plane Issues

Headscale Won't Start

Symptoms:

  • Docker container exits immediately
  • "Address already in use" errors
  • Database errors

Solutions:

  1. Check logs
docker logs headscale
  1. Check port conflicts
sudo lsof -i :8080
sudo lsof -i :9090
sudo lsof -i :3478
  1. Verify config syntax
cat config/config.yaml
# Check for YAML syntax errors
  1. Check database
ls -la data/db.sqlite
sudo chown -R 1000:1000 data/
  1. Reset database (WARNING: deletes all data)
docker-compose down
rm data/db.sqlite
docker-compose up -d

Nodes Not Registering

Symptoms:

  • Nodes don't appear in headscale nodes list
  • Authentication succeeds but node not visible
  • "Node not found" errors

Solutions:

  1. Check node list
docker exec headscale headscale nodes list
  1. Check user exists
docker exec headscale headscale users list
  1. Create user if missing
docker exec headscale headscale users create analyst1
  1. Check ACLs
cat config/acl.yaml
# Verify ACLs allow the connection
  1. Register manually
# Get node key from client logs
docker exec headscale headscale nodes register --user analyst1 --key nodekey:abc123

DERP Relay Not Working

Symptoms:

  • Peers can't connect even via relay
  • "No DERP home" errors
  • All connections fail

Solutions:

  1. Check STUN port
sudo netstat -ulnp | grep 3478
docker logs headscale | grep STUN
  1. Test STUN
sudo apt install stun-client
stun mesh.yourdomain.com 3478
  1. Check DERP config
cat config/config.yaml | grep -A 20 "derp:"
  1. Verify DERP is enabled
derp:
  server:
    enabled: true
    stun_listen_addr: "0.0.0.0:3478"
  1. Check firewall
sudo ufw allow 3478/udp

Android App Issues

App Crashes

Symptoms:

  • App closes unexpectedly
  • "MESH has stopped" error
  • Can't open app

Solutions:

  1. Clear app data
  2. Settings > Apps > MESH > Storage > Clear Data

  3. Reconnect to mesh

  4. Reinstall app

adb uninstall com.barghest.mesh
adb install mesh-android.apk
  1. Check logs
adb logcat | grep MESH
adb logcat | grep AndroidRuntime
  1. Check Android version
  2. MESH requires Android 8.0+
  3. Update Android if possible

VPN Permission Denied

Symptoms:

  • "VPN permission required" error
  • Can't connect to mesh
  • Permission dialogue doesn't appear

Solutions:

  1. Grant permission manually
  2. Settings > Apps > MESH > Permissions

  3. Enable VPN permission

  4. Check for conflicting VPN

  5. Disconnect other VPN apps

  6. Only one VPN can be active at a time

  7. Reinstall app

  8. Uninstall and reinstall to reset permissions

Diagnostic commands

Collect Logs

# Analyst client logs
sudo journalctl -u mesh --since "1 hour ago" > mesh-logs.txt

# Control plane logs
docker logs headscale > headscale-logs.txt

# Android logs
adb logcat -d > android-logs.txt

Generate Bug Report

# Analyst client
sudo meshcli bugreport --diagnose > mesh-bugreport.txt

# Android
adb bugreport bugreport.zip

Network Diagnostics

# Check NAT type and DERP latencies
sudo meshcli netcheck

# Trace route to peer
sudo meshcli ping 100.64.X.X --verbose

# Check WireGuard interface
ip addr show tun0
sudo wg show

Connection Diagnostics

# Detailed status
sudo meshcli status --json | jq

# Check peer connectivity
for peer in $(sudo meshcli status --json | jq -r '.Peer[].TailscaleIPs[0]'); do
    echo "Pinging $peer..."
    sudo meshcli ping $peer --c 3
done

# Monitor connection changes
watch -n 1 'sudo meshcli status --peers'

Getting Help

If you can't resolve your issue:

  1. Check GitHub Issues: github.com/BARGHEST-ngo/mesh/issues
  2. Create Bug Report: Include logs, config, and steps to reproduce
  3. Contact BARGHEST: barghest.asia

Next steps