Smith (Agent Smith) GitHub release (latest SemVer)

Smith Fleet Management

Smith is an open-source fleet management system built in Rust for managing distributed IoT devices at scale. Born from managing thousands of devices across healthcare facilities, Smith provides the automation and reliability you need to deploy, monitor, and update your fleet with confidence.

Why Smith?

  • Built for Scale: Manage everything from hundreds to thousands of devices
  • Reliable: Designed for >99% uptime in critical environments
  • Seamless Updates: Deploy upgrades and rollbacks with zero manual intervention
  • Transparent: Open-source infrastructure you can trust and extend

Architecture

Smith consists of five main components:

  • smithd: Daemon that runs on each device to execute deployments and report status
  • updater: Daemon that keeps smithd up to date
  • api: Backend service managing deployment configurations and fleet status
  • dashboard: Visual interface to monitor your fleet in real-time
  • cli (sm): Command-line tool for fleet administrators

Getting Started

Visit our documentation to get started with Smith.

Contributing

We welcome contributions! Whether you're fixing bugs, adding features, or improving documentation, your help makes Smith better for everyone. Check out our issues or submit a PR.

Security

Security is a top priority for Smith. If you discover a security vulnerability, please email security@teton.ai. We operate a bug bounty program and have paid bounties for responsibly disclosed vulnerabilities. See our Security Policy for more details.

License

The Smith source and documentation are released under the Apache License 2.0

Getting Started

Installing the Smith CLI (Command Line Interface)

Download and install

Let's install sm the Smith CLI (Command Line Interface) on your system using the terminal commands below.

curl -fsSL https://docs.smith.teton.ai/install.sh | sh

Testing your installation

After installing the sm CLI, you can test your installation by running the following command:

sm --version

Use sm help to see help text documenting Smith's flags and usage.

Getting Started

Follow this guide to get started with local development.

Setup

First, you will need the following stuff installed/setup:

  • Docker
  • 1password
  • node

Then run make init. This will initialize the repo with all the stuff you need. At the time of writing this will

  • Create .env files (.env and dashboard/.env) for you. Please fill in the missing values

Starting the api

Run make up, then make migrate and finally make dev to start the api.

Try running

curl http://localhost:8080/health

to check that it's running as it should. It should return something like

I'm good: <version number>

Starting the dashboard

Run the following

cd dashboard
npm i
npm run dev

The dashboard should now be running on localhost:3000

Starting smithd

Just run make run, which will start smithd and create a magic.toml for you.

You can now open the dashboard, and see a device in the "Pending Approval" box. Press approve, and you should be golden.

Starting the CLI

Run

cargo run --bin sm

You probably need to run it some times initially since it needs to create some defaults. If it shows a list of commands, you are golden.

CLI Overview

Command line tool (sm)

Smith provides a command line tool for communicating with the Smith API to manage your fleet of devices.

This tool is named sm.

For configuration, sm looks for configuration files in the $HOME/.config/smith directory. You can manage multiple profiles and switch between them using the sm profile command.

This overview covers sm syntax, describes the command operations, and provides common examples. For details about each command, see the individual command reference pages.

Installation

See the Installation Guide for installation instructions.

Syntax

Use the following syntax to run sm commands from your terminal window:

sm [command] [TYPE] [NAME] [flags]

where command, TYPE, NAME, and flags are:

  • command: Specifies the operation that you want to perform on one or more resources, for example get, status, restart, logs.

  • TYPE: Specifies the resource type. Resource types support aliases and you can specify the full name or abbreviated forms. For example, the following commands produce the same output:

    sm get device ABC123
    sm get devices ABC123
    sm get d ABC123
    
  • NAME: Specifies the name of the resource (device serial number or ID). Names are case-sensitive. If the name is omitted, details for all resources are displayed, for example sm get device.

    When performing an operation on multiple resources, you can specify each resource by name or use filters:

    • To specify multiple resources by name:

      sm get device ABC123 XYZ789 DEF456
      
    • To filter resources by labels:

      sm get device --label env=production --label region=us-west
      sm get device -l env=production -l region=us-west
      
    • To filter resources by status:

      sm get device --online
      sm get device --offline
      
    • To search with partial matching:

      sm get device rpi nano --search
      sm get device rpi -s
      
  • flags: Specifies optional flags. For example, you can use the --json flag to output results in JSON format, or --nowait to queue commands asynchronously.

Common Resource Type Aliases

Resource TypeAliasesExample
devicedevices, dsm get d --online
serviceservices, svcsm status svc nginx ABC123
commandscmdssm get cmds ABC123

Device Selection

Most commands support flexible device selection through a common set of flags:

  • Positional IDs: Specify one or more device serial numbers or IDs
  • --label KEY=VALUE (-l): Filter by labels (can be used multiple times)
  • --online: Show only online devices (last seen < 5 minutes)
  • --offline: Show only offline devices (last seen >= 5 minutes)
  • --search (-s): Enable partial matching for device IDs (matches serial number, hostname, or model)

Examples

# Single device by serial number
sm get d ABC123

# Multiple devices
sm get d ABC123 XYZ789

# All devices with a label
sm get d -l env=production

# Search for devices (partial match)
sm get d rpi -s

# Multiple search terms
sm get d rpi nano -s

# Combine filters
sm get d -l env=staging --online

Authentication

Before using the CLI, you need to authenticate:

sm auth login

This will open your browser for authentication. Use --no-open to prevent automatic browser launch.

To logout:

sm auth logout

To view your current token:

sm auth show

Common Workflows

Checking device status

# List all devices
sm get d

# Check which devices are online
sm get d --online

# Check smithd status on a device
sm status d ABC123

# Check if updates are available
sm status d -l env=production

Troubleshooting

# Get device logs
sm logs ABC123

# Check service status
sm status svc smithd ABC123

# Restart a service
sm restart svc smithd ABC123

# Test network connectivity
sm test-network ABC123

# Run custom diagnostic command
sm run ABC123 -w -- dmesg | tail -n 50

Managing labels

# Add environment label
sm label ABC123 env=production

# Query by label
sm get d -l env=production

# Update multiple devices
sm label -s rpi region=warehouse-1

Tips

  • Use aliases to save typing: sm get d instead of sm get device, sm status svc instead of sm status service
  • Use short flags where available: -l for --label, -s for --search, -j for --json, -o for --output
  • Use --search (-s) flag for fuzzy matching when you don't remember the full serial number
  • Combine multiple labels to narrow down device selection: sm get d -l env=prod -l region=us
  • Use --nowait for bulk operations to queue commands asynchronously
  • Use sm get cmds to check the history of commands sent to a device
  • Most commands support --json (-j) flag for machine-readable output
  • Use sm agent-help to get a comprehensive markdown reference for automation

Command Reference

  • get - Retrieve information about resources
  • status - Get status information for devices or services
  • restart - Restart devices or services
  • logs - Get logs from devices
  • run - Run custom commands on devices
  • label - Set labels on devices
  • Other Commands - Additional utility commands

get

The get command retrieves information about resources in your fleet. Use it to inspect device details, view command history, and filter resources by various criteria.

Available Resources:

  • device (alias: d, devices) - View device information and metadata
  • commands (alias: cmds) - View command history for devices

Quick Examples

# List all devices
sm get d

# Get a specific device
sm get d ABC123

# Get only online devices
sm get d --online

# Search for devices by partial match
sm get d rpi -s

# View command history for a device
sm get cmds ABC123

# Filter devices by label
sm get d -l env=production

get device

Display information about one or more devices in your fleet. Without any arguments, lists all devices.

Usage

sm get device [DEVICE_SELECTOR] [OPTIONS]
sm get d [DEVICE_SELECTOR] [OPTIONS]        # Using alias

Device Selection: Specify devices using one or more of the following:

  • Device IDs: One or more serial numbers or IDs as positional arguments
  • Labels: Filter using --label or -l (can be used multiple times)
  • Online/Offline Status: Filter using --online or --offline
  • Partial Matching: Use --search or -s to enable partial matching on device IDs

Options

  • --output FORMAT (-o): Output format (wide, json, or field name like serial_number, id, ip_address)
  • --json (-j): Output as JSON (shorthand for --output json)
  • --label KEY=VALUE (-l): Filter by labels (can be used multiple times)
  • --online: Show only online devices (last seen < 5 minutes)
  • --offline: Show only offline devices (last seen >= 5 minutes)
  • --search (-s): Enable partial matching for device IDs

Examples

# List all devices
sm get d

# Get specific device
sm get d ABC123

# Get devices with custom output
sm get d -o wide

# Get only serial numbers
sm get d -o serial_number

# JSON output
sm get d -j

# Get all online devices
sm get d --online

# Search for devices by partial match
sm get d rpi -s

# Filter by multiple labels
sm get d -l env=production -l region=us-west

get commands

View the history of commands sent to devices. This shows recent operations like restarts, status checks, custom commands, and their current execution status. Useful for tracking what actions have been performed on your devices and checking if commands have completed.

Usage

sm get commands [DEVICE_SELECTOR] [OPTIONS]
sm get cmds [DEVICE_SELECTOR] [OPTIONS]    # Using alias

Device Selection: Specify devices using one or more of the following:

  • Device IDs: One or more serial numbers or IDs as positional arguments
  • Labels: Filter using --label or -l (can be used multiple times)
  • Online/Offline Status: Filter using --online or --offline
  • Partial Matching: Use --search or -s to enable partial matching on device IDs

Options

  • --limit N: Number of commands to show per device (default: 10)
  • --json (-j): Output as JSON (shorthand for --output json)
  • --label KEY=VALUE (-l): Filter by labels
  • --online: Show only online devices
  • --offline: Show only offline devices
  • --search (-s): Enable partial matching for device IDs

Examples

# Get last 10 commands for a device
sm get cmds ABC123

# Get last 50 commands
sm get cmds ABC123 --limit 50

# Get commands for multiple devices
sm get cmds -l env=production

# Search and get commands
sm get cmds rpi -s --limit 20

Command Output

The output shows:

  • Command ID: Unique identifier for the command
  • Issued At: When the command was sent
  • Command Type: What kind of command (Restart, FreeForm, UpdateVariables, etc.)
  • Status: Current status (Pending, Fetched, Completed, Cancelled)

For FreeForm commands, the actual command text is displayed (e.g., systemctl status smithd).

status

Get status information for devices or services.

status device

Get smithd status for devices (runs 'smithd status' command).

Shows comprehensive update status including:

  • Update/upgrade status (whether the system is up-to-date)
  • Installed package versions (currently running on the device)
  • Target package versions (versions that should be running)
  • Update status flag for each package

Usage

sm status device [DEVICE_SELECTOR] [OPTIONS]
sm status d [DEVICE_SELECTOR] [OPTIONS]        # Using alias

Options

  • --nowait: Don't wait for result, just queue the command (faster, use sm command <id> to check results later)
  • --label KEY=VALUE (-l): Filter by labels
  • --online: Show only online devices
  • --offline: Show only offline devices
  • --search (-s): Enable partial matching for device IDs

Examples

# Get status for a device
sm status d ABC123

# Get status for multiple devices (async)
sm status d -l env=production --nowait

# Check status for all online devices
sm status d --online

# Search and check status
sm status d rpi -s

status service

Get systemd service status on devices (runs 'systemctl status ').

Usage

sm status service <UNIT> [DEVICE_SELECTOR] [OPTIONS]
sm status svc <UNIT> [DEVICE_SELECTOR] [OPTIONS]      # Using alias

Arguments

  • UNIT: Service unit name (e.g., nginx, smithd, docker)

Options

  • --nowait: Don't wait for result, just queue the command
  • --label KEY=VALUE (-l): Filter by labels
  • --online: Show only online devices
  • --offline: Show only offline devices
  • --search (-s): Enable partial matching for device IDs

Examples

# Check nginx status on a device
sm status svc nginx ABC123

# Check smithd status on all production devices
sm status svc smithd -l env=production

# Check docker on multiple devices
sm status svc docker ABC123 XYZ789

# Search and check service status
sm status svc nginx rpi -s

restart

Restart devices or services.

restart device

Restart one or more devices.

Usage

sm restart device [DEVICE_SELECTOR] [OPTIONS]
sm restart d [DEVICE_SELECTOR] [OPTIONS]        # Using alias

Options

  • --yes (-y): Skip confirmation prompt
  • --nowait: Don't wait for result, just queue the command
  • --label KEY=VALUE (-l): Filter by labels
  • --online: Show only online devices
  • --offline: Show only offline devices
  • --search (-s): Enable partial matching for device IDs

Examples

# Restart a device (with confirmation)
sm restart d ABC123

# Restart multiple devices without confirmation
sm restart d -l env=staging -y

# Queue restart for all offline devices
sm restart d --offline --nowait

# Restart specific devices
sm restart d ABC123 XYZ789 -y

# Search and restart
sm restart d rpi -s -y

Confirmation

When restarting devices, the CLI will show a preview of up to 10 devices that will be affected. If there are more than 10 devices, it will show the first 10 and indicate how many more devices will be restarted.

You can skip this confirmation with the --yes (-y) flag.

restart service

Restart a systemd service on devices (runs 'systemctl restart ').

Usage

sm restart service <UNIT> [DEVICE_SELECTOR] [OPTIONS]
sm restart svc <UNIT> [DEVICE_SELECTOR] [OPTIONS]      # Using alias

Arguments

  • UNIT: Service unit name (e.g., nginx, smithd, docker)

Options

  • --yes (-y): Skip confirmation prompt
  • --nowait: Don't wait for result, just queue the command
  • --label KEY=VALUE (-l): Filter by labels
  • --online: Show only online devices
  • --offline: Show only offline devices
  • --search (-s): Enable partial matching for device IDs

Examples

# Restart nginx on a device
sm restart svc nginx ABC123

# Restart smithd on all production devices
sm restart svc smithd -l env=production -y

# Queue restart for multiple services
sm restart svc docker ABC123 XYZ789 --nowait

# Search and restart service
sm restart svc nginx rpi -s -y

logs

Get logs from devices (runs 'journalctl -r -n 500').

Usage

sm logs [DEVICE_SELECTOR] [OPTIONS]

Options

  • --nowait: Don't wait for result, just queue the command
  • --label KEY=VALUE (-l): Filter by labels
  • --online: Show only online devices
  • --offline: Show only offline devices
  • --search (-s): Enable partial matching for device IDs

Examples

# Get logs from a device
sm logs ABC123

# Queue logs command for later retrieval
sm logs ABC123 --nowait

# Get logs from devices by label
sm logs -l env=production

# Search and get logs
sm logs rpi -s

Notes

  • This command runs journalctl -r -n 500 on the device, which shows the last 500 log entries in reverse chronological order (newest first)
  • The logs are retrieved from the system journal (systemd journal)
  • You can use --nowait to queue the command and check results later with sm command <id>

run

Run custom commands on devices.

Usage

sm run [DEVICE_SELECTOR] [OPTIONS] -- <COMMAND>

Options

  • --device ID (-d): Specific device serial numbers or IDs to target (can be used multiple times)
  • --wait (-w): Wait for command results (polls until completion)
  • --label KEY=VALUE (-l): Filter by labels
  • --online: Show only online devices
  • --offline: Show only offline devices
  • --search (-s): Enable partial matching for device IDs

Examples

# Run a command on a device (async by default)
sm run ABC123 -- uptime

# Run and wait for results
sm run ABC123 -w -- df -h

# Run on multiple devices
sm run -l env=production -- systemctl status smithd

# Run on specific devices using --device flag
sm run -d ABC123 -d XYZ789 -- free -h

# Search and run
sm run rpi -s -- cat /proc/cpuinfo

# Complex commands with pipes
sm run ABC123 -w -- "dmesg | grep -i error | tail -n 20"

Notes

  • By default, commands are queued asynchronously and return immediately
  • Use --wait (-w) to poll for results until the command completes
  • Use -- to separate the device selector from the command to run
  • Commands are executed via smithd on the target devices
  • You can check command results later using sm command <device_id>:<command_id>
  • For commands with pipes or special shell characters, wrap the entire command in quotes

label

Set labels on devices.

Usage

sm label [DEVICE_SELECTOR] <KEY=VALUE>...

Arguments

  • KEY=VALUE: Labels to set on the devices (can specify multiple)

Options

  • --device ID (-d): Specific device serial numbers or IDs to target (can be used multiple times)
  • --label KEY=VALUE (-l): Filter by existing labels
  • --online: Show only online devices
  • --offline: Show only offline devices
  • --search (-s): Enable partial matching for device IDs

Examples

# Set a label on a device
sm label ABC123 env=production

# Set multiple labels
sm label ABC123 env=production region=us-west

# Set labels on multiple devices
sm label ABC123 XYZ789 env=testing

# Set labels on devices matching a filter
sm label -l region=us-east env=production

# Search and set labels
sm label rpi -s location=warehouse-1

# Set labels on all online devices
sm label --online status=active

Notes

  • Labels are key-value pairs that help organize and filter devices
  • Setting a label will overwrite any existing value for that key
  • You can use labels to filter devices in other commands (e.g., sm get d -l env=production)
  • Common label use cases:
    • Environment: env=production, env=staging, env=development
    • Region/Location: region=us-west, location=warehouse-1
    • Device type: type=gateway, type=sensor
    • Team/Owner: team=backend, owner=john

Other Commands

Additional utility commands for managing your Smith CLI and devices.

test-network

Test network speed for devices (downloads 20MB test file).

Usage

sm test-network [DEVICE_SELECTOR]

Examples

# Test network for a device
sm test-network ABC123

# Test network for all online devices
sm test-network --online

# Test network for devices by label
sm test-network -l env=production

command

Check command results by ID.

Usage

sm command <DEVICE_ID:COMMAND_ID>...

Examples

# Check a specific command result
sm command 123:456

# Check multiple commands
sm command 123:456 789:012

Notes

  • Command IDs are returned when you use --nowait flag
  • You can also get command IDs from sm get cmds <device>

tunnel

Tunnel into a device.

Usage

sm tunnel <SERIAL_NUMBER> [OPTIONS]

Options

  • --overview-debug: Setup for overview debug

Examples

sm tunnel ABC123
sm tunnel ABC123 --overview-debug

profile

Manage CLI profiles.

Usage

sm profile [PROFILE_NAME]

Examples

# Show current profile
sm profile

# Switch to a different profile
sm profile production

distributions

List distributions and releases.

Usage

# List distributions
sm distributions ls
sm distributions ls --json
sm distros ls               # Using alias

# List distribution releases
sm distributions releases
sm distros releases         # Using alias

releases

Commands related to releases.

Usage

sm releases <subcommand>

See sm releases --help for available subcommands.

completion

Generate shell completion scripts.

Usage

sm completion <SHELL>

Supported shells

  • bash
  • zsh
  • fish
  • powershell
  • elvish

Examples

# Generate bash completion
sm completion bash > /usr/local/etc/bash_completion.d/sm

# Generate zsh completion
sm completion zsh > ~/.zsh/completion/_sm

# Generate fish completion
sm completion fish > ~/.config/fish/completions/sm.fish

update

Update the CLI.

Usage

sm update [OPTIONS]

Options

  • --check: Check for updates without installing

Examples

# Check for updates
sm update --check

# Update the CLI
sm update

agent-help

Print all available commands in markdown format (useful for AI agents).

Usage

sm agent-help

This command outputs comprehensive documentation in markdown format, including all commands, flags, and examples. It's designed to be consumed by AI agents or automated systems.

Integrations

The smith-api includes several pre-built integrations that can be easily configured to enhance your application.

Available Integrations

These integrations are optional and can be enabled by setting specific environment variables in your deployment configuration.

Sentry API Error Reporting

Purpose: Provides real-time error tracking and monitoring for your API.

Configuration:

  • Set the SENTRY_URL environment variable with your Sentry DSN (Data Source Name)
  • Example: SENTRY_URL=https://abc123@sentry.io/123456

Benefits:

  • Automatically captures and reports API exceptions
  • Tracks performance issues
  • Provides detailed error context for faster debugging

Slack Notifications

Purpose: Sends automated notifications to your Slack workspace when important events occur.

Configuration:

  • Set the SLACK_HOOK_URL environment variable with your Slack Incoming Webhook URL
  • Example: SLACK_HOOK_URL=https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX

Benefits:

  • Real-time alerts when new devices register with the API
  • Keep your team informed of system activity
  • Customize notification content in Slack's webhook settings

Victoria Metrics

Purpose: Time-series database and monitoring solution for collecting and visualizing API metrics.

Configuration:

  • Set the VICTORIA_METRICS_URL environment variable with your Victoria Metrics instance URL
  • Set the VICTORIA_METRICS_AUTH_TOKEN environment variable with your authentication token
  • Example:
    VICTORIA_METRICS_URL=https://your-vm-instance.example.com
    VICTORIA_METRICS_AUTH_TOKEN=your-auth-token
    

Benefits:

  • High-performance metrics collection
  • Long-term storage of monitoring data
  • Compatible with Prometheus querying and visualization tools

IP-API Geolocation

Purpose: Automatically enriches device IP addresses with geolocation data including country, city, ISP, and coordinates.

Configuration:

  • Set the IP_API_KEY environment variable with your IP-API Pro key
  • Example: IP_API_KEY=your-pro-api-key

Features:

  • Smart Updates: Only updates geolocation data when it's older than 24 hours, minimizing API calls
  • Background Processing: Geolocation lookups happen asynchronously without blocking device ping responses
  • Comprehensive Data: Collects country, city, region, ISP, coordinates, proxy/hosting detection
  • Graceful Fallback: When no API key is configured, only stores IP addresses without geolocation data

Database Schema: The system automatically stores geolocation data in the ip_address table with the following fields:

  • continent, continent_code
  • country_code, country
  • region, city
  • isp
  • coordinates (PostgreSQL POINT type for latitude/longitude)
  • proxy, hosting (boolean flags)
  • created_at, updated_at (automatic timestamps)

Benefits:

  • Track device geographical distribution
  • Identify unusual network activity (proxy/hosting detection)
  • Generate location-based analytics and insights
  • Minimal impact on API performance due to smart caching

Implementation Example

Add these environment variables to your deployment configuration:

# Error Reporting
SENTRY_URL=https://your-sentry-dsn

# Event Notifications
SLACK_HOOK_URL=https://hooks.slack.com/services/your-webhook-url

# Metrics and Monitoring
VICTORIA_METRICS_URL=https://your-vm-instance.example.com
VICTORIA_METRICS_AUTH_TOKEN=your-auth-token

# IP Geolocation
IP_API_KEY=your-pro-api-key

Additional Information

For more details on configuring these integrations or for troubleshooting, refer to each provider's documentation linked above.