Stempipe Tutorial: Features, Use Cases, How It Works

TL;DR, What is Stempipe?

Steampipe is an open-source command-line tool that turns cloud APIs into a queryable database.

Managing multiple APIs across cloud environments can be complex, often requiring your teams to learn vendor-specific tools and write custom scripts. Steampipe simplifies this process by providing a unified SQL interface to your cloud infrastructure. Instead of juggling different SDKs, you can use standard SQL queries to run security audits, check infrastructure inventories, and build automation workflows. Using a unified SQL interface provides live, read-only data directly from the APIs, which removes the need for custom ETL pipelines and helps reduce the learning curve for your team.

Developed by Turbot, Steampipe provides a consistent way to manage multi-cloud environments and can also connect to SaaS APIs and local resources through plugins.

API Security Best Practices [Cheat Sheet]

Download the Wiz API Security Best Practices Cheat Sheet and fortify your API infrastructure with proven, advanced techniques tailored for secure, high-performance API management.

Your work email here

Download cheat sheet

At-A-Glance

• GitHub: https://github.com/turbot/steampipe 

• License: AGPL 3.0

• Primary Language: Go

• Stars: 7.5k ⭐  

• Last Release: v2.2.0 September 24, 2025

• Topics/Tags: security, compliance, cloud, sql, policy-as-code

Common use cases

1. Security Compliance and Auditing: Security teams automate compliance checks against frameworks like CIS and NIST, querying for misconfigurations such as public S3 buckets or permissive firewall rules using Steampipe’s SQL-based “mods” across their entire multi-cloud estate. 

2. Cloud Asset Inventory: Engineers maintain a real-time inventory of all cloud resources, enabling them to track deployments, identify untagged assets, and enforce consistent configuration policies across different cloud providers.

3. Cloud Cost Optimization: Organizations proactively reduce cloud spend by querying for underutilized or orphaned resources like unattached volumes, idle VMs, and old snapshots.

4. DevOps and CI/CD Automation: Integrated into CI/CD pipelines, Steampipe acts as a policy-as-code gate, validating infrastructure changes against security rules before deployment to prevent misconfigurations from reaching production.

5. Multi-Cloud Data Analysis: Platform teams generate unified reports by querying data from AWS, Azure, and GCP simultaneously, providing a single-pane-of-glass view.

wiz academy

Top API Security Solutions (and how to choose the right one)

Read more

How does Stempipe work?

Steampipe uses a plugin-based architecture to let you query different APIs with standard PostgreSQL. When you run a SQL query, the Steampipe engine routes the request to the correct plugin. The plugin acts as a translator, making real-time API calls to the target service and turning the JSON response into a virtual database table. The embedded PostgreSQL server then runs your query against these dynamic tables, giving you access to live data without any ETL pipeline setup.

• Plugins: Each plugin is a self-contained Go plugin built with the Steampipe Plugin SDK. The plugin manages API authentication, defines table schemas, and handles the logic for fetching and transforming data from a specific API.

• Query Engine: The query engine parses SQL statements, determines which plugins and connections are needed, and coordinates the data flow from the API to the database in real time.

• Embedded PostgreSQL: Steampipe includes a Postgres server that uses Foreign Data Wrapper (FDW)-like technology. Using this mechanism, external API data from plugins appear as native database tables for complex querying.

Core Capabilities

1. Multi-Engine Plugin System: Steampipe's architecture uses over 140 plugins to provide a unified SQL interface for various data sources, including cloud providers like AWS, Azure, and GCP, as well as SaaS platforms. The plugin system handles API complexities like authentication, pagination, and rate limiting, allowing a single query to work across the CLI, PostgreSQL FDW mode, and other integration engines.

2. Zero-ETL Real-Time Querying: The tool queries APIs directly and in real time, removing the need for ETL pipelines or intermediary ingestion layers. Querying APIs directly ensures that your results, including complex JOINs across different providers, reflect the current state of the infrastructure.

3. Integrated Compliance Frameworks: Steampipe includes a library of mods and controls mapped to security and compliance frameworks like CIS Benchmarks, NIST, and SOC 2. These built-in modules help you automate compliance checks and security posture management and produce reports that are “audit-friendly” and dashboard-ready.

4. Flexible Configuration and Connection Management: Using HCL (HashiCorp Configuration Language), Steampipe helps you manage connections for query scenarios across multiple accounts, regions, and cloud platforms from a single configuration. Steampipe provides workspace organization for different environments and manages credentials securely, with caching to optimize performance.

5. Extensive Integration and Automation: Designed for DevOps workflows, Steampipe integrates with CI/CD tools, GitHub Actions, and containerized environments. The tool supports various output formats (JSON, CSV) for use with other scripts and tools, while its PostgreSQL compatibility allows it to operate as a Postgres FDW or integrate with BI platforms for automation.

Limitations

1. API Performance Dependency: As a real-time query tool, performance is directly tied to the latency and availability of the underlying service APIs. Queries can be slow or fail entirely during provider outages, and aggressive scanning may trigger API rate limiting, disrupting operations.

2. Steep Learning Curve for Non-SQL Users: The tool's primary interface is SQL. While powerful for those with database experience, it presents a significant learning barrier for DevOps, security, or operations personnel who are not proficient in writing complex SQL queries, potentially limiting broader team adoption.

3. Coverage Gaps from Plugin Ecosystem: The tool's effectiveness is dependent on the breadth, depth, and maintenance of its plugin ecosystem. If a required service lacks a plugin, or an existing plugin does not support specific resources or properties, those assets remain invisible to queries.

4. Performance Constraints on Large-Scale Environments: Executing real-time API calls across environments with hundreds of thousands of resources can be significantly slower than querying a dedicated data warehouse. Complex, cross-account analytical queries may face performance bottlenecks or timeouts not present in traditional ETL-based systems.

5. Potential for Incurring High API Costs: Frequent and broad queries can generate a high volume of API calls (e.g., LIST, DESCRIBE), which may be billable on major cloud platforms. Without careful management of query scope and frequency, automated scans could lead to unexpected increases in cloud provider bills.

Get Started

Step 1: Download and install Steampipe.

For macOS:

brew install turbot/tap/steampipe

For Linux or Windows (WSL2):

sudo /bin/sh -c "$(curl -fsSL https://steampipe.io/install/steampipe.sh)"

Step 2: Install a plugin for your desired service (example: Hacker News).

steampipe plugin install hackernews

Step 3: Start the interactive query shell.

steampipe query

Step 4: Run a test query (example: Hacker News).

select * from hackernews_new limit 10;

Catch code risks before you deploy

Learn how Wiz Code scans IaC, containers, and pipelines to stop misconfigurations and vulnerabilities before they hit your cloud.

Work Email*

First Name*

Last Name*

Country

Phone Number*

Company*

Keep me updated about Wiz product releases, industry news, and events (You can unsubscribe at any time)

Subscribe me to the Wiz blog digest emails

Submit

For information about how Wiz handles your personal data, please see our Privacy Policy.

Your work email here

Get a demo

Alternatives

FAQ