ReductStore:面向机器人与工业 IoT 的时间索引对象存储
reductstore/reductstore
ReductStore 是面向机器人与工业 IoT 的高性能时间索引对象存储,可按时间戳与标签存储并查询图像、视频、传感器读数、日志、文件和 ROS bag 等原生二进制数据。
🚀 60k+ downloads • 🏭 100+ production deployments • 📦 1 PB+ managed data
⭐ 350+ GitHub stars • 🛠️ 4+ years of active development
Make robotics and industrial data queryable
ReductStore is high-performance, time-indexed object storage for robotics and industrial IoT.
Store terabytes of images, video, sensor readings, logs, files, and ROS bags in their native binary formats. Every record is organized by timestamp and can be enriched with labels, so you can retrieve exactly the data you need by time range and operational context.
Instead of combining a time-series database, generic object storage, metadata indexes, and custom retention jobs, ReductStore provides one system for ingesting, organizing, retaining, and querying multimodal time-series data.
We built ReductStore for robotics and industrial systems that continuously produce images, video, sensor readings, logs, files, and ROS bags. These workloads need the time index of a time series database and the payload flexibility of object storage without maintaining both systems.
| Native support for continuous multimodal data | ReductStore | MinIO AIStor | TimescaleDB | MongoDB |
|---|---|---|---|---|
| Store arbitrary binary payloads | ✅ Native records | ✅ Native objects | 🟡 BYTEA rows |
🟡 BinData or GridFS |
| Index binary data by timestamp | ✅ Built in | ❌ Object key index | ✅ Hypertable time index | 🟡 Schema and index required |
| Query by time range and metadata | ✅ Time and label query | ❌ No online time/metadata query Listing or batch inventory |
🟡 Requires schema + indexes | 🟡 Indexed timestamp/metadata GridFS for >16 MiB |
| Manage a fixed disk as a ring buffer | ✅ Capacity based FIFO quota | ❌ Age based expiration | ❌ Time based retention | 🟡 Capped collections |
| Replicate selected data upstream | ✅ Entry and label conditions | 🟡 Prefix and object tag rules | ❌ No native selective replication Streaming replication is full copy |
❌ Application pipeline required |
| Compress and delete historical data | ✅ Lifecycle policies | 🟡 Expiration/tiering Compression on PUT |
✅ Columnstore and retention policies | 🟡 TTL expiration |
ReductStore keeps the timestamp, labels, and binary payload in one record. The same labels drive queries, deletion policies, and replication, so edge devices can retain a bounded local history and send only relevant data upstream.
This comparison covers native capabilities for continuously recorded multimodal data. A partial result means the capability requires a custom schema, restricted rule model, batch workflow, or additional setup rather than being a first class feature of the storage model.
When You Should Use It
ReductStore is ideal for scenarios where you have a continuous stream of binary data and your application has metadata that can be attached to the data as labels. If you ingest binary data with labels into a ReductStore instance, you can use it to:
- Query objects by time range and labels, e.g. "give me all the images from camera-1 over the last hour where the device status is 'error'".
- Replicate only selected data to another ReductStore instance, e.g. "replicate all the data from camera-1 where the device status is 'error' to the cloud instance for further analysis".
- Manage the data lifecycle with policies, e.g. "keep all the data from camera-1 for 30 days, then compress it and keep it for another 60 days before deleting it".
In such scenarios, ReductStore can help you manage and transfer your data efficiently based on its metadata, without combining a TSDB and blob storage. This keeps your architecture simple and efficient.
Who It Is For
- Robotics platforms that need to replay camera frames, telemetry, and logs around incidents or model failures
- Industrial IoT pipelines that collect binary payloads at the edge and forward only filtered data upstream
- Edge AI systems that need retention, labeling, and historical retrieval without building a custom storage stack
Get Started
The quickest way to get up and running is with Docker:
mkdir -p ./data
sudo chown -R 10001:10001 ./data
docker run -p 8383:8383 -v ${PWD}/data:/data \
-e RS_API_TOKEN=my-secret-token reduct/store:latestOr download a Linux binary directly from the latest release:
curl -LO https://github.com/reductstore/reductstore/releases/latest/download/reductstore.x86_64-unknown-linux-gnu.tar.gz tar -xzf reductstore.x86_64-unknown-linux-gnu.tar.gz mkdir -p ./data RS_DATA_PATH=./data RS_API_TOKEN=my-secret-token ./reductstore
For a more in-depth guide, visit the Getting Started and Download sections.
After initializing the instance, you can start writing and querying data immediately. Here's a Python sample:
import asyncio from reduct import Client async def main(): async with Client("http://localhost:8383", api_token="my-secret-token") as client: bucket = await client.create_bucket("my-bucket", exist_ok=True) await bucket.write( "camera-1", b"hello, reductstore", timestamp="2024-01-01T10:00:00Z", labels={"device": "camera-1"}, ) async for record in bucket.query( "camera-1", start="2024-01-01T10:00:00Z", stop="2024-01-01T10:00:01Z", when={"&device": {"$eq": "camera-1"}}, ): print((await record.read_all()).decode()) asyncio.run(main())
Next Steps
Learn more and pick the next piece you need:
Build From Source
ReductStore uses a vendored protoc binary through Cargo, so contributors do not need to install Protocol Buffers manually. To check the workspace locally, run:
cargo check --workspace
When You Should Not Use It
ReductStore is not the best fit for every data storage problem. You should consider another solution when:
- You have only numerical data that can be easily ingested into a time-series database.
- You have only blob data to store and do not need to access it as historical data. In this case, object storage or a file system may be a better fit.
- You need a message broker. Although ReductStore provides subscription and publishing functionality, it is designed for data storage and streaming, not message queueing.
Community & Contribution
If you've found a bug, have ideas, built something with ReductStore, or want to contribute directly, here are the best places to jump in:
- Questions and Ideas: Join our Discourse community to ask questions, share ideas, and collaborate with fellow ReductStore users.
- Bug Reports: Open an issue on our GitHub repository. Please provide as much detail as possible so we can address it effectively.
- First Contributions: Pick a task from good first issues or help wanted.
- Pull Requests: Open PRs only for an existing issue. Comment on the issue and get assigned before you start work. Use the repository pull request template when you open the PR, and update
CHANGELOG.mdfor user-visible changes. If you want to propose a new feature or workflow, open an issue first or start the discussion on the community forum before writing code. - Show Your Work: Share your projects, benchmarks, and lessons learned on our Discourse community.
- Support the Project: If ReductStore is useful to you, give us a ⭐ on GitHub.
Contributors
Thanks to everyone who has contributed to ReductStore.
Your support fuels our passion and drives us to keep improving.
Together, let's build better storage for robotics and industrial data. 🚀
来源:GitHub · github.com