VDS Compression: Unlock Seismic Data Storage Efficiency and Usability

Morten Ofstad, Chief Architect at Bluware

In the world of seismic data management, storage efficiency and access speed are crucial challenges. Bluware’s VDS has emerged as a groundbreaking solution, offering sophisticated compression technologies that set new industry standards.

Advanced Compression Technologies

TruLossless™ Technology – Bluware’s compression solution enables complete roundtrip conversion back and forth from VDS to SEG-Y, ensuring data integrity while maximizing storage efficiency. This approach guarantees that no information is lost during the compression process, providing confidence in the data’s fidelity.

Adaptive Streaming – Unlike traditional systems that require multiple format conversions and copies of data, VDS adaptive streaming technology streams data to applications in real-time without duplicating the data, so it is available instantly. Adaptive streaming delivers precisely the signal quality needed for each workflow to dramatically improve data access speeds, enabling seamless integration into diverse geoscience processes.

Compression Modes and Implementation

Adaptive Compression: A Bluware VDS Exclusive

VDS adaptive compression technology utilizes a bi-orthogonal wavelet transform. This sophisticated approach:

• Encodes coefficients adaptively
• Functions similarly to a wavelet soft-threshold noise removal filter
• Creates smoother data at higher compression ratios, benefiting workflows like horizon interpretation

Compression Formats

VDS supports multiple compression methods, each optimized for specific use cases:

1. Wavelet Compression:
   • Optimized for floating-point data
   • Offers both lossless and lossy options
   • Supports adaptive decompression for diverse workflows
2. TruLossless Compression
   • Applies highest quality preset, Virtually Lossless, to the data.
   • The difference between the original input data and Virtually Lossless compression is stored enabling VDS to be converted back to a SEG-Y that is binary (bitwise) equal to the original input SEG-Y.
   • Excellent for data preservation, as it provides 25% reduction in space while preserving the original SEG-Y.
3. Virtually Lossless Compression
   • Achieves up to 75% data size reduction
   • Suitable for all workflows
4. Near Lossless Compression
   • Achieves up to 90% size reduction
   • Suitable for most interactive workflows
   • Maintains high visualization quality
5. Uncompressed:
   • Useful when the available bandwidth is very high, such as running on cloud VMs that have a direct connection to the object store
   • Raw data storage for maximum I/O performance
   • Supports constant-region optimization
   • Available in both OpenVDS and Bluware VDS
6. Run-Length Encoding (RLE):
   • Ideal for data with repeated values
   • Highly efficient for compression and decompression
7. Zip Compression:
   • Perfect for text data and SEG-Y trace headers
   • Best suited for rarely accessed archival data

Performance Optimization

VDS compression enhances performance across various scenarios:

• Reduced Data Transfer Volume: Often outweighs the decompression overhead, making workflows more efficient.
• GPU-Accelerated Decompression: Powered by the Bluware Engine, this feature delivers exceptional performance.
• Distributed Scaling for Network Optimization: Particularly effective in distributed environments where speed and efficiency are critical.

Smart Data Organization

To further optimize data management, VDS employs a sophisticated “bricked” storage format:

• Default Brick Size: 128×128×128 samples
• 2D Slice Organization: Optional for specific workflows
• Neighboring Sample Queries: Optimized for faster access and processing

Best Practices for Implementation

To maximize the benefits of VDS compression, follow these guidelines:

1. Match compression methods to data characteristics.
2. Use adaptive compression with a near-lossless preset for interactive workflows.
3. Implement lossless compression for archival purposes.
4. Choose RLE for binary mask data.
5. Reserve ZIP compression for non-continuous data like trace headers.
6. Consider workflow requirements when selecting compression quality.

VDS for Cost-Efficient Storage

VDS and OpenVDS+ provide significant cost-saving optimizations, crucial for large datasets in cloud storage. Other formats lack similar cost-efficiency features, potentially increasing storage expenses.

VDS can be stored as a traditional file on disk and is optimized for cloud object storage, making it highly cost-effective and massively scalable. VDS is served without the need for a dedicated server. Using Bluware’s APIs, data on object store is accessed just like a regular file based VDS. Users can achieve significant cost savings related to computing. 

Storing VDS

VDS can be stored in two forms including file based and cloud-based object store such as AWS S3 or Microsoft Azure BLOB store.

While the individual bricks inside a VDS file are identical to the bricks stored in a cloud-based object-store, they are organized differently. Moving between file-based and object-store based is a simple conversion process.

File-Based VDS (VDS Only)

The file based VDS exists on disk as a single file with the extension ‘.vds’. This is used for non-cloud deployments and for data exchange in VDS format. These files are accessed and managed as regular files on disk.

Cloud-Based Object-Store VDS (OpenVDS and VDS)

Object storage architecture was designed to allow for massive amounts of data to be stored without the complexities involved in managing traditional networked file systems. Object storage delivers secure, high-performance access to data over the internet. Data might potentially be duplicated in multiple geographical regions allowing for fast access from different continents.

When storing VDS on object, each brick is stored as a separate object with a common prefix.

Serverless Architecture (OpenVDS and VDS)

One of the key benefits of VDS on object store, is that data is served without the need for a dedicated server. Using our APIs, data on object stores is accessed just like a regular file based VDS. This is a significant benefit in terms of cost, particularly in the cloud, where there is a cost associated with compute capability.

Get Flexibility for Your Modern Geoscience Workflows

By offering multiple compression options with varying degrees of data preservation and performance optimization, VDS provides flexibility for modern geoscience workflows.

Whether you’re prioritizing perfect data preservation, interactive performance, or storage efficiency, VDS’s compression toolkit has you covered. The system’s ability to adapt compression levels based on use case—from web visualization to detailed interpretation—makes it an invaluable tool for organizations dealing with large seismic datasets.

As the industry continues to generate ever-larger volumes of data, sophisticated compression capabilities will become increasingly crucial for efficient data management and analysis.

Unlock the potential of your seismic data in VDS and set a new standard for performance and efficiency in the field.