Laurent Clerc
CGG
Petroleum exploration gets usable information quicker
Advances in seismic data processing provide dramatic increases in the accuracy and range of information available for petroleum exploration. Processing applications are introducing functions such as anisotropic modeling to improve the accuracy of deep offshore exploration and advanced wave equation migration analysis techniques to provide enhanced images of complex geology.
What are first required for these kinds of advances are highly specific software packages. Sophisticated analytical approaches by themselves are not enough to provide real advances to the ultimate users of seismic data. Users also need to have data available within the time and budget constraints that the contemporary business environment dictates.
 Project space requirements vary from 1 to 50 terabytes over the course of a project. |
null
For that to occur, the analytical tools need to be supported by equal advances in processing methodologies, storage architecture, and data management technology. CGG's recent experience with installing an advanced storage area network (SAN) directly translated into significant efficiency and performance gains by allowing the storage system to provide better support for the company's heterogeneous data processing system.
Before looking at the changes that the SAN system provided, we need to talk about the processing environment and the way that the former CGG storage architecture was used to support it. Seismic processing is a step-by-step process of refining a large volume of noisy and redundant data into a clear picture of the underground, each step requiring up to a few thousand jobs on multiple systems.
A few years ago, those steps occurred after stack (when all the energy contributing to each cell has been added). Today they occur mostly before stack to ensure maximum accuracy. This means that a new data set of roughly the same size as the original is now generated for each of the five to 10 pre-stack steps needed in a project. The raw storage needed for each operation is huge, and the final storage needed for a single project can be anywhere between 10 and 100 terabytes (TB). Since it is fairly common to have multiple projects running at the same time on any given site, the total storage requirement can quickly grow to hundreds of terabytes, with huge throughput requirement.
CGG'c Geocluster software has been designed to run on multiple platforms and operating systems to ensure that it is always possible to take advantage of the best available price/performance combination. This implies that at any given time, multiple operating systems are used in production on sites, some being phased out, others introduced as replacements. It is important that the storage solutions work on a heterogeneous and ever changing environment,
The system that CGG initially used to manage this immense data access and management job was direct-attached storage on each of multiple high-performance Unix servers. Each server acted both as a processing system that carried out analytical operations and a server that spooled the multi-gigabyte (GB) files to other servers and to several thousand clustered PC-nodes over GB Ethernet or high performance parallel interface networks.
When files were not being actively processed, they were stored in high-capacity automated tape libraries. These libraries were linked to direct-attached disk using hierarchical storage management (HSM) software, which ran on each server. In this architecture, multiple storage containers were distributed across platforms on the site, creating a significant overhead when data needed to be transferred from one system to the other. This also resulted in a setup that was less reliable and somewhat difficult to upgrade; however, in view of the throughput and capacity requirements, it was hard to do better at the time.
The storage subsystem needed to be separated from the processing subsystem, while maintaining throughput, stability, and price/performance ratios. From a purely technical point of view, the expectations were to:
- Get more utilization out of the existing processing resources
- Reduce the time spent managing load balancing between the systems
- Shorten the project's turnaround
- Create a more resilient system
- Allow seamless and transparent upgrades of processing and storage systems.
Conventional SANs, even though they can provide sharing of physical storage systems, don't handle sharing of data and files. Instead, users provision the disk storage so that discrete volumes are each dedicated to a specific server. To provide shared data access, a distributed file system was required on top of the SAN architecture, and it had to integrate an HSM module to handle the amount of data required.
New architecture
Very few such products exist today, and eventually, StorNext File System, part of ADIC's StorNext Management Suite software, emerged as both a complete and efficient solution. SNMS is designed to allow data to be consolidated on centralized disk storage so that the data can be shared between the different operating systems, giving users concurrent high-performance access to all the files, while transparently handling tape to disk migration.
 CGG formerly used direct-attached storage on multiple high-performance Unix servers to handle data access and management. |
null
The new SAN architecture connects disk and tape storage resources (multiple 13-TB arrays of fiber channel RAID, and two ADIC AML automated tape libraries) into a SAN, connecting them to a switched fabric made up of fiber channel switches.
The primary Unix/Linux/Windows servers and clusters of PC platforms are also connected through 200-mb host bus adapters (HBAs). In addition to providing common access to the data stored on the shared disk resources, the SNMS software provides automated data management and protection capabilities. Based on policies developed around project requirements, CGG uses SNMS to automatically migrate reference data from disk to tape.
Differences
The previous data management system migrated data between disk and tape, and that capability is also provided with the full implementation of SNMS software. The difference from the former system is that now the function is centralized, supports all of the operating systems at once, and moves data over the SAN fabric rather than over conventional network topologies.
null
The improvements in the storage architecture, created by the combination of a SAN storage model and the StorNext File System, are significant:
- Performances have improved. All data is visible locally at wire speed without any CPU overhead, and HSM functions (and related tape and disk I/O activity) are concentrated on a single system, tuned accordingly
- Load balancing issues have gone. Applications run wherever CPU is available because data access is not an issue
- Reliability and serviceability of the system has increased, as individual processing systems can stop and start without any impact on data availability on other systems
- Upgrades and evolutions of both the storage and processing subsystems can now take place whenever required, with virtually no impact on production.
SAN transition
The transition from direct-attached to SAN storage was carried out while the site was running, and the change occurred in stages to preserve production. The first step was to install the fiber channel hardware infrastructure and to map servers to specific disks to retain the direct-attach storage functionality. The second step was to add the SNMS software layer, bringing different servers and storage resources on-line in sequence.
In the new system, software moves files between disk and tape based on automated, customized policies ensuring that data is available and protected on automated tape storage.
Benefits
Jobs are processed more rapidly, and more jobs can be processed at once with less management overhead. The groundwork has been laid for more advances. System capacity and performance has been enhanced, and it will be simpler to change the processing architecture as new technologies become available.
