As the cloud has grown in recent years, there has been a movement to increase data and server management and maintenance efficiency by migrating data (i.e., internal documents, design data, and email) that had been managed on internal servers to the cloud. In addition, as shown by the spread in the use of digital technology such as IoT and AI, there are high expectations for the ways that work and business will be transformed by the analysis and use of large volumes of data, including camera images from factories and other on-site locations, and log data from devices. Given this, there has been explosive growth in the volume of data passing through WAN lines between clouds, spurring a need for next-generation WAN acceleration technology capable of huge data transfers at high-speed between clouds.
Figure 1 : Use of WAN acceleration technology in a cloud environment
WAN acceleration technologies improve effective transfer speeds by reducing the volume of data through compression or deduplication of the data to be transferred. When transferring data at even higher speeds using 10Gbps network lines, the volume of data needing to be processed is so great that the compression and deduplication processing speed in the server bottlenecks. Therefore, in order to improve real-time operation, there is a need for either CPUs that can operate at higher speeds, or for WAN acceleration technology with faster processing speeds.
About the Newly Developed Technology
Fujitsu Laboratories has now developed WAN acceleration technology that can achieve real-time operation usable in the cloud even with speeds of 10Gbps or more, using server-mounted FPGAs as accelerators. Efficient operations with WAN acceleration technology are accomplished by using an FPGA to process a portion of the processing for which the computation is heavy and for which it is difficult to improve processing speed in the CPU, when performing compression or deduplication for WAN acceleration processing, and by efficiently connecting the CPU with the FPGA accelerator. Details of the technology are as follows.
- FPGA parallelization technology using highly parallel dedicated computational units
Fujitsu Laboratories has developed FPGA parallelization technology that can significantly reduce the processing time required for data compression and deduplication by deploying dedicated computational units specialized for data partitioning, feature value calculation, and lossless compression processing in a FPGA in a highly parallel configuration, and by enabling highly parallel operation of the computational units by delivering data at the appropriate times based on predictions of the completion of each calculation.
Figure 2 : Implementation of WAN acceleration processing using server equipped with FPGA
- Technology to optimize the flow of processing between CPU and FPGA
Previously, in determining whether to apply lossless compression to data based on the identification of duplication in that data, it was necessary to read the data twice, both before and after the duplication identification was executed on the FPGA, increasing overhead and preventing the system from delivering sufficient performance. Now, by consolidating the processing handoff onto the FPGA, handling both the preprocessing for duplication identification and the compression processing on the FPGA, and using a processing sequence that controls how the compression processing results are reflected on the CPU based on the results of the duplication identification, this technology reduces the overhead between the CPU and FPGA from reloading the input data and from control exchanges. This reduces the waiting time due to the handoff of data and control between the CPU and FPGA, delivering efficient coordinated operation of the CPU and FPGA accelerator.
Figure 3 : Summary of the method for reducing overhead between CPU and FPGA
Fujitsu Laboratories deployed this newly developed technology in servers installed with FPGAs, confirming acceleration approximately thirty times the performance of CPU processing alone. Fujitsu Laboratories evaluated the transfer speed for a high volume of data in a test environment where the servers were connected with 10Gbps connections, and in a test simulating the regular backup of data, including documents and video, confirmed that this technology achieved transfer speeds up to 40Gbps, an industry record. This technology has significantly improved data transfer efficiency over WAN connections, enabling high-speed data transfers between clouds, such as data sharing and backups, making possible the creation of next-generation cloud services that share and use large volumes of data between a variety of companies and locations.
Figure 4 : Sample evaluation results using trial data
Fujitsu Laboratories will continue to evaluate this technology in practical environments, deploying this technology in virtual appliances that can be used in cloud environments. Fujitsu Laboratories aims to make this technology available as a product of Fujitsu Limited during fiscal 2018.