Last month I had the opportunity to join professors, students and industry leaders in a workshop of the Industry-Academia Partnership (IAP) at Cornell University, addressing the future of the data center and cloud. The workshop was a unique opportunity for a flash storage solutions provider, such as SanDisk®, to interact with current (and future) leaders in academia and industry across the entire solution stack – from components, to software and data center infrastructure. Moreover, it was an inspiring dialog about developing the technologies and the revolutionary changes that are required for the underlying hardware and software infrastructures of compute, storage and networking.
Together with Cavium, SanDisk had the honor to sponsor the student Best Poster Award. There were 10 poster submissions by Cornell students on cloud related work, representing the contributions of more than 30 students.
The Best Poster Award for a Graduate Student went to PhD student Qi Huang of Cornell for his work “An Analysis of Facebook Photo Caching” that addresses instrumenting and characterizing of a service stack, analyzing caching algorithms at each level of that stack, and their effectiveness across data centers.
Qi’s research serves as a window to some of the coming changes, requirements and challenges of data centers, and in particular the cloud. With the volume of data increasing at a phenomenal rate and the rapidly changing means in which consumers and as enterprises use the internet, new workloads arise that bring on challenges for data processing, multi-user environments, mobile devices and storage demands.
As data centers grow to hyperscale size, efficiency is key not only for improved performance, latency, capacity, and persistence, but also in reducing the overall power consumption of data centers and TCO. In order to enable these efficiencies, our technologies will need to address, and improve, every level of the hardware and software stack: from the building blocks of hardware architecture, to intelligent software that enables virtualization and acceleration, and ending with applications uniquely architected to take advantage of hardware features.
From Challenges To Solutions: Flash Optimized Data Centers
Flash is a fast growing solution and industry disruptor in data centers, as it enables accelerated performance, can manage redundancy and concurrent accesses better, requires lower power demands and space than existing technologies and can offer a far better TCO solution (you can learn more about how to leverage SSD to achieve cost efficiencies by downloading our solutions brief: Maximize Savings with Enterprise SSDs). One of many exciting perspectives in the development of flash is that its use case now covers the entire data center: from hot to warm to cold data storage.
The problem is that many applications realize limited benefits from flash without system level optimization. Technology inside SanDisk offers a substrate for flash-optimized applications with multiple management, API and key features that help optimize access paths, workloads and concurrency. SanDisk has unique flash optimized stack capability, and in our labs today we have this operating across several application classes (in-memory data grid, databases, message queues, etc.). We see the flash optimization of applications yielding near in-DRAM performance but with in-flash capacity and cost, enabling server consolidation.
The next generation of solutions and engineers are investigating new paths to unleash efficiencies by reexamining how we architect our hardware, software and applications to take better advantage of every level of our infrastructures. An event such as the IAP workshop is an opportunity to discuss how research and industry solutions can come together to enable overall solutions such as an Exascale data center (achieving ExaFlop/ExaOP performance) that consumes no more than 10 megawatts. Flash memory and storage technology is playing a big role in enabling this and our future cloud data centers.