Observatory joins Ceph Foundation to advance open source storage

Yesterday in Berlin SARAO joined 30 other members in the establishment of the Ceph Foundation, to manage the massive growth in data and information generated from cloud, container and artificial intelligence applications.

The Linux Foundation, a non-profit organization enabling mass innovation through open source, announced that over 30 global technology leaders are forming a new foundation to support the Ceph open source project community. The Ceph project develops a unified distributed storage system providing applications with object, block, and file system interfaces.

"Ceph has a long track record of success when it comes to helping organisations effectively manage high growth and expanding data storage demands," said Jim Zemlin, Executive Director of the Linux Foundation. "This partnership will assist us to store and retrieve the huge volumes of data that will be collected by the MeerKAT radio telescope," says Dr. Rob Adam, Managing Director of SARAO.

The MeerKAT is a 64-antenna array radio telescope that has been built on the SKA site in the Karoo, and which will be integrated into the first phase of the Square Kilometre Array. MeerKAT has the capacity to process 275 gigabytes per second in real time - that equates to approximately 58 DVDs per second. SARAO currently uses Ceph to synthesise a ~20 petabyte object-based storage system, for the data generated by the MeerKAT radio telescope array.

Ceph is used by cloud providers and enterprises around the world, including financial institutions (Bloomberg, Fidelity), cloud service providers (Rackspace, Linode), academic and government institutions (Massachusetts Open Cloud), telecommunications infrastructure providers (Deutsche Telekom), auto manufacturers (BMW), software solution providers (SAP, Salesforce), and many more.

Ceph is also used by Rook, a Cloud Native Computing Foundation project that brings seamless provisioning of file, block and object storage services into the Kubernetes environment, running the Ceph storage infrastructure in containers alongside applications that are consuming that storage.

Efficient, agile, and massively scalable, Ceph significantly lowers the cost of storing enterprise information in the private cloud and provides high availability to any object, file, and block data. Block and file storage are critical to any IT infrastructure organisation and are important components of infrastructure platforms like OpenStack and Kubernetes.

The growth of new cloud, container and artificial intelligence/machine learning applications are driving the increased use of Ceph. Ceph combined with analytics and machine learning can, for example, enable enterprises to comb through mass amounts of unstructured data to quickly spot patterns in customer behaviour, online customer conversations, and potential noncompliance scenarios.

"A guiding vision for Ceph is to be the state of the art for reliable, scale-out storage, and to do so with 100% open source," said Sage Weil, Ceph co-creator, project leader, and chief architect at Red Hat for Ceph.

"While early public cloud providers popularised self-service storage infrastructure, Ceph brings the same set of capabilities to service providers, enterprises, and individuals alike, with the power of a robust development and user community to drive future innovation in the storage space. Today's launch of the Ceph Foundation is a testament to the strength of a diverse open source community coming together to address the explosive growth in data storage and services."

"Ceph was designed and built for scalability, initially with supercomputers and later with cloud infrastructure in mind. A key design premise was that the storage system needs to provide a highly reliable and available service in a dynamic and increasingly heterogeneous hardware environment where everything can potentially fail," said Carlos Maltzahn of University of California, Santa Cruz, a co-founder of the research project that first created Ceph over a decade ago.

Thomas Bennett, Senior Software Systems Engineer at SARAO says: "SARAO uses Ceph in concert with locally manufactured hardware to lower storage capital expenditure for the MeerKAT storage infrastructure. In order to share our experiences and showcase Ceph, the SARAO storage team is in the process of establishing a Cape Town Ceph community forum."

Related Links
South African Radio Astronomy Observatory
Stellar Chemistry, The Universe And All Within It

Tweet

Thanks for being there;
We need your help. The SpaceDaily news network continues to grow but revenues have never been harder to maintain.

With the rise of Ad Blockers, and Facebook - our traditional revenue sources via quality network advertising continues to decline. And unlike so many other news sites, we don't have a paywall - with those annoying usernames and passwords.

Our news coverage takes time and effort to publish 365 days a year.

If you find our news sites informative and useful then please consider becoming a regular supporter or for now make a one off contribution.

SpaceDaily Monthly Supporter
$5+ Billed Monthly

paypal only

SpaceDaily Contributor
$5 Billed Once

credit card or paypal

Multimessenger links to NASA's Fermi Mission show how luck favors the prepared
Greenbelt MD (SPX) Nov 09, 2018
In 2017, NASA's Fermi Gamma-ray Space Telescope played a pivotal role in two important breakthroughs occurring just five weeks apart. But what might seem like extraordinary good luck is really the product of research, analysis, preparation and development extending back more than a century. On Aug. 17, 2017, Fermi detected the first light ever seen from a source of gravitational waves - ripples in space-time produced, in this event, by the merger of two superdense neutron stars. Just five weeks la ... read more