AI helping to save the whales
When right whales started dying in the North Atlantic Ocean, researchers turned to IBM and AI to determine why. Using the deep learning platform called DeepSense, researchers are able to take advantage of rapidly evolving deep learning tools, frameworks and other software technologies to identify conservation strategies for right whales, which have been classified as endangered species by both the US and Canadian governments.
Now your forward-looking organization, which employs a smart and diverse group of researchers, can use state-of-the-art AI compute clusters and deep learning through multiple parallel-computing paradigms to solve your pressing challenges too.
Deep learning for research
Researchers and engineers across the scientific spectrum are increasingly becoming interested in experimenting with deep learning algorithms for their research. Deep learning is making an impact on several scientific applications that traditionally relied on numerical simulations such as seismic wave inversion and fluid flow simulations. Consequently, deep learning workloads are rapidly evolving as mainstream applications for parallel computing infrastructure. On the other hand, the convergence of traditional high-performance computing (HPC) and big data analytics is also gaining momentum among the academic and scientific communities.
Indeed, a recent report by leading researchers in the HPC community called for large-scale infrastructure design efforts (including both hardware and software) targeting the convergence of the above three compute patterns.
Modern parallel computing environments need to account for the simultaneous execution of these diversified workloads and their varied resource requirements. Apart from the FLOPS-hungry deep learning algorithms, the preprocessing steps for deep learning such as exploratory data analysis are also an integral part of modern workloads. For certain deep learning applications, it could be desirable to host the data sources (large-scale scientific data formats such as HDF5, databases, data lakes and so on) within the compute environment to facilitate the necessary data retrieval rates.
The DeepSense cluster
IBM Systems Lab Services helps clients create effective AI environments using IBM Systems infrastructure solutions and open source tools and technologies. Figure 1 shows a sample software infrastructure design developed by our consultants for the DeepSense cluster at Dalhousie University in Halifax, Nova Scotia, Canada.
The design in Figure 1 accommodates deep learning workloads, database maintenance and querying, and traditional numerical simulations in a single HPC environment. For deep learning workloads, this design enables efficient pipelining between data preparation and the learning epochs. The open source GPU databases supporting geospatial data (MapD, PostGres with PG-STROM) are used to accelerate the data querying and retrieval. The IBM Elastic Storage Server delivers an IBM Spectrum Scale software-defined storage platform providing as much as 2500 PB of storage space and sufficient I/O bandwidth for the HPC applications. The IBM Spectrum Suite of products (Spectrum Conductor and Spectrum LSF) provide job scheduling services for the diverse scientific workloads discussed above. The Spectrum Conductor schedules the GPU resources for Spark workloads and for the database queries.
An oceanic save
This compute environment is used for hosting and maintaining a peta-scale database of S-AIS data that can be queried by researchers on demand. The research on right whale conservation is using the AIS data along with acoustic sensors deployed in the ocean for an oceanic intervention. Other deep learning applications hosted by this compute environment span a variety of scientific fields including ocean engineering, bioinformatics and natural language processing.
IBM Systems Lab Services will consult with the scientific minds across your organization that are dealing with the inherent ambiguity in their research and customize the compute environments for their anticipated research applications as well as future needs.
Reach out to Lab Services today for support on your AI, big data and HPC projects for scientific research.
To learn more about DeepSense and the research being done at Dalhousie University, contact DeepSense Executive Director Kevin Dunn.
The post AI helping to save the whales appeared first on IBM IT Infrastructure Blog.