CodeFlare: An open-source framework for machine learning

IBM introduced on Wednesday its latest open-source serverless framework, with the solemn purpose of simplifying integration and efficient scaling of big data and artificial intelligence (AI) workflows into its hybrid cloud.

IBM’s recent technology, CodeFlare, is designed at the rank of a promising open-source distributed computing framework for machine learning (ML) applications commonly referred to as Ray.

CodeFlare will have the ability to drastically minimize time to set up, run, and scale ML tests. By applying the framework to evaluate and optimize roughly 100,000 pipelines for training machine learning models purposes, it will reduce the required time to execute each pipeline from 4 hours to 15 minutes.

The main purpose of this modern creation is to expand and rise the capabilities of CodeFlare Ray by inserting certain elements to make scaling workflow more accessible and understandable.

As big data and machine learning analytics are on the rise in every industry, tasks are developing more complicated features, IBM commented on their research blog.

“To create a machine learning model today, researchers and developers have to train and optimize the model first. This might involve data cleaning, feature extraction, and model optimization. CodeFlare simplifies this process using a Python-based interface for what’s called a pipeline – by making it simpler to integrate, parallelize and share data,” IBM stated in its research blog.

Since CodeFlow offers a Python-based interface to manage pipelines across multiple platforms, it will allow these pipelines to be shared and parallelized in manycomputing systems and incorporated with different cloud-native ecosystems through adapters.

To further elaborate, IBM data pipelines with an instant delivery of virtual database copies to work with near real-time data for data analytics, application testing, AI model training and testing and data virtualization.

The multinational technology company created its recent framework to help users go beyond splitting tasks for two-to-one end-to-end pipelines to integrate and scale, as a substitute of adopting a container with a database-friendly interface, such as Python.

It will provide users with a more simplified approach of integrating and measuring the entire pipeline while providing an integrated runtime and programming interface.

“The motivation behind the framework is the emergence of convergent workflows that combine AI and machine learning, data analysis and modeling, and the increasing complexity of integrating modalities beyond individual steps,” said Nagpurkar, the director of cloud platform research at IBM Research Center.

So, what does this mean for developers?

By implementing CodeFlare open-source framework, developers can anticipate a framework that saves time and effort in creating pipelines implemented in IBM’s hybrid cloud, instead of having to imitate and simulate their efforts or struggles.

According to IBM’s blog, when a user implements a framework to evaluate and optimize nearly 100,000 pipelines for training machine learning models, CodeFlair will minimize the amount of time needed implement every pipeline from four hours to one minute.