Let's begin with the challenges associated with machine learning application development. As many of you are probably painfully aware, developing machine learning applications is complex. To expand on that a bit, let's look at the typical machine learning lifecycle. It's a forest that brought this that all begins with the collection of the raw data. After data has been collected, is cleaned and prefix. This prefix data is used to fit or train a model. Finally, this model is deployed to a production environment to satisfy a use case for either business or production precinct. This production application often receive new data that was not part of the original training dataset, which kick starts the next iteration of this life cycle. This may sound straightforward. We have a forest that brought this, but several layers of complexity actually makes this a daunting task for many companies. If I try to implement any particular phase of this lifecycle, I first notice that many tools are available for each individual estates, but no single tool implements all four. This means that practically as a developer, I am combining [inaudible] and preparation like Kafka with model training breakwaters like TensorFlow, with deployment and vitamin say Kubernetes, and then repeating the process each time I add a new tool. Where we can successfully graph by blinds to combine various tools, the work is still not complete. We also know that hyper parameter tuning is a major element of the machine learning lifecycle and must be supported. Models are highly sensitive to configuration parameters referred to as hyper parameters that dramatically affect the performance. Selecting the appropriate set of parameters can produce a model that will revolutionize a production use case, but failure to select good parameters, might produce a result that is no better or even worse than this work. For that, when you are developing a platform that implements this lifecycle, you allow data scientists to explore this parameter space. Additionally a scale, this becomes a problem when implementing a solution inside of this life cycle. Although pipeline goal might work for a few developers, this pipeline goal can fail to scale properly to large companies, and this becomes increasingly problematic as the number of machine learning practitioners increase. Finally and very important, the model exchange and governments is a major problem intrinsic to the machine learning lifecycle. It's not sufficient to train a model once in a black box, deployed and forget where it came from. Every model within your organization needs a cycle bit limits or track record of the hyper parameters that were used to try that model, as well as the sole score, performance metrics and information about who trained the model and when. This is particularly important for organizations that are beginning to leverage machine learning in highest putting environments such as the financial sector. Now that we have expand on the difficulties associated with these lifecycle, it's clear that we don't have a single step process. Instead, we have a significant platform development challenge for many organizations that are attempting to leverage machine learning today. Do mind me asking, are there any platform solution right now that they standardized the lifecycle? We previously identified the effects from Google and there are others, such as Facebook, EverLearner and [inaudible]. These platforms sometimes limit the set of tools, programming languages, and algorithms that data scientists can leverage when building the models. For example, the effects may be an excellent platform from grading TensorFlow models, but it is a less app for developers who want to write classical models using frameworks like [inaudible] , for example. This lead data bricks engineers to a motivating question. Can they provide the benefits of standardized lifecycle compared to these platforms, but also in an open manner that would allow data scientists to bring the set of tools, languages, and algorithms that they require on a daily basis?
