Colab, MLflow and papermill

Machine learning with the maximum of free GPU currently available plus the ability to keep a neat log of your data science experiments. Interested? My article shows a deep dive solution.

Quick summary

ColabMLflow and papermill are individually great. Together they form a dream team.

Colab is great for running  notebooks, MLflow keeps records of your results and papermill can parametrise a notebook, run it and save a copy.

All three are backed by top tier American companies, Colab by Google, MLflow by Databricks and papermill by Netflix.

Outline

Background and Motivation

Let me ask a few rhetorical questions (and give you my answers):

What’s your most comfortable way of running machine learning notebooks for experiments:  Colab.

What’s your easiest way of running machine learning experiments where you need a decent size GPU:  Colab.

What’s an elegant way of keeping track of ML experiments:  MLfLow.

What’s a nice way of turning Jupyter-style ML notebooks into reproducible and loggable code :  Papermill.

Code

My code is published on github, as usual:

https://github.com/hfwittmann/colab-mlflow-papermill

My code owes thanks to this repository (but without the Colab part):

https://github.com/floscha/papermill-mlflow-template

Continue reading “Colab, MLflow and papermill”