A person overwhelmed by boxes by Cottonbro

Note: If you don’t want to read the blah-blah context and history stuff then you can jump to the recommendations

The Problem

The need for virtual python environments becomes fairly obvious early in most Python developers’ careers when they switch between two projects and realise that they have incompatible dependences (e.g. project1 needs scikit-learn-0.21 and project2 needs scikit-learn-0.24). Unlike other mainstream languages like Javascript(Node.js) and Java (with Maven) where dependencies are stored locally to the project, Python dependencies are installed at system or environment level and affect all projects that are using the same environment.


A beige analog compass by Ylanite Koppens

Introduction

Open machine learning research is undergoing something of a reproducibiltiy crisis. In fairness it’s not usually the authors’ fault — or at least not entirely. We’re a fickle industry and the tools and frameworks were ‘in vogue’ and state of the art a couple of years ago are now obsolete. Furthermore, academics and open source contributors are under no obligation to keep their code up to date. It is often left up to the reproducer to figure out how to breathe life back into older work.


MLFlow is a powerful open source MLOps platform with built in framework for serving your trained ML models as REST APIs. The REST framework will load data provided in a JSON or CSV format compatible with pandas and pass this directly into your model. This can be handy when your model is expecting a tabular list of numerical and categorical features. However it is less clear how to serve with models and pipelines that are expecting unstructured text data as their primary input. …


Introduction

When you’re working with large datasets, storing them in git alongside your source code is usually not an optimal solution. Git is famously, not really suited to large files and whilst general purpose solutions exist (Git LFS being perhaps the most famous and widely adopted solution), DVC is a powerful alternative that does not require a dedicated LFS server and can be used directly with a range of cloud storage systems as well as traditional NFS and SFTP-backed filestores all listed out here.


In an ever-growing digital landscape filled with more content than a person can consume in their lifetime, recommendation engines are a blessing but can also be a a curse and understanding their strengths and weaknesses is a vital skill as part of a balanced media diet.

If you remember when connecting to the internet involved a squawking modem and images that took 5 minutes to load then you probably discovered your favourite musician after hearing them on the radio, reading about them in NME being told about them by a friend. Likewise you probably discovered your favourite TV show by watching live terrestrial TV, your favourite book by taking a chance at your local library and your favourite movie at a cinema. …


In this article I’ll show you how to use free open source free and open source software to build productivity automation workflows to make Evernote and IFTTT jealous whilst keeping your data safely on your own machine(s).

Introduction

Many people are familiar with apps like Evernote and more recently Notion that allow you to keep notes on pretty much everything — almost like a second “digital” brain. There are also plenty of mature tools like IFTTT and Zapier that allow you to chain together the many digital resources in your life. E.g. …


Things are scary and unprecedented but as the great philosopher Jeff Goldblum once said “life finds a way”. We will find new ways to thrive.

TL;DR — just read the bolded bits!


I recently ran into an issue where the newest version of Torch (as of writing 1.4.0) requires a newer version of CUDA/Nvidia Drivers than I have installed.


In recent weeks and months the impending global climate catastrophe has been at the forefront of many peoples’ minds. Thanks to movements like Extinction Rebellion and high profile environmentalists like Greta Thunberg and David Attenborough as well as damning reports from the IPCC, it finally feels like momentum is building behind significant reduction of carbon emissions. That said, knowing how we can help on an individual level beyond driving and flying less still feels very overwhelming.

The Energy Issue

A recent study by Strubel et al. (2019) gave insight into exactly how much energy certain neural architectures require to train. Their findings show…


Filament build and deploy enterprise AI applications on behalf of incumbent institutions in finance, biotech, facilities management and other sectors. James Ravenscroft, CTO at Filament, writes about the challenges of enterprise software deployment and the opportunities presented by Kubernetes and Google’s Anthos offering.

It is a big myth that bringing a software package to market starts and ends with developers and testers. One of the most important, complex and time consuming parts of enterprise software projects is around packaging up the code and making it run across lots of different systems: commonly and affectionately termed “DevOps” in many organisations.

The Role of Devops in B2C and B2B Software companies

DevOps engineers for consumer-facing software have a pretty tough job. They engineer and maintain complex pipelines of automation that take the latest version of the developers code, run tests on it and then build a smorgasbord of installers and self-extracting packages to work on…

James Ravenscroft

Ml and NLP Geek, Saxophonist, foodie and explorer. I was born in Bermuda and I Live in the UK, PHD Student at Warwick and CTO at Filament.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store