Deep Learning Frameworks in Python

Deep learning has been booming in the recent years, mostly due to how well it scales with large amounts of data. We are producing more and more data each year, and as our data grows, so does our need for methods of regression and classification.

Another phenomenon that is simultaneously occurring is the popularity of Python, specifically its the fastest growing language in 2018 (at least on stack overflow). Originally written as a hobby project by Guido van Rossum in the late 80, Python was founded on very good principles that both lower the barrier to entry and promote readability. These traits are big factors in why python has grown and stuck around all these years.

This article will go through the two major python deep learning frameworks, detail the background, and weigh the pros and cons of each.

Pytorch

Background

Pytorch was developed by Facebook to be python-based at its core, which helps greatly with development. Many other frameworks are heavily invested in other languages like C++, as there are speed benefits in doing so. Pytorch borrows concepts and developments from Torch (written in Lua) and in March 2018 was merged with Caffe2 which was also created by Facebook

Pros

• Debugging is easiest in Pytorch compared to the other two libraries since it is written in native Python, the error messages are more meaningful and allow you to do a stack trace and use the python debugger.

• Pytorch is CUDA compatible which allows GPU computation ( usually more efficient than CPUs for Deep Learning). You can even use multiple GPUs.
• The Dynamic Graph allows more flexibility in computation structure, which can lead to faster speeds depending on the use case, and variable length sequences allow more complex structures.
• Pytorch has interfaces from multiple languages but to work on mobile, you must convert your model into a different framework (Caffe2 for android).

Cons

• While still quite large and well adopted now, Pytorch doesn’t have as large of a user-base as Tensorflow, which also means that there are community packages/capabilities that work well for Tensorflow that won’t work as well as Pytorch.
• Pytorch came out with its 1.0 December 7, 2018, which is quite recent. This implies that the code-base has room for growth. With the rapid pace of machine learning, this could mean near-term major changes.

Tensorflow

Background

Tensorflow by Google was originally released on November 9, 2015. Similar to Pytorch it allows for various machine learning computations, especially Deep Learning. Tensorflow builds off of an internal tool called DistBelief.

Pros

• Tensorflow allows very fine control of every aspect of deep learning.
• Tensorflow is CUDA compatible and you can even use multiple GPUs and CPUs.
• Keras is a very powerful interface for creating a tensorflow Neural Network with ease and Tensorboard is an amazing visualization tool.
• Tensorflow supports multiple languages – python, Go, Javascript, etc and even mobile!

Cons

• Debugging can be very difficult as the error messages are not as descriptive. Also, for full debugging, you must use a specialized tool – tfdbg and you can’t use the built-in python debugger.

• The codebase is disorganized which makes it hard to determine the best way to accomplish a task.

Comparison

Both libraries are very full-featured, with a slight edge to Tensorflow. Both libraries have very large communities and many compatible helper libraries. Tensorflow being so low level is difficult for beginners, but Keras fills that gap. Pytorch came later to the industry, so it was able to utilize some of the lessons learned by Tensorflow.

Pytorch is nicer to write in python, and the variable length sequences are great for NLP type applications among others. Tensorflow is more mature and has more surrounding packages to offer while being more portable to other languages/devices.

There are other packages as well, but for 95% of the readers, Keras/Tensorflow and Pytorch will be all you need.