TensorFlow 2.0 - A comprehensive platform that supports machine learning workflows

TensorFlow has is an open source library software library, which has been predominantly used for developing machine learning applications, specifically applications based on the neural networks. Currently, TensorFlow is used for both, research works as well as the production of applications. TensorFlow was developed by Google Brain, and the initial release of TensorFlow happened in the year of 2015, named as TensorFlow 1.X.  TensorFlow is available on the leading desktop as well as mobile platforms. It can be deployed, right from desktops to clustered nodes, to mobile & edge devices. 

 

Even though TensorFlow has been one of the most effective platforms for developing machine learning applications, the platform itself is deemed as quite complex and unstructured. One needs to have a deeper level of expertise on TensorFlow, in order to work on this platform. TensorFlow 2.0 brings some critical improvements over the previous version that is 1.X. TensorFlow 2.0 makes the platform much simpler and facilitates ease of use. This article will discuss the new architecture of TensorFlow 2.0, and the improvements which it brings with it. It also discusses the key concepts, which make TensorFlow 2.0 more effective. It also discusses the process of upgrading 1.X to 2.0.

 

 

New Features of TensorFlow 2.0

Some of the key concepts that make TensorFlow 2.0 effective are mentioned below. These are the key changes which make TensorFlow 2.0 much more productive:

• API Cleanup: In the latest version of TensorFlow, some of the API’s have been either removed or they have been moved. This includes removal of tf.app, tf.flags, and tf.logging, and has been replaced by absl-py. Even tf.* name has been completely removed, and the lesser used functions have been shifted to some of the sub packages. While some of them have replaced, only get an equivalent of them in 2.0. This includes tf. Summary and tf.keras.metrics. The API Cleanup makes the platform much more structured and simpler, that enhances the overall productivity.
• Eager Execution: With TensorFlow 2.0 one can expect an imperative environment for programming with eager execution. Eager execution enables the evaluation of operations immediately, rather than doing later. The evaluation is done even without the building the graphs. The evaluation operations return concrete values, without constructing the computational graphs, that can be run later. This makes debugging extremely easy and helps one to get started with the platform. Eager execution makes use of Python control flow instead of the graph control flow, which makes the flow control much more natural.
• No More Global namespaces: Unlike TensorFlow 1.X, the newer version is not going to rely anymore on the global namespaces. So, no more global namespaces in TensorFlow 2.0. So, whenever tf.Variable() is called, it will be there in the default graph. As a result, the variables can be easily tracked in the newer environment, provided that one knows the name of the variable with which it has been created.
• No more Sessions, only functions: In TensorFlow 2.0, we will see the transformation of sessions into functions. So one can call a function with the inputs specified, the function will return the output. In TensorFlow 2.0, one can make use of Python function for just in time compilation, where it runs as a single graph. By running into graph mode, both performances, as well as portability, is ensured. Through graph mode, the function can be optimized, and it can be exported or reimported accordingly.
• Refractor Code into small functions: In TensorFlow 2.0, the code is refactored in smaller functions. These functions are called upon whenever they are required, implementing just in time compilation. It should be noted that tf.function () should be used for only those computations which are high level as well as complex, for rest tf.function () might not be used.
• Manage variables with Keras layers and models: Use the flexibility as well as the convenience of the Keras models and layers to manage the variables. Make use of the “variables” and the “trainable variables” to make the management of local variable much more flexible, while they are being used.
• Combine tf.data.Datasets and @tf.function: We can use tf.data.Dataset, to stream the training data from the source. In the eager mode, datasets are iterable, and work just like the other Python iterables. One can wrap the code in tf. function (), in order to use features like dataset async prefetching and async dataset streaming. This replaces the Python iteration with equivalent graph operations, that can be accomplished by using Autograph.
• Use tf.metrics to aggregate data and tf.summary to log it: Aggregate the data with the help of tf.metrics (), before they can be logged as summaries. Metrics have state, as they keep on accumulating the values, and then they return a cumulative result when the function .result () called. The accumulated values can be easily cleared from the metrics, by calling the function .reset_states(). Similarly, tf. summary can be used for logging summaries.

 

 

The New Architecture of TensorFlow 2.0

The new architecture in TensorFlow 2.0 brings certain improvements over TensorFlow 1.X. These improvements will definitely make TensorFlow a machine language ecosystem of AI enabled technologies. Some of the key features are listed below:

• The first big feature of the improved architecture of TensorFlow 2.0 is, Keras is going to be the primary API to the TensorFlow. In the previous versions, there were issues such as lack of flexibility in extension and modification during implementation, difficulty in designing deep learning systems, etc. However, with the adoption of Kera as the primary API for the TensorFlow, it makes TensorFlow much more flexible and less complex. Keras is an open source project and is much simpler to use. Hence, with this improvement, designing as well as the implementation of deep learning systems will be much easier and simpler. 
•  The workflow in TensorFlow 2.0 is going to be much more simplified as well as integrated. The new workflow is much flexible in comparison to the older one. One can use tf.data for loading data and pre-processing. Similarly, Keras can be used for model construction as well as validation. On the other hand, tf.function can be used for DAG graph based execution. Thus TensorFlow 2.0 has become much simpler, even a beginner can start developing deep learning models on this platform.
• With the newer version of TensorFlow 2.0, there is expanded support for mobile platforms, that includes both Android as well as iOS. It also includes the support for Web (JavaScript), TF Lite, TF Edge, along with the IoT. With the expanded support, multiple deployment options are available on these platforms. For WebAssembly, SIMD+ support is available. Similarly, in Colab, support for Swift is available. Swift is used on the iOS platform. 

 

[Related Page: Installing TensorFlow ]

Apart from this, there is extended support available for data input pipelines, along with the data visualization libraries in JavaScript. The Footprint for Mobile Computing and IoT is going to be much smaller and simpler, with the integrated flow. The integrated workflow will implement an extended as well as better support for audio as well as text-based models. The revamped architecture for extended support for mobile has been shown below.

 

 

Before using a "TensorFlow Lite model" in the mobile app one needs to choose a pre-trained model, convert the model to a TensorFLow Lite format, and finally, integrate the model in the app. Once the conversion is done, the .tflite is generated which extends the APIs for the respective platforms, that is Android and iOS. As the Android apps are written in Java, and the core TensorFlow library is written in C++, a JNI library is provided as an interface. While for iOS platform no interpreter is required.

TensorFlow 2.0 is going to have unified programming models along with unified methodologies. This will be done using Keras, especially the methodologies used for coding deep learning network. This includes Symbolic or Declarative APIs and Imperative APIs (Subclassing). Symbolic or Declarative API aids in the development of models symbolically, by the description of it “Directed Acyclic Graph” (DAG). This helps in better visualization of the model. Also, the debugging errors might appear only during the compile time. With the help of imperative API, subclassing can be done without any complexities. 

One of the brand-new offerings of TensorFlow 2.0 is AIY (Artificial Intelligence for Yourself). TensorFlow AIY helps the consumers to implement AI for their projects as well as gadgets by themselves without any complex coding or complex models. It is a DIY (Do it Yourself) for consumers, where the consumers themselves can implement AI as per their requirement.

 

[Related Page: Technologies to Upskill Your Career]

 

 

The Benefits of TensorFlow 2.0 Updates

With TensorFlow 2.0 there are multiple benefits available for the user or the developer. Some of the key benefits of using TensorFlow 2.0 update are:

• First of all, by making Keras as the primary API, the platform has become much simpler as well as easy for the users. The new platform is much more structured, with a natural flow. Hence, it is expected that someone who has even lesser experience in TensorFlow can also develop deep learning models on the fly. 
• With the help of API cleanup, the APIs are much more structured as the APIs which are not relevant or redundant, are either removed permanently or moved under different packages. Both, Keras API, and restructuring of the existing APIs’ make TensorFlow 2.0 much more productive. This has also made the APIs more consistent. This includes unified RNN and unified optimizers. 
• With Eager execution, Python control flow is used that makes the flow control much more natural, and better integrates with the Python runtime. TensorFlow 2.0 has intuitive higher-level APIs, which facilitates flexible development of deep learning models.

 

[Related Page: Object Detection Using Tensorflow]

Effectiveness of TensorFlow 2.0

Some of the key concepts that make TensorFlow 2.0 effective are mentioned below. These are the key changes which make TensorFlow 2.0 much more productive:

• API Cleanup: In the latest version of TensorFlow, some of the API’s have been either removed or moved. This includes removal of tf.app, tf.flags, and tf.logging, and has been replaced by absl-py. Even tf.* name has been completely removed, and the lesser used functions have been shifted to some of the sub packages. While some of them have replaced, only get an equivalent of them in 2.0. This includes tf. Summary and tf.keras.metrics. The API Cleanup makes the platform much more structured and simpler, that enhances the overall productivity.
• Eager Execution: With TensorFlow 2.0 one can expect an imperative environment for programming with eager execution. Eager execution enables the evaluation of operations immediately, rather than doing later. The evaluation is done even without the building the graphs. The evaluation operations return concrete values, without constructing the computational graphs, that can be run later. This makes debugging extremely easy and helps one to get started with the platform. Eager execution makes use of Python control flow instead of the graph control flow, which makes the flow control much more natural.
• No More Global namespaces: Unlike TensorFlow 1.X, the newer version is not going to rely anymore on the global namespaces. So, no more global namespaces in TensorFlow 2.0. So, whenever tf.Variable() is called, it will be there in the default graph. As a result, the variables can be easily tracked in the newer environment, provided that one knows the name of the variable with which it has been created.
• No more Sessions, only functions: In TensorFlow 2.0, we will see the transformation of sessions into functions. So one can call a function with the inputs specified, the function will return the output. In TensorFlow 2.0, one can make use of Python function for just in time compilation, where it runs as a single graph. By running into graph mode, both performances, as well as portability, is ensured. Through graph mode, the function can be optimized, and it can be exported or re-imported accordingly.
• Refractor Code into small functions: In TensorFlow 2.0, the code is refactored in smaller functions. These functions are called upon whenever they are required, implementing just in time compilation. It should be noted that tf.function () should be used for only those computations which are high level as well as complex, for rest tf.function () might not be used.
• Manage variables with Keras layers and models: Use the flexibility as well as the convenience of the Keras models and layers to manage the variables. Make use of the “variables” and the “trainable variables” to make the management of local variable much more flexible, while they are being used.
• Combine tf.data.Datasets and @tf.function: We can use tf.data.Dataset, to stream the training data from the source. In the eager mode, datasets are iterable, and work just like the other Python iterables. One can wrap the code in tf. function (), in order to use features like dataset async prefetching and async dataset streaming. This replaces the Python iteration with equivalent graph operations, that can be accomplished by using Autograph.
• Use tf.metrics to aggregate data and tf.summary to log it: Aggregate the data with the help of tf.metrics (), before they can be logged as summaries. Metrics have state, as they keep on accumulating the values, and then they return a cumulative result when the function .result () called. The accumulated values can be easily cleared from the metrics, by calling the function .reset_states(). Similarly, tf. summary can be used for logging summaries.

 

[Related Page: What Is Artificial Neural Network And How It Works?]

Differences between TensorFlow 1.x and TensorFlow 2.0

Some of the key differences between TensorFlow 1.X and TensorFlow 2.0 are mentioned in the table below:

 

TensorFlow 1.X	TensorFlow 2.0
In TensorFlow 1.X, users have to manually construct an abstract syntax tree by calling tf. * API, followed by a manual compilation of the abstract syntax tree.	In 2.0, this has been replaced by Eager Execution, where the abstract syntax tree or the graph is not constructed initially but is done later done from the output returned from the evaluation of the operations.
TensorFlow 1.X relied on the global namespaces.	No more global variables in TensorFlow 2.0. Local variables can be created and tracked with its name during its entire lifecycle.
Computations are done on a pre-emptive basis, where the selected tensors with functions are loaded beforehand, even if it is not required.	The code is refactored and is made into much smaller and modular functions. These functions are called only when they are required, and not preemptively.
Complex platform with structure lacking natural flow. Not suitable for novice users.	Simplified platform with Keras as the main API, with more structured as well as natural flow. Perfect for novice users.
Autograph is not available 1.X. , and for conversion into graph mode, tf.cond and tf.while_loop needs to be used.	The autograph feature is available, and the same is used to convert data-dependent control flow into graph-mode.

 

[Related Page: Machine Learning Datasets]

How to Migrate from TensorFlow 1.x to TensorFlow 2.0

It is absolutely possible to migrate the existing code in 1.X to TensorFlow 2.0. By doing this, one can make use of the improvements available in 2.0, for their code. 

Upgrade Script 

The first option will be to run the automatic conversion script or the upgrade script, as it is known. Since, TensorFlow 2.0 has got multiple changes, and manually make those changes to the script will be a difficult task. Hence, this is where the upgrade script is extremely handy, as it seamlessly upgrades your existing code to the latest version of TensorFlow. This could be easily done with the help of tf_upgrade_v2, which contains the upgrade from legacy version to 2.0. You don’t need to download this utility separately, as it is included in pip install in TensorFlow 2.0. 

Once the upgrade process is initiated, the script converts the existing TensorFlow 1.x Python scripts to the latest version. However, it must be noted that some of the API symbols cannot be upgraded, simply by replacing the string. Hence, the upgrade script also checks the compatibility with the help of compatibility module compat.v1. This module will check the compatibility. This includes checking the name of namespaces, symbols, etc. It should be noted that some of the APIs which was there in 1.X is no longer available in TensorFlow 2.0, hence they might not be upgraded successfully. Once the compatibility is determined, only after that the automatic conversion script proceeds with the upgrade.

 

 

Step by Step summary of the overall upgrade process:

1. Execute the upgrade script.
2. Remove the contrib symbols.
3. Switch the existing models to object-oriented style, with Keras API.
4. Make use of tf.keras or tf.estimator, for training as well as evaluation of the loops wherever it is applicable.
5. Use custom loop wherever applicable,  but this has to be done without sessions and collections, as they are not available in TensorFlow 2.0.

 

[Related Page: Machine Learning Algorithms]

What to expect after the upgrade?

• Reduced number of lines of code.
• The increased amount of simplicity.
• The increased amount of clarity.
• Easy debugging process