Updated March 22, 2023
Introduction to TensorFlow
In this article, we will see an outline on TensorFlow Architecture. TensorFlow is a cross-platform library, and there are some common concepts like operations, sessions, and computation graphs. TensorFlow Serving makes new algorithms and experiments easy to deploy while retaining the same database architecture and APIs.
What is Tensorflow Architecture?
Tensorflow is a commonly used deep learning library, which is built by the Google brain team that was working on Artificial Intelligence. It was developed as it can be used on multiple CPUs, GPUs, and can run on mobile devices and supports different wrapper classes like python, C++ or Java and Tensors, which describe the linear relation between vectors, scalars, and other tensors.
Tensors are nothing but multidimensional arrays. For understanding the TensorFlow architecture and features, first, we need to understand some terms.
- Tensorflow Servables
- Servable Versions
- Servable Streams
- Tensorflow Model
- Tensorflow Loaders
- Tensorflow Sources
- Tensorflow Manager
- Tensorflow Core
- Life of Servable
- Batcher In Tensorflow architecture
1. Tensorflow Servables
- Tensorflow Servables are common objects that are mainly used to perform computation. The size of the TensorFlow servable is very flexible. Servables are central uncompleted units in TensorFlow serving.
- A servable unit can contain anything like it can contain tuples, lookup tables. Servable objects can be of any type and any interface. These features are very flexible and easy to make future improvements such as Asynchronous modes of operations, Streaming results, and Experimental APIs.
2. Servable Versions
- Servable versions are used for maintaining versions of TensorFlow servable. It can handle one or more versions of Servables. It can help configure the new algorithms, handle weights, and other changes can be done, data can be loaded.
- It also can change and manage versions, enable multiple servable versions at a time.
3. Servable Streams
It is a collection of different versions of Servables. It stores it in an increasing sequence of versions.
4. Tensorflow Models
- A TensorFlow model can contain one or more TensorFlow Servables. It is mainly used in machine learning tasks containing more than one algorithm that contains different weights according to model, lookup tables, metrics and table embeddings.
- It can be served in many different ways such that there are different ways to serve the lookup table.
5. Tensorflow Loaders
- Tensorflow loader is an API that manages the TensorFlow Servables life cycle. It provides a common infrastructure for some of the learning algorithms.
- The main use of the TensorFlow loader is to load and unload Servables using standardized loader APIs.
6. Tensorflow Sources
- Sources are work as a plugin module. Tensorflow source does the basic handling of servable like the finding of servable, provide servable on the request.
- Each reference can provide one or more servable streams at a time. Each source provides a loader instance for handling the version of the stream. And each servable stream is loaded on the version based.
- Source have different states that can be shared with different versions and to multiple Servables. Can also handle update or change between versions.
7. Tensorflow Managers
- Tensorflow manager handles the end to the end life cycle of servable.
- Loading Servable, Serving Servable, Unloading Servable.
- A manager gets a response from all servable sources and also monitor all version of it. The manager can also refuse or postpone the upload.
- GetServableHandle() function provides an interface that helps to handle managerial things about TensorFlow Servables.
8. Tensorflow Core
Tensorflow Core contains:
- Lifecycle Metrics.
- Tensorflow serving core takes servables and loaders as an object.
9. Life of a Servable
The whole procedure is explained about how servable is used and the whole life cycle of servable:
- Source create loaders for different servable versions. The client requested that the manager and manager send the loaders aspired version, load and served back to the client.
- Loader contains metadata that can be used to load the Servables. A callback is used to notify the manager about the servable source versions. The manager follows a version policy that helps to configure and decide what action to be needed next.
- The manager also checks the loaders’ security; if it is safe, it will give the necessary resources to the loader and permit to load the new version. The manager provides Severable on the client request. It will request a specific version explicitly or direct the latest version after that manager returns a handle for access the servable.
- The Dynamic manager handles the servable versions, apply the version policy and decides the version or latest version needs to be loaded. The dynamic manager allocates memory to the loader. Loader initiates the TensorFlow graph and updates the weights of the TensorFlow graph.
- When a client requests the servable and checks the client’s version, then the Dynamic manager returns that handler with the requested version of Servable.
10. Batcher in Tensorflow Architecture
- Batching is a procedure of handling multiple requests into a single request.
- Using this procedure, we can reduce the cost of performing inferences, especially the acceleration of hardware like GPU, TPU.
- Tensorflow servable contains a Batcher widget, by using this widget client can easily batch their multiple specific inference requests into a single batch request. So they can run this procedure efficiently.
Conclusion – TensorFlow Architecture
Tensorflow Architecture represents a decentralized system. It shows how different components work and serve their roles in a decentralized system. Serving on the client’s request and using batching multiple operations parallel. Some of TensorFlow architecture’s best features are batching the operations, hardware acceleration, dynamic manager options, loaders, sources, and servable streams.
Recommended Articles
This is a guide to TensorFlow Architecture. Here we discuss What is TensorFlow and some of the best features in TensorFlow Architecture. You can also go through our other suggested articles to learn more –
