Updated March 14, 2023
Introduction to Cloudera Manager
Cloudera Manager is the component of Cloudera Data Platform CDP. It is an end-to-end application to manage CDH clusters, it provides visibility and control of each part of the CDH cluster, by improving performance, enhancing the quality of services, reducing administrative costs, and increasing compliance. This application is used to configure, manage, and monitor CDP Cloud Base clusters and Cloudera Runtime service. It also automates the installation process, reduces deployment time, gives cluster-wide a real-time view of services and hosts running by providing a single central console by incorporating changes in configuration across clusters. Let us look wider into the topic and get to know how Cloudera Manager works, server, and other management services.
How does Cloudera Manager work?
To understand the working the user needs to know its inner workings or the Vocabulary and the Architecture of Cloudera Manager.
- Deployment is the major part, of clusters. Clusters are the collection of Coherent CDH hosts, and hosts are organized into racks.
- Services are the instances of a specific system with many roles, each assigned to a single host.
- A common point is the class type/ class instance nature of ‘role’ as well as ‘service’. Similar to other programming languages, “string” indicates the type or the instance of the type.
(“java.lang.String”) or (“hello world”).
Agent Server Architecture
It runs a central server, i.e., Cloudera Manager Server, also known as “SCM Server” and “CMF Server” in the past that hosts UI Web Server and application logic for managing CDH.
- Anything related to the installation of CDH, start and stop of services, and configuring services is managed by Cloudera Manager Server.
- Cloudera Manager agents are installed on each managed host and are responsible for starting, stopping Linux processes, triggering various installations paths, unpacking configurations, and monitoring hosts.
- Heartbeat makes the primary communication channel. Agent sends heartbeat, by default every 15 seconds to server to get know what agent is been doing.
- Agent when heartbeat’s saying, “Here’s what am up to!”, and the server in response says “Here is what you should be doing”. Hence both agent and server end up in reconciliation.
- If the user stopped service via UI, the agent will be stopping relevant processes. And if the process fails to start, the server marks the start command being failed.
Cloudera Manager Server:
Cloudera Manager Server maintains the entire state of the cluster, users can divide into “model” and “runtime” state, both stored in Cloudera Manager Server’s database.
- Model State: It is supposed to be what needs to run where, with configurations. It has 17 hosts, each supposed to run Data Node, which is known as Model State. User can integrate with the model through configuration screens.
- Runtime State: It is what processes are running, where, and what commands are being executed currently. Runtime state will include exact configuration files that need to run the process. When the user clicks “Start” in Cloudera Manager, the server gathers all the configurations for services and roles, validates, and then generated the configuration files by storing them in databases.
Cloudera Manager Monitoring and Other Management Services.
Cloudera itself manages some of its helpers, which include Activity Monitor, Host Monitor, Service Monitor, Reports Manager, Event Server, and Alert Publisher. These are managed separately for isolation and scalability.
- Activity Monitor: It collects information about activities that are run by MapReduce service, is not added as default.
Service Monitor: It collects health and metric information regarding services and activity information from Impala and YARN services. - Host Monitor: It collects the health and metric information of hosts.
- Event Server: It aggregates the relevant Hadoop events and makes them available for searching and alerting.
- Reports Manager: It generates reports that provide a historical view of disk utilization by users, directory, user groups, processing activities by YARN and user pool, HBase tables, namespaces. It is not added in Cloudera Express.
- Alert Publisher: It delivers and generates alerts for certain types of events.
Cloudera manages each as a separate role instead of considered as part of the Cloudera Manager Server for isolation and scalability. Additionally, Cloudera Management Service provides Navigator Audit Server, Navigator Metadata Server as roles for Cloudera Navigator.
To display Cloudera Management Service Status,
Select Clusters Cloudera Management Service
On Home Status Tab in Cloudera Management Service Table click on Cloudera Management Service Link
To Start Cloudera Management Service, minimum requirement is to have Cluster Administration,
Select Clusters Cloudera Management Service
Select Actions Start
On Home Status Tab click on the down arrow to the right of Cloudera Management Service click Start
Command details window will show up the progress of roles starting, when it gets completed with n/n successful subcommands, the task is said to be completed. Click Close
To Stop Cloudera Management Service, the minimum requirement is to have Cluster Administration,
Select Clusters Cloudera Management Service
Select Actions Stop
On Home Status Tab click on the down arrow to the right of Cloudera Management Service click Stop
Command details window will show up the progress of roles stopping, when it gets completed with n/n successful subcommands, the task is said to be completed. Click Close
Restarting Cloudera Management Service, minimum requirement is to have Cluster Administration,
Select Clusters Cloudera Management Service
Select Actions Restart
On Home Status Tab click on the down arrow to the right of Cloudera Management Service click Restart
Command details window will show up the progress of roles stopping and then starting when it gets completed with n/n successful subcommands, the task is said to be completed. click Close
Conclusion
With this, we shall conclude the topic “Cloudera Manager”. We have seen what Cloudera Manager means and how it works. We have also seen the Agent Server Architecture and Cloudera Manager Server, which has Runtime and Model State. Have also listed out each of the Other Management Services by Cloudera Manager. Last but not the least, have seen a few commands to Start/ Stop Cloudera Management Service, Restart, or to display Cloudera Management Service. This would help you to get an overview of what is Cloudera Manager and its components. Thanks! Happy Learning!!
Recommended Articles
This is a guide to Cloudera Manager. Here we discuss the introduction How does Cloudera Manager works, Agent Server Architecture, Cloudera Manager Monitoring, and Other Management Services. You may also have a look at the following articles to learn more –
