top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

The inter process communication between different nodes in Hadoop uses

0 votes
2,744 views
AREST API
BRPC
CRMI
DIP Exchange

Correct Option: 2  
posted Nov 30, 2017 by anonymous

Looking for an answer?  Promote on:
Facebook Share Button Twitter Share Button LinkedIn Share Button

Similar Questions
+1 vote
Hadoop: HTTPS communication between nodes, how can I confirm ?

We are trying to measure performance between HTTP and HTTPS version on Hadoop DFS, Mapreduce and other related modules.

As of now, we have tested using several metrics on Hadoop HTTP Mode. Similarly we are trying to test the same metrics on HTTPS Platform. Basically our test suite cluster consists of one Master Node and two Slave Nodes.

We have configured HTTPS connection and now we need to verify whether Nodes are communicating directly through HTTPS. Tried checking logs, clusters webhdfs ui, health check information, dfs admin report but of no help. Since there is only limited documentation available in HTTPS, we are unable to verify whether Nodes are communicating through HTTPS.

Hence any experts around here can shed some light on how to confirm HTTPS communication status between nodes (might be with mapreduce/DFS).

+1 vote
Two hadoop nodes on same machine while a second machine not joining the cluster

I have a test cluster of two machines, on both of them hadoop is installed. I have configured the hadoop cluster but on admin UI (as in the below picture) I see that two nodes are running on the same master machine, and that the other machine has no Hadoop node.

On master machine following services are running:

~$ jps 26310 ResourceManager 27593 Jps 26216 DataNode 26135 NameNode 26557 NodeManager 26701 JobHistoryServer

On the slave machine:

~$ jps 2614 DataNode 2920 Jps 2707 NodeManager

I don't why the slave is not joining the cluster (It was before). I tried to shutdown all servers on both machines and format HDFS then restarting everything but that didnot help. Any help to figure whats causing that behavior is appreciated.

+1 vote
Client and NN communication in Hadoop?

I have a Hadoop cluster running with 3 slaves and 1 Master. Slaves are Datanodes and running Tasktarckers. Namenode is running jobtracker and secondary namenode. I am running sample mapreduce after loading a file into HDFS.

According to Hadoop architecture ,before writing a file into HDFS , client will contact the Namenode and get the location details of DNs and then client directly write the file into DN.

What is this client ? is it an application running on Namenode ? Is user and client both are different ? How can I see the messages between client and datanodes?

...