top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

HDFS dfsclient: possible to force storage datanode ?

+2 votes
190 views

We have 52 DNs and more hundred clients they are store and read data from hdfs. One rack has 3 DNs and about 15 clients.

It is possible to force (if space is available) that these 15 clients prefer the 3 DNs located in there own rack to store and read data?

racklocation.conf with "org.apache.hadoop.net.NetworkTopology" is allready in use, but this does not help in this case

posted Aug 20, 2014 by anonymous

Looking for an answer?  Promote on:
Facebook Share Button Twitter Share Button LinkedIn Share Button

Similar Questions
0 votes
How to get info about which data in hdfs or file system that a MapReduce job visits?

I was trying to implement a Hadoop/Spark audit tool, but l met a problem that I can't get the input file location and file name. I can get username, IP address, time, user command, all of these info from hdfs-audit.log. But When I submit a MapReduce job, I can't see input file location neither in Hadoop logs or Hadoop ResourceManager.

Does hadoop have API or log that contains these info through some configuration ?If it have, what should I configure?

0 votes
What happens to a read operation when the file is moved to trash in HDFS?

I have a basic question regarding the HDFS file read. I want to know what happens, when the following steps are followed:

  1. Client opens the file for reading and starts reading the file.
  2. In the meantime, someone deletes the file and file moves to the trash folder

Will Step 1. succeed? I feel, since the client has already opened the file and file still exists in .trash, the client should continue to read the file.

0 votes
Unable to use ./hdfs dfsadmin -report with HDFS Federation

Our hadoop cluster is using HDFS Federation, but when use the following command to report the HDFS status

$ ./hdfs dfsadmin -reportreport: FileSystem viewfs://nsX/ is not an HDFS file system
Usage: hdfs dfsadmin [-report] [-live] [-dead] [-decommissioning]

It gives me the following message that viewfs is NOT HDFS filesystem. Then how can I proceed to report the hdfs status

0 votes
Can anyone provide me steps to grant a particular user access equivalnent to that of "hdfs" user in hadoop.

The reason behind this is I want to have my custom user who can create anything on the entire hdfs file system (/).
I tried couple of links however, none of them were useful. Is there any way by adding/modifying some property tags I can do that ?

...