HDFS NFS Gateway configuration help needed

We have a requirement to access Apache Ranger audit data from a non-HDFS host. Our ranger setup uses elastic for the live data and this is archived to HDFS on the same cluster. The non-HDFS node is in the same subnet but is not HDFS-aware.

I read through the Apache documentation for Hadoop and, specifically, the part for setting up NFS gateway.

https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html

What I do not understand about this is that it is using standard NFS commands on the NFS server but nowhere does it indicate how that relates to HDFS access. NFS reads the /etc/exports list but HDFS files and directories will not be accessible as standard Unix files. So I cannot see how any exports list will contain meaningful configuration data - even though the docs do not mention editing this file anyway.

For this core-site config I used the existing hdfs superuser and the hosts are just *

<property>
  <name>hadoop.proxyuser.nfsserver.groups</name>
  <value>root,users-group1,users-group2</value>
  <description>
         The 'nfsserver' user is allowed to proxy all members of the 'users-group1' and
         'users-group2' groups. Note that in most cases you will need to include the
         group "root" because the user "root" (which usually belonges to "root" group) will
         generally be the user that initially executes the mount on the NFS client system.
         Set this to '*' to allow nfsserver user to proxy any group.
  </description>
</property>

<property>
  <name>hadoop.proxyuser.nfsserver.hosts</name>
  <value>nfs-client-host1.com</value>
  <description>
         This is the host where the nfs gateway is running. Set this to '*' to allow
         requests from any hosts to be proxied.
  </description>
</property>

So our proxyuser exists already (default names: hue, hive, flink, etc)

<property>
  <name>hadoop.proxyuser.hdfs.groups</name>
  <value>*</value>
</property>

<property>
  <name>hadoop.proxyuser.hdfs.hosts</name>
  <value>*</value>
</property>

For the hdfs-site.xml config, I added

<property>
  <name>nfs.exports.allowed.hosts</name>
  <value>* rw</value>
</property>
<property>
  <name>nfs.superuser</name>
  <value>hdfs</value>
</property>
<property>
  <name>nfs.export.point</name>
  <value>/ranger</value>
</property>

So for that specific ranger data I would have to use hdfs dfs -ls /ranger but, for Linux, this is contained somewhere on various datanodes as /mnt/dataxx/dn/current/blah/something.

I don't understand how showmount would work for HDFS.

For a command of : mount -t nfs -o vers=3,proto=tcp,nolock,noacl,sync nfsserver:/ /mnt/ranger-data

On the NFS server the remote mount generates:-

rpc.mountd: refused mount request from <client> for / (/): unmatched host

and on the client we see:-

mount.nfs: access denied by server while mounting nfsserver:/

What I do not understand about this is that it is using standard NFS commands on the NFS server but nowhere does it indicate how that relates to HDFS access.

That does not seem to be the case. Indeed step 1 of starting the service, in that documentation page, tells you to stop the system-provided NFS server (and its helper services) – because you will be starting Hadoop's own NFS server in its place.

In the end, an NFS server is just something that talks the (fairly simple) NFS protocol on TCP port 2049 (or on UDP port whatever for legacy servers). So it seems that Hadoop has an NFS server that's written in Java, listens on port 2049, and serves data directly out of HDFS that way – without interacting with the host's filesystem at all.

(This is similar to "NFS-Ganesha" which is another userspace NFS server which can serve directly out of Ceph or Gluster.)

All that remains running out of standard NFS components is rpcbind, aka the portmapper, which is not strictly "part of NFS" – it is a generic component for various SunRPC-based services, all of which can register with the portmapper to announce that they're providing service X on TCP/UDP port Y.

It just happens that NFS is practically the sole remaining SunRPC-based service nowadays... so rpcbind gets associated with NFS because there is no longer any reason to have it running besides NFS. But Unix used to have several more network services which relied on the same portmapper, such as NIS/YP or rwalld.

(Although even NFSv4 doesn't use the portmapper anymore, it aggregates all of the sub-protocols into a single protocol over TCP and has settled on a static port for that, so it's really just NFSv3 that needs rpcbind.)

The other standard command, showmount, isn't an NFS server command – it is a small NFS client which interrogates the specified server through the network, which means it can talk to any NFS server, not just the system built-in one. In the docs, it is talking to the "HDFS Gateway" process.