ibis.hdfs_connect

ibis.hdfs_connect(host='localhost', port=50070, protocol='webhdfs', use_https='default', auth_mechanism='NOSASL', verify=True, session=None, **kwds)

Connect to HDFS.

Parameters
hoststr

Host name of the HDFS NameNode

portint

NameNode’s WebHDFS port

protocolstr,

The protocol used to communicate with HDFS. The only valid value is 'webhdfs'.

use_httpsbool

Connect to WebHDFS with HTTPS, otherwise plain HTTP. For secure authentication, the default for this is True, otherwise False.

auth_mechanismstr

Set to NOSASL or PLAIN for non-secure clusters. Set to GSSAPI or LDAP for Kerberos-secured clusters.

verifybool

Set to False to turn off verifying SSL certificates.

sessionOptional[requests.Session]

A custom requests.Session object.

Returns
WebHDFS

Notes

Other keywords are forwarded to HDFS library classes.