ibis.hdfs_connect

ibis.hdfs_connect(host='localhost', port=50070, protocol='webhdfs', use_https='default', auth_mechanism='NOSASL', verify=True, **kwds)

Connect to HDFS

Parameters:

host : string, Host name of the HDFS NameNode

port : int, NameNode’s WebHDFS port (default 50070)

protocol : {‘webhdfs’}

use_https : boolean, default ‘default’

Connect to WebHDFS with HTTPS, otherwise plain HTTP. For secure authentication, the default for this is True, otherwise False

auth_mechanism : string, Set to NOSASL or PLAIN for non-secure clusters.

Set to GSSAPI or LDAP for Kerberos-secured clusters.

verify : boolean, Set to False to turn off verifying SSL certificates.

(default True)

Other keywords are forwarded to hdfs library classes

Returns:

client : WebHDFS