ibis.hdfs_connect

ibis.hdfs_connect(host='localhost', port=50070, protocol='webhdfs', use_https='default', auth_mechanism='NOSASL', verify=True, session=None, **kwds)

Connect to HDFS.

Parameters
  • host (str) – Host name of the HDFS NameNode

  • port (int) – NameNode’s WebHDFS port

  • protocol (str,) – The protocol used to communicate with HDFS. The only valid value is 'webhdfs'.

  • use_https (bool) – Connect to WebHDFS with HTTPS, otherwise plain HTTP. For secure authentication, the default for this is True, otherwise False.

  • auth_mechanism (str) – Set to NOSASL or PLAIN for non-secure clusters. Set to GSSAPI or LDAP for Kerberos-secured clusters.

  • verify (bool) – Set to False to turn off verifying SSL certificates.

  • session (Optional[requests.Session]) – A custom requests.Session object.

Notes

Other keywords are forwarded to HDFS library classes.

Returns

Return type

WebHDFS