Making Hadoop Accessible to your Employees with LDAP

Making Hadoop Accessible to your Employees with LDAP

Last updated on July 9th 2015

 

Hue easily integrates with your corporation’s existing identity management systems and provides authentication mechanisms for SSO providers. By changing a few configuration parameters, your employees can start doing big data analysis in their browser by leveraging an existing security policy.

 

This blog post details the various features and capabilities available in Hue for LDAP:

  1. Authentication
  2. Search bind
  3. Direct bind

Importing users

Importing groups

Synchronizing users and groups

  1. Attributes synchronized
  2. Useradmin interface
  3. Command line interface

LDAP search

Case sensitivity

LDAPS/StartTLS support

Debugging

Notes

Summary

1.    Authentication

The typical authentication scheme for Hue takes of the form of the following image:

image

Passwords are saved into the Hue databases.

 

With the Hue LDAP integration, users can use their LDAP credentials to authenticate and inherit their existing groups transparently. There is no need to save or duplicate any employee password in Hue:

image

There are several other ways to authenticate with Hue: PAM, SPNEGO, OpenID, OAuth, SAML2, etc. This section details how Hue can authenticate against an LDAP directory server.

 

When authenticating via LDAP, Hue validates login credentials against a directory service if configured with this authentication backend:

 

[desktop]
 [[auth]]
 backend=desktop.auth.backend.LdapBackend

The LDAP authentication backend will automatically create users that don’t exist in Hue by default. Hue needs to import users in order to properly perform the authentication. The password is never imported when importing users. The following configuration can be used to disable automatic import:

[desktop]
  [[ldap]]
  create_users_on_login=false

The purpose of disabling the automatic import is to only allow to login a predefined list of manually imported users.

 

The case sensitivity of the authentication process is defined in the “Case sensitivity” section below.

 

Note

If a user is logging in as A before enabling LDAP auth and then after enabling LDAP auth logs in as B,  all workflows, queries etc will be associated with the user A and be unavailable. The old workflows would need to have their owner fields changed to B: this can be done in the Hue shell.

 

There are two different ways to authenticate with a directory service through Hue:

  1. Search bind
  2. Direct bind

1.1.    Search bind

The search bind mechanism for authenticating will perform an ldapsearch against the directory service and bind using the found distinguished name (DN) and password provided. This is, by default, used when authenticating with LDAP. The configurations that affect this mechanism are outlined in “LDAP search”.

1.2.    Direct bind

The direct bind mechanism for authenticating will bind to the ldap server using the username and password provided at login. There are two options that can be used to choose how Hue binds:

  1. nt_domain – Domain component for User Principal Names (UPN) in active directory. This active directory specific idiom allows Hue to authenticate with active directory without having to follow LDAP references to other partitions. This typically maps to the email address of the user or the users ID in conjunction with the domain.
  2. ldap_username_pattern – Provides a template for the DN that will ultimately be sent to the directory service when authenticating.

 

If ‘nt_domain’ is provided, then Hue will use a UPN to bind to the LDAP service:

 

[desktop]
  [[ldap]]
  nt_domain=example.com

Otherwise, the ‘ldap_username_pattern’ configuration is used (the <username> parameter will be replaced with the username provided at login):

 

[desktop]
    [[ldap]]
    ldap_username_pattern="uid=<username>,ou=People,DC=hue-search,DC=ent,DC=cloudera,DC=com"

Typical attributes to search for include:

 

  1. uid
  2. sAMAccountName

 

To enable direct bind authentication, the ‘search_bind_authentication’ configuration must be set to false:

 

[desktop]
    [[ldap]]
    search_bind_authentication=false

2.    Importing users

If an LDAP user needs to be part of a certain group and have a particular set of permissions, then this user can be imported via the Useradmin interface:image

 

As you can see, there are two options available when importing:

  1. Distinguished name
  2. Create home directory

 

If ‘Create home directory’ is checked, when the user is imported their home directory in HDFS will automatically be created, if it doesn’t already exist.

 

If ‘Distinguished name’ is checked, then the username provided must be a full distinguished name (eg: uid=hue,ou=People,dc=gethue,dc=com). Otherwise, the Username provided should be a fragment of a Relative Distinguished Name (rDN) (e.g., the username “hue” maps to the rDN “uid=hue”). Hue will perform an LDAP search using the same methods and configurations as defined in the “LDAP search” section. Essentially, Hue will take the provided username and create a search filter using the ‘user_filter’ and ‘user_name_attr’ configurations. For more information on how Hue performs LDAP searches, see the “LDAP Search” section.

 

The case sensitivity of the search and import processes are defined in the “Case sensitivity” section.

3.    Importing groups

Groups are importable via the Useradmin interface. Then, users can be added to this group, which would provide a set of permissions (e.g. accessing the Impala application). This function works almost the exact same way as user importing, but has a couple of extra features.

image

As the above image portrays, not only can groups be discovered via DN and rDN search, but users that are members of the group and members of the group’s subordinate groups can be imported as well. Posix groups and members are automatically imported if the group found has the object class ”posixGroup”.

4.    Synchronizing users and groups

Users and groups can be synchronized with the directory service via the Useradmin interface or via a command line utility. The images from the previous sections use the words “Sync” to indicate that when a name of a user or group that exists in Hue is being added, it will in fact be synchronized instead. In the case of importing users for a particular group, new users will be imported and existing users will be synchronized. Note: Users that have been deleted from the directory service will not be deleted from Hue. Those users can be manually deactivated from Hue via the Useradmin interface.

The groups of a user can be synced when he logs in (to keep its permission in sync):

[desktop]
  [[ldap]]
  # Synchronize a users groups when they login
  ## sync_groups_on_login=false

4.1.    Attributes synchronized

Currently, only the first name, last name, and email address are synchronized. Hue looks for the LDAP attributes ‘givenName’, ‘sn’, and ‘mail’ when synchronizing.  Also, the ‘user_name_attr’ config is used to appropriately choose the username in Hue. For instance, if ‘user_name_attr’ is set to “uid”, then the “uid” returned by the directory service will be used as the username of the user in Hue.

4.2.    Useradmin interface

The “Sync LDAP users/groups” button in the Useradmin interface will  automatically synchronize all users and groups.image

4.3.    Command line interface

Here’s a quick example of how to use the command line interface to synchronize users and groups:

<hue root>/build/env/bin/hue sync_ldap_users_and_groups

5.    LDAP search

There are two configurations for restricting the search process:

  1. user_filter – General LDAP filter to restrict the search.
  2. user_name_attr – Which attribute will be considered the username to search against.

 

Here is an example configuration:

[desktop]
    [[ldap]]
    [[[users]]]
    user_filter=”objectClass=*”
    user_name_attr=uid

    # Whether or not to follow referrals
    ## follow_referrals=false

With the above configuration, the LDAP search filter will take on the form:

(&(objectClass=*)(uid=<user entered usename>))

6.    Case sensitivity

Hue can be configured to ignore the case of usernames as well as force usernames to lower case via the ‘ignore_username_case’ and ‘force_username_lowercase’ configurations. These two configurations are recommended to be used in conjunction with each other. This is useful when integrating with a directory service containing usernames in capital letters and unix usernames in lowercase letters (which is a Hadoop requirement). Here is an example of configuring them:

[desktop]

[desktop]
    [[ldap]]
    ignore_username_case=true
    force_username_lowercase=true

7.    LDAPS/StartTLS support

Secure communication with LDAP is provided via the SSL/TLS and StartTLS protocols. It allows Hue to validate the directory service it’s going to converse with. Practically speaking, if a Certificate Authority Certificate file is provided, Hue will communicate via LDAPS:

[desktop]
    [[ldap]]
    ldap_cert=/etc/hue/ca.crt

The StartTLS protocol can be used as well (step up to SSL/TLS):

[desktop]
    [[ldap]]
    use_start_tls=true

 

8.    Debugging

Get more information when querying LDAP and use the ldapsearch tool:

[desktop]
    [[ldap]]
    debug=true

    # Sets the debug level within the underlying LDAP C lib.
    ## debug_level=255

    # Possible values for trace_level are 0 for no logging, 1 for only logging the method calls with arguments,
    # 2 for logging the method calls with arguments and the complete results and 9 for also logging the traceback of method calls.
    trace_level=0

Note

Make sure to add to the Hue server environment:

DESKTOP_DEBUG=true
DEBUG=true

 

9.    Notes

  1. Setting “search_bind_authentication=true” in the hue.ini will tell Hue to perform an LDAP search using the bind credentials specified in the hue.ini (bind_dn, bind_password). Hue will then search using the base DN specified in “base_dn” for an entry with the attribute, defined in “user_name_attr”, with the value of the short name provided in the login page. The search filter, defined in “user_filter” will also be used to limit the search. Hue will search the entire subtree starting from the base DN.
  2. Setting  ”search_bind_authentication=false” in the hue.ini will tell Hue to perform a direct bind to LDAP using the credentials provided (not bind_dn and bind_password specified in the hue.ini). There are two effective modes here:
    1. nt_domain is specified in the hue.ini: This is used to connect to an Active Directory directory service. In this case, the UPN (User Principal Name) is used to perform a direct bind. Hue forms the UPN by concatenating the short name provided at login and the nt_domain like so: “<short name>@<nt_domain>”. The ‘ldap_username_pattern’ config is completely ignore.
    2. nt_domain is NOT specified in the hue.ini: This is used to connect to all other directory services (can even handle Active Directory, but nt_domain is the preferred way for AD). In this case, ‘ldap_username_pattern’ is used and it should take on the form “cn=<username>,dc=example,dc=com” where <username> will be replaced with whatever is provided at the login page.
  3. The UserAdmin app will always perform an LDAP search when manage LDAP entries and will then always use the “bind_dn”, “bind_password”, “base_dn”, etc. as defined in the hue.ini.
  4. At this point in time, there is no other bind semantics supported other than SIMPLE_AUTH. For instance, we do not yet support MD5-DIGEST, NEGOTIATE, etc. Though, we definitely want to hear from folks what they use so we can prioritize these things accordingly!

10.    Summary

The Hue team is working hard on improving security. Upcoming LDAP features include: Import nested LDAP groups and multidomain support for Active Directory. We hope this brief overview of LDAP in Hue will help you make your system more secure, more compliant with current security standards, and open up big data analysis to many more users!

As always, feel free to contact us at [email protected] or @gethue!

29 Comments

  1. Kulssaka 5 years ago

    Nice, I needed this to understand all the options I found in Cloudera Manager to make HUE to use LDAP 🙂

  2. Alex 4 years ago

    Really need more logging please. When setting this up I get it failing – but no clue as to why it is failing.

    • Hue Team 4 years ago

      It is there in 3.8 now, cf the new ‘Debugging’ section 😉

  3. Gary Hassan 4 years ago

    Hi,

    I am configuring LDAP with Hue. Please provide me step-by-step approach to integrate Hue with LDAP.

    Thanks,

    -Gary

  4. wangmeng 4 years ago

    Hi , for Hue3.6 LDAP login in , the Hue user is personal empolyee of company such as [email protected] .
    However , the Hadoop services behind Hue such as Hive/MapReduce/HDFS are restricted by HadoopServiceUser (such as BI_Hadoop) rather than LDAP user. The LDAP user and HadoopServiceUser are competely different in my company. So the personal LDAP user “wangmeng” does not have the authority to access Hadoop services .
    Is there any way to mapping LDAP user to corresponding HadoopServiceUser. Thanks.

  5. Darshan 3 years ago

    hi
    Im trying to sync ldap users, groups, adding ldap users to existing groups and sync ldap groups via CLI. is there a possibility?

  6. Dale 3 years ago

    Having trouble synchronising LDAP groups. Users are working but not groups.

    I want to sync the group “Hadoop Users” in the “Hadoop Security Groups” OU therefore hue.ini should be:
    group_filter=”OU=Hadoop Security Groups,OU=AD,OU=Example”

    Then in the “Add/Sync LDAP group”:
    Name: Hadoop Users

    However I get the “Could not get LDAP details for groups in pattern Hadoop Users” error. Is there a better way to do this?

    Thanks.

    • Dale 3 years ago

      I have looked at the /dump_config page and noticed that the groups configuration is not getting passed through.

      https://www.dropbox.com/s/khtla5vlzm6ge4b/hue_groups.png?dl=0

      I am using Hue 3.10.

      Thanks.

      • Hue Team 3 years ago

        Either your are not updating the correct hue.ini file (path on top of /dump_config path), or you did not un-comment the good section?

        • Dale 3 years ago

          Thanks for the reply.
          The path at the top of the /dump_config page says: desktop/conf/hue.ini
          I am definitely editing the hue.ini file in desktop/conf/.

          What do you mean by the “good” section? All the LDAP section is un-commented other than the ldap_servers section.

          I am also seeing the exact same thing for the desktop_secret_key field; I’ve entered a value in hue.ini and there is an error on the About Hue page for the lack of a desktop security key.

          If I was using the wrong hue.ini filen the surely none of the other configuration would be passed through? The Hadoop configuration part has been passed through and is working.

  7. Goutam Torvi 2 years ago

    Is there a way to disable login? I just want the user to be able to access without login.

  8. Ostaer 2 years ago

    all the users have joined in default group now.Whether there is a setting of groups mapping,for example,a group named “RD” in LDAP map to group “dev” in hue.

  9. Chandra C 1 year ago

    Hi Team,

    I was able to set up the LDAP integration with Hue. For some reason, when some users logged out, it is showing ‘my email id’ instead of ‘username’ in login placeholder and showing pass word with xxxxxx. Any idea? Anyone had similar issue?

    I am using: CDH 5.9 and Hue 3.11 version

  10. Abraham 6 months ago

    Hello Folks,

    I am currently intergrating HUE with LDAP for our enterprise EMR in AWS. Currently it makes the first user that login to hue after emr is launched the Superuser.I want to set the super user to members of a Active Directory group.Could you guys guide me how to approach the issue.

    Best Regards,

    Timothy Abraham

Leave a reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.