How to Integrate Unix users and groups

How to Integrate Unix users and groups

Hue allows you to authenticate with several services like your company LDAP, OAuth, OpenId, SAML etc. This blog post covers how to integrate with linux account names in Hue by synchronizing with the underlying system.

Tutorial

The following will guide you in importing Linux accounts and groups into Hue:

  1. Ensure the ‘hadoop’ group is on the system. See the ‘Hadoop group’ section below to learn to how to verify this in different environments.
  2. From the command line, execute the command build/env/bin/hue useradmin_sync_with_unix. This will import users and groups from the machine Hue is on.
  3. Important: as a Hue administrator, give a password to each imported member. Users will not be able to login until a password has been provided to them. If you want to re-use Linux user password, you should look at the PAM backend instead (caveat: it can only authenticate the user who is running the Hue server (this being normal PAM behaviour in Linux) unless we run Hue server as root, which is not recommended. LDAP is the alternative recommended solution).

Here is a quick video demonstrating the above:

 

From the Hue root (/use/lib/hue by default or /opt/cloudera/parcels/CDH/lib/hue/ with CM):

build/env/bin/hue useradmin_sync_with_unix

 

If using CM, export this variable in order to point to the correct database:

Where <id> is the most recent ID in that process directory for hue-HUE_SERVER.

A quick way to get the correct directory is to use this script:

export HUE_CONF_DIR="/var/run/cloudera-scm-agent/process/`ls -alrt /var/run/cloudera-scm-agent/process | grep HUE | tail -1 | awk '{print $9}'`"

 

Command line interface

useradmin_sync_with_unix comes with a few useful command line arguments:

  • –min-uid – The minimum linux user ID that will be imported (inclusive). The default value is 500.
  • –max-uid – The maximum linux user ID that will be imported (exclusive). The default value is 65334.
  • –min-gid – The minimum linux group ID that will be imported (inclusive). The default value is 500.
  • –max-gid – The maximum linux group ID that will be imported (exclusive). The default value is 65334.
  • –check-shell – A boolean flag  to see if the users shell is set to /bin/false.

Hadoop group

To verify the hadoop group exists, you can use the ‘getent’ command:

getent group | grep hadoop

To add the hadoop group, you can use the ‘groupadd’ command:

groupadd hadoop

Conclusion

We hope this utility opens up your Hadoop cluster to your users and simplifies administration.

Have any suggestions? Feel free to tell us what you think through hue-user or @gethue!

15 Comments

  1. dale 3 years ago

    “build/env/bin/hue useradmin_sync_with_unix” works perfectly and syncs users correctly.

    But does this command have to be run every time a new user is added to the linux environment? I am seeing that it does not auto-synchronise when new users are added. Cheers.

    • Hue Team 3 years ago

      Yes, this command needs to be run every time you want to sync them.

  2. Anu V Das 2 years ago

    Hi Hue Team,

    I am getting following error when i try to sync. Can you help me please ?

    /opt/cloudera/parcels/CDH/lib/hue/build/env/bin/hue useradmin_sync_with_unix
    Traceback (most recent call last):
    File “/opt/cloudera/parcels/CDH/lib/hue/build/env/bin/hue”, line 9, in
    load_entry_point(‘desktop==3.7.0’, ‘console_scripts’, ‘hue’)()
    File “/opt/cloudera/parcels/CDH/lib/hue/desktop/core/src/desktop/manage_entry.py”, line 57, in entry
    execute_from_command_line(sys.argv)
    File “/opt/cloudera/parcels/CDH/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/__init__.py”, line 399, in execute_from_command_line
    utility.execute()
    File “/opt/cloudera/parcels/CDH/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/__init__.py”, line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
    File “/opt/cloudera/parcels/CDH/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/base.py”, line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
    File “/opt/cloudera/parcels/CDH/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/base.py”, line 285, in execute
    output = self.handle(*args, **options)
    File “/opt/cloudera/parcels/CDH/lib/hue/apps/useradmin/src/useradmin/management/commands/useradmin_sync_with_unix.py”, line 47, in handle
    sync_unix_users_and_groups(min_uid, max_uid, min_gid, max_gid, check_shell)
    File “/opt/cloudera/parcels/CDH/lib/hue/apps/useradmin/src/useradmin/views.py”, line 503, in sync_unix_users_and_groups
    if subprocess.call([pw_shell, “-c”, “echo”], stdout=subprocess.PIPE) != 0:
    File “/usr/lib64/python2.6/subprocess.py”, line 478, in call
    p = Popen(*popenargs, **kwargs)
    File “/usr/lib64/python2.6/subprocess.py”, line 642, in __init__
    errread, errwrite)
    File “/usr/lib64/python2.6/subprocess.py”, line 1238, in _execute_child
    raise child_exception
    OSError: [Errno 2] No such file or directory

    • Hue Team 2 years ago

      This fails because your user does not have a shell set:

      e.g. you should have this working first:
      python
      Python 2.7.6 (default, Jun 22 2015, 17:58:13)
      [GCC 4.8.2] on linux2
      Type "help", "copyright", "credits" or "license" for more information.
      >>> import pwd
      >>> pwd.getpwnam('romain')
      pwd.struct_passwd(pw_name='romain', pw_passwd='x', pw_uid=1000, pw_gid=1000, pw_gecos='Romain,,,', pw_dir='/home/romain', pw_shell='/bin/bash')

      • Anu V Das 2 years ago

        Hi ,

        Now i am working on my HUE server. Operating system is up to date and i am running useradmin_syn_with_unix as root. I am getting the same error.

        # yum update -y

        Loaded plugins: fastestmirror, refresh-packagekit
        Setting up Update Process
        Loading mirror speeds from cached hostfile
        No Packages marked for Update

        # python
        Python 2.6.6 (r266:84292, Jul 23 2015, 15:22:56)
        [GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
        Type “help”, “copyright”, “credits” or “license” for more information.
        >>> import pwd
        >>> pwd.getpwnam(‘root’)
        pwd.struct_passwd(pw_name=’root’, pw_passwd=’x’, pw_uid=0, pw_gid=0, pw_gecos=’root’, pw_dir=’/root’, pw_shell=’/bin/bash’)
        >>> quit()

        # whoami
        root

        • Hue Team 2 years ago

          Does /bin/bash -c echo works?

          • Anu V Das 2 years ago

            No, I am not getting any output for /bin/bash -c echo command.

          • BloqueNegro 1 year ago

            Hi, I have the same issue.

            I get
            >>> import pwd
            >>> pwd.getpwnam(‘root’)
            pwd.struct_passwd(pw_name=’root’, pw_passwd=’x’, pw_uid=0, pw_gid=0, pw_gecos=’root’, pw_dir=’/root’, pw_shell=’/bin/bash’)
            >>> quit()

            and no output for /bin/bash -c echo

            I can’t find anything about that on the net – do you have an idea what’s happening here?

  3. Drz_Ecstasy 2 years ago

    Hi,
    I have successfully installed Hue in HDINSIGHT Cluster and it is working as expected.

    I have one question, can we bypass the initial “the first user who logs in to Hue can choose any username and password and automatically becomes an administrator” page and create admin user from script itself, so the initial popup doesn’t appear?

  4. LKS 2 years ago

    ive had an error executing: build/env/bin/hue useradmin_sync_with_unix

    Error: Password not present
    Traceback (most recent call last):
    File “build/env/bin/hue”, line 12, in
    load_entry_point(‘desktop==3.9.0’, ‘console_scripts’, ‘hue’)()

    i am not shure how this came to be, it is a fresh installation with just an admin user. i restarted the hue since the user was added.
    but here is the Solution:
    export $HUE_SECRET_KEY=myHueAdminPassword

  5. Kiera 1 year ago

    Hi, We have a cloudera cluster with MIT kerberos installed on it.
    When I try to use useradmin_sync_with_unix, it gives an error : Password not present with the following trace:

    Traceback (most recent call last):
    File “build/env/bin/hue”, line 12, in
    load_entry_point(‘desktop==3.9.0’, ‘console_scripts’, ‘hue’)()
    File “/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p1464.1349/lib/hue/desktop/core/src/desktop/manage_entry.py”, line 57, in entry
    execute_from_command_line(sys.argv)
    File “/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p1464.1349/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/_init_.py”, line 399, in execute_from_command_line
    utility.execute()
    File “/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p1464.1349/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/_init_.py”, line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
    File “/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p1464.1349/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/_init_.py”, line 261, in fetch_command
    commands = get_commands()
    File “/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p1464.1349/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/_init_.py”, line 107, in get_commands
    apps = settings.INSTALLED_APPS

  6. Ryan Gault 7 months ago

    Using Cloudera 5.12, the script runs successfully but the users aren’t coming over to Hue.

    [[email protected] hue]# build/env/bin/hue useradmin_sync_with_unix
    [11/Sep/2017 15:56:20 +0000] settings DEBUG DESKTOP_DB_TEST_NAME SET: /opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hue/desktop/desktop-test.db
    [11/Sep/2017 15:56:20 +0000] settings DEBUG DESKTOP_DB_TEST_USER SET: hue_test
    [11/Sep/2017 15:56:21 +0000] __init__ INFO Couldn’t import snappy. Support for snappy compression disabled.
    [11/Sep/2017 15:56:21 +0000] decorators INFO AXES: BEGIN LOG
    [11/Sep/2017 15:56:21 +0000] decorators INFO Using django-axes 1.5.0
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user centos from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user httpfs from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user keytrustee from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user hdfs from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user keyadmin from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user systest from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user kms from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user jenkins from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user mapred from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user llama from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user sentryadm from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user tomcat from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user centos-user from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user yarn from Unix
    [11/Sep/2017 15:56:21 +0000] views INFO Synced user impala from Unix

  7. Ryan Gault 7 months ago

    I figured it out. I had to manually edit the hue.ini and change from the default sqllite DB to the MySQL instance I was using.

Leave a reply

Your email address will not be published. Required fields are marked *

*