Apache Sentry made easy with the new Hue Security App

Apache Sentry made easy with the new Hue Security App

Hi Hadoop Sheriffs,

In order to support the growth of the Apache Sentry project and make it easier to secure your cluster, a new app was added into Hue. Sentry privileges determine which Hive / Impala databases and tables a user can see or modify. The Security App let’s you create/edit/delete Roles and Privileges directly from your browser (there is no sentry-provider.ini file to edit anymore).

Here is a video showing how the app works:

Main features:

  • Bulk edit roles and privileges
  • Visualize/edit roles and privileges on a database tree
  • WITH GRANT OPTION support
  • Impersonate a user to see which databases and table he can see

hue-sentry

To have Hue point to a Sentry service and another host, modify these hue.ini properties:

[libsentry]
  # Hostname or IP of server.
  hostname=localhost

  # Port the sentry service is running on.
  port=8038

  # Sentry configuration directory, where sentry-site.xml is located.
  sentry_conf_dir=/etc/sentry/conf

Hue will also automatically pick up the server name of HiveServer2 from the sentry-site.xml file of /etc/hive/conf.

 

And that’s it, you can know specify who can see/do what directly in a Web UI! The app sits on top of the standard Sentry API and so it fully compatible with Sentry. Next planned features will bring Solr Collections, HBase privilege management as well as more bulk operations and a tighter integration with HDFS.

As usual, feel free to continue to send us questions and feedback on the hue-user list or @gethue!

Notes

To be able to edit roles and privileges in Hue, the logged-in Hue user needs to belong to a group in Hue that is also an admin group in Sentry (whatever UserGroupMapping Sentry is using, the corresponding groups must exist in Hue or need to be entered manually). For example, our ‘hive’ user belongs to a ‘hive’ group in Hue and also to a ‘hive’ group in Sentry:

<property>
  <name>sentry.service.admin.group</name>
  <value>hive,impala,hue</value>
 </property>

 

Notes

  • Create a role in the Sentry app through Hue
  • Grant privileges to that role such that the role can see the database in the Sentry app
  • Create a group in Hue with the same name as the role in Sentry
  • Grant that role to a user in Hue
  • Ensure that the user in Hue has an equivalent O/S level
  • Ensure a user has an O/S level account on all hosts and that user is part of a group with the same name as the group in Hue (this assumes that the default ShellBasedUnixGroupsMapping is set for HDFS in CM)

 

Notes

We are using CDH5.2+ with Kerberos MIT and Sentry configured. The app also works in non secure mode.

Our users are:

  • hive (admin) belongs to the hive group
  • user1_1 belongs to the user_group1 group
  • user2_1 belongs to the user_group2 group

We synced the Unix users/groups into Hue with these commands:

export HUE_CONF_DIR="/var/run/cloudera-scm-agent/process/`ls -alrt /var/run/cloudera-scm-agent/process | grep HUE | tail -1 | awk '{print $9}'`"

build/env/bin/hue useradmin_sync_with_unix --min-uid=1000

If using the package version and has the CDH repository register, install sentry with:

sudo apt-get install sentry

If using Kerberos, make sure ‘hue’ is allowed to connect to Sentry in /etc/sentry/conf/sentry-site.xml:

<property>
    <name>sentry.service.allow.connect</name>
    <value>impala,hive,solr,hue</value>
</property>

Here is an example of sentry-site.xml

Here is an example of sentry-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
 <property>
   <name>sentry.service.security.mode</name>
   <value>none</value>
 </property>
 <property>
   <name>sentry.service.admin.group</name>
   <value>hive,romain</value>
 </property>
 <property>
   <name>sentry.service.allow.connect</name>
   <value>impala,hive,solr</value>
 </property>
 <property>
   <name>sentry.store.jdbc.url</name>
   <value>jdbc:derby:;databaseName=sentry_store_db;create=true</value>
 </property>
 <property>
   <name>sentry.store.jdbc.driver</name>
   <value>org.apache.derby.jdbc.EmbeddedDriver</value>
 </property>
 <property>
   <name>sentry.store.jdbc.password</name>
   <value>aaa</value>
 </property>
</configuration>

For testing purposes, here is how to create the initial Sentry database:

[email protected]:~/projects/hue$ sentry --command schema-tool -initSchema -conffile /etc/sentry/conf/sentry-site.xml -dbType derby

And start the service:

sentry --command service  -conffile /etc/sentry/conf/sentry-site.xml

Note
In Sentry 1.5, you will need to specify a ‘entry.store.jdbc.password’ property in the sentry-site.xml, if not you will get:

Caused by: org.apache.sentry.provider.db.service.thrift.SentryConfigurationException: Error reading sentry.store.jdbc.password

46 Comments

  1. vishakha sadawarte 2 years ago

    Here, In this post, Is it assumed that Apache sentry is already installed? I am unable to find sentry-site.xml or sentry-store-site.xml in my machine after hue installation.

    • Hue Team 2 years ago

      Yes, of course 🙂

  2. omar 2 years ago

    Hi, i have a problem , when i use “desktop.auth.backend.AllowFirstUserDjangoBackend” this backend.
    i have been used kdc Kerberos with out ldap.
    i logging in with a user named ketty, this account was created by hue admin, and create group named ketty with hue admin.
    but i use hive user to grant privileges to ketty . ketty can not see any database or tables.
    i do like this
    create role ketty;
    grant all on database test to role ketty;
    grant role ketty to group ketty;
    how can i fixed it ?

    • Hue Team 2 years ago

      You need to create ‘ketty’ at the Unix level, he needs to exist for real in the system, not just in Hue.

  3. kimchi 2 years ago

    Hi,
    I’m trying to run the demo in the quickstart.cloudera vm (CDH 5.4).
    As I define Privileges for Tables, I first select a table and I see the message “No privileges found for the selected item”.
    I then click on ‘Roles’ in the left menu and I can see ‘There are currently no roles defined’ and I can see no way of defining one in the GUI.

    Can you help ?

    Best.
    Kimchi

    • Hue Team 2 years ago

      Please see the red note above, you probably need to configure it

    • White 2 years ago

      Hi,
      I have the same problem.Do you have any solutions?
      Thx

      • obaid 12 months ago

        Hi,

        I had same issue. Solved following red Notes.
        The trick was “whatever UserGroupMapping Sentry is using, the corresponding groups must exist in Hue or need to be entered manually”. You need to have the group on OS/LDAP and in Hue as well.

        Thanks

  4. Yunpeng 2 years ago

    Hello,

    I noticed that we should sync the accounts in Hue with those in OS level.
    Does it mean that if we want to use LDAP to access Hue, we should sync all the accounts in LDAP with OS (of all the nodes in cluster)?

    Thanks

    • Hue Team 2 years ago

      When using Kerberos you need to login in Hue with a user that exists in the whole cluster. If you are using LDAP, you need to make sure that a LDAP user exists, I think you can configure your OS to sync with LDAP directly.

      • Mohamed 1 year ago

        I am facing the same problem. Could you plz help me to sync my cluster with LDAP.
        Thanks

  5. Jyotsna 2 years ago

    Hi,

    I am unable to view the privileges in sentry through HUE.
    Could you please let me know how to configure the privileges in Sentry so that it will be available in HUE.

    I could not see “Click to add more privileges+ option ”

    Could you please help me on this.

    • Hue Team 2 years ago

      Hi,
      your user/group should be a Sentry admin group, like specified in the post (search for: “To be able to edit roles and privileges in Hue”)

  6. Anu V Das 1 year ago

    Hi Hue Team,

    Even i was also facing the same problem. But after creating a “hive” group in Hue I could see the “Click to add more privileges+ option” . However now my problem is if i click on “Roles” on the left i see the roles assigned to user1 and user2, but if i click on “Browse” I still see “No privileges found for the selected item. Click here to add some”. More over if i login as user1 or user2 it is not showing any Databases or Tables even though i added roles. So i think privileges are not taking any effect. !
    Any help is appreciated.

    • Hue Team 1 year ago

      Are user1 or user2 added to the correct groups?
      ‘hive’ is the Sentry super user, you should see all the privileges / roles.

  7. Zack 1 year ago

    Hi, Thanks for the tutorial, how did you manage to login as hive? it is a system user and is created during install so we dont have a password? Secondly if I am using Ldap as a backend for users other than system user and at the same time use local groups for system users and services ( while still UnixShellBaseGroup mapping class) how do I map hive user and group to Hue because after ldap configuration it is picking up all users from ldap server?

  8. Dippi 11 months ago

    Hello.
    I am using Cloudera Director to provision cloudera manager and CDH on the instances in my google cloud cluster.
    My question is regarding one of the notes in the article above. The note I am referring to is the following:
    “To be able to edit roles and privileges in Hue, the logged-in Hue user needs to belong to a group in Hue that is also an admin group in Sentry”

    Question: When I go to Hue UI first time, it asks me to create credentials that will be used for superuser, I used admin/admin for it. So far so good.

    Now, when I go to Manage Users option in the Hue UI (top right in the menu), it does not show any other users such as hive etc. Also, the ‘admin’ user shows up as belonging to the group – default. However, as I understand, in order to configure Sentry roles using Hue, I should be logged in as a user which belongs to a group that is also an admin group in sentry-site.xml file. The ‘default’ group is not in the list of admin groups in sentry.

    Given this scenario, how should I go about configuring sentry roles/databases through Hue. Thanks.

    • Simran 10 months ago

      Hi, Same question. Did you find a solution?

  9. Simran 10 months ago

    I get No databases found. Permissions could be missing. Why is it? Since I am logging in using default user hue that belongs to default group, I added that to the list of users for admin access to sentry . It allowed me to defined roles for groups but what next? In hue editor, I can’t see any tables and can’t run any queries

    • Author
      Hue Team 10 months ago

      Is the default group a real Unix group? Probably not. Could you use the ‘hive’ group for example or any other exiting Unix group?

      • Simran 10 months ago

        hive group is not an existing user group on my system. I added and created it in hue though. So, I now have a group hive and a user with name hive in hue but not on my centos machine. I checked that through command :

        getent group | grep hadoop

        and I get:

        hadoop:x:510:zookeeper,hdfs,yarn,mapred

        I tried running:

        sudo build/env/bin/hue useradmin_sync_with_unix

        from directory

        /opt/cloudera/parcels/CDH/lib/hue/

        . The command gives no errors, just moves to the next line in command line as if it was executed but I don’t see the users synced in hue.

        echo $HUE_CONF_DIR gives:

        /var/run/cloudera-scm-agent/process/365-hue-HUE_SERVER

        but can’t execute command

        sudo build/env/bin/hue useradmin_sync_with_unix

        says no such file or directory and users still not synced.

  10. Simran 10 months ago

    Also, I tried:

    groupadd hive
    It says:

    groupadd: group ‘hive’ already exists

    but doesn’t show hive with this command :

    getent group | grep hadoop

  11. Simran 10 months ago

    One more thing, if I try to execute commands through beeline, I get:

    0: jdbc:hive2://SERVER_NAME.com:10000/d> show databases;
    Error: Error while compiling statement: FAILED: InvalidConfigurationException hive.server2.authentication can’t be none in non-testing mode (state=42000,code=40000)

    I don’t understand how to establish authentication mechanism here. Why do we need some other authentication mechanism when we have sentry. Also, I have tried to set it to KERBEROS, I don’t see hive.server2.authentication property set anywhere in my hive-site.xml . I am using cloudera

  12. Naidu 9 months ago

    this post helped me to understand Sentry.

    Thanks.

  13. Kiera 6 months ago

    Hello, thanks for the great tutorial but even granting roleA all the permission in a databaseA and adding roleA for groupA and userA is part of group A. When I logged in as userA in hue, it will still tell me I am missing insert and select permission. When I used the beeline cli, as userA I am able to see roleA in my but it is not being in effect. What can I do?

    This is the grant permission: server=server1->db=databaseA->action=ALL

    • Author
      Hue Team 6 months ago

      Is userA and its group existing at the Unix level?

      • Kiera 5 months ago

        Yes, I created them on both side so I’m not sure what is wrong. UserB has a default admin_role for hive group, userB is able to do everything such as selecting the tables and granting roles. However, when I grant roleA to userA nothing is happening.

  14. Shobhit 2 months ago

    I am not able to see sentry Tables in Hue Do I need to enable sentry for Hive as well ?
    Currently I enabled it for Hue Only

    • Author
      Hue Team 2 months ago

      Yes, of course. It’s fairly easy if you use Cloudera Manager

  15. xiaomin 4 weeks ago

    How to certified the port?

    • Author
      Hue Team 4 weeks ago

      You mean how do you make sure Sentry is running on port 8038? Or is it security related?

  16. xiaomin 4 weeks ago

    I mean in the file ‘hue.ini’, about libsentry configure,i set the port is default ,and restart hue ,but on the hue interface is not show the sentry option,so i think the port probably is wrong.

  17. xiaomin 4 weeks ago

    Create a role in the Sentry app through Hue,on this step,in the sentry app ,i can not see the add role butoon,what should i do?

    • Author
      Hue Team 4 weeks ago

      Make sure you have the right permissions for using the Sentry app with your user

  18. xiaomin 3 weeks ago

    I have solved the problem,thank you!

  19. xiaomin 2 weeks ago

    I use jdbc to connect impala with username and password ,without ldap or kerberos,but i can not connect success,what should i do ?

    • Author
      Hue Team 2 weeks ago

      Which error are you seeing? Hue does not use JDBC and authenticates directly in Plain, LDAP or Kerberos mode.

Leave a reply

Your email address will not be published. Required fields are marked *

*