How to count the documents of a user?
Sometimes, it is convenient to administrate Hue directly via the command line. While investigating while http://demo.gethue.com was slow, we discovered that the demo user had more than 85 000 documents! This was a quick way to validate this and delete the extra ones.
On the command line:
.<span class="hljs-meta-keyword">/build/</span>env<span class="hljs-meta-keyword">/bin/</span>hue shell
If using Cloudera Manager, as a root user launch the shell.
Export the configuration directory:
<span class="hljs-built_in">export</span> HUE_CONF_DIR=<span class="hljs-string">"/var/run/cloudera-scm-agent/process/`ls -alrt /var/run/cloudera-scm-agent/process | grep HUE_SERVER | tail -1 | awk '{print <span class="hljs-variable">$9</span>}'`"</span>
<span class="hljs-built_in">echo</span> <span class="hljs-variable">$HUE_CONF_DIR</span>
> /var/run/cloudera-scm-agent/process/2061-hue-HUE_SERVER
Get the process id:
<span class="hljs-attribute">lsof</span> -i :<span class="hljs-number">8888</span>|grep -m1 hue|awk <span class="hljs-string">'{ print <span class="hljs-variable">$2</span> }'</span>
> <span class="hljs-number">14850</span>
In order to export all Hue’s env variables:
<span class="hljs-keyword">for</span> line <span class="hljs-keyword">in</span> `strings /proc/$(lsof -i :8888|grep -m1 hue|awk <span class="hljs-string">'{ print $2 }'</span>)/environ|egrep -v <span class="hljs-string">"^HOME=|^TERM=|^PWD="</span>`;<span class="hljs-keyword">do</span> <span class="hljs-built_in">export</span> <span class="hljs-variable">$line</span>;<span class="hljs-keyword">done</span>
And finally launch the shell by:
HUE_IGNORE_PASSWORD_SCRIPT_ERRORS=<span class="hljs-number">1</span> <span class="hljs-regexp">/opt/</span>cloudera<span class="hljs-regexp">/parcels/</span>CDH<span class="hljs-regexp">/lib/</span>hue<span class="hljs-regexp">/build/</span>env<span class="hljs-regexp">/bin/</span>hue shell
> <span class="hljs-string">ALERT:</span> This appears to be a CM Managed environment
> <span class="hljs-string">ALERT:</span> HUE_CONF_DIR must be set when running hue commands <span class="hljs-keyword">in</span> CM Managed environment
> <span class="hljs-string">ALERT:</span> Please run <span class="hljs-string">'hue <command> --cm-managed'</span>
Then use the Python code to access a certain user information:
Python <span class="hljs-number">2.7</span>.<span class="hljs-number">6</span> (default, Oct <span class="hljs-number">26</span> <span class="hljs-number">2016</span>, <span class="hljs-number">20</span>:<span class="hljs-number">30</span>:<span class="hljs-number">19</span>)
Type <span class="hljs-string">"copyright"</span>, <span class="hljs-string">"credits"</span> or <span class="hljs-string">"license"</span> <span class="hljs-keyword">for</span> more information.
IPython <span class="hljs-number">5.2</span>.<span class="hljs-number">0</span> -- An enhanced Interactive Python.
? -> Introduction and overview of IPython<span class="hljs-string">'s features.
%quickref -> Quick reference.
help -> Python'</span>s own help system.
object? -> Details about <span class="hljs-string">'object'</span>, use <span class="hljs-string">'object??'</span> <span class="hljs-keyword">for</span> extra details.
from django.contrib.auth.models import User
from desktop.models import Document2
user = User.objects.get(username=<span class="hljs-string">'demo'</span>)
Document2.objects.documents(user=user).count()
<span class="hljs-keyword">In</span> [<span class="hljs-number">8</span>]: Document2.objects.documents(user=user).count()
Out[<span class="hljs-number">8</span>]: <span class="hljs-number">1167</span>
<span class="hljs-keyword">In</span> [<span class="hljs-number">10</span>]: Document2.objects.documents(user=user, perms=<span class="hljs-string">'own'</span>).count()
Out[<span class="hljs-number">10</span>]: <span class="hljs-number">1166</span>
<span class="hljs-keyword">In</span> [<span class="hljs-number">11</span>]: Document2.objects.documents(user=user, perms=<span class="hljs-string">'own'</span>, include_history=True).count()
Out[<span class="hljs-number">11</span>]: <span class="hljs-number">7125</span>
<span class="hljs-keyword">In</span> [<span class="hljs-number">12</span>]: Document2.objects.documents(user=user, perms=<span class="hljs-string">'own'</span>, include_history=True, include_trashed=True).count()
Out[<span class="hljs-number">12</span>]: <span class="hljs-number">7638</span>
<span class="hljs-keyword">In</span> [<span class="hljs-number">13</span>]: Document2.objects.documents(user=user, perms=<span class="hljs-string">'own'</span>, include_history=True, include_trashed=True, include_managed=True).count()
Out[<span class="hljs-number">13</span>]: <span class="hljs-number">31408</span>
Out[<span class="hljs-number">14</span>]:
(<span class="hljs-number">85667</span>L,
{u<span class="hljs-string">'desktop.Document'</span>: <span class="hljs-number">18524</span>L,
u<span class="hljs-string">'desktop.Document2'</span>: <span class="hljs-number">31409</span>L,
u<span class="hljs-string">'desktop.Document2Permission'</span>: <span class="hljs-number">556</span>L,
u<span class="hljs-string">'desktop.Document2Permission_groups'</span>: <span class="hljs-number">277</span>L,
u<span class="hljs-string">'desktop.Document2Permission_users'</span>: <span class="hljs-number">0</span>L,
u<span class="hljs-string">'desktop.Document2_dependencies'</span>: <span class="hljs-number">15087</span>L,
u<span class="hljs-string">'desktop.DocumentPermission'</span>: <span class="hljs-number">1290</span>L,
u<span class="hljs-string">'desktop.DocumentPermission_groups'</span>: <span class="hljs-number">0</span>L,
u<span class="hljs-string">'desktop.DocumentPermission_users'</span>: <span class="hljs-number">0</span>L,
u<span class="hljs-string">'desktop.Document_tags'</span>: <span class="hljs-number">18524</span>L})
Et voila, it is much cleaner now!
