Installing HUE 4

Hue是一個開源的Apache Hadoop UI系統(tǒng)，最早是由Cloudera Desktop演化而來，由Cloudera貢獻(xiàn)給開源社區(qū)，它是基于Python Web框架Django實(shí)現(xiàn)的。通過使用Hue我們可以在瀏覽器端的Web控制臺上與Hadoop集群進(jìn)行交互來分析處理數(shù)據(jù)，例如操作HDFS上的數(shù)據(jù)，運(yùn)行MapReduce Job等等。HUE的特性如下圖所示：

HUE的架構(gòu)圖如下：

1、安裝依賴軟件包
本文搭建HUE的平臺是Oracle Linux 7.4。

[root@hdp01 ~]# yum -y install gcc-c++ asciidoc cyrus-sasl-devel cyrus-sasl-gssapi krb5-devel libxml2-devel libxslt-devel mysql-devel openldap-devel python-devel sqlite-devel openssl-devel gmp-devel libffi libffi-devel MySQL-python mysql-community-devel cyrus-sasl-plain

如果這些軟件包不安裝，在編譯的過程中會報錯，尤其是mysql相關(guān)的軟件包，還有cryus-sasl。
2、安裝HUE
HUE的下載地址可以從http://gethue.com/官網(wǎng)找 。HUE的安裝很簡單，如下：

[hadoop@hdp01 u02]$ tar -xzf hue-4.1.0.tgz 
[hadoop@hdp01 u02]$ cd hue-4.1.0
[hadoop@hdp01 hue-4.1.0]$ make apps

3、創(chuàng)建MySQL數(shù)據(jù)庫和用戶
默認(rèn)情況下，HUE使用的是自帶的sqlite3，測試環(huán)境可以用用，但是正式環(huán)境就需要使用諸如oracle、MySQL、PostgreSQL等數(shù)據(jù)庫了。

mysql> create database hue;
mysql> create user hue identified by 'abcABC@12';
mysql> grant all privileges on *.* to hue@'%' identified by 'abcABC@12';
mysql> flush privileges;

4、編輯HUE配置文件
HUE的配置文件位于$HUE_HOME/desktop/conf目錄，文件名為hue.ini。這里把我配置的內(nèi)容貼出來，如下：

[desktop]
    secret_key=
    http_host=0.0.0.0
    http_port=8888
    is_hue_4=true
    time_zone=Asia/Shanghai
    django_debug_mode=false
    http_500_debug_mode=false
    server_user=hadoop
    server_group=hadoop
    default_user=hadoop
    default_hdfs_superuser=hadoop
    enable_server=yes
  [[database]]
    engine=mysql
    host=192.168.120.92     --數(shù)據(jù)庫主機(jī)
    port=3306               --MySQL端口
    user=hue                --數(shù)據(jù)庫用戶
    password=abcABC@12      --數(shù)據(jù)庫密碼
    name=hue                --數(shù)據(jù)庫名稱
[hadoop]
  [[hdfs_clusters]]
    [[[default]]]
    fs_defaultfs=hdfs://hdp01:9000
    webhdfs_url=http://hdp01:50070/webhdfs/v1
    security_enabled=false
    [[[default]]]
    resourcemanager_host=hdp01
    resourcemanager_port=8032
    submit_to=True
    security_enabled=false
    resourcemanager_api_url=http://hdp01:8088
    proxy_api_url=http://hdp01:8088
    history_server_api_url=http://hdp01:19888
  [[mapred_clusters]]
    [[[default]]]
    submit_to=False
[beeswax]
    hive_server_host=hdp01.thinkjoy.tt
    hive_server_port=10000
    hive_conf_dir=/u01/hive/conf
[metastore]
    enable_new_create_table=true
[filebrowser]
    archive_upload_tempdir=/tmp
    show_download_button=false
    show_upload_button=false
[sqoop]
    server_url=http://192.168.120.96:12000/sqoop
    sqoop_conf_dir=/u01/sqoop/conf
[hbase]
--這個cluster名稱隨便叫，如果hbase沒有開啟thriftserver，通過hue訪問hbase會報無法訪問xxx的9090端口錯誤。
  hbase_clusters=(Cluster|hdp02:9090),(Cluster|hdp03:9090),(Cluster|hdp04:9090)
    hbase_conf_dir=/u01/hbase/conf
    truncate_limit = 500
    thrift_transport=buffered
[search]
    solr_url=http://hdp01:8983/solr/
[zookeeper]
  [[clusters]]
    [[[default]]]
    host_ports=hdp01:2181,hdp02:2181,hdp03:2181,hdp04:2181
[liboozie]
    oozie_url=http://192.168.120.101:11000/oozie
[libzookeeper]
    ensemble=hdp01:2181,hdp02:2181,hdp03:2181,hdp04:2181
###########################################################################
# Settings for the RDBMS application
###########################################################################
[librdbms]
    [[databases]]
      [[[mysql]]]
        nice_name="MySQL DB"
        name=hue
        engine=mysql
        host=192.168.120.92
        port=3306
        user=hue
        password=abcABC@12

Hbase開啟thrift服務(wù)，如下：

[hadoop@hdp01 ~]$ hbase-daemons.sh start thrift

5、初始化MySQL數(shù)據(jù)庫
如果使用了RDBMS數(shù)據(jù)庫，則在開啟服務(wù)之前，必須初始化數(shù)據(jù)庫，否則在訪問HUE的管理界面會報錯。另外初始化的過程中已創(chuàng)建了HUE的管理員賬戶：hadoop。

hadoop@hdp01 ~]$ /u01/hue/build/env/bin/hue syncdb
Syncing...
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_openid_auth_nonce
Creating table django_openid_auth_association
Creating table django_openid_auth_useropenid
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table django_admin_log
Creating table south_migrationhistory
Creating table axes_accessattempt
Creating table axes_accesslog

You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use 'hadoop'): 
Email address: 
Password: 
Password (again): 
Superuser created successfully.
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

Synced:
 > django.contrib.auth
 > django_openid_auth
 > django.contrib.contenttypes
 > django.contrib.sessions
 > django.contrib.sites
 > django.contrib.staticfiles
 > django.contrib.admin
 > south
 > axes
 > about
 > filebrowser
 > help
 > impala
 > jobbrowser
 > metastore
 > proxy
 > rdbms
 > zookeeper
 > indexer
 > dashboard

Not synced (use migrations):
 - django_extensions
 - desktop
 - beeswax
 - hbase
 - jobsub
 - oozie
 - pig
 - search
 - security
 - spark
 - sqoop
 - useradmin
 - notebook
(use ./manage.py migrate to migrate these)
[hadoop@hdp01 ~]$ /u01/hue/build/env/bin/hue migrate
Running migrations for django_extensions:
 - Migrating forwards to 0001_empty.
 > django_extensions:0001_empty
 - Loading initial data for django_extensions.
Installed 0 object(s) from 0 fixture(s)
Running migrations for desktop:
 - Migrating forwards to 0026_change_is_trashed_default_to_false.
 > pig:0001_initial
 > oozie:0001_initial
 > oozie:0002_auto__add_hive
 > oozie:0003_auto__add_sqoop
 > oozie:0004_auto__add_ssh
 > oozie:0005_auto__add_shell
 > oozie:0006_auto__chg_field_java_files__chg_field_java_archives__chg_field_sqoop_f
 > oozie:0007_auto__chg_field_sqoop_script_path
 > oozie:0008_auto__add_distcp
 > oozie:0009_auto__add_decision
 > oozie:0010_auto__add_fs
 > oozie:0011_auto__add_email
 > oozie:0012_auto__add_subworkflow__chg_field_email_subject__chg_field_email_body
 > oozie:0013_auto__add_generic
 > oozie:0014_auto__add_decisionend
 > oozie:0015_auto__add_field_dataset_advanced_start_instance__add_field_dataset_ins
 > oozie:0016_auto__add_field_coordinator_job_properties
 > oozie:0017_auto__add_bundledcoordinator__add_bundle
 > oozie:0018_auto__add_field_workflow_managed
 > oozie:0019_auto__add_field_java_capture_output
 > oozie:0020_chg_large_varchars_to_textfields
 > oozie:0021_auto__chg_field_java_args__add_field_job_is_trashed
 > oozie:0022_auto__chg_field_mapreduce_node_ptr__chg_field_start_node_ptr
 > oozie:0022_change_examples_path_format
 - Migration 'oozie:0022_change_examples_path_format' is marked for no-dry-run.
 > oozie:0023_auto__add_field_node_data__add_field_job_data
 > oozie:0024_auto__chg_field_subworkflow_sub_workflow
 > oozie:0025_change_examples_path_format
 - Migration 'oozie:0025_change_examples_path_format' is marked for no-dry-run.
 > desktop:0001_initial
 > desktop:0002_add_groups_and_homedirs
 > desktop:0003_group_permissions
 > desktop:0004_grouprelations
 > desktop:0005_settings
 > desktop:0006_settings_add_tour
 > beeswax:0001_initial
 > beeswax:0002_auto__add_field_queryhistory_notify
 > beeswax:0003_auto__add_field_queryhistory_server_name__add_field_queryhistory_serve
 > beeswax:0004_auto__add_session__add_field_queryhistory_server_type__add_field_query
 > beeswax:0005_auto__add_field_queryhistory_statement_number
 > beeswax:0006_auto__add_field_session_application
 > beeswax:0007_auto__add_field_savedquery_is_trashed
 > beeswax:0008_auto__add_field_queryhistory_query_type
 > beeswax:0009_auto__add_field_savedquery_is_redacted__add_field_queryhistory_is_reda
 > desktop:0007_auto__add_documentpermission__add_documenttag__add_document
 > desktop:0008_documentpermission_m2m_tables
 > desktop:0009_auto__chg_field_document_name
 > desktop:0010_auto__add_document2__chg_field_userpreferences_key__chg_field_userpref
 > desktop:0011_auto__chg_field_document2_uuid
 > desktop:0012_auto__chg_field_documentpermission_perms
 > desktop:0013_auto__add_unique_documenttag_owner_tag
 > desktop:0014_auto__add_unique_document_content_type_object_id
 > desktop:0015_auto__add_unique_documentpermission_doc_perms
 > desktop:0016_auto__add_unique_document2_uuid_version_is_history
 > desktop:0017_auto__add_document2permission__add_unique_document2permission_doc_perm
 > desktop:0018_auto__add_field_document2_parent_directory
 > desktop:0019_auto
 > desktop:0020_auto__del_field_document2permission_all
 > desktop:0021_auto__add_defaultconfiguration__add_unique_defaultconfiguration_app_is
 > desktop:0022_auto__del_field_defaultconfiguration_group__del_unique_defaultconfigur
 > desktop:0023_auto__del_unique_defaultconfiguration_app_is_default_user__add_field_d
 > desktop:0024_auto__add_field_document2_is_managed
 > desktop:0025_auto__add_field_document2_is_trashed
 > desktop:0026_change_is_trashed_default_to_false
 - Migration 'desktop:0026_change_is_trashed_default_to_false' is marked for no-dry-run.
 - Loading initial data for desktop.
Installed 0 object(s) from 0 fixture(s)
Running migrations for beeswax:
 - Migrating forwards to 0014_auto__add_field_queryhistory_is_cleared.
 > beeswax:0009_auto__chg_field_queryhistory_server_port
 > beeswax:0010_merge_database_state
 > beeswax:0011_auto__chg_field_savedquery_name
 > beeswax:0012_auto__add_field_queryhistory_extra
 > beeswax:0013_auto__add_field_session_properties
 > beeswax:0014_auto__add_field_queryhistory_is_cleared
 - Loading initial data for beeswax.
Installed 0 object(s) from 0 fixture(s)
Running migrations for hbase:
 - Migrating forwards to 0001_initial.
 > hbase:0001_initial
 - Loading initial data for hbase.
Installed 0 object(s) from 0 fixture(s)
Running migrations for jobsub:
 - Migrating forwards to 0006_chg_varchars_to_textfields.
 > jobsub:0001_initial
 > jobsub:0002_auto__add_ooziestreamingaction__add_oozieaction__add_oozieworkflow__ad
 > jobsub:0003_convertCharFieldtoTextField
 > jobsub:0004_hue1_to_hue2
 - Migration 'jobsub:0004_hue1_to_hue2' is marked for no-dry-run.
 > jobsub:0005_unify_with_oozie
 - Migration 'jobsub:0005_unify_with_oozie' is marked for no-dry-run.
 > jobsub:0006_chg_varchars_to_textfields
 - Loading initial data for jobsub.
Installed 0 object(s) from 0 fixture(s)
Running migrations for oozie:
 - Migrating forwards to 0027_auto__chg_field_node_name__chg_field_job_name.
 > oozie:0026_set_default_data_values
 - Migration 'oozie:0026_set_default_data_values' is marked for no-dry-run.
 > oozie:0027_auto__chg_field_node_name__chg_field_job_name
 - Loading initial data for oozie.
Installed 0 object(s) from 0 fixture(s)
Running migrations for pig:
- Nothing to migrate.
 - Loading initial data for pig.
Installed 0 object(s) from 0 fixture(s)
Running migrations for search:
 - Migrating forwards to 0003_auto__add_field_collection_owner.
 > search:0001_initial
 > search:0002_auto__del_core__add_collection
 > search:0003_auto__add_field_collection_owner
 - Loading initial data for search.
Installed 0 object(s) from 0 fixture(s)
? You have no migrations for the 'security' app. You might want some.
Running migrations for spark:
 - Migrating forwards to 0001_initial.
 > spark:0001_initial
 - Loading initial data for spark.
Installed 0 object(s) from 0 fixture(s)
Running migrations for sqoop:
 - Migrating forwards to 0001_initial.
 > sqoop:0001_initial
 - Loading initial data for sqoop.
Installed 0 object(s) from 0 fixture(s)
Running migrations for useradmin:
 - Migrating forwards to 0008_convert_documents.
 > useradmin:0001_permissions_and_profiles
 - Migration 'useradmin:0001_permissions_and_profiles' is marked for no-dry-run.
 > useradmin:0002_add_ldap_support
 - Migration 'useradmin:0002_add_ldap_support' is marked for no-dry-run.
 > useradmin:0003_remove_metastore_readonly_huepermission
 - Migration 'useradmin:0003_remove_metastore_readonly_huepermission' is marked for no-dry-run.
 > useradmin:0004_add_field_UserProfile_first_login
 > useradmin:0005_auto__add_field_userprofile_last_activity
 > useradmin:0006_auto__add_index_userprofile_last_activity
 > useradmin:0007_remove_s3_access
 > useradmin:0008_convert_documents
 - Migration 'useradmin:0008_convert_documents' is marked for no-dry-run.
Starting document conversions...

Finished running document conversions.

 - Loading initial data for useradmin.
Installed 0 object(s) from 0 fixture(s)
Running migrations for notebook:
 - Migrating forwards to 0001_initial.
 > notebook:0001_initial
 - Loading initial data for notebook.
Installed 0 object(s) from 0 fixture(s)

5、啟動HUE服務(wù)

[hadoop@hdp01 ~]$ /u01/hue/build/env/bin/supervisor &

啟動后，通過瀏覽器訪問HUE服務(wù)器的8888端口，如下：

PS:如果要啟用中文界面，需修改HUE_HOME/desktop/core/src/desktop/settings.py文件，將LANGUAGE_CODE = 'en-us'的值改為'zh-CN'，然后make apps即可。
參考文獻(xiàn)：
1、安裝Hue后的一些功能的問題解決干貨總結(jié)