网站建设插件,网站检测,网站建设公司有前途吗,怎样开始学做自媒体视频教程#xff1a;哔哩哔哩网站#xff1a;黑马大数据Hadoop入门视频教程 总时长#xff1a;14:22:04教程资源: https://pan.baidu.com/s/1WYgyI3KgbzKzFD639lA-_g 提取码: 6666【P001-P017】大数据Hadoop教程-学习笔记01【大数据导论与Linux基础】【17p】【P018-P037】大… 视频教程哔哩哔哩网站黑马大数据Hadoop入门视频教程 总时长14:22:04教程资源: https://pan.baidu.com/s/1WYgyI3KgbzKzFD639lA-_g 提取码: 6666【P001-P017】大数据Hadoop教程-学习笔记01【大数据导论与Linux基础】【17p】【P018-P037】大数据Hadoop教程-学习笔记02【Apache Hadoop、HDFS】【20p】【P038-P050】大数据Hadoop教程-学习笔记03【Hadoop MapReduce与Hadoop YARN】【13p】【P051-P068】大数据Hadoop教程-学习笔记04【数据仓库基础与Apache Hive入门】【18p】【P069-P083】大数据Hadoop教程-学习笔记05【Apache Hive DML语句与函数使用】【15p】【P084-P096】大数据Hadoop教程-学习笔记06【Hadoop生态综合案例陌陌聊天数据分析】【13p】目录
01【数据仓库基本概念】
P051【01-课程内容大纲学习目标】
P052【02-数据仓库概念与起源发展由来】
P053【03-数据仓库主要特征--面向主题、集成、非易失、时变】
P054【04-数仓主流开发语言--SQL介绍】
02【Apache Hive入门】
P055【05-Apache hive软件介绍与Hadoop关系】
P056【06-场景设计--Hive功能模拟实现底层猜想】
P057【07-Apache hive--架构图、各组件功能】
03【Apache Hive安装部署】
P058【08-Apache hive安装部署--metadata与metastore、远程模式介绍】
P059【09-Apache hive安装部署--与Hadoop整合、MySQL安装】
P060【10-Apache hive安装部署--配置文件修改编辑】
P061【11-Apache hive安装部署--metastore服务启动方式】
P062【12-Apache hive--新老客户端使用与hiveserver2服务】
P063【13-Apache hive--DataGrip连接Hiveserver2】
04【Hive SQL语言DDL建库、建表】
P064【14-Apache hive--数据库与建库、切换库操作】
P065【15-Apache hive--表与建表sql语句--数据类型、分隔符指定语法】
P066【16-Apache hive--表与建表sql语句--默认分隔符使用】
05【Hive Show语法】
P067【17-Apache hive--常见的show语法】
P068【18-Apache hive--注释comment中文乱码解决】 01【数据仓库基本概念】
P051【01-课程内容大纲学习目标】 目录 数据仓库基本概念Apache Hive入门Apache Hive安装部署Hive SQL语言DDL建库、建表学习目标 掌握数据仓库是什么、解决什么理解数据仓库有什么特点理解SQL编程语言的概念、优点掌握Apache Hive架构原理、组件掌握Apache Hive客户端使用掌握Apache Hive的建库、建表SQL语法P052【02-数据仓库概念与起源发展由来】 数仓概念 数据仓库英语Data Warehouse简称数仓、DW是一个用于存储、分析、报告的数据系统。 数据仓库的目的是构建面向分析的集成化数据环境分析结果为企业提供决策支持Decision Support。 联机事务处理系统OLTP正好可以满足上述业务需求开展其主要任务是执行联机事务处理。其基本特征是前台接收的用户数据可以立即传送到后台进行处理并在很短的时间内给出处理结果。 关系型数据库RDBMS是OLTP典型应用比如Oracle、MySQL、SQL Server等。 如数仓定义所说,数仓是一个用于存储、分析、报告的数据系统目的是构建面向分析的集成化数据环境。我们把这种面向分析、支持分析的系统称之为OLAP联机分析处理系统。当然数据仓库是OLAP系统的一种实现。 P053【03-数据仓库主要特征--面向主题、集成、非易失、时变】 数仓主要特征面向主题、集成性、非易失性、时变性。 P054【04-数仓主流开发语言--SQL介绍】 SQL语言介绍 结构化查询语言Structured Query Language简称SQL是一种数据库查询和程序设计语言用于存取数据以及查询、更新和管理数据。 02【Apache Hive入门】
P055【05-Apache hive软件介绍与Hadoop关系】 Hive介绍 Apache Hive是一款建立在Hadoop之上的开源数据仓库系统可以将存储在Hadoop文件中的结构化、半结构化数据文件映射为一张数据库表基于表提供了一种类似SQL的查询模型称为Hive查询语言HQL用于访问和 分析存储在Hadoop文件中的大型数据集。Hive核心是将HQL转换为MapReduce程序然后将程序提交到Hadoop群集执行。Hive由Facebook实现并开源。P056【06-场景设计--Hive功能模拟实现底层猜想】 对Hive的理解 Hive能将数据文件映射成为一张表这个映射是指什么文件和表之间的对应关系。 Hive软件本身到底承担了什么功能职责SQL语法解析编译成为MapReduce。 Hive架构图P057【07-Apache hive--架构图、各组件功能】 Apache Hive架构图HIve组件 用户接口元数据存储Driver驱动程序包括语法解析器、计划编译器、优化器、执行器执行引擎03【Apache Hive安装部署】
P058【08-Apache hive安装部署--metadata与metastore、远程模式介绍】 元数据Metadata又称中介数据、中继数据为描述数据的数据data about data主要是描述数据属性property的信息用来支持如指示存储位置、历史数据、资源查找、文件记录等功能。 Hive Metadata Hive Metadata即Hive的元数据。包含用Hive创建的database、table、表的位置、类型、属性字段顺序类型等元信息。元数据存储在关系型数据库中。如hive内置的Derby、或者第三方如MySQL等。metastore服务配置有3种模式内嵌模式、本地模式、远程模式。 metastore远程模式 在生产环境中建议用远程模式来配置Hive Metastore。在这种情况下其他依赖hive的软件都可以通过Metastore访问hive。由于还可以完全屏蔽数据库层因此这也带来了更好的可管理性/安全性。 Remote Metastore远程模式P059【09-Apache hive安装部署--与Hadoop整合、MySQL安装】 步骤 Step1MySQL安装Step2上传解压Hive安装包node1安装即可Step3修改hive-env.shStep4新增hive-site.xmlStep5添加驱动、初始化连接成功
Last login: Wed Feb 22 19:31:24 2023 from 192.168.88.1
[rootnode1 ~]# cat /export/server/hadoop-3.3.0/etc/hadoop/core-site.xml
?xml version1.0 encodingUTF-8?
?xml-stylesheet typetext/xsl hrefconfiguration.xsl?
!--Licensed under the Apache License, Version 2.0 (the License);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an AS IS BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License. See accompanying LICENSE file.
--!-- Put site-specific property overrides in this file. --configuration
!-- 设置默认使用的文件系统 Hadoop支持file、HDFS、GFS、ali|Amazon云等文件系统 --
propertynamefs.defaultFS/namevaluehdfs://node1:8020/value
/property!-- 设置Hadoop本地保存数据路径 --
propertynamehadoop.tmp.dir/namevalue/export/data/hadoop-3.3.0/value
/property!-- 设置HDFS web UI用户身份 --
propertynamehadoop.http.staticuser.user/namevalueroot/value
/property!-- 整合hive 用户代理设置 --
propertynamehadoop.proxyuser.root.hosts/namevalue*/value
/propertypropertynamehadoop.proxyuser.root.groups/namevalue*/value
/property
!-- 文件系统垃圾桶保存时间 单位分 --
propertynamefs.trash.interval/namevalue1440/value
/property
/configuration
[rootnode1 ~]# rpm -qa|grep mariadb
mariadb-libs-5.5.64-1.el7.x86_64
您在 /var/spool/mail/root 中有新邮件
[rootnode1 ~]# rpm -e mariadb-libs-5.5.64-1.el7.x86_64 --nodeps
[rootnode1 ~]# rpm -qa|grep mariadb
[rootnode1 ~]# mkdir /export/software/mysql
[rootnode1 ~]# cd /export/software/mysql
[rootnode1 mysql]# ll
总用量 533040
-rw-r--r-- 1 root root 545832960 2月 23 11:20 mysql-5.7.29-1.el7.x86_64.rpm-bundle.tar
[rootnode1 mysql]# tar xvf mysql-5.7.29-1.el7.x86_64.rpm-bundle.tar
mysql-community-embedded-devel-5.7.29-1.el7.x86_64.rpm
mysql-community-test-5.7.29-1.el7.x86_64.rpm
mysql-community-embedded-5.7.29-1.el7.x86_64.rpm
mysql-community-embedded-compat-5.7.29-1.el7.x86_64.rpm
mysql-community-libs-5.7.29-1.el7.x86_64.rpm
mysql-community-client-5.7.29-1.el7.x86_64.rpm
mysql-community-server-5.7.29-1.el7.x86_64.rpm
mysql-community-devel-5.7.29-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.29-1.el7.x86_64.rpm
mysql-community-common-5.7.29-1.el7.x86_64.rpm
[rootnode1 mysql]# yum -y install libaio
已加载插件fastestmirror, langpacks
Determining fastest mirrors* base: mirrors.tuna.tsinghua.edu.cn* extras: mirrors.tuna.tsinghua.edu.cn* updates: mirrors.tuna.tsinghua.edu.cn
base | 3.6 kB 00:00:00
extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/3): extras/7/x86_64/primary_db | 249 kB 00:00:00
(2/3): base/7/x86_64/primary_db | 6.1 MB 00:00:02
(3/3): updates/7/x86_64/primary_db | 19 MB 00:00:12
软件包 libaio-0.3.109-13.el7.x86_64 已安装并且是最新版本
无须任何处理
[rootnode1 mysql]# rpm -ivh mysql-community-common-5.7.29-1.el7.x86_64.rpm mysql-community-libs-5.7.29-1.el7.x86_64.rpm mysql-community-client-5.7.29-1.el7.x86_64.rpm mysql-community-server-5.7.29-1.el7.x86_64.rpm
警告mysql-community-common-5.7.29-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中... ################################# [100%]
正在升级/安装...1:mysql-community-common-5.7.29-1.e################################# [ 25%]2:mysql-community-libs-5.7.29-1.el7################################# [ 50%]3:mysql-community-client-5.7.29-1.e################################# [ 75%]4:mysql-community-server-5.7.29-1.e################################# [100%]
[rootnode1 mysql]# mysqld --initialize
[rootnode1 mysql]# chown mysql:mysql /var/lib/mysql -R
[rootnode1 mysql]# systemctl start mysqld.service
[rootnode1 mysql]# cat /var/log/mysqld.log
2023-02-23T03:43:16.580339Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2023-02-23T03:43:18.234540Z 0 [Warning] InnoDB: New log files created, LSN45790
2023-02-23T03:43:18.421383Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2023-02-23T03:43:18.500035Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 360b5b52-b32c-11ed-888d-000c29340b53.
2023-02-23T03:43:18.501254Z 0 [Warning] Gtid table is not ready to be used. Table mysql.gtid_executed cannot be opened.
2023-02-23T03:43:19.887964Z 0 [Warning] CA certificate ca.pem is self signed.
2023-02-23T03:43:20.162146Z 1 [Note] A temporary password is generated for rootlocalhost: z(39#?pF?O
2023-02-23T03:43:57.841346Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2023-02-23T03:43:57.872210Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.29) starting as process 37730 ...
2023-02-23T03:43:57.896542Z 0 [Note] InnoDB: PUNCH HOLE support available
2023-02-23T03:43:57.896662Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2023-02-23T03:43:57.896669Z 0 [Note] InnoDB: Uses event mutexes
2023-02-23T03:43:57.896721Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2023-02-23T03:43:57.896731Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2023-02-23T03:43:57.896736Z 0 [Note] InnoDB: Using Linux native AIO
2023-02-23T03:43:57.917078Z 0 [Note] InnoDB: Number of pools: 1
2023-02-23T03:43:57.919925Z 0 [Note] InnoDB: Using CPU crc32 instructions
2023-02-23T03:43:57.955368Z 0 [Note] InnoDB: Initializing buffer pool, total size 128M, instances 1, chunk size 128M
2023-02-23T03:43:58.036981Z 0 [Note] InnoDB: Completed initialization of buffer pool
2023-02-23T03:43:58.079394Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2023-02-23T03:43:58.107082Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2023-02-23T03:43:58.137269Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2023-02-23T03:43:58.137408Z 0 [Note] InnoDB: Setting file ./ibtmp1 size to 12 MB. Physically writing the file full; Please wait ...
2023-02-23T03:43:58.269290Z 0 [Note] InnoDB: File ./ibtmp1 size is now 12 MB.
2023-02-23T03:43:58.271662Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2023-02-23T03:43:58.271702Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2023-02-23T03:43:58.292372Z 0 [Note] InnoDB: 5.7.29 started; log sequence number 2630592
2023-02-23T03:43:58.309770Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2023-02-23T03:43:58.327667Z 0 [Note] Plugin FEDERATED is disabled.
2023-02-23T03:43:58.354784Z 0 [Note] InnoDB: Buffer pool(s) load completed at 230223 11:43:58
2023-02-23T03:43:58.416170Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2023-02-23T03:43:58.416216Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
2023-02-23T03:43:58.417327Z 0 [Warning] CA certificate ca.pem is self signed.
2023-02-23T03:43:58.417392Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
2023-02-23T03:43:58.418168Z 0 [Note] Server hostname (bind-address): *; port: 3306
2023-02-23T03:43:58.418225Z 0 [Note] IPv6 is available.
2023-02-23T03:43:58.418237Z 0 [Note] - :: resolves to ::;
2023-02-23T03:43:58.418255Z 0 [Note] Server socket created on IP: ::.
2023-02-23T03:43:58.430589Z 0 [Note] Event Scheduler: Loaded 0 events
2023-02-23T03:43:58.430915Z 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: 5.7.29 socket: /var/lib/mysql/mysql.sock port: 3306 MySQL Community Server (GPL)
[rootnode1 mysql]# mysql -u root -p
Enter password:
ERROR 1045 (28000): Access denied for user rootlocalhost (using password: YES)
[rootnode1 mysql]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.29Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type help; or \h for help. Type \c to clear the current input statement.mysql alter user user() identified by hadoop;
Query OK, 0 rows affected (0.00 sec)mysql use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -ADatabase changed
mysql GRANT ALL PRIVILEGES ON *.* TO root% IDENTIFIED BY hadoop WITH GRANT OPTION;
Query OK, 0 rows affected, 1 warning (0.00 sec)mysql FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)mysql ^DBye
[rootnode1 mysql]# systemctl status mysqld
● mysqld.service - MySQL ServerLoaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)Active: active (running) since 四 2023-02-23 11:43:58 CST; 4min 1s agoDocs: man:mysqld(8)http://dev.mysql.com/doc/refman/en/using-systemd.htmlProcess: 37728 ExecStart/usr/sbin/mysqld --daemonize --pid-file/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (codeexited, status0/SUCCESS)Process: 37708 ExecStartPre/usr/bin/mysqld_pre_systemd (codeexited, status0/SUCCESS)Main PID: 37730 (mysqld)CGroup: /system.slice/mysqld.service└─37730 /usr/sbin/mysqld --daemonize --pid-file/var/run/mysqld/mysqld.pid2月 23 11:43:56 node1.itcast.cn systemd[1]: Starting MySQL Server...
2月 23 11:43:58 node1.itcast.cn systemd[1]: Started MySQL Server.
[rootnode1 mysql]# systemctl enable mysqld
[rootnode1 mysql]# systemctl list-unit-files | grep mysqld
mysqld.service enabled
mysqld.service disabled
[rootnode1 mysql]#
P060【10-Apache hive安装部署--配置文件修改编辑】 在Windows中远程连接Linux中的MySQL数据库 192.168.88.151、node1.itcast.cn 连接成功
Last login: Thu Feb 23 12:10:56 2023
[rootnode1 ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.29 MySQL Community Server (GPL)Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type help; or \h for help. Type \c to clear the current input statement.mysql use mysql
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -ADatabase changed
mysql show tables;
---------------------------
| Tables_in_mysql |
---------------------------
| columns_priv |
| db |
| engine_cost |
| event |
| func |
| general_log |
| gtid_executed |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| innodb_index_stats |
| innodb_table_stats |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| server_cost |
| servers |
| slave_master_info |
| slave_relay_log_info |
| slave_worker_info |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
---------------------------
31 rows in set (0.00 sec)mysql select user,password,host from user;
ERROR 1054 (42S22): Unknown column password in field list
mysql select host,user from user;
--------------------------
| host | user |
--------------------------
| % | root |
| localhost | mysql.session |
| localhost | mysql.sys |
| localhost | root |
--------------------------
4 rows in set (0.00 sec)mysql update user set host% where user root;
ERROR 1062 (23000): Duplicate entry %-root for key PRIMARY
mysql systemctl stop firewalld.service- ^DBye
[rootnode1 ~]# systemctl stop firewalld.service
[rootnode1 ~]# systemctl disable firewalld.service
[rootnode1 ~]# //查看开放的端口号
-bash: //查看开放的端口号: 没有那个文件或目录
[rootnode1 ~]# firewall-cmd --list-all
FirewallD is not running
[rootnode1 ~]#
[rootnode1 ~]# //设置开放的端口号
-bash: //设置开放的端口号: 没有那个文件或目录
[rootnode1 ~]# firewall-cmd --add-servicehttp --permanent
FirewallD is not running
[rootnode1 ~]# firewall-cmd --add-port3306/tcp --permanent
FirewallD is not running
[rootnode1 ~]#
[rootnode1 ~]# //重启防火墙
-bash: //重启防火墙: 没有那个文件或目录
[rootnode1 ~]# firewall-cmd --reload
FirewallD is not running
[rootnode1 ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.29 MySQL Community Server (GPL)Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type help; or \h for help. Type \c to clear the current input statement.mysql use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -ADatabase changed
mysql
mysql select Host,User from user;
--------------------------
| Host | User |
--------------------------
| % | root |
| localhost | mysql.session |
| localhost | mysql.sys |
| localhost | root |
--------------------------
4 rows in set (0.00 sec)mysql flush privileges;
Query OK, 0 rows affected (0.00 sec)mysql ^DBye
[rootnode1 ~]#
P061【11-Apache hive安装部署--metastore服务启动方式】
[rootnode1 ~]# start-all.sh
Starting namenodes on [node1]
上一次登录四 2月 23 14:02:35 CST 2023从 192.168.88.1pts/1 上
Starting datanodes
上一次登录四 2月 23 14:23:50 CST 2023pts/0 上
Starting secondary namenodes [node2]
上一次登录四 2月 23 14:23:53 CST 2023pts/0 上
Starting resourcemanager
上一次登录四 2月 23 14:24:00 CST 2023pts/0 上
Starting nodemanagers
上一次登录四 2月 23 14:24:13 CST 2023pts/0 上
[rootnode1 ~]# /export/server/apache-hive-3.1.2-bin/bin/hive --service metastore
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2023-02-23 14:27:37: Starting Hive Metastore Server
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
^C[rootnode1 ~]# nohup /export/server/apache-hive-3.1.2-bin/bin/hive --service metastore
[1] 35181
[rootnode1 ~]# nohup: 忽略输入并把输出追加到nohup.out[rootnode1 ~]# jps
28691 ResourceManager
35683 Jps
28916 NodeManager
27689 NameNode
27885 DataNode
35181 RunJar
[rootnode1 ~]# ll
总用量 20
-rw-r--r-- 1 root root 2 2月 21 21:14 1.txt
-rw-r--r-- 1 root root 4 2月 22 11:03 666.txt
-rw-------. 1 root root 1340 9月 11 2020 anaconda-ks.cfg
-rw-r--r-- 1 root root 34 2月 21 21:36 hello.txt
-rw------- 1 root root 1072 2月 23 14:28 nohup.out
[rootnode1 ~]# kill-9 35181
-bash: kill-9: 未找到命令
[rootnode1 ~]# kill 35181
[rootnode1 ~]# jps
28691 ResourceManager
28916 NodeManager
37508 Jps
27689 NameNode
27885 DataNode
[1] 退出 143 nohup /export/server/apache-hive-3.1.2-bin/bin/hive --service metastore
[rootnode1 ~]#
P062【12-Apache hive--新老客户端使用与hiveserver2服务】 第一代客户端 node3 第一代客户端 node1 /export/server/apache-hive-3.1.2-bin/bin/hive 连接成功
Last login: Thu Feb 23 11:14:41 2023 from 192.168.88.1
[rootnode3 ~]# /export/server/apache-hive-3.1.2-bin/bin/hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/export/server/jdk1.8.0_241/bin:/export/server/hadoop-3.3.0/bin:/export/server/hadoop-3.3.0/sbin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID bd951b9d-11c9-43d5-ae6d-3b82a7a96823Logging initialized using configuration in jar:file:/export/server/apache-hive-3.1.2-bin/lib/hive-common-3.1.2.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID cd48f8fb-6073-4ddf-a5cb-2b619eba70cf
hive show da
data date date( date_add( date_format( date_sub( datediff( datetime day( dayofmonth( dayofweek(
hive show databases;
OK
default
Time taken: 0.923 seconds, Fetched: 1 row(s)
hive show tables;
OK
Time taken: 0.09 seconds
hive 您在 /var/spool/mail/root 中有新邮件
[rootnode3 ~]# 第二代客户端 node3 /export/server/apache-hive-3.1.2-bin/bin/beeline ! connect jdbc:hive2://node1:10000 root 然后直接回车 [rootnode3 ~]# /export/server/apache-hive-3.1.2-bin/bin/beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 3.1.2 by Apache Hive
beeline ! connect jdbc:hive2://node1:10000
Connecting to jdbc:hive2://node1:10000
Enter username for jdbc:hive2://node1:10000: root
Enter password for jdbc:hive2://node1:10000:
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://node1:10000 show databases;
INFO : Compiling command(queryIdroot_20230223144903_ef22457c-2764-4ead-9755-f84aaa2505f2): show databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223144903_ef22457c-2764-4ead-9755-f84aaa2505f2); Time taken: 1.407 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223144903_ef22457c-2764-4ead-9755-f84aaa2505f2): show databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223144903_ef22457c-2764-4ead-9755-f84aaa2505f2); Time taken: 0.07 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
----------------
| database_name |
----------------
| default |
----------------
1 row selected (2.172 seconds)
0: jdbc:hive2://node1:10000 show tables;
INFO : Compiling command(queryIdroot_20230223144915_bf4a1a40-2124-46b6-acfd-b036b91149b9): show tables
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223144915_bf4a1a40-2124-46b6-acfd-b036b91149b9); Time taken: 0.039 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223144915_bf4a1a40-2124-46b6-acfd-b036b91149b9): show tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223144915_bf4a1a40-2124-46b6-acfd-b036b91149b9); Time taken: 0.025 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
-----------
| tab_name |
-----------
-----------
No rows selected (0.119 seconds)
0: jdbc:hive2://node1:10000 Closing: 0: jdbc:hive2://node1:10000
您在 /var/spool/mail/root 中有新邮件
[rootnode3 ~]#
P063【13-Apache hive--DataGrip连接Hiveserver2】 DataGrip 2020.2.2 x64 04【Hive SQL语言DDL建库、建表】
P064【14-Apache hive--数据库与建库、切换库操作】 Hive数据模型总览 选择正确的方向比盲目努力更重要。 P065【15-Apache hive--表与建表sql语句--数据类型、分隔符指定语法】 --创建数据库并切换使用
create database if not exists itheima;
use itheima;--1、创建一张表将射手结构化数据文件在Hive中映射成功
-- 表名
-- 字段 名称 类型 顺序
-- 字段之间的分隔符需要指定--ddl create table
create table itheima.t_archer(id int comment ID编号,name string comment 英雄名称,hp_max int comment 最大生命,mp_max int comment 最大法力,attack_max int comment 最高物攻,defense_max int comment 最大物防,attack_range string comment 攻击范围,role_main string comment 主要定位,role_assist string comment 次要定位
) comment 王者荣耀射手信息
row format delimited
fields terminated by \t; -- 字段之间的分隔符是tab键 制表符select * from t_archer; P066【16-Apache hive--表与建表sql语句--默认分隔符使用】 node1 start-all.sh /export/server/apache-hive-3.1.2-bin/bin/hive --service metastore前台启动metastore服务 nohup /export/server/apache-hive-3.1.2-bin/bin/hive --service metastore 后台启动metastore服务 jps nohup /export/server/apache-hive-3.1.2-bin/bin/hive --service hiveserver2 jps node3 /export/server/apache-hive-3.1.2-bin/bin/beeline ! connect jdbc:hive2://node1:10000 root 然后直接回车 连接成功
Last login: Thu Feb 23 21:06:42 2023 from 192.168.88.1
[rootnode1 ~]# jps
1813 Jps
[rootnode1 ~]# start-all.sh
Starting namenodes on [node1]
上一次登录四 2月 23 21:12:56 CST 2023从 192.168.88.1pts/1 上
Starting datanodes
上一次登录四 2月 23 21:13:22 CST 2023pts/0 上
Starting secondary namenodes [node2]
上一次登录四 2月 23 21:13:25 CST 2023pts/0 上
Starting resourcemanager
上一次登录四 2月 23 21:13:31 CST 2023pts/0 上
Starting nodemanagers
上一次登录四 2月 23 21:13:39 CST 2023pts/0 上
[rootnode1 ~]# jps
2272 NameNode
3400 NodeManager
3162 ResourceManager
2477 DataNode
3903 Jps
[rootnode1 ~]# nohup /export/server/apache-hive-3.1.2-bin/bin/hive --service metastore
[1] 4670
[rootnode1 ~]# nohup: 忽略输入并把输出追加到nohup.out[rootnode1 ~]# jps
2272 NameNode
4880 Jps
3400 NodeManager
3162 ResourceManager
2477 DataNode
4670 RunJar
[rootnode1 ~]# nohup /export/server/apache-hive-3.1.2-bin/bin/hive --service hiveserver2
[2] 5027
[rootnode1 ~]# nohup: 忽略输入并把输出追加到nohup.out[rootnode1 ~]# jps
2272 NameNode
5171 Jps
3400 NodeManager
3162 ResourceManager
5051 VersionInfo
2477 DataNode
4670 RunJar
[rootnode1 ~]# jps
2272 NameNode
5027 RunJar
7383 Jps
3400 NodeManager
3162 ResourceManager
2477 DataNode
4670 RunJar
[rootnode1 ~]#
连接成功
Last login: Thu Feb 23 21:01:31 2023 from 192.168.88.1
[rootnode3 ~]# /export/server/apache-hive-3.1.2-bin/bin/beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 3.1.2 by Apache Hive
beeline ! connect jdbc:hive2://node1:10000
Connecting to jdbc:hive2://node1:10000
Enter username for jdbc:hive2://node1:10000: root
Enter password for jdbc:hive2://node1:10000:
23/02/23 21:15:26 [main]: WARN jdbc.HiveConnection: Failed to connect to node1:10000
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://node1:10000: java.net.ConnectException: 拒绝连接 (Connection refused) (state08S01,code0)
beeline 您在 /var/spool/mail/root 中有新邮件
[rootnode3 ~]# /export/server/apache-hive-3.1.2-bin/bin/beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/export/server/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/export/server/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 3.1.2 by Apache Hive
beeline ! connect jdbc:hive2://node1:10000
Connecting to jdbc:hive2://node1:10000
Enter username for jdbc:hive2://node1:10000: root
Enter password for jdbc:hive2://node1:10000:
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://node1:10000 show databases;
INFO : Compiling command(queryIdroot_20230223211656_6489cb28-8689-4b0f-8ee6-6503535f4cbd): show databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223211656_6489cb28-8689-4b0f-8ee6-6503535f4cbd); Time taken: 1.802 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223211656_6489cb28-8689-4b0f-8ee6-6503535f4cbd): show databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223211656_6489cb28-8689-4b0f-8ee6-6503535f4cbd); Time taken: 0.085 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
----------------
| database_name |
----------------
| default |
| itheima |
----------------
2 rows selected (2.589 seconds)
0: jdbc:hive2://node1:10000 use itheima;
INFO : Compiling command(queryIdroot_20230223211710_5f6217f0-2d03-431b-8dee-23cb6965e677): use itheima
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling command(queryIdroot_20230223211710_5f6217f0-2d03-431b-8dee-23cb6965e677); Time taken: 0.045 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223211710_5f6217f0-2d03-431b-8dee-23cb6965e677): use itheima
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223211710_5f6217f0-2d03-431b-8dee-23cb6965e677); Time taken: 0.015 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
No rows affected (0.157 seconds)
0: jdbc:hive2://node1:10000 show tables;
INFO : Compiling command(queryIdroot_20230223211726_dd7ca710-179b-47e7-9cf8-a8396d2edf1b): show tables
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223211726_dd7ca710-179b-47e7-9cf8-a8396d2edf1b); Time taken: 0.036 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223211726_dd7ca710-179b-47e7-9cf8-a8396d2edf1b): show tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223211726_dd7ca710-179b-47e7-9cf8-a8396d2edf1b); Time taken: 0.035 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
-----------
| tab_name |
-----------
| t_archer |
-----------
1 row selected (0.122 seconds)
0: jdbc:hive2://node1:10000 在hive库里面用sql建成一张数据表以后在hdfs里面就是一个文件夹把文本传输进去如果都能够对应的上在hive这边客户端就能查到刚才文本映射出来的表结构数据。 --创建数据库并切换使用
create database if not exists itheima;
use itheima;--1、创建一张表将射手结构化数据文件在Hive中映射成功
-- 表名
-- 字段 名称 类型 顺序
-- 字段之间的分隔符需要指定--ddl create table
create table itheima.t_archer(id int comment ID编号,name string comment 英雄名称,hp_max int comment 最大生命,mp_max int comment 最大法力,attack_max int comment 最高物攻,defense_max int comment 最大物防,attack_range string comment 攻击范围,role_main string comment 主要定位,role_assist string comment 次要定位
) comment 王者荣耀射手信息
row format delimited
fields terminated by \t; -- 字段之间的分隔符是tab键 制表符select * from t_archer;create table itheima.t_archer1(id int comment ID编号,name string comment 英雄名称,hp_max int comment 最大生命,mp_max int comment 最大法力,attack_max int comment 最高物攻,defense_max int comment 最大物防,attack_range string comment 攻击范围,role_main string comment 主要定位,role_assist string comment 次要定位
);--使用默认分隔符建表 \001 非打印字符
create table t_team_ace_player(id int,team_name string,ace_player_name string
);
-- row format delimited
-- fields terminated by \001;select * from t_team_ace_player;--使用默认分隔符建表 \001 非打印字符
create table t_team_ace_player2(id int,team_name string,ace_player_name string
)
row format delimited
fields terminated by \t;select * from t_team_ace_player2;
05【Hive Show语法】
P067【17-Apache hive--常见的show语法】 0: jdbc:hive2://node1:10000 show databases;
INFO : Compiling command(queryIdroot_20230223211656_6489cb28-8689-4b0f-8ee6-6503535f4cbd): show databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223211656_6489cb28-8689-4b0f-8ee6-6503535f4cbd); Time taken: 1.802 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223211656_6489cb28-8689-4b0f-8ee6-6503535f4cbd): show databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223211656_6489cb28-8689-4b0f-8ee6-6503535f4cbd); Time taken: 0.085 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
----------------
| database_name |
----------------
| default |
| itheima |
----------------
2 rows selected (2.589 seconds)
0: jdbc:hive2://node1:10000 use itheima;
INFO : Compiling command(queryIdroot_20230223211710_5f6217f0-2d03-431b-8dee-23cb6965e677): use itheima
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling command(queryIdroot_20230223211710_5f6217f0-2d03-431b-8dee-23cb6965e677); Time taken: 0.045 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223211710_5f6217f0-2d03-431b-8dee-23cb6965e677): use itheima
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223211710_5f6217f0-2d03-431b-8dee-23cb6965e677); Time taken: 0.015 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
No rows affected (0.157 seconds)
0: jdbc:hive2://node1:10000 show tables;
INFO : Compiling command(queryIdroot_20230223211726_dd7ca710-179b-47e7-9cf8-a8396d2edf1b): show tables
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223211726_dd7ca710-179b-47e7-9cf8-a8396d2edf1b); Time taken: 0.036 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223211726_dd7ca710-179b-47e7-9cf8-a8396d2edf1b): show tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223211726_dd7ca710-179b-47e7-9cf8-a8396d2edf1b); Time taken: 0.035 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
-----------
| tab_name |
-----------
| t_archer |
-----------
1 row selected (0.122 seconds)
0: jdbc:hive2://node1:10000 show databases;
INFO : Compiling command(queryIdroot_20230223214926_9c11c9cf-81d6-460e-8cc5-ada1f4ba3b2a): show databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223214926_9c11c9cf-81d6-460e-8cc5-ada1f4ba3b2a); Time taken: 0.015 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223214926_9c11c9cf-81d6-460e-8cc5-ada1f4ba3b2a): show databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223214926_9c11c9cf-81d6-460e-8cc5-ada1f4ba3b2a); Time taken: 0.006 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
----------------
| database_name |
----------------
| default |
| itheima |
----------------
2 rows selected (0.048 seconds)
0: jdbc:hive2://node1:10000 show schemas;
INFO : Compiling command(queryIdroot_20230223214947_ea4ee4e1-3e8d-41c8-9ab9-3612885a12bd): show schemas
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223214947_ea4ee4e1-3e8d-41c8-9ab9-3612885a12bd); Time taken: 0.02 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223214947_ea4ee4e1-3e8d-41c8-9ab9-3612885a12bd): show schemas
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223214947_ea4ee4e1-3e8d-41c8-9ab9-3612885a12bd); Time taken: 0.006 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
----------------
| database_name |
----------------
| default |
| itheima |
----------------
2 rows selected (0.047 seconds)
0: jdbc:hive2://node1:10000 show tables;
INFO : Compiling command(queryIdroot_20230223215003_3e56b709-c5fc-47c5-9791-d6f34cb53e3d): show tables
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223215003_3e56b709-c5fc-47c5-9791-d6f34cb53e3d); Time taken: 0.04 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215003_3e56b709-c5fc-47c5-9791-d6f34cb53e3d): show tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215003_3e56b709-c5fc-47c5-9791-d6f34cb53e3d); Time taken: 0.009 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
---------------------
| tab_name |
---------------------
| t_archer |
| t_archer1 |
| t_team_ace_player |
| t_team_ace_player2 |
---------------------
4 rows selected (0.068 seconds)
0: jdbc:hive2://node1:10000 use def
default deferrable deferred defined
0: jdbc:hive2://node1:10000 use default;
INFO : Compiling command(queryIdroot_20230223215035_62f80942-23c8-413b-a4de-6ea52723e6f1): use default
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling command(queryIdroot_20230223215035_62f80942-23c8-413b-a4de-6ea52723e6f1); Time taken: 0.024 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215035_62f80942-23c8-413b-a4de-6ea52723e6f1): use default
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215035_62f80942-23c8-413b-a4de-6ea52723e6f1); Time taken: 0.007 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
No rows affected (0.042 seconds)
0: jdbc:hive2://node1:10000 show tables;
INFO : Compiling command(queryIdroot_20230223215037_65d2f780-be33-42f3-96ba-002eec6ff1e3): show tables
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223215037_65d2f780-be33-42f3-96ba-002eec6ff1e3); Time taken: 0.024 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215037_65d2f780-be33-42f3-96ba-002eec6ff1e3): show tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215037_65d2f780-be33-42f3-96ba-002eec6ff1e3); Time taken: 0.008 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
-----------
| tab_name |
-----------
-----------
No rows selected (0.045 seconds)
0: jdbc:hive2://node1:10000 show tables in itheima;
INFO : Compiling command(queryIdroot_20230223215118_6fab9cbb-be95-417e-9549-0474cad5742f): show tables in itheima
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223215118_6fab9cbb-be95-417e-9549-0474cad5742f); Time taken: 0.032 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215118_6fab9cbb-be95-417e-9549-0474cad5742f): show tables in itheima
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215118_6fab9cbb-be95-417e-9549-0474cad5742f); Time taken: 0.009 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
---------------------
| tab_name |
---------------------
| t_archer |
| t_archer1 |
| t_team_ace_player |
| t_team_ace_player2 |
---------------------
4 rows selected (0.06 seconds)
0: jdbc:hive2://node1:10000 desc formatted t_team_ace_player;
Error: Error while compiling statement: FAILED: SemanticException [Error 10001]: Table not found t_team_ace_player (state42S02,code10001)
0: jdbc:hive2://node1:10000 desc formatted t_team_ace_player;
Error: Error while compiling statement: FAILED: SemanticException [Error 10001]: Table not found t_team_ace_player (state42S02,code10001)
0: jdbc:hive2://node1:10000 desc formatted t_archer;
Error: Error while compiling statement: FAILED: SemanticException [Error 10001]: Table not found t_archer (state42S02,code10001)
0: jdbc:hive2://node1:10000 show tables;
INFO : Compiling command(queryIdroot_20230223215309_27c866fa-f738-4072-96bb-8270f94f7540): show tables
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223215309_27c866fa-f738-4072-96bb-8270f94f7540); Time taken: 0.028 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215309_27c866fa-f738-4072-96bb-8270f94f7540): show tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215309_27c866fa-f738-4072-96bb-8270f94f7540); Time taken: 0.008 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
-----------
| tab_name |
-----------
-----------
No rows selected (0.053 seconds)
0: jdbc:hive2://node1:10000 show tables in itheima;
INFO : Compiling command(queryIdroot_20230223215315_84c2bd7d-00a5-47c5-b118-37af449380f8): show tables in itheima
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223215315_84c2bd7d-00a5-47c5-b118-37af449380f8); Time taken: 0.03 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215315_84c2bd7d-00a5-47c5-b118-37af449380f8): show tables in itheima
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215315_84c2bd7d-00a5-47c5-b118-37af449380f8); Time taken: 0.01 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
---------------------
| tab_name |
---------------------
| t_archer |
| t_archer1 |
| t_team_ace_player |
| t_team_ace_player2 |
---------------------
4 rows selected (0.068 seconds)
0: jdbc:hive2://node1:10000 desc formatted t_team_ace_player;
Error: Error while compiling statement: FAILED: SemanticException [Error 10001]: Table not found t_team_ace_player (state42S02,code10001)
0: jdbc:hive2://node1:10000 use itheima;
INFO : Compiling command(queryIdroot_20230223215329_836ff206-4117-4e17-971f-6a90beddc017): use itheima
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling command(queryIdroot_20230223215329_836ff206-4117-4e17-971f-6a90beddc017); Time taken: 0.023 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215329_836ff206-4117-4e17-971f-6a90beddc017): use itheima
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215329_836ff206-4117-4e17-971f-6a90beddc017); Time taken: 0.011 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
No rows affected (0.046 seconds)
0: jdbc:hive2://node1:10000 show tables;
INFO : Compiling command(queryIdroot_20230223215335_72de1e84-a59d-4a58-9a2a-f224716b9796): show tables
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223215335_72de1e84-a59d-4a58-9a2a-f224716b9796); Time taken: 0.027 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215335_72de1e84-a59d-4a58-9a2a-f224716b9796): show tables
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215335_72de1e84-a59d-4a58-9a2a-f224716b9796); Time taken: 0.01 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
---------------------
| tab_name |
---------------------
| t_archer |
| t_archer1 |
| t_team_ace_player |
| t_team_ace_player2 |
---------------------
4 rows selected (0.055 seconds)
0: jdbc:hive2://node1:10000 desc formatted t_team_ace_player;
INFO : Compiling command(queryIdroot_20230223215339_61a932de-0793-42ba-8d6d-e501707939ec): desc formatted t_team_ace_player
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col_name, type:string, comment:from deserializer), FieldSchema(name:data_type, type:string, comment:from deserializer), FieldSchema(name:comment, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223215339_61a932de-0793-42ba-8d6d-e501707939ec); Time taken: 0.044 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215339_61a932de-0793-42ba-8d6d-e501707939ec): desc formatted t_team_ace_player
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215339_61a932de-0793-42ba-8d6d-e501707939ec); Time taken: 0.133 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
---------------------------------------------------------------------------------------------------------------------------------------
| col_name | data_type | comment |
---------------------------------------------------------------------------------------------------------------------------------------
| # col_name | data_type | comment |
| id | int | |
| team_name | string | |
| ace_player_name | string | |
| | NULL | NULL |
| # Detailed Table Information | NULL | NULL |
| Database: | itheima | NULL |
| OwnerType: | USER | NULL |
| Owner: | root | NULL |
| CreateTime: | Thu Feb 23 21:31:11 CST 2023 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Retention: | 0 | NULL |
| Location: | hdfs://node1:8020/user/hive/warehouse/itheima.db/t_team_ace_player | NULL |
| Table Type: | MANAGED_TABLE | NULL |
| Table Parameters: | NULL | NULL |
| | COLUMN_STATS_ACCURATE | {\BASIC_STATS\:\true\,\COLUMN_STATS\:{\ace_player_name\:\true\,\id\:\true\,\team_name\:\true\}} |
| | bucketing_version | 2 |
| | numFiles | 0 |
| | numRows | 0 |
| | rawDataSize | 0 |
| | totalSize | 0 |
| | transient_lastDdlTime | 1677159071 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| Storage Desc Params: | NULL | NULL |
| | serialization.format | 1 |
---------------------------------------------------------------------------------------------------------------------------------------
33 rows selected (0.21 seconds)
0: jdbc:hive2://node1:10000 desc formatted t_archer;
INFO : Compiling command(queryIdroot_20230223215354_49faf959-b5e0-46b8-bd99-8e65c5d62add): desc formatted t_archer
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col_name, type:string, comment:from deserializer), FieldSchema(name:data_type, type:string, comment:from deserializer), FieldSchema(name:comment, type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling command(queryIdroot_20230223215354_49faf959-b5e0-46b8-bd99-8e65c5d62add); Time taken: 0.044 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryIdroot_20230223215354_49faf959-b5e0-46b8-bd99-8e65c5d62add): desc formatted t_archer
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryIdroot_20230223215354_49faf959-b5e0-46b8-bd99-8e65c5d62add); Time taken: 0.044 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
---------------------------------------------------------------------------------------------------------------------------------------
| col_name | data_type | comment |
---------------------------------------------------------------------------------------------------------------------------------------
| # col_name | data_type | comment |
| id | int | ID?? |
| name | string | ???? |
| hp_max | int | ???? |
| mp_max | int | ???? |
| attack_max | int | ???? |
| defense_max | int | ???? |
| attack_range | string | ???? |
| role_main | string | ???? |
| role_assist | string | ???? |
| | NULL | NULL |
| # Detailed Table Information | NULL | NULL |
| Database: | itheima | NULL |
| OwnerType: | USER | NULL |
| Owner: | root | NULL |
| CreateTime: | Thu Feb 23 17:21:18 CST 2023 | NULL |
| LastAccessTime: | UNKNOWN | NULL |
| Retention: | 0 | NULL |
| Location: | hdfs://node1:8020/user/hive/warehouse/itheima.db/t_archer | NULL |
| Table Type: | MANAGED_TABLE | NULL |
| Table Parameters: | NULL | NULL |
| | COLUMN_STATS_ACCURATE | {\BASIC_STATS\:\true\,\COLUMN_STATS\:{\attack_max\:\true\,\attack_range\:\true\,\defense_max\:\true\,\hp_max\:\true\,\id\:\true\,\mp_max\:\true\,\name\:\true\,\role_assist\:\true\,\role_main\:\true\}} |
| | bucketing_version | 2 |
| | comment | ???????? |
| | numFiles | 0 |
| | numRows | 0 |
| | rawDataSize | 0 |
| | totalSize | 0 |
| | transient_lastDdlTime | 1677144078 |
| | NULL | NULL |
| # Storage Information | NULL | NULL |
| SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL |
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat | NULL |
| OutputFormat: | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL |
| Compressed: | No | NULL |
| Num Buckets: | -1 | NULL |
| Bucket Columns: | [] | NULL |
| Sort Columns: | [] | NULL |
| Storage Desc Params: | NULL | NULL |
| | field.delim | \t |
| | serialization.format | \t |
---------------------------------------------------------------------------------------------------------------------------------------
41 rows selected (0.114 seconds)
0: jdbc:hive2://node1:10000
P068【18-Apache hive--注释comment中文乱码解决】 hive将数据保存在mysql中而mysql默认的编码不支持中文默认编码 Latin1兼容ascii。 --注意下面sql语句是需要在MySQL中执行修改Hive存储的元数据信息metadata。
use hive3;
show tables;alter table hive3.COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;
alter table hive3.TABLE_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8;
alter table hive3.PARTITION_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8 ;
alter table hive3.PARTITION_KEYS modify column PKEY_COMMENT varchar(4000) character set utf8;
alter table hive3.INDEX_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8;
--创建数据库并切换使用
create database if not exists itheima;
use itheima;--1、创建一张表将射手结构化数据文件在Hive中映射成功
-- 表名
-- 字段 名称 类型 顺序
-- 字段之间的分隔符需要指定--ddl create table
create table itheima.t_archer(id int comment ID编号,name string comment 英雄名称,hp_max int comment 最大生命,mp_max int comment 最大法力,attack_max int comment 最高物攻,defense_max int comment 最大物防,attack_range string comment 攻击范围,role_main string comment 主要定位,role_assist string comment 次要定位
) comment 王者荣耀射手信息
row format delimited
fields terminated by \t; -- 字段之间的分隔符是tab键 制表符select * from t_archer;--查看表结构
desc formatted t_archer;--删除表sql
drop table t_archer;create table itheima.t_archer1(id int comment ID编号,name string comment 英雄名称,hp_max int comment 最大生命,mp_max int comment 最大法力,attack_max int comment 最高物攻,defense_max int comment 最大物防,attack_range string comment 攻击范围,role_main string comment 主要定位,role_assist string comment 次要定位
);--使用默认分隔符建表 \001 非打印字符
create table t_team_ace_player(id int,team_name string,ace_player_name string
);
-- row format delimited
-- fields terminated by \001;select * from t_team_ace_player;--使用默认分隔符建表 \001 非打印字符
create table t_team_ace_player2(id int,team_name string,ace_player_name string
)
row format delimited
fields terminated by \t;select * from t_team_ace_player2;
