什么网站做英语翻译练习,国外可以做会员网站的网站,苏州园区一站式服务中心,一体化企业管理软件一、环境说明
1.1CI/CD
CI即为持续集成(Continue Integration,简称CI)#xff0c;用通俗的话讲#xff0c;就是持续的整合版本库代码编译后制作应用镜像。建立有效的持续集成环境可以减少开发过程中一些不必要的问题、提高代码质量、快速迭代等;(Jenkins)
CD即持续交付Con…
一、环境说明
1.1CI/CD
CI即为持续集成(Continue Integration,简称CI)用通俗的话讲就是持续的整合版本库代码编译后制作应用镜像。建立有效的持续集成环境可以减少开发过程中一些不必要的问题、提高代码质量、快速迭代等;(Jenkins)
CD即持续交付Continuous Delivery和持续部署Continuous Deployment用通俗的话说即可以持续的部署到生产环境给客户使用这里分为两个阶段持续交付我理解为满足上线条件的过程但是没有上线持续部署即为上线应用的过程;(k8s) 1.2 系统架构
研发人员通过Git将代码上传至Gitlab项目管理员审核代码后合并代码
Jenkins通过webhook获取Gitlab上的项目代码并通过maven打包构建镜像如果构建报错则可以通过邮件或钉钉等通知相关人员
镜像构建完成后Jenkins自动将镜像pull到Harbor镜像仓库
Harbor镜像仓库通过将镜像push到k8s集群运行k8s通过编排管理实现镜像服务的伸缩和高可用。
1.3 软硬件环境
角色主机名ip地址备注masterk8s-master192.168.250.217 k8s主节点、8核CPU、4GB(CD) nodek8s-node1192.168.250.218 k8s从节点、8核CPU、4GB nodek8s-node2192.168.250.219 k8s从节点、8核CPU、4GB GitlabJenkinsDockerjenkins192.168.250.188 8核CPU、12GBGitlab最少需要4G(CI) docker镜像仓库:harborharbor192.168.250.220 8核CPU、4GB
软件版本linuxcentos7docker26.1.3k8s1.22.2harbor2.0.6Jenkins2.457(尽量别用latest版本安装 可能导致插件安装失败)gitlab10.7.5 二、环境准备
在所有节点操作
2.1 关闭NetworkManager
NetworkManager会和network启动是发生冲突导致network启动失败不能远程访问虚拟机 systemctl stop NetworkManager #临时关闭 systemctl disable NetworkManager #永久关闭网络管理命令 systemctl start network.service #开启网络服 2.2 设置静态ip 2.2.1查看网络配置 ifconfig #查看网络配置 2.2.2 修改网络配置文件 vi /etc/sysconfig/network-scripts/ifcfg-ens33 1选择动态分配或者静态分配 BOOTPROTOstatic #dhcp自动分配ip static静态ip 2开启自动打开网卡 ONBOOTyes #开启启动必须是yes TYPEEthernet PROXY_METHODnone BROWSER_ONLYno BOOTPROTOstatic #dhcp自动分配ip static静态ip DEFROUTEyes IPV4_FAILURE_FATALno IPV6INITyes IPV6_AUTOCONFyes IPV6_DEFROUTEyes IPV6_FAILURE_FATALno IPV6_ADDR_GEN_MODEstable-privacy NAMEens33 UUID9c922cc8-b3ed-419a-88bd-6fd756e04880 DEVICEens33 ONBOOTyes #开启启动必须是yes IPADDR192.168.163.152 #静态ip NETMASK255.255.255.0 # GATEWAY192.168.163.2 #网关 DNS1192.168.163.0 #DNS DNS2114.114.114.114 2.2.3重启服务 systemctl restart network 2.2.4查看网络状态 systemctl status network.service 图示 ifconfig 2.3 修改主机名
hostnamectl set-hostname k8s-master #k8s-master 为主机名称
2.4 所有机器关闭防火墙 systemctl stop firewalld #关闭 systemctl disable firewalld #开机不自启 systemctl status firewalld #查看状态 2.5 所有机器关闭selinux sed -i s/enforcing/disabled/ /etc/selinux/config setenforce 0 systemctl status firewalld 2.6 所有机器关闭swap swapoff -a # 临时关闭 sed -ri s/.*swap.*/#/ /etc/fstab #永久关闭 swapon -s #查看swapon分区 2.7 为所有节点安装docker yum install wget.x86_64 -y rm -rf /etc/yum.repos.d/* wget -O /etc/yum.repos.d/centos7.repo http://mirrors.aliyun.com/repo/Centos-7.repo wget -O /etc/yum.repos.d/epel-7.repo http://mirrors.aliyun.com/repo/epel-7.repo wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo yum install docker-ce-20.10.11 -y systemctl start docker systemctl enable docker docker version 配置docker加速器 mkdir -p /etc/docker tee /etc/docker/daemon.json -EOF { registry-mirrors: [https://2tefyfv7.mirror.aliyuncs.com] } EOF systemctl daemon-reload systemctl restart docker 三、部署k8s集群
3.1 添加主机名与ip的对应关系k8s-master、k8s-node1、k8s-node2
cat /etc/hosts EOF
192.168.163.151 k8s-master
192.168.163.152 k8s-node1
192.168.163.153 k8s-node2
EOF source /etc/profile 3.2 将桥接的ipv4流量传递到iptables的链k8s-master、k8s-node1、k8s-node2 cat /etc/sysctl.d/k8s.conf EOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 EOF 3.3 安装kubeadm、kubelet、kubectlk8s-master、k8s-node1、k8s-node2 cat EOF /etc/yum.repos.d/kubernetes.repo [kubernetes] nameKubernetes baseurlhttps://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled1 gpgcheck1 repo_gpgcheck1 gpgkeyhttps://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF yum install kubelet-1.22.2 kubeadm-1.22.2 kubectl-1.22.2 -y systemctl enable kubelet systemctl start kubelet kubectl version 3.4 修改docker的配置k8s-master、k8s-node1、k8s-node2 #镜像加速 registry-mirrors cat /etc/docker/daemon.json EOF { exec-opts: [native.cgroupdriversystemd], registry-mirrors: [https://2tefyfv7.mirror.aliyuncs.com] } EOF systemctl daemon-reload systemctl restart docker.service systemctl restart kubelet.service systemctl status kubelet.service 这里从节点的kubelet.service状态报codeexited, status1/FAILURE是正常的 集群没有初始化导致的
3.5 部署master节点主节点k8s-master
3.5.1 集群初始化 kubeadm init \ --apiserver-advertise-address192.168.163.151 \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.22.2 \ --control-plane-endpoint k8s-master \ --service-cidr172.16.0.0/16 \ --pod-network-cidr10.244.0.0/16 这段命令是用于将一个工作节点worker node加入到已存在的 Kubernetes 集群中的过程。 kubeadm init 是用来初始化 Kubernetes 集群的命令。下面是一些常用的参数及其说明 --apiserver-advertise-address: 指定用于通告的 API 服务器的 IP 地址。 --apiserver-bind-port: 指定 API 服务器绑定的端口。 --control-plane-endpoint: 指定控制平面的端口。 --image-repository: 设置 Kubernetes 镜像仓库的地址。 --kubernetes-version: 设置 Kubernetes 版本。 --pod-network-cidr: 设置 Pod 网络的 CIDR 范围。 --service-cidr: 设置服务的 CIDR 范围。 1).遇到报错 2).解决办法关闭swap swapoff -a # 临时关闭 sed -ri s/.*swap.*/#/ /etc/fstab #永久关闭 1).遇到报错 Here is one example how you may list all Kubernetes containers running in docker: - docker ps -a | grep kube | grep -v pause Once you have found the failing container, you can inspect its logs with: - docker logs CONTAINERID error execution phase wait-control-plane: couldnt initialize a Kubernetes cluster To see the stack trace of this error execute with --v5 or higher 2).解决办法 rm -rf /etc/containerd/config.toml systemctl restart containerd 如果初始化失败可以重新初始化 kubeadm reset 此命令将删除当前集群的状态信息并使其回到初始状态
初始化成功
记下红框中内容添加节点时需要 3.5.2 按照指示执行 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config 初始化时 该命令有输出 3.5.3 查看kubelet.service状态 systemctl status kubelet.service 3.5.4 查看节点状态为notready kubectl get nodes 3.5.5 安装网络插件
官方文档https://github.com/flannel-io/flannel # 最好手动提前拉取所需镜像 docker pull quay.io/coreos/flannel:v0.14.0 wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml kubectl apply -f kube-flannel.yml kubectl get pods --all-namespaces 3.5.6 添加node节点
在k8s-node1和k8s-node1节点执行 docker pull quay.io/coreos/flannel:v0.14.0 kubeadm join k8s-master:6443 --token yt11th.b5wzq4lkjxcsljpg \ --discovery-token-ca-cert-hash sha256:f36824a14f1acb9413246e8f5f0cdf5f6060066f7e4eec6d27edd69004499f9f 在主节点k8s-master查看节点状态为notready 四、部署Gitlab192.168.163.155
4.1 配置docker加速器 mkdir -p /etc/docker tee /etc/docker/daemon.json -EOF { registry-mirrors: [https://2tefyfv7.mirror.aliyuncs.com] } EOF systemctl daemon-reload systemctl restart docker 4.2 安装并配置Gitlab docker pull beginor/gitlab-ce 4.2.1 创建共享卷目录 mkdir -p /data/gitlab/etc/ /data/gitlab/log/ /data/gitlab/data chmod 777 /data/gitlab/etc/ /data/gitlab/log/ /data/gitlab/data/ 4.2.2 创建 gitlab 容器 docker run -itd --namegitlab --restartalways --privilegedtrue -p 8443:443 -p 80:80 -p 222:22 -v /data/gitlab/etc:/etc/gitlab -v /data/gitlab/log:/var/log/gitlab -v /data/gitlab/data:/var/opt/gitlab beginor/gitlab-ce docker ps 切记:这里的端口要设置成80要不push项目会提示没有报错如果宿主机端口被占用需要把这个端口腾出来
4.2.3 关闭容器修改配置文件 docker stop gitlab cat /data/gitlab/etc/gitlab.rb |grep external_url sed -i /external_url GENERATED_EXTERNAL_URL/a external_url\thttp://192.168.163.155 /data/gitlab/etc/gitlab.rb cat /data/gitlab/etc/gitlab.rb |grep external_url external_url ‘http://192.168.163.155’ gitlab_rails[‘gitlab_ssh_host’] ‘192.168.163.155’’ cat /data/gitlab/etc/gitlab.rb |grep gitlab_ssh_host sed -i /gitlab_ssh_host/a gitlab_rails[gitlab_ssh_host] 192.168.163.155 /data/gitlab/etc/gitlab.rb cat /data/gitlab/etc/gitlab.rb |grep gitlab_ssh_host gitlab_rails[gitlab_shell_ssh_port] 222 cat /data/gitlab/etc/gitlab.rb | grep gitlab_shell_ssh sed -i /gitlab_shell_ssh_port/a gitlab_rails[gitlab_shell_ssh_port] 222 /data/gitlab/etc/gitlab.rb cat /data/gitlab/etc/gitlab.rb | grep gitlab_shell_ssh vim /data/gitlab/data/gitlab-rails/etc/gitlab.yml 4.2.4 修改完配置文件之后。直接启动容器 docker start gitlab docker ps 在宿主机所在的物理机访问http://192.168.163.155/ 会自动跳转到修改密码(root用户),如果密码设置的没有满足一定的复杂性则会报500需要从新设置
4.2.5 新建SSH密钥
1.进入gitlab容器 docker exec -it gitlab /bin/sh 2.生成密钥 ssh-keygen 一路回车就可以完成 创建SSH 密钥可以看到 ~/.ssh目录下生成了 id_rsa 和 id_rsa.pub 两个文件
3.查看密钥 cat ~/.ssh/id_rsa.pub 4.gitlab配置密钥 5.推出容器 exit 五、部署harbor192.168.163.156
5.1 安装docker-compose yum install -y docker-compose 配置docker加速器 mkdir -p /etc/docker tee /etc/docker/daemon.json -EOF { registry-mirrors: [https://2tefyfv7.mirror.aliyuncs.com] } EOF systemctl daemon-reload systemctl restart docker 5.2 使用安装包安装 harbor
harbor安装包harbor 解压安装harbor tar -zxvf harbor-offline-installer-v2.0.6.tgz docker load -i harbor/harbor.v2.0.6.tar.gz 5.3 修改配置文件 cd harbor/ cp harbor.yml.tmpl harbor.yml vim harbor.yml harbor.yml设置IP和用户名密码 注释写https
5.4 执行./prepare ./install.sh 初始化harbor ./prepare ./install.sh [rootharbor ~]# cd harbor/ [rootharbor harbor]# cp harbor.yml.tmpl harbor.yml [rootharbor harbor]# vim harbor.yml [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# ./prepare prepare base dir is set to /root/harbor ERROR:root:Error: The protocol is https but attribute ssl_cert is not set Error happened in config validation... [rootharbor harbor]# ls common common.sh harbor.v2.0.6.tar.gz harbor.yml harbor.yml.tmpl input install.sh LICENSE prepare [rootharbor harbor]# prepare bash: prepare: 未找到命令... [rootharbor harbor]# ./prepare prepare base dir is set to /root/harbor Error happened in config validation... ERROR:root:Error: The protocol is https but attribute ssl_cert is not set [rootharbor harbor]# vim harbor.yml [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# ./prepare prepare base dir is set to /root/harbor Error happened in config validation... ERROR:root:Error: The protocol is https but attribute ssl_cert is not set [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# vim harbor.yml [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# clear [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# ./prepare prepare base dir is set to /root/harbor WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https Generated configuration file: /config/log/logrotate.conf Generated configuration file: /config/log/rsyslog_docker.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/registryctl/config.yml Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml Generated and saved secret to file: /data/secret/keys/secretkey Successfully called func: create_root_cert Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# [rootharbor harbor]# ./install.sh [Step 0]: checking if docker is installed ... Note: docker version: 26.1.3 [Step 1]: checking docker-compose is installed ... Note: docker-compose version: 1.18.0 [Step 2]: loading Harbor images ... Loaded image: goharbor/notary-server-photon:v2.0.6 Loaded image: goharbor/clair-photon:v2.0.6 Loaded image: goharbor/clair-adapter-photon:v2.0.6 Loaded image: goharbor/harbor-portal:v2.0.6 Loaded image: goharbor/harbor-core:v2.0.6 Loaded image: goharbor/harbor-db:v2.0.6 Loaded image: goharbor/harbor-jobservice:v2.0.6 Loaded image: goharbor/redis-photon:v2.0.6 Loaded image: goharbor/notary-signer-photon:v2.0.6 Loaded image: goharbor/harbor-log:v2.0.6 Loaded image: goharbor/harbor-registryctl:v2.0.6 Loaded image: goharbor/trivy-adapter-photon:v2.0.6 Loaded image: goharbor/chartmuseum-photon:v2.0.6 Loaded image: goharbor/prepare:v2.0.6 Loaded image: goharbor/nginx-photon:v2.0.6 Loaded image: goharbor/registry-photon:v2.0.6 [Step 3]: preparing environment ... [Step 4]: preparing harbor configs ... prepare base dir is set to /root/harbor WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https Clearing the configuration file: /config/log/logrotate.conf Clearing the configuration file: /config/log/rsyslog_docker.conf Clearing the configuration file: /config/nginx/nginx.conf Clearing the configuration file: /config/core/env Clearing the configuration file: /config/core/app.conf Clearing the configuration file: /config/registry/passwd Clearing the configuration file: /config/registry/config.yml Clearing the configuration file: /config/registryctl/env Clearing the configuration file: /config/registryctl/config.yml Clearing the configuration file: /config/db/env Clearing the configuration file: /config/jobservice/env Clearing the configuration file: /config/jobservice/config.yml Generated configuration file: /config/log/logrotate.conf Generated configuration file: /config/log/rsyslog_docker.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/registryctl/config.yml Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml Creating harbor-log ... done Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir Creating harbor-db ... done Creating harbor-core ... done Creating network harbor_harbor with the default driver Creating nginx ... done Creating harbor-portal ... Creating redis ... Creating registryctl ... Creating registry ... Creating harbor-db ... Creating harbor-core ... Creating harbor-jobservice ... Creating nginx ... ✔ ----Harbor has been installed and started successfully.---- [rootharbor harbor]# 5.5 查看相关镜像 docker ps 5.6 访问测试
http://192.168.163.156 5.7 CI服务器的docker配置jenkins:192.168.163.155
这里因为我们要在192.168.163.155(CI服务器)上push镜像到192.168.163.156(私仓)所有需要修改CI服务器上的Docker配置。 cat /etc/docker/daemon.json vim /etc/docker/daemon.json [rootjenkins ~]# cat /etc/docker/daemon.json { registry-mirrors: [https://2tefyfv7.mirror.aliyuncs.com], insecure-registries: [192.168.163.156] } [rootjenkins ~]# 加载使其生效 systemctl daemon-reload systemctl restart docker CI机器简单测试一下 docker login 192.168.163.156 测试上传镜像 docker tag beginor/gitlab-ce 192.168.163.156/library/gitlab-ce #beginor/gitlab-ce为本地镜像 可以 自定义 docker images docker push 192.168.163.156/library/gitlab-ce 5.8 CD服务器的docker配置k8s-master、k8s-node1、k8s-node2 cat /etc/docker/daemon.json vim /etc/docker/daemon.json cat /etc/docker/daemon.json 加载使其生效 systemctl daemon-reload systemctl restart docker CI机器简单测试一下 5.9 harbor开机启动
编辑/lib/systemd/system/harbor.service vim /lib/systemd/system/harbor.service [Unit] DescriptionHarbor Afternetwork.service docker.service systemd-networkd.service systemd-resolved.service Requiresdocker.service Documentationhttp://github.com/vmware/harbor [Service] Typesimple Restarton-failure RestartSec5 ExecStart/usr/bin/docker-compose -f /root/harbor/docker-compose.yml up ExecStop/usr/bin/docker-compose -f /root/harbor/docker-compose.yml down [Install] WantedBymulti-user.target 重载配置 chmod 755 /lib/systemd/system/harbor.service systemctl restart harbor systemctl status harbor systemctl enable harbor 六、安装配置jenkinsjenkins:192.168.163.155
6.1 镜像jenkins拉取 docker pull jenkins/jenkins:2.457 6.2 创建共享卷修改所属组和用户,和容器里相同 mkdir /jenkins chown 1000:1000 /jenkins # 这里为什么要改成 1000是因为容器里是以 jenkins 用户的身份去读写数据而在容器里jenkins 的 uid 是 1000
6.3 创建 jenkins 容器 docker run -dit -p 8080:8080 -p 50000:50000 --name jenkins --privilegedtrue --restartalways -v /jenkins:/var/jenkins_home jenkins/jenkins:2.457 docker ps | grep jenkins 访问jenkins http://192.168.163.155:8080
关闭Jenkins容器因为要修改配置 docker stop jenkins 6.4 配置jenkins环境
更换国内清华大学镜像,Jenkins下载插件特别慢更换国内的清华源的镜像地址会快不少 cat /jenkins/hudson.model.UpdateCenter.xml sed -i s#updates.jenkins.io/update-center.json#mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json#g /jenkins/hudson.model.UpdateCenter.xml cat /jenkins/hudson.model.UpdateCenter.xml “https://www.google.com/” 替换为 “https://www.baidu.com/” yum install -y jq cat /jenkins/updates/default.json | jq .connectionCheckUrl cat /jenkins/updates/default.json | jq keys 替换 sed -i s#https://www.google.com/#https://www.baidu.com/#g /jenkins/updates/default.json 重启docker获取登录密匙 docker start jenkins cat /jenkins/secrets/initialAdminPassword 需要修改jenkins绑定的docker的启动参数ExecStart/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd/run/containerd/containerd.sock vim /lib/systemd/system/docker.service 修改镜像库启动参数后需要重启docker systemctl daemon-reload systemctl restart docker 6.5 配置Jenkins 安装插件
安装和配置docker插件
依此点击Manage Jenkins-Manage Plugins-AVAILABLE-Search 搜索docker、docker-build-step 需要修改harbor(192.168.163.156)的docker的启动参数ExecStart/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd/run/containerd/containerd.sock vim /lib/systemd/system/docker.service 修改镜像库启动参数后需要重启docker systemctl daemon-reload systemctl restart docker 以上docker配置是为了Build / Publish Docker Image时使用定义了用那个服务器的docker执行docker命令
jenkins 安全设置
后面 gitlab 要和 jenkins 进行联动所以必须要需要对 jenkins 的安全做一些设置依次点击 系统管理-全局安全配置-授权策略勾选匿名用户具有可读权限
七、Jenkins发布打包成Docker的spring boot项目到K8s集群
7.1 创建spring boot 项目 pom.xml ?xml version1.0 encodingUTF-8? project xmlnshttp://maven.apache.org/POM/4.0.0 xmlns:xsihttp://www.w3.org/2001/XMLSchema-instance xsi:schemaLocationhttp://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd modelVersion4.0.0/modelVersion parent groupIdorg.springframework.boot/groupId artifactIdspring-boot-starter-parent/artifactId version2.3.7.RELEASE/version /parent groupIdcom.example/groupId artifactIddemo/artifactId version0.0.1-SNAPSHOT/version namedemo/name descriptiondemo/description properties java.version1.8/java.version java.version1.8/java.version !--换成你的harbor仓库地址-- docker.repostory192.168.163.156/docker.repostory !--换成你的仓库项目名称-- docker.registry.namedemo/docker.registry.name docker.image.tag0.0.1/docker.image.tag docker.maven.plugin.version1.4.10/docker.maven.plugin.version /properties dependencies dependency groupIdorg.springframework.boot/groupId artifactIdspring-boot-starter-web/artifactId /dependency dependency groupIdorg.projectlombok/groupId artifactIdlombok/artifactId optionaltrue/optional /dependency dependency groupIdorg.springframework.boot/groupId artifactIdspring-boot-starter-test/artifactId scopetest/scope /dependency /dependencies build finalNamedemo/finalName plugins plugin groupIdorg.springframework.boot/groupId artifactIdspring-boot-maven-plugin/artifactId configuration excludes exclude groupIdorg.projectlombok/groupId artifactIdlombok/artifactId /exclude /excludes /configuration /plugin plugin groupIdcom.spotify/groupId artifactIddockerfile-maven-plugin/artifactId version${docker.maven.plugin.version}/version !--以下配置依赖docker环境 maven编译打包时会生成docker镜像 本地编译时先注释掉 上传git时在打开-- executions execution iddefault/id goals goalbuild/goal goalpush/goal /goals /execution /executions configuration contextDirectory${project.basedir}/contextDirectory useMavenSettingsForAuthtrue/useMavenSettingsForAuth repository${docker.repostory}/${docker.registry.name}/${project.artifactId}/repository !--换成你的harbor仓库账号密码-- usernameadmin/username passwordroot123456/password tag${docker.image.tag}/tag buildArgs JAR_FILEtarget/${project.build.finalName}.jar/JAR_FILE /buildArgs /configuration /plugin /plugins resources !-- 指定 src/main/resources下所有文件及文件夹为资源文件 -- resource directorysrc/main/resources/directory targetPath${project.build.directory}/classes/targetPath includes include**/*/include /includes filteringtrue/filtering /resource /resources /build /project 创建测试接口 package com.example.demo.api; import lombok.AllArgsConstructor; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; /** * author lgp * description: 测试PI * date 2023/9/6 */ RestController AllArgsConstructor RequestMapping(/api) public class testApi { GetMapping(/test) public String test() { return 测试1112233552219122120911; } } 启动项目
访问测试接口
添加Dockerfile文件 该文件时docker创建容器的文件名称不可改变
#添加依赖环境前提是将Java8的Docker镜像从官方镜像仓库pull下来然后上传到自己的Harbor私有仓库中 FROM 192.168.163.156/library/java:8
#指定镜像制作作者可自己随意设置 MAINTAINER admin #运行目录 VOLUME /tmp #将本地的文件拷贝到容器一般在项目的target目录下要根据项目自己修改 ADD target/*jar app.jar #启动容器后自动执行的命令 ENTRYPOINT [ java, -Djava.security.egdfile:/dev/./urandom, -jar, /app.jar ] 添加demo.yaml文件
该文件是k8s创建pod的配置文件 apiVersion: apps/v1 kind: Deployment metadata: name: demo labels: app: demo spec: replicas: 1 selector: matchLabels: app: demo template: metadata: labels: app: demo spec: imagePullSecrets: #使用imagePullSecrets参数指定镜像拉取秘钥 - name: harbor-registry #使用我们的secret即harbor-registry containers: - name: demo image: 192.168.163.156/demo/demo:0.0.1 ports: - containerPort: 8080 #修改成你项目的端口 imagePullPolicy: Always #镜像拉取策略每次都拉去最新的镜像 --- apiVersion: v1 kind: Service metadata: name: demo labels: app: demo spec: ports: - name: http port: 8080 #修改成你项目的端口 nodePort: 30001 ##容器暴露的访问端口要确定无冲突否则需修改 type: NodePort selector: app: demo 项目上传到gitlab
7.2 拉取java8镜像
在jenkins服务器(192.168.163.155)上拉取java8镜像然后推送至harbor仓库一会打包构建docker镜像的时候会用到。 docker pull java:8 docker tag java:8 192.168.163.156/library/java:8 docker login 192.168.163.156 docker push 192.168.163.156/library/java:8 7.3 配置 jenkins容器到其他服务的ssh免密登录
jenkins执行构建命令时需要ssh访问jenkins容器所在服务器192.168.163.155和k8s主节点服务器192.168.163.151所以需要配置ssh免密登录
在jenkins容器所在服务器192.168.163.155上进入jenkins容器 docker exec -it jenkins bash 生产密钥 ssh-keygen -t rsa 将公钥复制到jenkins容器所在服务器192.168.163.155 ssh-copy-id root192.168.163.155 将公钥复制到k8s主节点服务器192.168.163.151)
7.4 k8s 创建私有镜像仓库Secretk8s主节点服务器192.168.163.151
k8s需要从harbor私有进行仓库拉取镜像时需要创建Secret配置harbor私有镜像仓库地址、账号密码
#docker-registry是关键字 harbor-registry是自定义的名称与demo.yaml中的imagePullSecrets.name值一致 kubectl -n default create secret docker-registry harbor-registry --docker-emailbaidu.comexample --docker-usernameroot --docker-passwordroot123456 --docker-server192.168.163.156 kubectl get secret harbor-registry #查看secret 7.5 安装jdk和maven(jenkins:192.168.163.155)
安装mvn
cd /usr/local/ wget https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz tar -zxvf apache-maven-3.6.3-bin.tar.gz 安装jdk
yum install -y java-1.8.0-openjdk-devel 配置环境变量
vim /etc/profile MAVEN_HOME/usr/local/apache-maven-3.6.3 JAVA_HOME/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-1.el7_9.x86_64 ##这里的路径注意使用你安装的jdk的版本号 CLASS_PATH.:$JAVA_HOME/lib PATH$MAVEN_HOME/bin:$JAVA_HOME/bin:$PATH 刷新环境变量
source /etc/profile 检查是否安装成功
mvn -v java -version 7.6 使用jenkins构建发布项目 #使用jenkins宿主机的maven打包 /jenkins/workspace是jenkins 拉取git项目的目录 创建jenkins容器时已经做好了映射 ssh -tt root192.168.163.155 docker rmi 192.168.163.156/demo/demo:0.0.1 /usr/local/apache-maven-3.6.3/bin/mvn -f /jenkins/workspace/demo/pom.xml clean install -Dmaven.test.skiptrue ssh -tt root192.168.163.155 docker login 192.168.163.156 docker push 192.168.163.156/demo/demo:0.0.1 scp /var/jenkins_home/workspace/demo/demo.yaml root192.168.163.151:/ ssh -tt root192.168.163.151 /usr/bin/kubectl delete -f /demo.yaml #第一次执行时 由于没有该pod 会报错 所以第一次先注释掉 ssh -tt root192.168.163.151 /usr/bin/kubectl apply -f /demo.yaml 镜像仓库
查看k8s pod
kubectl get pod --all-namespaces 访问应用192.168.163.151:30001/api/test
方式二 构建 比较麻烦 依赖于 6.5中的安装和配置docker插件 分为三步 1.执行 shell maven编译生成jar包 2.Build / Publish Docker Image 生成镜像并上传至镜像仓库 3.执行 shell 远程k8s更新pod
以上项目部署CI/CD 还需要手动在jenkins中构建项目下面整合Jenkins和gitlab使用gitlab的webhook功能实现代码提交到gitlab后 自动构建部署项目
7.7 配置 gitlab 和 jenkins 的联动
Jenkins安装gitlab插件
修改项目构建配置
gitlab配置
回到需要自动构建的项目
保存后测试
返回200说明成功
验证下自动构建是否成功
现在是第39次构建修改代码提交gitlab
提交后开始自动构建
访问应用192.168.163.151:30001/api/test
至此搭建GitlabJenkinsDockerHarborK8s集群搭建CICD平台完成
