服装商店的网站建设要求,window服务器如何做网站访问,搜索量最大的关键词,宁波市建设银行网站导语#xff1a; k8s通过psp限制nvidia-plugin插件的使用。刚开始接触psp 记录一下 后续投入生产测试了再完善。
通过apiserver开启psp 静态pod会自动更新
# PSP(Pod Security Policy) 在默认情况下并不会开启。通过将PodSecurityPolicy关键词添加到 --enbale-admission-plu…导语 k8s通过psp限制nvidia-plugin插件的使用。刚开始接触psp 记录一下 后续投入生产测试了再完善。
通过apiserver开启psp 静态pod会自动更新
# PSP(Pod Security Policy) 在默认情况下并不会开启。通过将PodSecurityPolicy关键词添加到 --enbale-admission-plugins 配置数组后可以开启PSP权限认证功能。
# /etc/kubernetes/manifests/kube-apiserver.yaml 在NodeRestriction后添加PodSecurityPolicy
- --enable-admission-pluginsNodeRestriction,PodSecurityPolicy直接创建容器测试
lung.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: lunglabels:k8s-app: lungk8s-med-type: biz-internel
spec:strategy:type: Recreatereplicas: 1selector:matchLabels:k8s-app: lungtemplate:metadata:labels:k8s-app: lungspec:
# runtimeClassName: nvidia
# hostPID: truecontainers:- name: lungimage: nvidia/cuda:11.3.0-base-ubi8command: [sh,-c,tail -f /dev/null ]#command: [sh,-c,for i in ls /srv/conf-drwise220531;do rm -rf /root/lung/$i/conf ln -s /srv/conf-drwise220531/$i/conf /root/lung/$i/ ;done rm -rf /root/lung/Release/path.conf /root/lung/path.conf ln -s /srv/conf-drwise220531/Release/path.conf /root/lung/Release/ ln -s /root/lung/Release/path.conf /root/lung/ sh /root/aiclassifier/startup.sh sh /root/lung/startup.sh ]
# securityContext:
# privileged: trueenv:- name: NVIDIA_DRIVER_CAPABILITIESvalue: compute,utility,video,graphics,display- name: NVIDIA_VISIBLE_DEVICESvalue: allvolumeMounts:- mountPath: /dev/shmname: dshmvolumes:- name: dshmemptyDir:medium: MemorysizeLimit: 1Gi
#deepwise-operator
# serviceAccountName: deepwise-operator# 创建deployment测试 发下会有psp的问题
# 注意开启PodSecurityPolicy功能后即使没有使用任何安全策略都会使得创建pods包括调度任务重新创建pods失败kubectl apply -f lung -n deepwise 创建对应的资源限制策略
4.nvidia-plugin.yaml
# 显卡驱动的限制
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:name: psp-nvidia
spec:privileged: falsefsGroup:rule: RunAsAnyrunAsUser:rule: RunAsAnyseLinux:rule: RunAsAnysupplementalGroups:rule: RunAsAnyvolumes:- *hostPID: falsehostIPC: falsehostNetwork: false---
apiVersion: v1
kind: ServiceAccount
metadata:namespace: kube-systemname: nvidiaoperator---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: psp-permissive-nvidianamespace: kube-system
rules:- apiGroups:- extensionsresources:- podsecuritypoliciesresourceNames:- psp-nvidiaverbs:- use
---apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:name: psp-permissive-nvidianamespace: kube-system
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: psp-permissive-nvidia
subjects:- kind: ServiceAccountname: nvidiaoperatornamespace: kube-system
---
apiVersion: apps/v1
kind: DaemonSet
metadata:name: nvidia-device-plugin-daemonsetnamespace: kube-system
spec:selector:matchLabels:name: nvidia-device-plugin-dsupdateStrategy:type: RollingUpdatetemplate:metadata:# This annotation is deprecated. Kept here for backward compatibility# See https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/annotations:scheduler.alpha.kubernetes.io/critical-pod: labels:name: nvidia-device-plugin-dsspec:runtimeClassName: nvidiatolerations:# This toleration is deprecated. Kept here for backward compatibility# See https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/- key: CriticalAddonsOnlyoperator: Exists- key: nvidia.com/gpuoperator: Existseffect: NoSchedule# Mark this pod as a critical add-on; when enabled, the critical add-on# scheduler reserves resources for critical add-on pods so that they can# be rescheduled after a failure.# See https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/priorityClassName: system-node-criticalcontainers:- image: harbor.deepwise.com/terra-k8s/k8s-device-plugin:v0.10.0name: nvidia-device-plugin-ctrargs: [--fail-on-init-errorfalse]securityContext:allowPrivilegeEscalation: falsecapabilities:drop: [ALL]volumeMounts:- name: device-pluginmountPath: /var/lib/kubelet/device-pluginsvolumes:- name: device-pluginhostPath:path: /var/lib/kubelet/device-pluginsserviceAccountName: nvidiaoperator5.nvida-psp.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:name: psp-nvidia
spec:privileged: falsefsGroup:rule: RunAsAnyrunAsUser:rule: RunAsAnyseLinux:rule: RunAsAnysupplementalGroups:rule: RunAsAnyvolumes:- *hostPID: falsehostIPC: falsehostNetwork: false---
apiVersion: v1
kind: ServiceAccount
metadata:namespace: kube-systemname: nvidiaoperator---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: psp-permissive-nvidianamespace: kube-system
rules:- apiGroups:- extensionsresources:- podsecuritypoliciesresourceNames:- psp-nvidiaverbs:- use
---apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:name: psp-permissive-nvidianamespace: kube-system
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: psp-permissive-nvidia
subjects:- kind: ServiceAccountname: nvidiaoperatornamespace: kube-system6.deepwise-psp.yaml
# 用户的限制
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:name: psp-deepwise
spec:privileged: falsefsGroup:rule: RunAsAnyrunAsUser:rule: RunAsAnyseLinux:rule: RunAsAnysupplementalGroups:rule: RunAsAnyvolumes:- *hostPID: falsehostIPC: falsehostNetwork: false---
apiVersion: v1
kind: ServiceAccount
metadata:namespace: deepwisename: deepwise-operator---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: psp-permissive-deepwisenamespace: deepwise
rules:- apiGroups:- extensionsresources:- podsecuritypoliciesresourceNames:- psp-deepwiseverbs:- use
---apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:name: psp-permissive-deepwisenamespace: deepwise
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: psp-permissive-deepwise
subjects:- kind: ServiceAccountname: deepwise-operatornamespace: deepwise7.runtimeclass.yaml
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:name: nvidia
handler: nvidia如果是docker运行时handler需要调整为docker。使用containerd则不需要调整。参考https://opni.io/setup/gpu/
重新创建lung的deployment
# 加上runtimeClassName: nvidia
# 加上serviceAccountName: deepwise-operator
apiVersion: apps/v1
kind: Deployment
metadata:name: lunglabels:k8s-app: lungk8s-med-type: biz-internel
spec:strategy:type: Recreatereplicas: 1selector:matchLabels:k8s-app: lungtemplate:metadata:labels:k8s-app: lungspec:runtimeClassName: nvidia
# hostPID: truecontainers:- name: lungimage: nvidia/cuda:11.3.0-base-ubi8command: [sh,-c,tail -f /dev/null ]#command: [sh,-c,for i in ls /srv/conf-drwise220531;do rm -rf /root/lung/$i/conf ln -s /srv/conf-drwise220531/$i/conf /root/lung/$i/ ;done rm -rf /root/lung/Release/path.conf /root/lung/path.conf ln -s /srv/conf-drwise220531/Release/path.conf /root/lung/Release/ ln -s /root/lung/Release/path.conf /root/lung/ sh /root/aiclassifier/startup.sh sh /root/lung/startup.sh ]
# securityContext:
# privileged: trueenv:- name: NVIDIA_DRIVER_CAPABILITIESvalue: compute,utility,video,graphics,display- name: NVIDIA_VISIBLE_DEVICESvalue: allvolumeMounts:- mountPath: /dev/shmname: dshmvolumes:- name: dshmemptyDir:medium: MemorysizeLimit: 1Gi
#deepwise-operatorserviceAccountName: deepwise-operator参考文档
https://kubernetes.io/zh-cn/docs/reference/access-authn-authz/admission-controllers/
https://blog.csdn.net/tushanpeipei/article/details/121940757
https://blog.csdn.net/weixin_45081220/article/details/125407608
