目录

Kubernetes搭建记录📝

Kubernetes搭建记录📝

工具🔧准备

安装kubectl

Kubernetes 命令行工具,kubectl,使得你可以对 Kubernetes 集群运行命令。 你可以使用 kubectl 来部署应用、监测和管理集群资源以及查看日志。

话不多说上包管理器
Snap

1
snap install kubectl --classic
1
brew install kubectl

考虑到后期经常需要敲kubectl,可以设置一个别名,同时配置shell补全功能

1
2
echo 'alias k=kubectl' >>~/.zshrc
echo 'complete -F __start_kubectl k' >>~/.zshrc

本地Kubernetes搭建

Docker for Desktop内K8s

通过Preferences开启Kubernetes,在Kubernetes中设置enable即可, 之后会pull images,但有极大可能遇到images下载失败导致一直处于starting状态,建议移除相关镜像重下,不过本次尝试下来似乎移除重启Docker没有效果。

https://libget.com/gkirito/blog/image/2021/image-20210304sbKUz3ye%402x.png

出现情况:一直处于这个状态

https://libget.com/gkirito/blog/image/2021/image-20210304OFCpQ5Fz%402x.png

重试几次未好在Issues中找到一位大佬操作:

https://libget.com/gkirito/blog/image/2021/image-20210304IrjrX6b9%402x.png

直接暴力点,移除~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/ 下的Docker.raw文件
警告
Docker.raw为docker image文件存放位置,删除会导致所有其他镜像的删除,但是能解决问题(至少我是这样最后成功run了Kubernetes)

minikube

安装

1
brew install minikube

开启

1
minikube start

等待镜像pull完,一个小型k8s集群就搭建好了, 启动完成后,kubectl的config就会自动切到minikube,这时我们便可以进行正常操作了

https://libget.com/gkirito/blog/image/2021/image-20210304d9GXAqPS%402x.png

可以通过该命令来查看k8s dashboard

1
minikube dashboard

https://libget.com/gkirito/blog/image/2021/image-20210304qLyTQJrr%402x.png

可以尝试创建一个Deployment,可以通过指定replicas来设置pod

1
kubectl create deployment hello-node --image=k8s.gcr.io/echoserver:1.4 --replicas=2

然后可以通过查看Deployment和pod看到结果

1
kubectl get deployments,pods

https://libget.com/gkirito/blog/image/2021/image-20210304LlCgMxEh%402x.png

默认情况下,Pod 只能通过 Kubernetes 集群中的内部 IP 地址访问。 要使得 hello-node 容器可以从 Kubernetes 虚拟网络的外部访问,你必须将 Pod 暴露为 Kubernetes Service。

1
kubectl expose deployment hello-node --type=LoadBalancer --port=8080

这时查看Service,便能看到端口暴露信息

1
kubectl get services

https://libget.com/gkirito/blog/image/2021/image-202103049uzDZEGo%402x.png

对于支持负载均衡器的云服务平台而言,平台将提供一个外部 IP 来访问该服务。 在 Minikube 上,LoadBalancer 使得服务可以通过命令 minikube service 访问。

1
minikube service hello-node

https://libget.com/gkirito/blog/image/2021/image-20210304WBVSE96J%402x.png

https://libget.com/gkirito/blog/image/2021/image-20210304wROeku1e%402x.png

补充:
Minikube 有一组内置的 插件, 可以在本地 Kubernetes 环境中启用、禁用和打开

1
minikube addons list

https://libget.com/gkirito/blog/image/2021/image-20210304O9ZrdZyu%402x.png
启用与禁用插件

1
2
minikube addons enable metrics-server
minikube addons disable metrics-server

最后可以用k8s删除service和deployment,暂停minikube

1
2
3
4
kubectl delete service hello-node
kubectl delete deployment hello-node

minikube stop

AWS上Kubernetes搭建(EKS方案)

AWS平台工具🔧安装

AWS CLI 版本 2

  1. 下载对应系统的AWS CLI https://awscli.amazonaws.com/AWSCLIV2.pkg

  2. 安装完以后,在Terminal终端验证

    1
    2
    
    which aws
    aws --version
    

    https://libget.com/gkirito/blog/image/2021/image-20210304MKrh1STt%402x.png

  3. 配置 AWS CLI

    配置前准备:
    首先配置AWS CLI需要aws access keyaws secret access key,由于root账号没法获得这个,所以需要去IAM建立user

    1. 首先去Groups创建一个新的Group,在Permissions上,可以把EKS(Elastic Kubernetes Service)相关的都选上

    2. Users添加一个新的user,填入名字;把Programmatic accessAWS Management Console access都选上,可做如下配置:

      https://libget.com/gkirito/blog/image/2021/image-20210305MR2Hy31F%402x.png

    3. 下一步勾选之前创建的Group;之后再往下一步,设置Tags(随意)

    4. 确定之后点击创建

    注意:创建成功后需要记住Secret access keyPassword(特别是password自动生成情况),这两种之后便无法再次查看,同时下载 .csv 文件备份;点击完成后,还要再去Users下,点击创建的user,获得User ARN并保存

    https://libget.com/gkirito/blog/image/2021/image-20210305s1GkRwsw%402x.png

    根据上面方式获得aws access keyaws secret access key后,在Terminal上执行

    1
    
    aws configure
    

    https://libget.com/gkirito/blog/image/2021/image-20210305pAYmjSiO%402x.png

    根据要求填入即可

eksctl

安装

1
brew install weaveworks/tap/eksctl

并通过

1
eksctl version

确定安装成功

通过平台面板创建EKS

  1. 先登录之前创建的子账号(IAM用户)

    https://libget.com/gkirito/blog/image/2021/image-20210305MaEbyGnk%402x.png

    账户ID(12位)即之前保存的User ARN中间的12位数字

  2. 登录以后选择IAM,在Roles下添加新的role,选择EKS –> EKS-Cluster,点击下一步添加权限,权限默认即可,之后添加Tags,最后为这个role添加名字

  3. 创建VPCIPv4 CIDR block可以填:192.168.0.0/16;之后在VPC下创建Subnets,选择之前创建的VPC,在IPv4 CIDR block中选择一个192.168下的段地址,然后重复创建多个,分别用不同的段,如:192.168.0.0/24、192.168.10.0/24、192.168.20.0/24

    https://libget.com/gkirito/blog/image/2021/image-20210305izQM4JKA%402x.png

  4. 创建Security Groups,先创建EKS需要的基础,Outbound rules中默认即可,Inbound rules中随机选一个,然后点击创建,再点击Edit inbound rules,修改source选择刚刚创建的Security Groups,保存

  5. 选择Elastic Kubernetes Service,输入Cluster name后点击下一步,选择Kubernetes版本、Cluster Service Role选择刚刚创建的role,然后下一步;选择之前创建的VPC相关内容,Cluster endpoint access选为public,之后继续下一步,Control Plane Logging上可选都打开;最后review确定没有问题后选择创建

    https://libget.com/gkirito/blog/image/2021/image-20210305czZUfRqP%402x.png

通过eksctl创建EKS

eksctl创建会比之前手动面板创建方便不少,根据需求填入相应参数,就会自动创建,可以不用再关注于VPC、IAM等role等方面。
可以通过

1
eksctl create cluster -h

获知一些相关参数信息

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
Create a cluster

Usage: eksctl create cluster [flags]

General flags:
  -n, --name string               EKS cluster name (generated if unspecified, e.g. "hilarious-wardrobe-1614914845")
      --tags stringToString       Used to tag the AWS resources. List of comma separated KV pairs "k1=v1,k2=v2" (default [])
  -r, --region string             AWS region
      --with-oidc                 Enable the IAM OIDC provider
      --zones strings             (auto-select if unspecified)
      --version string            Kubernetes version (valid options: 1.15, 1.16, 1.17, 1.18, 1.19) (default "1.18")
  -f, --config-file string        load configuration from a file (or stdin if set to '-')
      --timeout duration          maximum waiting time for any long-running operation (default 25m0s)
      --install-vpc-controllers   Install VPC controller that's required for Windows workloads
      --fargate                   Create a Fargate profile scheduling pods in the default and kube-system namespaces onto Fargate

Initial nodegroup flags:
      --nodegroup-name string          name of the nodegroup (generated if unspecified, e.g. "ng-87f0eb81")
      --without-nodegroup              if set, initial nodegroup will not be created
  -t, --node-type string               node instance type
  -N, --nodes int                      total number of nodes (for a static ASG) (default 2)
  -m, --nodes-min int                  minimum nodes in ASG (default 2)
  -M, --nodes-max int                  maximum nodes in ASG (default 2)
      --node-volume-size int           node volume size in GB (default 80)
      --node-volume-type string        node volume type (valid options: gp2, gp3, io1, sc1, st1) (default "gp3")
      --max-pods-per-node int          maximum number of pods per node (set automatically if unspecified)
      --ssh-access                     control SSH access for nodes. Uses ~/.ssh/id_rsa.pub as default key path if enabled
      --ssh-public-key string          SSH public key to use for nodes (import from local path, or use existing EC2 key pair)
      --enable-ssm                     Enable AWS Systems Manager (SSM)
      --node-ami string                'auto-ssm', 'auto', 'static' (deprecated) or an AMI id (advanced use)
      --node-ami-family string         'AmazonLinux2' for the Amazon EKS optimized AMI, or use 'Ubuntu2004' or 'Ubuntu1804' for the official Canonical EKS AMIs (default "AmazonLinux2")
  -P, --node-private-networking        whether to make nodegroup networking private
      --node-security-groups strings   attach additional security groups to nodes
      --node-labels stringToString     extra labels to add when registering the nodes in the nodegroup. List of comma separated KV pairs "k1=v1,k2=v2" (default [])
      --node-zones strings             (inherited from the cluster if unspecified)
      --instance-prefix string         add a prefix value in front of the instance's name
      --instance-name string           overrides the default instance's name
      --disable-pod-imds               Blocks IMDS requests from non host networking pods
      --managed                        Create EKS-managed nodegroup
      --spot                           Create a spot nodegroup (managed nodegroups only)
      --instance-types strings         Comma-separated list of instance types (e.g., --instance-types=c3.large,c4.large,c5.large

Cluster and nodegroup add-ons flags:
      --install-neuron-plugin    install Neuron plugin for Inferentia nodes (default true)
      --install-nvidia-plugin    install Nvidia plugin for GPU nodes (default true)
      --asg-access               enable IAM policy for cluster-autoscaler
      --external-dns-access      enable IAM policy for external-dns
      --full-ecr-access          enable full access to ECR
      --appmesh-access           enable full access to AppMesh
      --appmesh-preview-access   enable full access to AppMesh Preview
      --alb-ingress-access       enable full access for alb-ingress-controller

VPC networking flags:
      --vpc-cidr ipNet                 global CIDR to use for VPC (default 192.168.0.0/16)
      --vpc-private-subnets strings    re-use private subnets of an existing VPC
      --vpc-public-subnets strings     re-use public subnets of an existing VPC
      --vpc-from-kops-cluster string   re-use VPC from a given kops cluster
      --vpc-nat-mode string            VPC NAT mode, valid options: HighlyAvailable, Single, Disable (default "Single")

AWS client flags:
  -p, --profile string         AWS credentials profile to use (overrides the AWS_PROFILE environment variable)
      --cfn-role-arn string    IAM role used by CloudFormation to call AWS API on your behalf
      --cfn-disable-rollback   for debugging: If a stack fails, do not roll it back. Be careful, this may lead to unintentional resource consumption!

Output kubeconfig flags:
      --kubeconfig string               path to write kubeconfig (incompatible with --auto-kubeconfig) (default "/Users/gkirito/.kube/config")
      --authenticator-role-arn string   AWS IAM role to assume for authenticator
      --set-kubeconfig-context          if true then current-context will be set in kubeconfig; if a context is already set then it will be overwritten (default true)
      --auto-kubeconfig                 save kubeconfig file by cluster name, e.g. "/Users/gkirito/.kube/eksctl/clusters/hilarious-wardrobe-1614914845"
      --write-kubeconfig                toggle writing of kubeconfig (default true)

Common flags:
  -C, --color string   toggle colorized logs (valid options: true, false, fabulous) (default "true")
  -h, --help           help for this command
  -v, --verbose int    set log level, use 0 to silence, 4 for debugging and 5 for debugging with AWS debug logging (default 3)

Use 'eksctl create cluster [command] --help' for more information about a command.

我们可以简单创建一个4个node的集群

1
2
3
4
5
6
7
8
eksctl create cluster \
 --name g4-test \
 --version 1.19 \
 --with-oidc \
 --region ap-southeast-1 \
 --nodes 4 \
 --nodegroup-name g4-node-test \
 --ssh-access

由于创建需要十几分钟,大概接下来一段时间都是waiting状态

https://libget.com/gkirito/blog/image/2021/image-20210305o7nlJwQq%402x.png

创建后kube config配置

可以通过AWS CLI快速切换集群配置

1
aws eks --region ap-southeast-1 update-kubeconfig --name t

https://libget.com/gkirito/blog/image/2021/image-20210305Dfiwbap0%402x.png

AWS上Kubernetes搭建(EC2手动搭建)

通过自建EC2服务器,设置搭建K8s集群

这部分内容过多,还未整理,到目前为止还未完全搭建完成