从源码部署基于 cri-o 的 Kubernetes 集群

2018, Aug 02    

系统环境

系统版本：Fedora Release 28
内核版本：4.17.9-200.fc28.x86_64

从源码构建必要组件

安装依赖和工具：

$ dnf install git go make libseccomp-devel gpgme-devel libassuan-devel btrfs-progs-devel device-mapper-devel ostree-devel glibc-static rsync tar

下载源码并安装组件：

runc:

$ mkdir -p ~/go/src/github.com/opencontainers
$ cd  ~/go/src/github.com/opencontainers
$ git clone https://github.com/opencontainers/runc
$ make && make install

cri-o：

$ mkdir -p ~/go/src/github.com/kubernetes-incubator
$ cd ~/go/src/github.com/kubernetes-incubator
$ git clone https://github.com/kubernetes-incubator/cri-o
$ make install.tools && make binaries && make install.bin
$ mkdir /etc/crio
$ mkdir -p /etc/containers/oci/hooks.d

cni 插件：

$ mkdir -p ~/go/src/github.com/containernetworking
$ cd ~/go/src/github.com/containernetworking
$ git clone https://github.com/containernetworking/plugins
$ ./build.sh
$ mkdir -p /opt/cni/bin
$ cp bin/* /opt/cni/bin/
$ mkdir -p /etc/cni/net.d

hyperkube：

$ mkdir -p ~/go/src/k8s.io
$ cd -p ~/go/src/k8s.io
$ git clone https://github.com/kubernetes/kubernetes
$ make all WHAT=cmd/hyperkube
$ cp _output/hyperkube /usr/local/bin/
$ mkdir -p /etc/kubernetes

etcd：

$ mkdir -p ~/go/src/github.com/coreos
$ cd ~/go/src/github.com/coreos
$ git clone https://github.com/coreos/etcd
$ ./build
$ cp ./bin/*  /usr/local/bin/

配置并启动集群

cni 配置

/etc/cni/net.d/99-loopback.conf：

{
    "cniVersion": "0.3.1",
    "type": "loopback"
}

/etc/cni/net.d/10-mynet.conf：

{
    "cniVersion": "0.3.1",
    "name": "mynet",
    "type": "bridge",
    "bridge": "cni0",
    "isGateway": true,
    "ipMasq": true,
    "ipam": {
        "type": "host-local",
        "subnet": "10.88.0.0/16",
        "routes": [
            {"dst": "0.0.0.0/0"}
        ]
    }
}

oci 配置

/etc/containers/policy.json：

{
    "default": [
        {
            "type": "insecureAcceptAnything"
        }
    ],
    "transports": {
        "docker": {}
    }
}

crio 配置

/etc/crio/crio.conf 可以使用命令 “crio config –default > /etc/crio/crio.conf” 创建，然后修改 registries 即可：

registries = [
	"docker.io"
]

/etc/crio/seccomp.json 配置来自于 github.com/kubernetes-incubator/cri-o/seccomp.json，不再列出。

kubernetes 配置

/etc/kubernetes/kubeconfig.yaml：

apiVersion: v1
kind: Config
clusters:
- cluster:
    server: http://127.0.0.1:8080
contexts: []
current-context: ""
preferences: {}
users: []

启动各个组件：

crio &
etcd —data-dir /root/default.etcd &
hyperkube apiserver --etcd-servers=http://127.0.0.1:2379 &
hyperkube kubelet --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --kubeconfig=/etc/kubernetes/kubeconfig.yaml &
hyperkube controller-manager  --kubeconfig=/etc/kubernetes/kubeconfig.yaml &
hyperkube scheduler  --kubeconfig=/etc/kubernetes/kubeconfig.yaml &
hyperkube proxy  --kubeconfig=/etc/kubernetes/kubeconfig.yaml &

创建 secret

由于 default 分区默认创建的 service account 没有关联 secret，导致无法创建 Pod。

创建 secret：

apiVersion: v1
kind: Secret
metadata:
  annotations:
    kubernetes.io/service-account.name: default
  name: default-token
  namespace: default
type: kubernetes.io/service-account-token

修改 default 分区默认的 service account 的 secrets：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: default
secrets:
- name: default-token

运行应用

创建 deployment：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
spec:
  replicas: 1 
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

然后通过 “hyperkube kubectl get pod” 即可查看创建出来的 Pod。

其他

skopeo：

$ mkdir -p ~/go/src/github.com/projectatomic
$ cd ~/go/src/github.com/projectatomic
$ git clone https://github.com/containers/skopeo
$ make binary-local && make install-binary

skopeo 可以用来查看和下载镜像：

skopeo inspect docker://docker.io/fedora
skopeo --insecure-policy copy docker://internal-registry.com/library/server:v1.0.0 dir:./server/ --src-creds username:password

coredns：

$ mkdir -p ~/go/src/github.com/coredns
$ cd ~/go/src/github.com/coredns
$ git clone https://github.com/coredns/coredns
$ make
Twitter Facebook Google+
# Kubernetes