Qemu customize

mika published on 2023-04-25 included in Linux

Customize img

OCP etcd metrics

mika published on 2023-04-07 included in Openshift

Recommended etcd practices

https://docs.openshift.com/container-platform/4.12/scalability_and_performance/recommended-host-practices.html#recommended-etcd-practices_recommended-host-practices

The histogram_quantile(0.99, rate(etcd_network_peer_round_trip_time_seconds_bucket[2m])) metric reports the round trip time for etcd to finish replicating the client requests between the members. Ensure that it is less than 50 ms.

Metrics to monitor

https://access.redhat.com/articles/6967785#metrics

Monitor Leadership changes:

This is expected as per result of installation/upgrade process or day1/2 operations (as result of Machine Config daemon operations), but we don’t expect to see it happening during normal operations. etcdHighNumberOfLeaderChanges alert can help us to identify that situation. Prometheus query could also be used (sum(rate(etcd_server_leader_changes_seen_total[2m]))). If happening during normal operation, I/O and network metrics can help us to identify the root cause.

Installer-provisioned installation

mika published on 2023-03-24 included in Openshift

IPI

Get openshift installer, openshift cli and pull-secret from https://console.redhat.com/openshift
Create install-config.yaml $ openshift-install create install-config --dir ./cluster

Here is a sample install-config.yaml for vSphere IPI

additionalTrustBundlePolicy: Proxyonly
apiVersion: v1
baseDomain: example.com
compute:
- architecture: amd64
  hyperthreading: Enabled
  name: worker
  platform: {}
  replicas: 3
controlPlane:
  architecture: amd64
  hyperthreading: Enabled
  name: master
  platform: {}
  replicas: 3
metadata:
  creationTimestamp: null
  name: mmayeras
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  machineNetwork:
  - cidr: 10.10.0.0/24
  networkType: OVNKubernetes
  serviceNetwork:
  - 172.30.0.0/16
platform:
  vsphere:
    apiVIPs:
    - 10.10.0.2
    cluster: your_cluster
    datacenter: your_datacenter
    defaultDatastore: your_datastore
    ingressVIPs:
    - 10.10.0.3
    network: your_network
    password: your_password
    username:  your_username
    vCenter: your_vcenter
publish: External
pullSecret: 'your_pull_secret'
sshKey: your_ssh_pub_key

Backup and Copy the install-config.yaml into the installation dir $ cp install-config.yaml{,.bak} && mv install-config.yaml ./cluster/
Launch the installer $ openshift-install create cluster --dir ./cluster

Openshift docs

mika published on 2023-03-24 included in Openshift

docs.openshift.com

Update Path Lab

Openshift Lifecycle

User-provisioned installation

mika published on 2023-03-24 included in Openshift

UPI

PXE Config

dnf install -y tftp-server syslinux-tftpboot httpd haproxy
wget https://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.tar.gz
wget https://raw.githubusercontent.com/leoaaraujo/openshift_pxe_boot_menu/main/files/bg-ocp.png -O /var/lib/tftpboot/bios/bg-ocp.png
tar xf syslinux-6.03.tar.gz 
cp syslinux-6.03/bios/core/pxelinux.0 /var/lib/tftpboot/bios/
cp syslinux-6.03/bios/com32/elflink/ldlinux/ldlinux.c32 /var/lib/tftpboot/bios/
cp syslinux-6.03/bios/com32/lib/libcom32.c32 /var/lib/tftpboot/bios/
cp syslinux-6.03/bios/com32/libutil/libutil.c32 /var/lib/tftpboot/bios/
cp syslinux-6.03/bios/memdisk/memdisk /var/lib/tftpboot/bios/
cp syslinux-6.03/bios/com32/modules/poweroff.c32 /var/lib/tftpboot/bios/
cp syslinux-6.03/bios/com32/modules/pxechn.c32 /var/lib/tftpboot/bios/
cp syslinux-6.03/bios/com32/modules/reboot.c32 /var/lib/tftpboot/bios/
cp syslinux-6.03/bios/com32/menu/vesamenu.c32 /var/lib/tftpboot/bios/
 
cp syslinux-6.03/efi64/efi/syslinux.efi /var/lib/tftpboot/efi64/
cp syslinux-6.03/efi64/com32/elflink/ldlinux/ldlinux.e64 /var/lib/tftpboot/efi64/
cp syslinux-6.03/efi64/com32/lib/libcom32.c32 /var/lib/tftpboot/efi64/
cp syslinux-6.03/efi64/com32/libutil/libutil.c32 /var/lib/tftpboot/efi64/
cp syslinux-6.03/bios/memdisk/memdisk /var/lib/tftpboot/efi64/
cp syslinux-6.03/efi64/com32/modules/poweroff.c32 /var/lib/tftpboot/efi64/
cp syslinux-6.03/efi64/com32/modules/pxechn.c32 /var/lib/tftpboot/efi64/
cp syslinux-6.03/efi64/com32/modules/reboot.c32 /var/lib/tftpboot/efi64/
cp syslinux-6.03/efi64/com32/menu/vesamenu.c32 /var/lib/tftpboot/efi64/


# wget https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/latest/rhcos-live-kernel-x86_64  -O /var/www/html/ 
# wget https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/latest/rhcos-live-initramfs.x86_64.img -O /var/www/html/
# wget https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/latest/rhcos-live-rootfs.x86_64.img **-O /var/www/html/
systemctl enable tftp.service httpd --now

/var/lib/tftpboot/efi64/pxelinux.cfg/default

UI vesamenu.c32
MENU BACKGROUND        bg-ocp.png  
MENU COLOR sel         4  #ffffff std 
MENU COLOR title       1  #ffffff  
TIMEOUT 120
PROMPT 0
MENU TITLE OPENSHIFT 4.x INSTALL BARE METAL PXE MENU 
LABEL INSTALL BOOTSTRAP   
  KERNEL http://192.168.0.10:8080/rhcos-live-kernel-x86_64
  APPEND initrd=http://192.168.0.10:8080/rhcos-live-initramfs.x86_64.img coreos.live.rootfs_url=http://192.168.0.10:8080/rhcos-live-rootfs.x86_64.img coreos.inst.install_dev=/dev/sda coreos.inst.ignition_url=http://192.168.0.10:8080/bootstrap.ign
LABEL INSTALL MASTER     
  KERNEL http://192.168.0.10:8080/rhcos-live-kernel-x86_64
  APPEND initrd=http://192.168.0.10:8080/rhcos-live-initramfs.x86_64.img coreos.live.rootfs_url=http://192.168.0.10:8080/rhcos-live-rootfs.x86_64.img coreos.inst.install_dev=/dev/sda coreos.inst.ignition_url=http://192.168.0.10:8080/master.ign
LABEL INSTALL WORKER    
  KERNEL http://192.168.0.10:8080/rhcos-live-kernel-x86_64
  APPEND initrd=http://192.168.0.10:8080/rhcos-live-initramfs.x86_64.img coreos.live.rootfs_url=http://192.168.0.10:8080/rhcos-live-rootfs.x86_64.img coreos.inst.install_dev=/dev/sda coreos.inst.ignition_url=http://192.168.0.10:8080/worker.ign
LABEL INSTALL EL
  KERNEL http://192.168.0.10:8080/el/vmlinuz
  APPEND initrd=http://192.168.0.10:8080/el/initrd.img inst.repo=http://192.168.0.10:8080/el/Packages/ inst.ks=http://192.168.0.10:8080/el/kickstart.cfg 

Create ignition config from install-config.yaml

apiVersion: v1
baseDomain: example.com
metadata:
  name: ocp
compute:
- hyperthreading: Enabled
  name: worker
  replicas: 0
controlPlane:
  hyperthreading: Enabled
  name: master
  replicas: 1
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16

platform:
  none: {}

fips: false

pullSecret: 'pull_secret'
sshKey: 'ssh-rsa'

$ openshift-install create ignition-configs --dir ./cluster

Ceph installation

mika published on 2023-02-15 included in Openshift

Requirements

Red Hat Enterprise Linux 8.4 EUS or later.
Ansible 2.9 or later.
A valid Red Hat subscription with the appropriate entitlements.
Root-level access to all nodes.
An active Red Hat Network (RHN) or service account to access the Red Hat Registry.

Create 3 RHEL 8 virtual machines

ceph1
ceph2
ceph3

Note

Installing Ceph on Virtual Machines is not recommendend for production use

Register servers to RHN

Find and attach Red Hat Ceph Storage pool

$ subscription-manager list --available --matches 'Red Hat Ceph Storage'    
$ subscription-manager attach --pool=POOL_ID

Enable server & extra repos

$subscription-manager repos --disable=*
subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms
subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms
subscription-manager repos --enable=rhceph-5-tools-for-rhel-8-x86_64-rpms
subscription-manager repos --enable=ansible-2.9-for-rhel-8-x86_64-rpms

Update system

$ dnf update -y