Category: <span>Uncategorized</span>

“I have lots of photo files. Since 2006, when I purchased my first digital camera, the number of photos has grown quickly, and after getting an iPhone, the number of photos exploded.

With the high number of photos, the number of backups grew as well.

I decided to organize all backups and create folders using the format YYYY-MM from the metadata of the photo files.”

Bellow the python script. The script runs on macos:

import os
import shutil
import datetime
import logging
import tkinter as tk
from tkinter import filedialog
from PIL import Image, ExifTags
import pillow_heif
import piexif
import struct

# Setup logging
logging.basicConfig(level=logging.DEBUG, format="%(asctime)s - %(levelname)s - %(message)s")

ATOM_HEADER_SIZE = 8
EPOCH_ADJUSTER = 2082844800  # Difference between Unix and QuickTime epoch

def get_file_date(file_path):
    try:
        if file_path.lower().endswith(".heic") and pillow_heif.is_supported(file_path):
            heif_file = pillow_heif.open_heif(file_path, convert_hdr_to_8bit=False)
            exif_data = heif_file.info.get("exif")
            if exif_data:
                exif_dict = piexif.load(exif_data)
                date_str = exif_dict["Exif"].get(piexif.ExifIFD.DateTimeOriginal)
                if date_str:
                    return datetime.datetime.strptime(date_str.decode("utf-8"), "%Y:%m:%d %H:%M:%S")
        
        elif file_path.lower().endswith((".jpg", ".jpeg")):
            with Image.open(file_path) as img:
                exif_data = img.getexif()
                if exif_data:
                    exif_dict = {ExifTags.TAGS.get(tag, tag): value for tag, value in exif_data.items()}
                    logging.debug(f"EXIF metadata for {file_path}: {exif_dict}")
                    
                    if "DateTimeOriginal" in exif_dict:
                        date_str = exif_dict["DateTimeOriginal"]
                    elif "DateTime" in exif_dict:
                        date_str = exif_dict["DateTime"]
                    else:
                        date_str = None
                        logging.warning(f"No DateTimeOriginal or DateTime found for {file_path}")
                    
                    if date_str:
                        try:
                            logging.debug(f"Extracted date string from EXIF: {date_str}")
                            return datetime.datetime.strptime(date_str, "%Y:%m:%d %H:%M:%S")
                        except ValueError as ve:
                            logging.error(f"Error parsing date for {file_path}: {ve}")
                    else:
                        logging.warning(f"DateTime metadata missing or unreadable for {file_path}")
                else:
                    logging.warning(f"No EXIF metadata found for {file_path}")
    
    except Exception as e:
        logging.error(f"Error extracting date from {file_path}: {e}")
    
    # If metadata exists but could not be parsed, use file birth time (creation date on macOS)
    file_stats = os.stat(file_path)
    file_birth_time = file_stats.st_birthtime
    logging.debug(f"Using file birth time for {file_path}: {datetime.datetime.fromtimestamp(file_birth_time)}")
    return datetime.datetime.fromtimestamp(file_birth_time)

def move_files_to_folders(source_folder):
    for filename in os.listdir(source_folder):
        file_path = os.path.join(source_folder, filename)
        if filename.lower().endswith((".jpg", ".jpeg", ".heic", ".mov")):
            date_taken = get_file_date(file_path)
            if date_taken:
                folder_name = date_taken.strftime("%Y-%m")
            else:
                logging.warning(f"Could not determine date for {file_path}, using 'unknown' folder.")
                folder_name = "unknown"
            
            dest_folder = os.path.join(source_folder, folder_name)
            os.makedirs(dest_folder, exist_ok=True)
            
            dest_file_path = os.path.join(dest_folder, filename)
            count = 1
            while os.path.exists(dest_file_path):
                name, ext = os.path.splitext(filename)
                dest_file_path = os.path.join(dest_folder, f"{name}_{count}{ext}")
                count += 1
            
            shutil.move(file_path, dest_file_path)
            logging.info(f"Moved {filename} to {dest_folder}")

if __name__ == "__main__":
    root = tk.Tk()
    root.withdraw()
    folder_selected = filedialog.askdirectory(title="Select the folder containing files")
    if folder_selected:
        move_files_to_folders(folder_selected)
        logging.info("File organization complete.")
    else:
        logging.warning("No folder selected.")

Uncategorized

I set up an OpenShift 4.16 cluster using UPI on top of VMware. The cluster has 3 Masters, 3 Worker Nodes, and 3 InfraNodes. The infra nodes were necessary to install IBM Storage Fusion.

After the setup, I needed to create a load balancer in front of the OpenShift cluster. There are several options, and one of them is HAProxy.

I just installed an RHEL 9 server, added 3 ips to the network card and setup the haproxy.

Prerequisites

  • A system running RHEL 9
  • Root or sudo privileges
  • A basic understanding of networking and load balancing

Step 1: Install HAProxy

First, update your system packages:

sudo dnf update -y

Then, install HAProxy using the package manager:

sudo dnf install haproxy -y

Verify the installation:

haproxy -v

Step 2: Configure HAProxy

The main configuration file for HAProxy is located at /etc/haproxy/haproxy.cfg. Open the file in a text editor:

sudo nano /etc/haproxy/haproxy.cfg

The configuration bellow was used for my cluster. Change the IP adresses to match

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

    # utilize system-wide crypto-policies
    #ssl-default-bind-ciphers PROFILE=SYSTEM
    #ssl-default-server-ciphers PROFILE=SYSTEM

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    tcp
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------

frontend api
    bind 192.168.252.171:6443
    default_backend controlplaneapi

frontend apiinternal
    bind 192.168.252.171:22623
    bind 192.168.252.171:22624
    default_backend controlplaneapiinternal

frontend secure
    bind 192.168.252.170:443
    default_backend secure

frontend insecure
    bind 192.168.252.170:80
    default_backend insecure

#---------------------------------------------------------------------
# static backend
#---------------------------------------------------------------------

backend controlplaneapi
    balance source
    server master-01  192.168.252.5:6443 check
    server master-02  192.168.252.6:6443 check
    server master-03  192.168.252.7:6443 check


backend controlplaneapiinternal
    balance source
    server master-01  192.168.252.5:22623 check
    server master-02  192.168.252.6:22623 check
    server master-03  192.168.252.7:22623 check
    server master-01  192.168.252.5:22624 check
    server master-02  192.168.252.6:22624 check
    server master-03  192.168.252.7:22624 check

backend secure
    balance source
    server worker-01  192.168.252.8:443 check
    server worker-02  192.168.252.9:443 check
    server worker-03  192.168.252.10:443 check
    server  worker-04   192.168.252.11:443 check
    server  worker-05   192.168.252.12:443 check
    server  worker-06   192.168.252.13:443 check

backend insecure
    balance roundrobin
    server worker-01  192.168.252.8:80 check
    server worker-02  192.168.252.9:80 check
    server worker-03  192.168.252.10:80 check
    server worker-04   192.168.252.11:80 check
    server worker-05   192.168.252.12:80 check
    server worker-06  192.168.252.13:80 check

Uncategorized

When you delete a node using the CLI, the node object is deleted in Kubernetes, but the pods that exist on the node are not deleted. Any bare pods not backed by a replication controller become inaccessible to OpenShift Container Platform. Pods backed by replication controllers are rescheduled to other available nodes. You must delete local manifest pods.

  • To delete the node from the UPI installation, the node must be firstly drained and then marked unschedulable prior to deleting it:

$ oc adm cordon <node_name>
$ oc adm drain <node_name> --force --delete-local-data --ignore-daemonsets
- Ensure also that there are no current jobs/cronjobs being ran or scheduled in this specific node as the draining does not take it into consideration.
- For Red Hat OpenShift Container Platform 4.7+, utilize the option `--delete-emptydir-data` in case `--delete-local-data` doesn't work. The `--delete-local-data` option is deprecated in favor of `--delete-emptydir-data`.

$ oc get node <node_name> -o yaml > backupnode.yaml

Before proceeding with deletion of the node, it needs to be under "power off" status:
$ oc delete node <node_name>

Although the node object is now deleted from the cluster, it can still rejoin the cluster after reboot or if the kubelet service is restarted. To permanently delete the node and all its data, you must decommission the node once it is in shutdown mode.

Once the node is deleted, it can be ready for a power-off activity, or if it is needed to rejoin the cluster, it could be possible to either restart the kubelet or create the yaml back:

$ oc create -f backupnode.yaml

In order to get the node back, it can also be back by restarting kubelet:

$ systemctl restart kubelet

If it is needed to destroy then all the data from the worker node to delete all the software installed, execute the following:

# nohup shred -n 25 -f -z /dev/[HDD]
This command will overwrite all data on /dev/[HDD] repeatedly, in order to make it harder for even very expensive hardware probing to recover the data. Command line parameter -z will overwrite this device with zeros at the end of cycle to re-write data 25 times (it can be overridden with -n [number]).

One should consider running this command from RescueCD.

In order to monitor the deletion of the node, get the kubelet live logs:

$ oc adm node-logs <node-name> -u kubelet

https://access.redhat.com/solutions/4976801

Uncategorized

Applying a specific node selector to all infrastructure components will guarantee that they will be scheduled on nodes with that label. See more details on node selectors in placing pods on specific nodes using node selectors, and about node labels in understanding how to update labels on nodes.

Our node label and matching selector for infrastructure components will be node-role.kubernetes.io/infra: "".

To prevent other workloads from also being scheduled on those infrastructure nodes, we need one of two solutions:

  • Apply a taint to the infrastructure nodes and tolerations to the desired infrastructure workloads.
    OR
  • Apply a completely separate label to your other nodes and matching node selector to your other workloads such that they are mutually exclusive from infrastructure nodes.

TIP: To ensure High Availability (HA) each cluster should have three Infrastructure nodes, ideally across availability zones. See more details about rebooting nodes running critical infrastructure.

TIP: Review the infrastructure node sizing suggestions

By default all nodes except for masters will be labeled with node-role.kubernetes.io/worker: "". We will be adding node-role.kubernetes.io/infra: "" to infrastructure nodes.

However, if you want to remove the existing worker role from your infra nodes, you will need an MCP to ensure that all the nodes upgrade correctly. This is because the worker MCP is responsible for updating and upgrading the nodes, and it finds them by looking for this node-role label. If you remove the label, you must have a MachineConfigPool that can find your infra nodes by the infra node-role label instead. Previously this was not the case and removing the worker label could have caused issues in OCP <= 4.3.

This infra MCP definition below will find all MachineConfigs labeled both “worker” and “infra” and it will apply them to any Machines or Nodes that have the “infra” role label. In this manner, you will ensure that your infra nodes can upgrade without the “worker” role label.

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
  name: infra
spec:
  machineConfigSelector:
    matchExpressions:
      - {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,infra]}
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/infra: ""

If you are not using the MachineSet API to manage your nodes, labels and taints are applied manually to each node:

Label it:

oc label node <node-name> node-role.kubernetes.io/infra=
oc label node <node-name> node-role.kubernetes.io=infra

Taint it:

oc adm taint nodes -l node-role.kubernetes.io/infra node-role.kubernetes.io/infra=reserved:NoSchedule node-role.kubernetes.io/infra=reserved:NoExecute

openshift Uncategorized

In the context of a Kubernetes cluster, CPU throttling still refers to the process of limiting the amount of CPU time a container or pod can use, but it’s slightly different than throttling within an individual CPU or device, as Kubernetes manages resources at the container level. Here’s a breakdown of how CPU throttling works in a Kubernetes environment:

1. CPU Resources in Kubernetes

Kubernetes allows you to specify how much CPU a container can request and how much it is allowed to consume. This is done through resource requests and limits:

  • CPU Request: The minimum CPU resource that the container is guaranteed to have.
  • CPU Limit: The maximum CPU resource the container can use.

Kubernetes uses CPU throttling to ensure that containers do not exceed their allocated CPU limits. If a container tries to use more CPU than it has been allocated (based on the CPU limit), Kubernetes will throttle the container’s CPU usage to prevent it from violating the resource limits.

2. How CPU Throttling Works in Kubernetes

  • CPU Requests: When a container is scheduled on a node, Kubernetes ensures that the requested CPU is available to the container. If the node doesn’t have enough available CPU, the pod may not be scheduled.
  • CPU Limits: If a container exceeds its CPU limit (i.e., tries to use more CPU than what is specified in the limit), Kubernetes throttles the container’s CPU usage. The system does this by applying CPU usage constraints (using mechanisms like CFS (Completely Fair Scheduler) in Linux) to ensure that the container doesn’t exceed its allocated CPU time.
  • CFS Throttling: The CFS quota system controls how much CPU a container can use. If a container tries to use more CPU than its allocated limit, the kernel uses a mechanism called CFS throttling. Essentially, the Linux kernel will temporarily stop a container from using the CPU until it is within the allowed usage range.
  • Exceeding Limits: If a container tries to use more CPU than its limit allows (e.g., 1 CPU core), Kubernetes will restrict or throttle the container, reducing its access to the CPU until it falls back within the limit.

3. Example: CPU Limits and Throttling

Suppose you define a pod in Kubernetes with the following resource configuration:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
  - name: example-container
    image: myimage
    resources:
      requests:
        cpu: "500m"      # 0.5 CPU core requested
      limits:
        cpu: "1000m"      # 1 CPU core max
  • Request: The container is guaranteed 500 milli-CPU (or 0.5 CPU core).
  • Limit: The container can burst up to 1000 milli-CPU (or 1 CPU core).

If the container tries to use more than 1 CPU core (e.g., if the workload spikes and tries to use 1.5 CPU cores), Kubernetes will throttle it back down to 1 core.

4. How Throttling Happens in Practice

  • Within Node: If there are multiple containers on the same node and they exceed their CPU limits, the Linux kernel (via CFS) enforces throttling to ensure that no container exceeds its CPU limit. This can cause delays or latency in container performance, especially when several containers are competing for CPU on a node.
  • Overcommitment: If a node is overcommitted (i.e., the sum of all container CPU limits exceeds the physical capacity of the node), Kubernetes will throttle the containers that try to exceed the available CPU capacity.

5. Monitoring CPU Throttling in Kubernetes

You can monitor CPU throttling in a Kubernetes cluster by observing certain metrics, such as:

  • container_cpu_cfs_throttled_seconds_total: This metric in Prometheus shows how much time a container has been throttled by the kernel (CFS throttling).
  • container_cpu_usage_seconds_total: This metric shows the total CPU usage by a container, which can help correlate throttling behavior with usage spikes.

You can query these metrics in Prometheus and Grafana to see if containers are being throttled and to identify performance bottlenecks.

6. What Happens When Throttling Occurs?

When CPU throttling happens, a container might experience:

  • Increased Latency: Throttling limits the amount of CPU time available to the container, leading to increased response times.
  • Reduced Performance: The container may be unable to process requests as quickly, affecting application performance.
  • Delays in Processing: If the container cannot access enough CPU resources, jobs that require more compute power will queue up and take longer to complete.

7. How to Avoid CPU Throttling in Kubernetes

To avoid CPU throttling and ensure containers have the necessary resources:

  • Proper Resource Allocation: Set appropriate CPU requests and limits. The request should reflect the expected CPU usage, while the limit should provide headroom for occasional spikes.
  • Monitor Resource Usage: Use monitoring tools like Prometheus, Grafana, and Kubernetes metrics server to observe resource usage and throttling events.
  • Avoid Overcommitment: Ensure that the sum of the CPU requests of all containers on a node doesn’t exceed the total CPU capacity of that node.
  • Horizontal Scaling: If a container regularly hits its CPU limits, consider scaling the application horizontally by adding more pods to distribute the load.

8. Conclusion

In a Kubernetes cluster, CPU throttling is primarily a mechanism to enforce resource limits and ensure that each container gets its fair share of CPU time, preventing any single container from monopolizing resources. While this helps maintain system stability and prevent resource exhaustion, it can result in performance degradation if a container is constantly throttled. Proper resource allocation and monitoring are key to avoiding excessive throttling and ensuring efficient operation of your Kubernetes workloads.

Uncategorized

Since June 2022 the Red Hat OpenShift operator index images (redhat/redhat/operator-index) have been served from registry.redhat.io using Quay.io as the backend. OpenShift itself already needs access to the Quay.io registry and CDN hosts as explained in its installation instructions, and so this change required no action from customers at that time.

We are extending this to all Red Hat container images. This allows customers to benefit from the high availability of the Quay.io registry while simplifying the way Red Hat delivers container images and paving the way for future enhancements.

More informaion

Uncategorized

I started a channel on youtube. The first video series will be about software that helps beginners to start using linux and after this series i will start to talk about my jorney learning new things like OpenShift, IBM Cloud, Cloud Paks and related IBM technologies.
I will record the videos in Brazilian Portuguese first and maybe i will create videos in English.
There is no much content about the things i will record in my native language and not everyone here in Brazil are able listen videos in English.
There is another blog too https://www.chuvisco.net.br were you can view the transcription of the video.

Uncategorized

I received an email yesterday from Docker. It’s a reminder about the end of grace period.

Hello,

As a reminder you’re receiving this email because on August 31, 2021 we updated the terms applicable to the Docker products or services you use.

On January 31, 2022, the grace period ends for free commercial use of Docker Desktop in larger enterprises. Companies with more than 250 employees OR more than $10 million USD in annual revenue now require a paid subscription to use Docker Desktop. Read the blog or visit our FAQ to learn more about these updates.

For me is not a problem anymore i remove Docker Desktop from my computers and install Podman.  No issues, no problems everything works.

Don’t need Docker Desktop anymore.

Uncategorized

Openshift comes with a set of default templates, you can use oc get templates -n openshift to show them
Each template contains specifc sections
  • The objects section: defines a list of resources that will be created
  • The parameters section: defines parameters that are used in the template objects
1 – Inspect the template file for the parameters
I export the postgresql-ephemeral to a yaml file using :  oc get template postgresql-ephemeral -o yaml -n openshift > postgresql.yaml 
Then inspect the yaml file  oc process --parameters -f <filename.yaml>
2 – Create the application using oc process
oc process -f postgresql.yaml -l app=mydb -p DATABASE_SERVICE_NAME=dbservice -p POSTGRESQL_USER=dbuser \
-p POSTGRESQL_PASSWORD=password -p POSTGRESQL_DATABASE=books | oc create -f -

Uncategorized

 

I have on my desk now 2 MacBooks Pro a monitor, a keyboard and a magic mouse. For the two computers to share the monitor, keyboard and mouse I should buy a KVM but I don’t want to have too many cables on the desk and also the cost of the KVM for the macbooks is too expensive for me in Brazil.

I found an interesting solution which was to connect each macbook using HDMI adapters to the monitor and use a software to switch the keyboard and mouse between the computers.

I’m testing Barrier and so far everything is working fine.

There is feature very cool :  You can copy and paste from one computer to another.

You can see this video and see how it works

Uncategorized