Devops on K-Life Hack | Seoul Gastronomy & Travel Guide

Technical Evolution Stages and Domain Architecture Analysis for Software Engineers

Tue, 26 May 2026 10:40:20 +0900

Systematic Analysis of Engineering Evolution Processes and Technical Requirements

In the modern software development ecosystem, engineer growth is not limited to mastering programming languages. There is a strong demand for the ability to deeply understand overall system architecture and drive operational automation. This analysis details the five-stage evolution process and the technical requirements for each domain, assuming a development environment based on automation tools such as Jenkins CI/CD.

1. Five-Stage Model of Engineering Evolution

Software engineer growth transitions incrementally from building basic logic to large-scale system design. The technical focus and objectives for each phase represent a structured progression.

Stage 1: Mastery of Basic Logic (Introductory Phase)

This stage focuses on understanding basic programming syntax and algorithms. It is a period for solidifying the foundations of memory management and control flow, including variable declarations, data types, conditional branching, loops, and function definitions. The technical goal involves becoming proficient in reading and writing code, cultivating logical thinking through the creation of calculators, number-guessing games, and simple automation scripts.

Stage 2: Modularization and Structuring (Basic Development Phase)

Transitioning from single scripts to reusable code requires appropriate function decomposition, namespace separation via modules, implementation of exception handling, and data persistence through file I/O. Engineers build the capacity to independently implement small-scale applications, such as blog pages or user registration interfaces, by effectively utilizing data structures like lists, dictionaries, and arrays.

Stage 3: Core Engineering and Data Flow (Intermediate Development Phase)

Professional service development necessitates learning data coordination and structural design between systems. The focus includes class design via Object-Oriented Programming (OOP), data management using Relational Databases (RDB) and SQL, and API design based on the HTTP protocol. Mastery of distributed version control using Git is also essential at this stage.

Stage 4: Production Operations and Quality Management (Practical Phase)

Shifting perspective from individual development to team development and production deployment involves logic optimization through code reviews, internal quality improvement via refactoring, and regression testing through test automation. Practical capabilities for stable service operation include integration into CI/CD pipelines, security measures such as authentication and vulnerability patching, and performance optimization.

Stage 5: System Architecture and Technical Leadership (Expert Phase)

Advanced technical decisions ensure high availability and scalability. This includes scalable configuration design using cloud infrastructure like AWS, GCP, or Azure, introduction of microservices architecture, operational automation via DevOps, and design for high traffic resistance. This stage requires exercising leadership to resolve technical bottlenecks and ensure long-term maintainability.

2. Tech Stacks and Responsibilities by Domain

As engineering evolves, specialization in specific domains becomes necessary. The primary technical elements define the core responsibilities of each domain.

Backend Development: Centered on Java (Spring), Python (Django), or Go, responsible for API design, DB schema design, and server-side performance optimization.
DevOps & Cloud: Utilizing Linux, Docker, Kubernetes, Jenkins, and Terraform, specializing in CI/CD automation, Infrastructure as Code (IaC), monitoring, and incident response.
Data Engineering: Using SQL, Python, Spark, and Kafka to build ETL processes, manage data pipelines, and ensure data quality.
AI & Machine Learning: Utilizing PyTorch, TensorFlow, and LangChain, covering everything from model training to MLOps including model deployment and monitoring.

3. Implementation Example of a CI/CD Pipeline using Jenkins

Continuous integration is essential in the practical phase. The Jenkinsfile structure automates builds, tests, and deployments.

pipeline {
 agent { label 'docker-node' }
 environment {
 APP_NAME = 'core-service-api'
 IMAGE_TAG = "${env.BUILD_ID}"
 }
 stages {
 stage('Source Checkout') {
 steps {
 checkout scm
 }
 }
 stage('Static Analysis') {
 steps {
 sh 'npm run lint'
 }
 }
 stage('Unit Testing') {
 steps {
 sh 'npm test -- --coverage'
 }
 }
 stage('Container Build &amp; Push') {
 steps {
 script {
 docker.withRegistry('https://registry.example.com', 'registry-credentials') {
 def customImage = docker.build("${APP_NAME}:${IMAGE_TAG}")
 customImage.push()
 }
 }
 }
 }
 stage('Staging Deployment') {
 steps {
 sh "kubectl set image deployment/${APP_NAME} ${APP_NAME}=registry.example.com/${APP_NAME}:${IMAGE_TAG} -n staging"
 }
 }
 }
 post {
 failure {
 echo 'Pipeline failed. Notification sent to engineering team.'
 }
 always {
 cleanWs()
 }
 }
}

4. Strategic Growth Roadmap

Establishment of a Common Foundation: Deeply understand one language such as Python or JavaScript and master history management with Git.
Domain Selection: Choose frontend for visual UIs, or backend and data engineering for logic and data structures.
Portfolio Construction: Prepare documentation that logically describes the reasons for technology selection, challenges faced, and solutions implemented.
Continuous Learning: Track transitions in frameworks and cloud-native technology trends while contributing through technical blogs and code reviews.

Key Takeaways

Engineer growth evolves through five stages, from basic logic to system architecture.
From Stage 4 onwards, production operation capabilities such as CI/CD and test automation become indispensable.
A balance between deepening the tech stack for a specific domain and cross-domain understanding is critical.
In practice, the ability to build automation pipelines using tools like Jenkins determines an engineer’s market value.

Building Local Kubernetes CI/CD with GitHub Actions and Self-hosted Runners: Overcoming Authentication and Network Boundaries

Mon, 25 May 2026 12:26:26 +0900

Optimizing Local Kubernetes Deployment with GitHub Actions and Self-hosted Runners

1. Background: Deployment Disconnect in Hybrid Environments

In modern microservices development, local Kubernetes environments such as Docker Desktop are critical assets that enable validation close to production. However, when attempting to deploy from GitHub Actions managed runners to a local cluster, two major barriers arise. First, the reachability issue from runners on the public cloud to cluster endpoints (kubernetes.docker.internal) within a private network. Second, PEM block parsing errors caused by broken line breaks or indentation when saving YAML-formatted Kubeconfig in GitHub Secrets.

This article details the construction process of a CI/CD pipeline that breaks through these boundaries and fully automates synchronization from Git Push to the local cluster.

2. Technology Selection and Trade-offs: Reasons for Adopting Self-hosted Runners

Cloud Runner + VPN/Tunneling (e.g., ngrok): A method to build a tunnel from the outside to the local network. While setup is easy, it carries high security risks, and bandwidth limits or latency become bottlenecks.

Self-hosted Runner (Adopted): A method where the GitHub Actions agent runs directly on the local machine. Since it operates inside the firewall, there is no need to open external ports, and it can directly access the local Docker daemon and K8s API. It also has the advantage of minimizing network costs when pulling pre-built images from a registry.

3. Implementation Details: Encapsulating Credentials and Runner Configuration

3.1 Ensuring Integrity via Base64 Encoding of Kubeconfig

Saving Kubeconfig directly in GitHub Secrets carries an extremely high probability of encountering the error: error: unable to load root certificates: unable to parse bytes as PEM block. To avoid this, we perform Base64 encoding at the binary level using PowerShell and inject it as a string.

# Convert Kubeconfig to Base64 string and output to file
$configPath = "$HOME\.kube\config"
$base64Config = [Convert]::ToBase64String([IO.File]::ReadAllBytes($configPath))
$base64Config | Out-File -FilePath "encoded_config.txt"

3.2 Workflow Definition for Windows Self-hosted Runners

When running a runner in a Windows environment, the default shell is PowerShell, so Linux-based commands (e.g., mkdir -p) will not work. The following implementation ensures idempotency.

jobs:
 deploy:
 runs-on: self-hosted
 steps:
 - name: Checkout code
 uses: actions/checkout@v4

 - name: Configure Kubeconfig
 shell: pwsh
 run: |
 $kubeDir = "$HOME\.kube"
 if (!(Test-Path $kubeDir)) { New-Item -ItemType Directory -Path $kubeDir }
 $decodedConfig = [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String("${{ secrets.KUBE_CONFIG_DATA }}"))
 $decodedConfig | Out-File -FilePath "$kubeDir\config" -Encoding ascii

 - name: Deploy to Local Kubernetes
 run: |
 kubectl apply -f ./k8s/deployment.yaml
 kubectl rollout status deployment/api-service

4. Operational Warnings and Workarounds (Operational Reality)

4.1 Optimizing ErrImagePull and imagePullPolicy

During development in a local environment, if a deployment is performed immediately after pushing an image to a registry, Kubernetes may reference an old cache if the tag is “latest,” or an ErrImagePull may occur due to a pull failure. To prevent this, the following settings are recommended in deployment.yaml:

imagePullPolicy: Always: Forces the registry to be checked every time. However, this increases network load.

imagePullPolicy: IfNotPresent: Effective when using locally built images as-is. If the Self-hosted Runner is operating on the same node as the cluster, the built image becomes immediately available, making this setting the most efficient.

4.2 Precautions for Persistent Volume Path Specification

⚠️ When using Docker Desktop for Windows, the path specified in hostPath must be the mount path within the Docker VM (/run/desktop/mnt/host/c/…) rather than the Windows format. If this is incorrect, a mount error will occur during container startup, and the shared directory will not be recognized correctly.

5. Results and Evaluation

By implementing this configuration, the following quantitative and qualitative improvements were confirmed:

Reduction in deployment time: Reduced the lead time from code push to reflection by approximately 70% compared to manual kubectl operations.
Improved environmental consistency: Established a consistent deployment pipeline independent of the developer’s local environment through dynamic Kubeconfig generation.
Enhanced security: Achieved bidirectional communication with GitHub Actions without allowing any inbound traffic from the outside.

Summary

This architecture combines the flexibility of GitHub Actions with the network advantages of Self-hosted Runners to eliminate deployment barriers in hybrid cloud environments. It serves as an effective solution to significantly reduce the operational burden (Ops Burden) during the transition from legacy operations centered on static configurations like Nginx to Kubernetes-native GitOps. Moving forward, transitioning to immutable tag management using GITHUB_RUN_NUMBER instead of the “latest” tag will be key to further improving reliability.