Chapter 4
Section titled “Chapter 4”Everything we’ve discussed so far—Deployments, Probes, Resource Limits—has been in the context of long-running services like web servers or APIs. The goal of Kubernetes in those cases is to prevent the application from ever stopping. But not all work fits this model. What about tasks that are supposed to run, finish their work, and then stop? Examples include:
- Running a database migration before deploying a new version of your app.
- Generating a daily report.
- Processing a video file that was just uploaded.
- Running a nightly backup script.
Using a
Deploymentfor these tasks would be a mistake. If the Pod successfully completes its job and exits, the Deployment will see this as a “failure” and immediately restart it, creating an endless and unwanted loop. For these run-to-completion workloads, we need different tools: Jobs and CronJobs.
Handling Batch Workloads with Jobs & CronJobs
Section titled “Handling Batch Workloads with Jobs & CronJobs”The Job Object
Section titled “The Job Object”A Job is a Kubernetes object whose purpose is to create one or more Pods and ensure that they run to completion successfully. Unlike a Deployment, a Job’s goal is completion, not continuous operation.
When the process running in a Pod created by a Job exits with a success code (exit code 0), the Job considers that Pod complete. It does not restart it. If the Pod fails (exits with a non-zero code), the Job can be configured to retry it a certain number of times before marking the entire Job as failed.
This is perfect for any one-off task that you need to execute on your cluster.
The CronJob Object
Section titled “The CronJob Object”A CronJob builds on top of the Job object to add a schedule. A CronJob is a controller that creates new Job objects on a repeating schedule that you define.
You provide a schedule in the standard cron format (e.g., 0 2 * * * for “every day at 2:00 AM”). At each scheduled time, the CronJob controller creates a new Job from a template you provide. That Job then creates a Pod, which runs to completion.
This is the Kubernetes-native equivalent of the classic Linux crontab or a scheduled task.
Part A: Running a One-off Job
Section titled “Part A: Running a One-off Job”Our Job will run a simple task: calculating the digits of Pi.
-
The Job YAML Create a file named
pi-job.yaml:pi-job.yaml apiVersion: batch/v1kind: Jobmetadata:name: pi-calculatorspec:# The .spec.template defines the Pod that will be created.template:spec:containers:- name: piimage: perl:5.34command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]# For Jobs, the restart policy should be OnFailure or Never, not Always.restartPolicy: OnFailure# If the Pod fails, the Job will retry it up to 4 times before giving up.backoffLimit: 4 -
Create and Watch the Job Apply the file and watch its status.
Terminal window kubectl apply -f pi-job.yamlkubectl get job pi-calculator --watchYou will see the
COMPLETIONScolumn change from0/1to1/1. This indicates the Job has successfully completed its task. Now, look at the Pod it created:Terminal window kubectl get pods# NAME READY STATUS RESTARTS AGE# pi-calculator-abcde 0/1 Completed 0 30sNotice the status is
Completed. The Pod has finished its work and stopped. -
Check the Output You can still get the logs from a completed Pod to see the result of its work.
Terminal window # Get the name of the pod created by the jobPOD_NAME=$(kubectl get pods -l job-name=pi-calculator -o jsonpath='{.items[0].metadata.name}')# Print the logs from that podkubectl logs $POD_NAMEYou will see the first 2000 digits of Pi printed to your screen.
-
Clean Up Jobs and their Pods are not automatically deleted after completion, so you can inspect their logs and status. You must clean them up manually.
Terminal window kubectl delete job pi-calculator
Part B: Running a Scheduled CronJob
Section titled “Part B: Running a Scheduled CronJob”Now let’s create a CronJob that runs every single minute.
-
The CronJob YAML Create a file named
hello-cronjob.yaml:hello-cronjob.yaml apiVersion: batch/v1kind: CronJobmetadata:name: hello-cronspec:# This schedule means "at every minute".schedule: "* * * * *"# This is the template for the Jobs that will be created.jobTemplate:spec:template:spec:containers:- name: helloimage: busybox:1.36command: ["/bin/sh", "-c", "date; echo Hello from the Kubernetes CronJob"]restartPolicy: OnFailure -
Create and Watch the CronJob Apply the file and then watch what happens.
Terminal window kubectl apply -f hello-cronjob.yaml# Watch the jobs being created every minutewatch kubectl get jobsEvery minute, on the minute, you will see a new Job appear in the list, created automatically by your CronJob.
NAME COMPLETIONS DURATION AGEhello-cron-276... 1/1 3s 2mhello-cron-276... 1/1 2s 1mhello-cron-276... 1/1 2s 10sYou can also watch the Pods being created and completing their work:
kubectl get pods. -
Clean Up Deleting the CronJob will stop any future Jobs from being created.
Terminal window kubectl delete cronjob hello-cron
Key Takeaways for Chapter 4:
Section titled “Key Takeaways for Chapter 4:”- Use Jobs for one-off tasks that need to run to completion, like data processing or migrations.
- Use CronJobs for repeating, scheduled tasks like backups or generating nightly reports.
- The
restartPolicyfor Pods in a Job template should beOnFailureorNever. - Understanding Jobs and CronJobs allows you to move beyond just long-running services and automate a wide variety of batch workloads within your cluster.
This covers the fundamentals of handling batch tasks. When you’re ready, let me know, and we’ll tackle our final chapter: Chapter 5: Managing Stateful Applications with StatefulSets.