Task Scheduling with Python

We run many tasks during our day to day work that can be automated instead of performed repetitively. A task scheduler allows you to run your task after a particular period of time and can be set to perform a task at any given time, be it weekends or daily. Scheduling can be useful to fetch data from a database or to store some data periodically. Scheduling can also be used to train machine learning models as new data comes in. Let’s see how we can achieve task scheduling using Python.

Python provides a generic scheduler module named schedule. By using this module, we can easily run our script after a fixed time. You can install it using the following command -

pip install schedule

Let’s see an example -

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
import schedule
import io
import sys
import time
import pickle
import os
import json
from urllib.request
import urlopen
from datetime
import date

today = date.today()

URL = 'https://api.exchangerate-api.com/v4/latest/USD'

def wait_for_internet_connection():
  while True:
  try:
  # # # Random url to check
for internet connection
response = urlopen('http://216.58.192.142', timeout = 1)
return
except Exception as e:
  # # # Logs can be captured
pass

def job():
  print("Loading...")
response = urlopen(URL, timeout = 5)
  .read()
response.decode('utf8')
  .replace("'", '"')
inr_rate = json.loads(response)['rates']['INR']
file = open('output.txt', 'a')
file.write(today.strftime("%d/%m/%Y") + ' - ' + str(inr_rate) + '\n')
file.close()

schedule.every()
  .day.at("10:30")
  .do(job) # # # Everyday at 10: 30
#schedule.every(5)
  .seconds.do(job) # # # Every 5 seconds
#schedule.every()
  .sunday.at("10:30")
  .do(job) # # # Every Sunday at 10: 30

while 1:
  wait_for_internet_connection()
schedule.run_pending()
time.sleep(1)

You can save this file as script.py. In this example, we are calling a REST API to get the conversion rate from USD to INR. We created a wait_for_internet_connection function to check for network connectivity. Otherwise, our script may cause an error and stop abruptly.

run_pending() function runs all jobs that are scheduled to run. We have to keep it in a loop so that it can be running all the time.
Suppose we want to get the conversion on a day to day basis and want to run it at a fixed time. We can achieve this with the every() function of the schedule module.

schedule.every().day.at("10:30").do(job)

This function takes an input which is the job that needs to be performed. Once the machine time reaches the scheduled time, it calls the do function which performs the job.

at(time_str) function takes a time string in HH:MM format.

To keep this script running, we need to open a terminal or console and run python script.py Now just leave it running.

That’s it. This script will execute the function based on your machine time when it reaches the given time period.

There are many other libraries which can also be used to perform scheduling. The only disadvantage of this approach is that in order to run the script in the background we have to keep our machine ON all the time. For production tasks, we can either build our own server and perform scheduling there or we can use some third party cloud-based services.

That’s all for now. Now it’s your turn to try scheduling for your routine tasks. Happy coding :)