Saturday 30 January 2016

Usage of Cron to trigger IIB Flow



  • Typically we may have  a requirement where the IIB flow needs to be triggered at a particular time in a Day and every day (Eg. 10PM everyday). And such requirement cannot be done in a straight way using the available nodes.

  • Some of the possible options are as below:

1. Use an MQ Input Node as starter of the flow. Write an Agent (Eg. Java Agent) that puts a message onto the Queue at desired time of the day every day. Once the message is dropped to the Queue, the flow gets triggered and things can be done subsequently. Drawbacks of this approach is, one is required to write the agent and ensure it is running all the time without fail.

2. Use a timer control node that keeps ticking / polling every X seconds. Have compute node wired to the timer control node and write some logic to check whether the current time falls between the desired time. If it falls move forward and do what it is to be done. Else, do nothing in the flow. Drawback of this approach is, the IIB dips into infinite loop checking the time unnecessarily.

3. With tradeoffs above, there is another approach of using a Cron Job running at operating system level of the IIB Server. This cron job will drop a dummy file onto a folder, where the folder is constantly polled by a File Input Node. As part of the basic property, when the file is visible to the File Node, the flow gets triggered and subsequent action can be done. With that visibility in mind, one can write a cron job to drop a file at a specified time in a day every day.

We will see how to implement the third one which is simple and reliable in steps ahead.

Before that, following is the typical flow implementation on the lines of the requirements stated above. However, to understand better, the use case and how it is implemented below:

Use Case requirement : Requirement is to move all the messages from one Queue to another Queue at EOD say 10PM every day.

How it is achieved : We have a file node as starter of the flow, which keeps polling a particular directory on the file system. Sooner a file is dropped onto the file node, flow gets triggered. The file node is wired to a Compute Node that literally does nothing. It just passes control to the MQ Get node where it tries to GET  single message from the Queue. Note that the Queue will be the source queue where messages will be emptied at end of the flow. The MQ Get node has four terminals, where useful terminal in this use case are OUT & NO MESSAGE.  We can see the OUT terminal is wired to the MQ Output (Dummy OP Node. Loops Back). What this step does is simply picks the message and drops to the Queue. We can notice that this MQ Output node is wired back to the compute node. It means the process stated above is repeated. And it is repeated until the MQ get finishes reading all the messages of Source Queue. Once it is done, the flow is routed to 'No Message' Terminal where the flow ends. At the end of the flow, we can have all the messages from a Source Queue moved to Destination Queue.



Below are the steps to do a cron job and push the file to a specified folder every day at a particular time in a day. Note that this setup is for LINUX environment.


Step 1 :  
Logon to Linux environment as below with credentials of root. Eg as below
root / system 123*
Step 2 :
 Step 3:
The next step is creating a script file with .sh  as extension  (eg:  cron.sh)
 Explanation :
 What we are doing above
 We ran a command called     'cat'
 This command when followed by ' > filename.sh' will create a file as specified.
 Note that it will allow us to enter some text after which we can save.
 However, our idea is to execute something automatically.
 Hence we write some valid commands in the file as below:
    cd /root    <changes to root>
    cat > test.txt    <creates a dummy file by name 'test.txt'

 Press Ctrl + D to exit the text window and save the contents. 
Step 4:
Type ls - lrt   command to see whether the file is created or not.

We can see above the file is created.  Note that inside this file we have another two simple commands, one of which again creates a file. 
Step 5:
 Though  the file  cron.sh  is created, however does not have permissions to execute a command. We can see above it does not have necessary permissions (-rw-r--r--)  Hence it is very important to give the permission to run.

 The command    chmod -R 775 cron.sh  will give that necessary permission to do that.

 Upon executing the command we can see below the permissions have been changed so as to execute. 
Step 6:
 Now that we have created a Shell script ( Dot Sh file), which has commands that can generate a dummy file, we need to use another command of Linux that can schedule a particular task at a particular time.
 The command is   crontab

 Type-in the command crontab -e , hit enter and hit  letter 'I'.
 Linux waits for user to enter 'Task-to-do'
 Type in following line. What it means is explained ahead.

45 18 29 01 5    / root/cron.sh

 We can see the same in the screenshot below:

 Explanation
The meaning of      45 18 29 01 5    / root/cron.sh   is as below

 45 indicate Minutes
 18 indicate Hour  (24 Hour format)
 29 indicate Date
 01 indicate Month
 5 indicate Day of the week (Friday)

 Here we are scheduling using the 'crontab' command and telling the OS to execute the file cron.sh  on 29 Jan at 18 Hours 45 Minutes exactly.

  The possible values at each position is explained below
 
Table: Crontab Fields and Allowed Ranges (Linux Crontab Syntax)
Field
Description
Allowed Value
MIN
Minute field
0 to 59
HOUR
Hour field
0 to 23
DOM
Day of Month
1-31
MON
Month field
1-12
DOW
Day Of Week
0-6
CMD
Command
Any command to be executed
 










 Above data would just execute the cron.sh file at a particular time only once on a particular day.
 Now, we can change the values as per the requirement we have i.e. 'Execute the cron.sh file every day at 18 45. Then the values would be as below.

45 18 * *  *    / root/cron.sh  

 We have replaced other positions with Asterix (*)

Step 7:
 Once we are done by pressing 'ESC' key we can exit the CRONTAB.
 We can see below the screen shot post-press of 'ESC' key.

Step 8:
We can list the Cron jobs that are configured use the command crontab -l


 After a wait till that particular time as configured in the Cron Job, we can see the script file generates the file automatically. Same can be seen as below the file 'test.sh'

 The screenshot however was taken out of previous implementations, hence don't go by the timing it is created.


With such simple tweek around, we can bind that folder and file to the file input node on IIB and do the things at a particular time of the day.





No comments:

Post a Comment