Intermediate Training for WLOADCTL: Error Handling, Inheritance, and Override

Workflow scheduling in WLOADCTL is not only about executing jobs in the correct order. In real production environments, network interruptions, temporary database unavailability, etc., can happen, and they all cause failures.

In this article, we'll explore WLOADCTL's fault-tolerance capabilities, including:

• Automatic retry mechanisms

• Error ignore strategies

• Retry interval control

• Practical fault-tolerance scenarios

• Attribute inheritance and override mechanisms

---

Understanding WLOADCTLFault-Tolerance Strategies

1. Automatic Retry

When a job fails, WLOADCTL can automatically rerun it a specified number of times.

This is useful because many failures are just temporary. A database connection timeout, network latency issue, or temporary file lock may disappear after a short delay.

2. Error Ignoring

After reaching all retry attempts, WLOADCTL allows two possible outcomes:

Option A: Stop the Workflow

The job remains in a failed state and downstream jobs will not execute.

Option B: Ignore the Error

The job is marked as a warning rather than a failure, allowing downstream jobs to continue executing.

This behavior is controlled through the ignoreerr attribute.

3. Retry Delay Control

In many cases, immediately retries don't work because external systems need some time to recover. WLOADCTL allows you to set a gap time between retry attempts.

---

Core Fault-Tolerance Attributes

WLOADCTL implements fault-tolerance using three attributes:

Attribute	Description
maxnum	Maximum retry count
ignoreerr	Whether errors should be ignored
errdelay	Delay between retries (seconds)

---

maxnum

Controls how many times a failed job could be retried, default value is set to be 1, which means the scheduler performs one execution attempt.

There is also one special value, A value of 0 means unlimited retries until the job succeeds.

ignoreerr

Determines whether WLOADCTL should ignore the failure after finishing all retry attempts.

Value	Meaning
Y	Ignore the error
N	Do not ignore the error

The default ignoreerr value is set to be N:

errdelay

Specifies the waiting time between retry attempts. The defualt unit is Seconds, default value is set to be 30

---

Configuration Examples

The code belolw shows unlimited retries:

<sh>
    <name>job2</name>
    <progname>$HOME/myshell.sh</progname>
    <maxnum>0</maxnum>
</sh>

The following code means: retry 3 times, wait 10 seconds between attempts, then ignore the error if still unsuccessful:

<sh>
    <name>job3</name>
    <progname>$HOME/myshell.sh</progname>
    <errdelay>10</errdelay>
    <maxnum>3</maxnum>
    <ignoreerr>Y</ignoreerr>
</sh>

---

Case Study #1: Using maxnum=0 as a Workflow Trigger Controller

Although unlimited retries are not commonly used, they could become valuable when implementing workflow start conditions.

Consider the following example:

<sh>
    <name>job1</name>
    <progname>$HOME/mystartctl.sh</progname>
    <maxnum>0</maxnum>
    <errdelay>60</errdelay>
</sh>

The script mystartctl.sh can contain any custom logic required before a workflow is allowed to start.

For example:

#!/bin/bash

if complex_condition_is_met
then
    exit 0
else
    exit 1
fi

Behavior:

• Condition not met → Exit 1 → Retry after 60 seconds

• Condition met → Exit 0 → Workflow continues

This turns the scheduler into an event-driven scheduler without requiring external trigger services.

---

Practical Scenario #2: Dynamic Error Ignore Strategies

The ignoreerr attribute supports not only static values but also variables and expressions.

Using Variables

<sh>
    <name>job2</name>
    <progname>$HOME/mydeal1.sh</progname>
    <maxnum>6</maxnum>
    <ignoreerr>$(ISIGNERR)</ignoreerr>
</sh>

This allows a runtime error-handling mechanism.

---

Using Expressions

<sh>
    <name>job2</name>
    <progname>$HOME/mydeal2.sh</progname>
    <maxnum>6</maxnum>
    <ignoreerr>dayofweek() in (0, 5, 6)</ignoreerr>
</sh>

In this example:

• Friday, Saturday, Sunday → Ignore errors

• Other days → Do not ignore errors

The mechanism above makes plans more flexible.

---

Improving ETL Development Efficiency with ignoreerr

ETL workflows often contain:

• Hundreds of jobs

• Long execution times

• Incomplete test datasets

• Frequent intermediate failures

As a result, developers spend significant time manually restarting workflows after individual job failures.

A common strategy is to temporarily ignore those failure:

ignoreerr=Y

By doning so, developers are able to validate the entire workflow structure before focusing on individual job failures. It enables a continuous workflow execution and faster integartion testing. Also, it is easier to identify downstream issues.

---

Understanding Default Values, Inheritance, and Override

WLOADCTL provides an inheritance model that significantly reduces configuration complexity.

Consider the following example:

<serial>
    <name>MainModul_rootnode</name>
 <!-- Sets the error count for the entire module process to 3 -->
    <maxnum>3</maxnum>  
    <begin>
        <name>MainModul_beginjob</name>
    </begin>
    <sh>
        <name>job1</name>
        <progname>$HOME/study/job1.sh</progname>
    </sh>
    <serial>
        <name>serial_group1</name>
<!-- Sets the agentid attribute for all jobs in the serial group "serial_group1" to magt1; all jobs will run on the agent magt1 -->
        <agentid>magt1</agentid>  
        <sh>
            <name>job2</name>
            <progname>$HOME/study/job1.sh</progname>
<!-- Does not inherit maxnum=3 from the parent; must be reloaded and reset to 10 -->
            <maxnum>10</maxnum>  
        </sh>
        <sh>
            <name>job3</name>
            <progname>$HOME/study/job1.sh</progname>
        </sh>
        <sh>
            <name>job4</name>
            <progname>$HOME/study/job1.sh</progname>
        </sh>
    </serial>
    <end>
        <name>MainModul_endjob</name>
    </end>
</serial>

---

Inheritance

Child nodes automatically inherit attributes defined by parent nodes.

Example:

<maxnum>3</maxnum>

All descendant jobs inherit this value except other specified values.

---

Override

A child node can redefine an inherited attribute.

Example:

<maxnum>10</maxnum>

For job2, the inherited value 3 is replaced with 10.

---

Why Inheritance Matters

• The inheritance mechanism provides several important benefits:

• Cleaner Configuration: Common settings only need to be defined once.

• Easier Maintenance: Changing a parent attribute updates all inheriting child jobs automatically.

• Better Consistency: Large workflows remain standardized and easier to manage.

More Powerful Workflow Design

Inheritance can be combined with:

• Retry policies

• Error handling rules

• Agent assignment

• Conditional branching

• Environment-specific configuration

This makes WLOADCTL workflows more scalable and significantly easier to operate.

---

Conclusion

Fault tolerance is a fundamental requirement for enterprise workflow scheduling.

Whether you're implementing complex startup conditions, reducing ETL testing overhead, or standardizing large workflow deployments, these mechanisms can dramatically improve both operational reliability and development efficiency.

Feel free to leave any comments below

← Prev | Scheduled, Fixed-Frequency, & Execution Plans

→ Next | lean, serial, and the ostr Mutual Exclusion Attribute