13 Jul 2022 Exception Handling in KNIME using Try-Catch Nodes
KNIME is an open-source data science platform enabling organisations to discover business value hidden in their data. It allows you to blend various data sources, as well as to perform ETL (Extraction, Transformation, Loading), modelling, data analysis and visualisations with no or low coding.
In this blog post, we are going to look at exception handling using the try-catch nodes in KNIME, with a working example.
What is an Exception and How is it Handled?
An exception is when a runtime error or event occurs during the execution of a job or programme that stops the natural flow of the process or instructions.
Exception handling is the process of handling these runtime malfunctions by carefully inspecting the possible causes of failure.
We can handle exceptions using try-catch in all available programming languages. We can also handle these exceptions using the try-catch nodes in the KNIME analytics platform.
Try-Catch Nodes in KNIME
There are different types of try-catch nodes in KNIME:
- Try-Catch data ports.
- Try-Catch variable ports.
- Catch generic port.
Try-Catch Data Ports:
These nodes are available under Workflow Control->Error Handling in the KNIME Analytics Platform node repository. The nodes between the try and catch nodes are prone to failure in runtime and this can be considered as a top branch:
The catch node has two inputs: this node forwards the output from the top branch through its first input node if the execution is successful. If the execution on the top branch fails, then the output from the bottom branch port will be forwarded through the second input port of the catch node. The second output port (which is a variable port) contains the information about the observed error.
Try-Catch Variable Ports:
These nodes are useful when we want to pass on the flow variables which are modified or created between the try and catch nodes, even in the case of node failures or errors. The nodes between the try and catch variable nodes are prone to failure in runtime and this can be considered as a top branch:
The catch node has two inputs, and unlike data ports the try-catch variable ports pass the flow variables from the first port if the execution is successful.
If the execution on the top branch fails (and a matching try node is connected before the failing node) then the input from the second port, i.e. the flow variables (ex: _error_stacktrace, _error_reason, _error_node, _error_caught) will be passed, containing information about the observed error.
Catch Generic Port:
This catch port will handle the failures on the data table connections. The output of the catch node is a successful connection from the top or bottom branches, based on the configuration. If the execution is successful from the top branch which is enclosed by a try node, then the catch node passes the connection result from the top branch. Otherwise it will pass the connection result from the bottom branch.
Example Implementation of Try-Catch block in KNIME:
Use Case: Let’s assume that an ERP system produces an end-of-day sales transactions extract and uploads the file into a SharePoint folder. The file must be processed in KNIME and loaded into a database daily. Business analysts use this processed information to create their sales analysis reports, and any errors in this automated process can impact these analyses and must be duly notified to the respective teams for prompt action. In this blog post, we will look at how KNIME Try-Catch can address some of the common problems.
Data Source: A CSV file has been uploaded into the SharePoint directory.
Input file format: Meat_and_Egg_Production_+ File generation date in ‘YYYY-MM-DD‘ format.
Ex: Meat_and_Egg_Production_2021-11-22.csv, Meat_and_Egg_Production_2021-11-23.csv etc.,
Solution and KNIME flow:
Configuration Block:
In this block, we are using nodes like date&time input, string input and string manipulation to create the current date and append it with the file name as a flow variable dynamically. Finally, the string to path (variable) node will be used to convert the file name into a path which can be used by the CSV reader.
Try-Catch Block:
In the Try-Catch block, Microsoft Authentication is used to establish a successful connection with SharePoint online. The file to be processed is available in a directory within SharePoint. The naming convention of the file is matched to the input format mentioned earlier. This setup is enclosed between try-catch nodes to handle any exceptions and act accordingly. Some common exceptions are authentication errors, file not available, etc.
Further Action Block:
In this block, we have configured a rule engine based on the success/failure from the catch block. If the file for a processing date (current day) is available, then we can process the file as per the business requirements (not shown in the workflow). We can send an email to the business team to notify them that the file has been processed successfully. In addition to this, we can also trigger an email to the business team with the details of the processed data. If any exception arises, we will send a failure email with the details of the error to the IT team for investigation and further action.
Rule created for file upload failure/success
The try-catch block takes care of the scenarios below efficiently:
Scenario 1: File is not available for a given day.
Desired action: Workflow to send an email with the File unavailability error.
Scenario 2: Authentication is unsuccessful.
Desired action: Workflow to send an email with the Authentication error.
Scenario 3: No errors.
Desired action: Workflow to send an email notifying the successful processing of the file.
Conclusion
KNIME enables us to handle exceptions within its analytics platform using the try-catch nodes. While we can use a programming language to do this, having the option to use a KNIME node simplifies exception handling greatly.
If you have any queries about your own platform, or just want to get deeper insights into your data, don’t hesitate to contact our team of experienced professionals.
