NiFi Adapter for NetSuite

Vishal Augustine,

Vice President, Product Development

How do you connect to NetSuite from NiFi, and with a connection that is highly configurable? How do you create a framework where connecting, transforming, and testing any new NetSuite end-point can be done in only a few minutes?

This can be achieved through a combination of NiFi, Python and config files. Below are the steps:

Step 1: Have a clear configuration approach and keep the configuration files ready.

In nifi.properties file, set the nifi.variable.registry.properties variable to your application’s “master” NiFi config file. In this case, the master config file is xfactrs.properties.

NiFi is designed where the NiFi instance has to be restarted if the .properties files are modified; only then will the variables in .properties file take effect. This is highly inconvenient in a production environment. To avoid such “restarts” for the NetSuite adapter changes and to keep the connections to each external system modular, it is recommended to create an interface-specific config file. Refer the config file xfactrs.properties as shown below. This is like a pointer to a pointer to a pointer.

Within xfactrs.properties file, add the following variables:

# Point to the NetSuite config file location ns_conf_file=/path/to/conf/netsuite.conf

# Point to a working directory

work_dir=/path/to/working/dir/tmp

# Point to the python script location

ns_adapter=/path/to/bin/erp_ns_generic_api.py

# Point to the python binary

python=/usr/bin/python3

Put all the NetSuite-specific variables into the NetSuite Config file – netsuite.conf

# NetSuite OAut1.0 authentication info/section

[connection]

token_key=<token key – obtain from the NetSuite Server side>

token_secret=<token secret – obtain from the NetSuite Server side>

cons_key=<consumer key – obtain from the NetSuite Server side>

cons_secret=<consumer secret – obtain from the NetSuite Server side>

realm=netsuite-realm-number/account-number

# Example of Functional/Interface section – GL

[GL]

# NetSuite End point

url=https://<realm>.suitetalk.api.netsuite.com/services/rest/query/v1/suiteql

http_method=POST

# NetSuite Query

suiteql=SELECT * FROM TransactionAccountingLine

Step 2: Connect to NetSuite using Postman and derive the Python connection code snippet

Use postman to test the basic connectivity and authentication. The above-mentioned SuiteQL could be kept empty. Use any simple end-point URL. Once the connection is successful, from Postman request/response, download the Python code snippet related to the connection. Opt for the Python snippet that uses the “request” module.

Note: This code snippet will be a useful reference in Step 3.

Step 3: Write/Test the Python code

There are numerous ways to connect to NetSuite using Python. While this article will not go into all of the details, here are a few pointers to keep in mind while writing the Python code:

1. The Python script should be able to read the above-mentioned netsuite.conf file.
2. NetSuite usually uses HMAC SHA256, and hence the OAuth signature generation has to keep using SignatureMethod_HMAC_SHA256.
3. Based on the script’s command-line argument, the Python script shall handle only one functional/interface section.

Step4: Configure NiFi

Configure NiFi to invoke the Python script using ExecuteStreamCommand Processor.

Depending on the type of NetSuite endpoint required, the corresponding section name can be invoked through a command-line argument from within the ExecuteStreamCommand Processor.

As a best practice, it is recommended to use the JOLT Transformation processor after ExecuteStreamCommand to make future transformations much faster.