Skip to main content

A Quick Talk On Generic Cloud App Integration With Informatica Cloud

The Informatica Cloud Integration platform is the industry's complete cloud integration platform as a service (iPaaS). Informatica has been continuously playing as the leader role in this space for the 4th consecutive year (Gartner's margic quadrant for enterprise integration platform). It's definitely your top choice for data integration management.

With the emerging SaaS application on the market, it's essential for integration platform to provide a flexible and efficient connector to integrate various cloud applications in a generic manner. Indeed there are many specific connectors built for individual applications by Informatica (Tier 1 and Tier 2 connectors) or vendors themselves (OEM connectors). But let's face it: most cloud applications evolves quickly on new features and their APIs get updated quicker than you realize. How quickly the connectors can turn around with the most recent API updates? The answer is definitely the connector update lags behind, as most connectors get updated once a year or longer than that.

For what's worth, Informatica introduced the Rest V2 connector in fall of 2016. The V2 connector is an successor of its ancestor which is just named as "Rest Connector". It allows to connect with the latest Rest API of the cloud application directly to read or write data. You can fully leverage it no matter the cloud application is a source, a target or a lookup system. It offers great improvements on flexibility, efficiency, ease of use besides performance. Considering the majority use of Rest API as the industry standard for emerging cloud services, this Rest V2 connector can fill in the role of a generic connector for cloud applications with Rest API. In this article, I will reveal the great features and caveats followed by uncovering some hidden gems of the Rest V2 connector that many may not have realized yet.

One of the great features of the Rest V2 is the introduction of hierarchy data support. As we know, the Rest API deals with either XML or JSON data which are hierarchy data structure. Before the appearance of Rest V2, transformation of hierarchy and relational data is cumbersome and even requiring writing your own script to transform the data as part of the integration process. At the Informatica fall release of 2016, the same time Rest V2 is introduced, the Hierarchy Builder/Parser is brought to life to fill the gap. Hierarchy builder is to build hierarchy data from relational data and hierarchy parse does the opposite way. Let's take a look at how hierarchy builder/parser works.

First, a hierarchy schema needs to be created. It is used by hierarchy builder/parser to present the data structure. To create a hierarchy schema, one will need to have a XML schema file (.xsd) or a JSON sample file (.json). Note the XML schema file is not necessarily provided by the vendor and one has to create it based on the XML payload.


After the creation of hierarchy schema, a hierarchy builder/parser can be used in a data mapping to map hierarchy data to relational data or vice versa.



That seems quite easy, compared to writing your own script to transform. But let's look at the Rest V2 connector as it's even better.

The core of Rest V2 connector configuration is the Swagger support. I have to thumb up on Informatica on the betting of Swagger because of its popularity and great potential as industry standard for API framework. Not all vendor provides swagger definition for their API. But no worry, Informatica provides this tool to you right inside your Informatica cloud web console. It's under Configure -> Swagger files. If you don't see this option, please contact with your Informatica support to enable it. You will need to have Rest V2 connector license to use it.

While creating the Swagger file, you basically defines the API details - URL, verb (POST/GET etc), authentication (username/password, token, etc), Headers, Query parameters, Body and sample response file. A couple of caveats in this process are:

  • The JSON response file is required even you are dealing with XML response. In this case, you will need to convert the XML response in JSON format. There are many online tools to do that.
  • If your raw body is XML format and you have issues creating the Swagger, after making sure nothing is wrong on your end, you may want to contact Informatica support requesting them to create the Swagger for you. Informatica support does have their own internal tool to create swagger file. Informatica will improve the Swagger tool based on my conversation with them and most likely it's no longer an issue soon.



After the Swagger file, create the Rest V2 connection which is quite easy as majority of the info needed for the API is filled in the Swagger file.


Now the Rest V2 Connector is ready to use in the data mapping. The Rest V2 can be used a source to read data or the target to write data in this case. Of course the API you put in the initial Swagger file has to comply with read or write operation. The hierarchy data mapping can be achieved in the Rest V2 Connector as a source/target itself. No need to use hierarchy builder or parser.

To save the best to the last, the great power of using Rest V2 connector is to use it as a midstream. Let's work on a use case: say I need to pull data from A system and push to B system via Rest API. If I were using Rest V2 as the target for B system. At the end of the flow, how do I know the data is successfully taken in B? I will need to start another pipeline or another data mapping task to examine the log file and take action from here. When using it as a midstream, the response of the Rest call can be transmitted to the next step for follow up action in the same streamline flow. This is a differentiator and made me believe all Rest V2 connector should not serve as target but midstream instead.

Another hidden gem on the Rest V2 connector as a midstream, is the capability of processing multiple Rest calls. This is definitely a great sweet feature. For any Rest call, one payload corresponds to one request and generates one response. If you have multiple payloads to be processed in one integration, essentially you need a loop function to loop through individual records and make the Rest call for each record. Without Rest V2 connector, this is impossible as there is no such loop function in Informatica data integration. The alternative would be to use Informatica Cloud Real Time (ICRT) which provides a workflow type of designer that a task can be jumped from one step to another based on certain condition - basically to achieve loop function. With Rest V2 connector, there is no additional configurations needed and it can process multiple records in one step - taking multiple records as inputs, make Rest calls for each record, and generate all response as outputs. While using it as a midstream, we can easily follow up on actions with the Rest call outputs in the same streamline flow.

Comments

shalu said…
Thanks a lot for explaining practically. Fantastic Post! RPA Training in Chennai. Get more information Robotics Process Automation Training in Chennai
Unknown said…

I appreciate your work on Informatica. It's such a wonderful read on Informatica.Keep sharing stuffs like this. I am also educating people on similar technologies so if you are interested to know more you can watch this:-
https://www.youtube.com/watch?v=56vMQ1lG-vc
Akash said…
Informatica training and Robotics Process Automation training is very popular nowadays. There are more institutes providing RPA training in Chennai. Learn to shine your skills.
Unknown said…
Super very nice post and thanks for sharing this useful information
Informatica Training in Chennai | Informatica Training Institute in Chennai
https://zappysys.com/blog/read-json-informatica-import-rest-api-json-file/

JSON / REST API is becoming more and more popular each day as everyone embrace cloud-centric services. Informatica Read JSON is primarily focused on Informatica users who want to do XML SOAP/ JSON / REST API Integration in Informatica. However many tips and techniques described here that will help to understand how to integrate XML SOAP / JSON / REST API in other ETL / Reporting apps such as Tableau, Power BI, SSRS, Talend, Excel and many more.
Unknown said…
https://zappysys.com/blog/read-json-informatica-import-rest-api-json-file/

JSON / REST API is becoming more and more popular each day as everyone embrace cloud-centric services. Informatica Read JSON is primarily focused on Informatica users who want to do XML SOAP/ JSON / REST API Integration in Informatica. However many tips and techniques described here that will help to understand how to integrate XML SOAP / JSON / REST API in other ETL / Reporting apps such as Tableau, Power BI, SSRS, Talend, Excel and many more.
likitha said…
This comment has been removed by the author.
rama said…
The post was amazing. It showcases your knowledge on the topic. Thanks for Posting.
CPHQ Online Training in Kabul. Get Certified Online|
CPHQ Training Classes in Al Farwaniyah
manisha said…
Thank you for sharing such great information very useful to us.
Informatica Training in Noida

Skein Tech said…
Wow, amazing blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your website is fantastic, let alone the content!

3d animation Company
Best Chatbot Development Company
Mobile app development in Coimbatore

easylearn said…
Hi,
Good job & thank you very much for the new information, i learned something new. Very well written. It was sooo good to read and usefull to improve knowledge. Who want to learn this information most helpful. One who wanted to learn this technology IT employees will always suggest you take hadoop training in bangalore. Because data science course in pune is one of the best that one can do while choosing the course.
Janardhan said…
I Got Job in my dream company with decent 12 Lacks Per Annum salary, I have learned this world most demanding course out there in the current IT Market from the Hkbk group of institutions experts who helped me a lot to achieve my dreams comes true. Really worth trying
James Zicrov said…
Thank you for putting up a descriptive post on a very useful aspect of Informatica. This actually helped me a lot to understand this topic.

Informatica Read Rest API
Roger Binny said…
This comment has been removed by the author.
Arya Steve said…
I think this is a really article. You make this information interesting and engaging. You give readers a lot to think about and I appreciate that kind of writing.ict solutions company bahrain
James Zicrov said…
I think that is actually a very nice and smart explanation about Informatica installation and its other useful aspects.

Informatica Read Rest API
Hi...
Nice Blog. Cloud Based technology are the most trending now a days. Informatica Cloud Integration platform provides us a complete cloud integration as Paas ( Platform as a service ). Also so many connectors are available to integrate your generic app with Integration Platform. Guidance you have provided is very useful for beginners.
pointraiser said…
A great article on Informatica. Your website has been a great help in helping me learn more about Data Integration and ETL.
senthilr7692 said…
This blog has the relevant data according to the topic and precise to the topic.
Data Science Course in Chennai
Data Science Course in Bangalore
Chris Lesnar said…
I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. In fact your creative writing abilities has inspired me to start my own BlogEngine blog now. Really the blogging is spreading its wings rapidly. Your write up is a fine example of it.
powerbi read rest