Authors: PdtUser1210 Skip navigation

Cloud Data Integration

10 Posts authored by: PdtUser1210

We kicked off the month of October by holding another session on cloud integration templates - this time building an entirely new one, as opposed to using a prebuilt template within Informatica Cloud. Our last session on templates focused on a specific customer use case - a Zuora to Great Plains template with some complex SQL Server stored procedures.

 

We went over some of the most important transformations (such as Joiners, Filters, Normalizers, and Routers), and what they're used for. We also went over certain "Template Stubs" (which consist of a combination of prebuilt transformations) that can help you be even more productive as you build your custom integration template. These template stubs broadly fell into 3 categories:

 

1) Accessing Data Sources

2) Transforming, Filtering, & Aggregating Data

3) Routing & Joining Data

4) Looking up Data

5) Writing to Targets

 

You can view the slides here as well as the recording featuring Naresh Govindaraj from our product management team. Tomorrow's TechTuesdays session on Oct. 8 at 830am is on an exciting topic, REST Web Services and JSON, and represents a snapshot of what's going on in the evolution of the internet.

Our TechTuesdays session on September 24th focused on what happens during large deployments of Salesforce or Oracle database clusters. Typically, before any such deployment takes place, stringent testing of the integration environment needs to happen to ensure that the production environment functions as smoothly as possible.

 

The only way to do this is to create a copy of the production data in the testing environment, and test out all the integration mappings there before pushing into production. At present, when using Salesforce, developers have two options:

  1. Use developer or configuration-only sandboxes
  2. Use full sandbox copies

 

Either approach carries with it a lot of time, cost, and/or risk. With developer-only sandboxes, test data sets are not representative of their final production datasets and as a result, can omit a lot of crucial integration scenarios. With full sandbox copies, costs skyrocket tremendously, and can lower the ROI of a Salesforce deployment.

 

An alternative solution instead, is to copy over only a subset of the main production data, and mask any personally identifiable or confidential information (such as credit cards, social security numbers, or dates of birth). The main demo component of TechTuesdays focused on this aspect.

 

Here are the slides for this session, as well as the recording. Tomorrow's TechTuesdays session focuses on building integration templates from scratch.

On our seventh Informatica Cloud #TechTuesdays session on September 17th, "Cloud Migration: Sandbox Best Practices", we talked about the importance of testing out your integration workflows before deploying them into production. Using an Informatica Cloud sandbox has three primary benefits, especially for companies focused on improving the productivity of their integration projects without compromising on security:

 

  • Security: The Informatica Cloud Secure Agent can be deployed into separate virtual LANs, each one demarcated away from any sensitive corporate IT resources
  • Streamlined Integration Lifecycle: With all the different stages in an integration lifecycle (development, testing, and production), having Informatica Cloud sandboxes, makes it easy to move approved and tested integrations from the testing phase, directly into production
  • Debugging: If any bugs have been encountered in a production environment, it is very easy to to isolate, trace, and resolve these bugs in the testing environment within sandboxes

 

Krupa Natarajan from our cloud product management team ran through a demo showing how to move objects from a sandbox environment to production, and ran through the following demo scenarios:

  • Having the 'Migrate Objects' functionality enabled by your Informatica Cloud Administrator
  • Credentialing, and access between sandbox and production accounts
  • Ensuring that users have the correct privileges to either read/write/update objects and tasks via Fine Grained Access Controls
  • The types of "objects" that would be migrated, be it an application's object, a connection name, a plugin, etc.
  • Keeping migration privileges separate from task-related privileges
  • Editing connections to point to a different connection, especially when having separate test accounts for your SaaS applications as well

 

You can download the recording to view the replay, as well as the slides. Tomorrow's TechTuesday session on 9/24 at 830am PDT will go one step further, and focus on Data Masking - this is a very useful feature to have when you want to copy a subset of your production data into a testing environment in order to mimic real-life scenarios, but wish to keep confidential information private and secure.

During our sixth Informatica Cloud #TechTuesdays session, "Reusable Integration: The Power of Integration Templates" on September 10, we had a customer, Intralinks show us their use case of integrating Zuora with Great Plains using Informatica Cloud's templates. The main integration scenario revolved around the need to access multiple invoice objects from Zuora and integrate them with the main financial and accounting components within Great Plains.

 

However, there was also the issue of certain legacy stored procedures from SQL Server that needed to be combined with the Zuora data before being integrated with Great Plains. Intralinks was able to use several joiner and filter conditions to make this happen via their template, as you can see below:

 

Intralinks Zuora GP.bmp

 

Amol Dongre from our Engineering team also covered how to import templates from the shared master org using object migration. These templates are being made available to all current Informatica Cloud customers so contact me at aviswanath@informatica.com if you'd like the password to this repository.

 

Here are the slides and the entire recording. Tomorrow's  session  on September 17 will focus on using sandboxes within Informatica Cloud and best practices for separating out your development, test, and production environments.

During our fifth Informatica Cloud #TechTuesdays session, "SOAP Web Services Made Simple" on August 20, we focused on the fact that there are several legacy on-premise systems that need to communicate with modern SaaS applications such as Salesforce. The easiest way to establish such a communication path is to put a SOAP web service wrapper around the legacy system, along with a WSDL that appropriately describes all the desired methods and operations.

 

We also discussed the fact that SOAP web services can be of two different kinds - procedural, or document style. A document-style web service is a bit more complicated than a procedural web service because they are not self-describing through the WSDL alone – you would need the WSDL as well as the XSD schema to use the service.

 

During the demo component we showcased the following aspects of using Informatica Cloud for SOAP web services:

  • Connecting to the main WSDL URL
  • Looking through the WSDL URL's binding information to put the appropriate service into the endpoint URL
  • Using the 5-step wizard to map each web service to the appropriate fields within a Salesforce object
  • Applying filters to a web service to extract only relevant information
  • Using task flows to execute consecutive web service calls

 

Here are the slides and entire recording featuring Bryan Plaster from our Cloud Labs team. Next week's  session on September 10 will focus on using integration templates, and will feature an actual customer taking us through a template use case.

Our fourth Informatica Cloud #TechTuesdays session, "Benefits of Using REST APIs in Integration", focused on the specific resources that Informatica Cloud's REST API provides, and the various scenarios in which they can be used. We first went over the most common kinds of requests (such as GET, POST, PUT, DELETE) that REST calls make, and then gave an overview of the resources that the Informatica Cloud REST API provides. These resources can be broadly grouped into the following categories:

  • Activities
  • Secure Agent Details
  • Jobs
  • Logins
  • Integration Templates
  • Workflow Details
  • Schedules

 

When using Informatica Cloud's REST API, it is very important to know what the header configuration looks like, and also to use only Version 2 of the REST API. The demo showcased the following scenarios:

 

  • Embedding integration into a SaaS Application
  • Job scheduling & monitoring
  • Administering new users

 

Here are the slides and entire recording featuring Amol Dongre from our Cloud Engineering team. Tomorrow's session will focus on using SOAP web services within Informatica Cloud.

Our third Informatica Cloud #TechTuesdays session, "Simplifying SAP Connectivity with Cloud Integration", focused on how cloud applications were rapidly gaining traction, especially in the CRM space, and needed to connect to SAP R/3 to access crucial customer, pricing, and product master data. The reason why integration to SAP is so important is because over 25% of ERP deployments in 2012 were based upon SAP (Souce: Forbes, Gartner).

 

http://b-i.forbesimg.com/louiscolumbus/files/2013/05/ERP-Market-Share-2012-Stats.jpg

 

Although SAP only grew 2.2% from 2011 to 2012 while other cloud-based  ERP vendors such as Workday, and NetSuite grew 114.7% and 34%  respectively over the same period, the fact remains that as a category,  SAP still holds the lions share of the ERP market. Coupled with the fact  that CRM is one of the fastest growing categories of cloud  applications, it's not hard to see why integration between cloud CRM  applications (such as Salesforce and Microsoft Dynamics CRM Online ) and  SAP is such a sought-after solution.We discussed three primary use  cases for SAP integration: data warehousing, data migration, and data  synchronization. Some of the most exciting features of Informatica  Cloud's

SAP connector that were shown on the demo was the ability to toggle between the  technical and business names in R/3, and how easy it was to extract data  from transparent, clustered, and pooled tables and views. The demo  zeroed in on the following themes:

 

  • Loading SAP customer info into Salesforce
  • Bulk extraction from clustered tables to flat files
  • Using Salesforce outbound messaging
  • Performing lookups

 

Here are the slides from the session, and the entire recording featuring Anand Peri, who takes care of product management for our SAP connector.

 

Session 4, which focused on using the Informatica Cloud REST API for integration just got over this morning and we'll have the slides and recordings up soon.

Our second Informatica Cloud #TechTuesdays session, "Accelerating Big Data Initiatives through Cloud Integration", shone a light upon the growing use of Big Data for datawarehousing projects. We cleared the confusion around many of the flavors of Big Data out there, such as Hadoop, and focused the discussion on Big Data providers in the cloud, such as Amazon RedShift.

 

Big Data has several use cases such as data warehousing, predictive analytics, machine data, and OLTP, and we decided to tackle the data warehousing use case. When looking at the industries that had the fastest adoption of Big Data, we found out, not suprisingly, that the banking, media, and government industry verticals led the way (Source: Forbes, Gartner).

 

http://blogs-images.forbes.com/louiscolumbus/files/2012/08/big-data-heat-map-by-industry.jpg

 

During the session, we discussed that the main drivers behind moving to cloud-based Big Data for data warehousing projects was because of the speed with which you could provision multiple database nodes. Other benefits involved saving on costs of provisioning multiple on-premise databases, as well as the ability to start petabyte-scale data warehousing projects a lot sooner. The demo itself touched on the following aspects of using Amazon RedShift:

 

  • Configuring RedShift for first-time use by downloading SQL Workbench
  • Ensuring that security groups were set up correctly
  • Writing to Amazon RedShift using the Informatica Cloud connector
  • Reading from Amazon RedShift using ODBC

 

Here are the slides from the session, and the entire recording featuring Vijay Narayanan, who takes care of product management for a lot of our new connectors.

 

Session 3, which focused on SAP integration just got over this morning and we'll have the slides and recordings up soon.

Last Tuesday we held our first ever Informatica Cloud #TechTuesdays session, "Better Business Intelligence: Blazing Fast Data Replication for Cloud Apps".

 

We focused on how cloud applications such as Salesforce were built from the ground up to allow rapid customization, and as a result, contain numerous custom objects and custom fields. As a consequence, when extracting data from Salesforce into a staging database, it is important to ensure that all the changed fields have propagated their way into the relevant mappings for the business intelligence process.

 

The demo focused on four critical elements of a data replication process:

 

  • Replicating Deleted & Archived Rows
  • Auto-creation of Database Tables
  • “Full Load” vs. “Incremental Load”
  • Using the Salesforce Bulk API

 

Here are the slides from the session, and the entire recording featuring Ron Lunasin who heads our Cloud Platform Adoption team.

 

Session 2 takes place tomorrow, Tuesday, July 30th at 830am PDT and will focus on Big Data, specifically how to perform data warehousing through the cloud with Amazon RedShift.

Informatica Cloud will be holding weekly webinars starting Tuesday July 23rd, covering a wide range of deeply technical issues with experts from our product management and engineering team. The first session is on performing BI with cloud applications such as Salesforce using Data Replication. Subsequent sessions focus on Big Data with Amazon Redshift, web services, SAP integration, integration templates, and much more.

 

You can view the full list of sessions here and register for any that you find interesting. The schedule currently runs until October 1, and will be updated with more sessions in late August.

 

Please submit any questions you'd like answered during the webinars about Informatica Cloud and we will do our best to answer them during a session that is relevant to the topic. You can submit your questions either on this Community site, the Informatica Cloud LinkedIn group, or by tagging us on the @infacloud twitter handle.