Informatica 10.0

2 posts

Informatica 10.0 Update 1 is now available

Posted by Vidya Mar 7, 2016

Informatica BDM 10.0 Update1

What are we announcing?

Informatica BDM is releasing update1 on top off v10.

Who would benefit from this release?

This release is for all customers and prospects who need big data management on V10. This release addresses some key features around performance and connectivity.

What’s in this release?

Informatica customer base we see a strong interest in our largest customers to start or expand their Big Data management solution. As part of this, we need to address some key challenges around performance, connectivity & PAM. In the update1 release, we are addressing some of the critical gaps around it. High-level themes for this release are:

Performance ( Hardening Blaze and adding more functionality)
Reuse ( Ability to generate a PC report on reusability)
PAM ( Updating to new versions)
Connectivity ( Improving Teradata & Netezza connectivity and many more)
Functionality ( Key functional gaps like HS2,update strategy etc)
Enterprise Information Catalog – Standard Edition

Details on each of these themes:

Performance
- Blaze
  - Hive source and target
  - Map Side Join
  - Partitioning support for small file
- Connectivity
  - Teradata, Netezza partitioning
- B2B DT Performance Improvements
Reuse of PC Mappings in BDM
- Reuse report
PAM Updates
- CDH 5.5
- IBM BI 4.1
- HDP 2.3
- MapR 4.0.2
Functionality
- Parallel workflow
- HiveServer2 & Update strategy support ( Update 3 merge in v10)
- BDE Util Enhancements.

Enterprise Information Catalog – Standard Edition

New Connectivity
- Tableau Scanner: Support Tableau Server as a metadata source.
- IBM Cognos Scanner: Support IBM’s BI reporting tool as a metadata source.
- Hive on MapR
New Hadoop Distro Support
- Starting Update 1, deployment is also supported on an existing Hortonworks cluster
Performance Enhancements
- Improved column profiling performance by ~3X
- Improved aggregate linking propagation performance resulting in >40% improvement in overall scanner execution.
PAM
- CDH 5.5, HDP 2.3

Big Data Management Version 10.0 Update 1 Release Notes

Informatica Live Data Map Version 1.0 Update 1 Release Notes

PAM for Informatica Platform v10-Update 1

You can download the HotFixes from here.

Informatica 10.0 is now available

Posted by Vidya2 Dec 7, 2015

Informatica 10.0 includes the following new capabilities:

Informatica Platform

Big Data Management

Informatica Blaze: Blaze is Informatica’s new high-performance, cluster-aware data integration engine based on YARN
Smart Executor: Ability to pick the best execution engine for big data workloads
Parameterization support for big data workloads
Dynamic mapping support for big data
Team-based Development and Versioning
Acquire exclusive object lock to prevent simultaneous edits.
Create object versions in an external version control system (such as Perforce or Subversion).
View object details for an earlier version in read-only mode.
Restore an earlier object version as the latest version.
Revert a check-out operation.
Perform administrative tasks such as reassigning checked-out objects to another user.

Partitioning

Support for MxN partitioning with the ability to distribute data across multiple partitions
Support for sort order
Cardinality-based costing to determine the number of partitions on each transformation
Partitioning in sorted merge join and sorted input aggregation
Maximum parallelism for a mapping and suggested parallelism value for transformations

Optimization

Branch pruning
Global predicate optimization
Top-down optimizer
Dataship join

Relational Pushdown

Full mapping pushdown
Update strategy support
Lookup support

Parameterization

Extensive parameter support in source, target, and transformation conditions and properties
Support for new parameter data types
Ability to bind parameters for reusable transformations and mapplets
Parameter Set – new object stored in repository and contains parameter names and values
Mapping output returning aggregated values from the mapping run to support incremental extract use cases

Hybrid-IT

New architecture separating service and compute components providing better resource usage
Node roles: Service, compute, service and compute
Support for running jobs in remote processes

Domain Services

Single instance system services created out of the box
Email, Resource Manager, Scheduler

Monitoring enhancements

Bird’s eye view of the domain and service health (current and past)
Monitor progress of jobs in detail with graphical visualizations
Summary view of past job executions
Introduction of new out-of-the-box Operator role and Operator group

Dynamic Mappings

Dynamic schema for relational and flat file
Extensive parameterization of source, targets, and transformations
Dynamic types and rule-based propagation of ports
Dynamic expressions support to automatically create multiple expression based on rules
Create target definition by using mapping flow and create DDL at runtime

Mapping enhancements

Target load ordering, commit interval, and stop on errors

Performance enhancements

Out-of-the-box performance enhancements of the data integration transformations
Dynamic switching of ASCII or Unicode mode at mapping run time
Variable length cache
Performance improvements in design-time metadata access and development

Usability improvements

Data object enhancements: Create DDL at design time, add target based on transformations, switch connection types
Canvas improvements, Test expressions, Flat file physical data object editor improvements
Marketplace integration

New Transformation: Hierarchical-to-Relational

Informatica Data Services

Improved top-down cost-based optimizer
Global predicate optimization to analyze mappings globally and push down predicates
Mapping pruning optimization improvements
Dataship-join optimization

Connectors

New SAP Connectors for Informatica

SAP Table Reader
SAP BW OHS Reader
SAP BW Loader

New Application and Technology Connectors for Informatica

JD Edwards EnterpriseOne Connector (Reader and Writer)
LDAP Connector (Reader and Writer)
Microsoft Dynamics CRM Connector (Reader and Writer)
ODATA Reader

Major Enhancements for Greenplum

Partitioning support for Greenplum native bulk writer in Informatica
Full parameterization support for Greenplum bulk writer in Informatica

Major Enhancements for Teradata

Teradata 15.1 certification
Teradata bulk read support in Informatica
Partitioning support for Teradata native bulk writer (with TTU 14.1 version)
Full parameterization support for Teradata native bulk writer

Major Enhancements for Netezza

Support for bulk read and write in Informatica
Netezza 7.2x and Windows 2012 R2 (7.1) certification

Major Enhancements on Platform

Full pushdown support for relational sources in Informatica Developer
Timestamp with Timezone datatype support for Oracle
Decimal data type with precision 38 support for relational databases (Oracle, DB2, My SQL, and SQL Server)
Design and runtime DDL generation by using Informatica Developer

Major Enhancements for Oracle

Upgrade from OCI 8 to OCI 11.2

Major Enhancements for SQL Server

SNAC (OLEDB) based connectivity deprecated with new ODBC connector

Enhancements to Technology Connectors for Informatica

HANA (ODBC) parameterization and full pushdown support
Parameterization support for social media adapters

Enhancements to Application SDK

Support for partitioning, parameterization, and pushdown filter
Reject file creation
Support for C-based adapters
Multiple NMO support

Informatica Data Quality

Rule Specifications

Additional functions - contains, search in list, default values
UI and UX improvements
Use existing rules within rule specifications
Validity periods
View linked assets (rules, reference tables)

Universal ID

UI enhancements with wizard-driven store creation and automatic configuration
New store policies
Turn match off during store management operations for better performance

NLP Features

New classifier model editor UI
Bulk labelling training data model
Engine updates:
- Classifier engine - v2.0.7
- Probabilistic engine v3.4.0

Exception Management

UX improvements
Faster response time
Seamless handling of simultaneous multiple tasks
Task progress information

Address Validation

Integrated with the latest Address Doctor engine v 5.7
Added NIS code for Belgium
Added FIAS_ID for Russia
Added UPRN for UK
Added multi-language support for Switzerland

Platform Capabilities

Scheduler

Schedule mapping and workflow runs at specified times in future
New schedules tab in Informatica Administrator
Infacmd command line support

Workflow

Standardize on Informatica BPM Technology
Functional improvements
Integration with monitoring
Return codes for infacmd execution

Business Glossary

Email Notifications

Change notifications are delivered to the user over emails
In case of approval voting, ability to navigate to the voting dashboard from email
Users can opt out of email (and regular) notifications

Approval Workflow

Provides a default workflow for BG asset approvals
Supports two-level voting approvals
Displays voting dashboard while the asset is in approval
Can be used with Business Initiatives

Support for Rich Text, Images, and Attachments

Format business terms and descriptions in more meaningful ways including:
- Bold, Italics, Underline, Bulleted Lists, etc.
- Hyperlinks to other Glossary Terms, internal sites, etc.
- Inline images
Users can also add Images, Diagrams, and PDF attachments for further information

Import and Export Enhancements

Clear separation of import/export of models from import/export of glossary assets
Include audit history and attachments during import/export
Summary of assets being imported/exported
Run the import in background and be notified after completion

Relationship View Diagram Enhancements

Dynamically expand or collapse related nodes
Highlight based on relationship type
Find in canvas
Display full asset name and relationship name

Analyst Tool Privileges

View Glossaries privilege introduced in Informatica Administrator
Enable bulk permission to users on assets based on category
Predefined permissions for OOTB roles

Default Approvers

Supports a preset of default approvers for term revisions including stakeholders, owners, and stewards of related assets
Configurable at the glossary level

Performance Improvements

Improved the average response time for Business Glossary approximately by 34x. 85% of the response time is sub 5 seconds in v10 compared to the 17% in v9.6.1. 0% of the response time is now over 10 seconds compared to the earlier 60%.
Importing glossaries from excel is ~5x faster compared to 9.6.1

Metadata Manager

New Catalog User Experience

New Catalog List View, which loads faster, scrolls less
Tree View is maintained for backward compatibility
Tabular view for Related Catalog Objects with filtering and sort options
Recent History: View and access recent objects used in the session

Improved Lineage Performance

Performance of MM lineage rendering improved by up to 50x
Uses a graph database (OrientDB) for lineage computation
Graph DB is bundled with MM
Ability to cancel a lineage calculation and rendering

Summary Lineage for PowerCenter Mappings

Summary Lineage shows only end-points of PowerCenter mappings by default
Users can drill-down to get detailed lineage flow

Object Filtering in Lineage

Filter Objects and Object Types from Lineage Diagram

New MDX for Tableau to extract Tableau Server Metadata

Support Tableau Server as a metadata source

Parameters Support for Informatica BDM

Support for Source/Target/Lookup Parameters: Connection/Schema/Table/File Location/File Name
Support for Parameter Sets
Will work with 10.0 repositories only

Cancel Lineage Diagram Rendering

Canceling Lineage Diagram Rendering on the UI will now clear any associated back-end processes resulting in reduced loads on MM server and faster performance

Impala Queries in Cloudera Navigator Resource

Impala Queries are now available as resources from Cloudera Navigator source.

PowerCenter Session Task Instances in Impact Summary

Include PC Session Names in Impact Summary for better traceability

Profiling and Discovery

New Profile UI and Visualizations

Profiling Results: Summary view
- Unified result view for column profiling results and data domain discovery
- Intelligent Facets are created dynamically based on the results
- View Input columns for Rules
- Comments and Tags at the column level
- Lazy loading of column/rule groups
Profiling Results: Detailed View
- New Value frequency graph
- Sparkline to see Null, Unique and Non-Unique value trends
- Drill down column highlighting and Data preview

Outlier Detection

Profiling will identify any pattern, value or frequency that does not fall in the expected range of majority values in a column. These will be identified as Outliers.
You can highlight the Outliers by choosing the “Highlight On” option in the UI.

Compare and Historical view

Understand data quality trends through time by comparing historical profile results
Compare column profiles and rule profile results between two profile runs.
Detailed comparisons including changes in data types, patterns, nulls and distinct counts.

Profiling/BG Integration

Link columns with Business Glossary terms to provide the right business context to analysts.
Users can see Glossary term recommendations based on inferred data domains.
Once linked to the term, the link to profiling object can be seen from the glossary term

Profiling XML/JSON

Profile XML and JSON files on native file systems or Hadoop
Ability to profile multiple XML/JSON files with same schema
Ability to select groups and columns for profiling

Scorecard Filters

Filter can be created for each Metric in the scorecard
Active filter from Profile will be carried forward to scorecard
Each filter metric will run in a separate mapping.

Column Level Tags and Comments

Add comments to a profile object or columns in the profile to provide ancillary information for further collaboration and analysis
Export comments as a virtual column to a CSV file along with the profile results. The CSV file contains all the comments for a profile and columns in the profile.
Ability to add tags to profile columns from both summary and detailed views

Support for New Platform Features

Team Based Development Support
Next Generation Mapping Support
Profiling Custom Data Types
Operator Persona Support
Enhanced Performance from Pushdown Optimization
Partitioning Support
Hybrid IT (Grid)

PowerExchange Mainframe & CDC

More Intelligence

While the ideal is point, click & acquire. Data from the mainframe has always been seen as “different”. IMG II (Intelligent Metadata Generation) is helping make mainframe data just as easy to process as anywhere else, with utility driven Datamap generation and datamap to file view filtering.

Enhanced Monitoring & Reporting

Remote PowerExchange logger benefits from more granular reporting around what it is doing, helping Administrators keep control of processes.

Filling the DataType gaps

LOB Datatypes are being introduced to the PowerExchange architecture. In Version 10 DB2 (z/OS) gets “Read” & “Look-up” support for this somewhat complicated datatype.

VSAM CDC

ESDS joins the family of CDC supported VSAM connectors

Oracle gets IOT support

“Index Organised Table” support for PowerExchange Oracle Express.

Minor Enhancements

PAM Adabas 8.3, Datacom V15
PowerExchange Navigator support for Restart token specification when using row-test against CDC data streams.

B2B Data Transformation

New Library Object to support industry standard message customization
Improvements to RunMapplet feature
Improvements in XmlToXlsx processor leveraging existing template to generate Excel file