Informatica 10.5 Release Announcement

Version 3

    Informatica技术超群(QQ群: 112443162) 杨晓东 整理。

    欢迎您加入Informatica技术超群(QQ群: 112443162, 92949669)。

     

    找工作,找项目,找老师,找朋友,找买卖,找理财,找旅游,找健康,找啥都有!

    Informatica技术超群(112443162),一个神奇的Q群!

     

     

    Informatica 10.5

    Release Announcement
    March 2021

     

    What are we announcing?

    The release of Informatica 10.5

    Who would benefit from this release?

    This release is for all customers and prospects who want to take advantage of the latest PowerCenter, Informatica Data Quality, Data Engineering Integration, Data Engineering Quality, Data Engineering Streaming, Data Privacy Management, Enterprise Data Catalog, Enterprise Data Preparation, and Test Data Management capabilities.

    What’s in this release?

    This release provides new features, latest ecosystem & connectivity support, security enhancements, cloud support, and performance enhancements while improving the user experience.

    Some of the key features include:

    Data Engineering Integration

    New Features

    • Mapping audit: You can validate the consistency and validity of the mapping jobs by creating audit rules and conditions. Rules can be scheduled before and after the mapping runs, either from the Developer tool or infacmd.
    • File manager utility: You can administer the preprocessing and file watching capabilities for a cloud ecosystem. The utility eliminates the need for a complex setup or custom script. You can use the credentials and connections set up within Informatica Administrator for file management.
    • Claire Recommendation: Enterprises that use Enterprise Data Catalog to tag columns as sensitive can share the information with developers. This information empowers developers to take appropriate masking actions to secure data in the Data Engineering pipeline.
    • CI/CD: You can compare objects across services in the same domain and between services across domains.
    • Debugging: The LogPacker utility can now aggregate logs from ephemeral cloud clusters and Spark job servers. A new API is also available to trigger and perform log collection programmatically.
    • Dynamic flattening in mappings: You can flatten complex hierarchical data types in dynamic mappings. Dynamic flattening improves the reusability of mappings when files are processed with hierarchical data types.
    • Multi-match lookups: Helps developers leverage the full functionality of feature-rich Informatica lookups in mappings that run on the Spark engine.
    • Databricks warm pool: Users of the Databricks pushdown capabilities can leverage warm pool instances to shorten cluster startup time with ephemeral or standard clusters from Data Engineering Integration.
    • File watcher: Added support for file watcher or file-preprocessing commands that allows file copy, read, list, rename, move, and watch.

     

    PAM Update

    • Databricks 5.5, 7.3
    • Google Dataproc 1.5
    • Cloudera CDP Public Cloud 7.2 (AWS and Azure)
    • Cloudera CDP Private Cloud 7.1.4,7.1.5
    • Microsoft Azure HD Insights 4.0
    • Hortonworks HDP 3.1.5
    • Cloudera CDH 6.3.4 (with EBF)
    • Amazon EMR 5.29, 6.1

     

    Spark PAM Update

    • Spark 3.0.1 support
      • Databricks 7.3
      • Amazon EMR 6.1

     

    PowerCenter

    • Parameterized connect string: Provides the ability to parameterize the connect string attribute in relational connections for the Oracle database.
    • Oracle Multitenant CDB/PDB support: Applies to the domain and the PowerCenter Repository Service.
    • Integration of system trace an stack trace: Improved debugging with the integration of system trace and stack trace collection on the Linux platform.
    • TLS-enabled Mail Server Support: Send secured emails using through SMTP servers that use TLS protocol.
    • GENERATE_UUID(). GENERATE_UUID( ) function for pushdown optimization with ODBC for GBQ.  Use for use cases such as autogenerating string values as surrogate keys in the tables.
    • Error logging. Improved error logging for the PowerCenter Integration Service to improve supportability.
    • Diffie-Hellman Ciphers: Supports Diffie-Hellman Ciphers in PowerCenter services to mitigate the Forward Secrecy issue.

     

    Platform

    Platform PAM

    • Oracle Database Multitenant (CDB/PDB) support for Informatica PowerCenter Domain, PowerCenter Repository Service, Model Repository Service, Data Integration Service, Informatica Data Service and the Analyst tool.
    • Database support added:
      • PostgreSQL 12.3
      • PostgreSQL 11.8
      • IBM DB2 11.5
    • Operating System support added:
      • RHEL 8
      • SUSE 15
      • Ubuntu 20.04
      • Amazon Linux 2.0.20210126
    • Database support removed:
      • PostgreSQL 11.7 & 10.6
      • SQL Server 2014
      • Oracle 12cR1 & 11gR2
      • IBM DB2 10.5
    • Operating System support removed:
      • Windows 2012 R2
      • SUSE 11
      • Ubuntu 16.04.5
      • Amazon Linux2.0.20200304
    • Operating System support deferred:
      • AIX 7.1 and 7.2
    • Java support updated:
      • OpenJDK Java 1.8.0_275
      • IBM JDK 8.0.6.20
    • Tomcat
      • Added Tomcat 9.0.41
      • Removed 7.x
    • Browser support updated:
      • Microsoft Edge Chromium - Version 86.0.622.38
      • Chrome Version - 86.0.4240.75
      • Removed Microsoft Edge Browser as it is replaced with Microsoft Edge Chromium .
      • Internet Explorer - 11.x
      • Safari 13.0.4

     

    Informatica Domain

    • SAML Enhancements
      • Request signing: SAML request signed by Informatica using the private key and the identity providers (IdP) can verify the signature using the public certificate.
      • Response signing: IdP sends signed SAML response using the private key and Informatica verifies the signature using the public certificate.
      • Encrypted assertion: IdP sends the encrypted assertion to Informatica, which then decrypts the assertion.
      • Configurable properties: Configure SAML authentication in the Administrator tool: gateway and worker nodes. Configure functionality such as the IdP URL, clock skew tolerance, and authentication context, as well as the above-mentioned functionality.
    • SAML identity providers (IdP):
      • OKTA SAML SSO
      • Oracle Access Manager (OAM) SAML SSO
      • Azure Active Directory Domain Services (ADDS) SAML SSO
    • Optimize LDAP sync and performance improvements.
    • Encryption strengthening to AES-256:
      • Update to AES-256 bit encryption key for secure data storage from AES-128. This enhanced encryption allows to encrypt sensitive data, such as passwords and secure connection parameters, before Informatica stores the data within the Informatica repositories with 256 bit encryption key.
      • Regeneration of the site key is removed to strengthen the security. The regeneration of the encryption key with keyword and domain name is removed and replaced with generation only once during initial setup. The site key is generated only once (regeneration with result in a new site key) and the same should be copied to each node or can be placed at a shared location accessible by each node. You cannot regenerate the site key. It is recommended to secure the site key.
      • Upgrade impact-  It is recommended to migrate the site key (to AES-256) after the Informatica domain upgrade. However, customer can choose not to upgrade to the latest AES-256 bit encryption and the previous behavior will be retained with AES-128 bit encryption key.
    • Enhanced audit logs for permission and privilege change. Display detailed permission and privilege level changes in the user activity logs.
    • Credential security: Credential data needs to be securely encrypted so that it cannot be modified and disallow elevation of privileges for logged-in user.
    • Azure SQL DB authentication with Azure Active Directory for Informatica Domain and Repository (PCRS and MRS).

     

    Installer

    • Web-based user interface for the installer is available for technical preview. Technical preview functionality is supported for evaluation purposes but is unwarranted and is not production-ready. Informatica recommends that you use in non-production environments only. Informatica intends to include the preview functionality in an upcoming release for production use, but might choose not to in accordance with changing market or technical circumstances. For more information, contact Informatica Global Customer Support.
    • Support for enhanced SAML authentication features: signed request, signed response, and encrypted assertion.

     

    Model Repository

    • Bitbucket load balancer
    • Version control systems (VCS) support added:
      • Azure DevOps
      • Visual SVN 4.3.1 (with 1.14.0 Apache SVN)
      • Perforce 2020.1
      • SVN  1.14.0
      • Bitbucket Server 7.7
      • GitHub Enterprise Server 2.22.4
      • GitLab 13.6.0
      • Removed - Visual SVN 4.0.2 , Perforce 2017.2 , Bitbucket 7.1 & 6.4 , Github Enterprise Server 2.20.5 , GitLab - 12.9.2

     

    Informatica Container Utility (Deployment Manager)

    Kubernetes (K8s)

    Support for the Informatica Domain in Azure Kubernetes Service (AKS) with new INFAK8s plugin. This is a new deployment utility with plugin based approach to support individual Informatica application services on Kubernetes cluster. The following functionality is available:

    • Node management with Installation and Configuration Update
    • Grid Service Manual & Auto Scaling
    • EBF Patching - EBF can be placed in shared volume and picked up by INFAK8s plugin (EBFs will be applied to all the nodes automatically)
    • Upgrade - Seamless upgrades into newer Informatica releases - 1 click task (post 10.5)
    • Pod and Node Recovery - In case of pod crash or cluster failure, auto recovery to last running state – no backup node required for non Grid services like the Model Repository Service and the PowerCenter Repository Service
    • 1-click domain shutdown and restart
    • Restrict CPU, memory usage per node
    • File streaming/Log retrieval from nodes
    • CPU and memory based auto-scaling for grid services
    • Managing CPU and memory usage limits per node
    • Ability to run node and service pre-startup and post-startup commands
    • Node and service recovery after K8s POD restart
    • Restoring the Model Repository Service and PowerCenter Repository Service backup from an existing installation
    • Container Image configuration - support for full image and custom service-based Images

     

    PAM support

    • Docker Container - CentOS & RHEL
    • Database - Oracle , SQL Server and PostgreSQL
    • Kubernetes - Azure Kubernetes Service (AKS) only
    • Available for Informatica Domain, PowerCenter, PowerCenter Repository Service,  PowerCenter Integration Service, Data Engineering Integration, Model Repository Service, Data Integration Service, monitoring Model Repository Service, and Email Service.

     

    Data Engineering Streaming (DES)

    Streaming Data Integration

    • Support for high precision decimal numbers
    • Support for logical data types in Avro data format
    • Support for periodic refresh of lookup cache in long running streaming mappings
    • Enhanced parsers for CSV, XML, JSON, and Avro data format for addressing complex use cases
    • Support for offset header port in Kafka source

     

    Enhanced Cloud streaming support

    • Google Ecosystem
      • Support for running streaming jobs on Google Dataproc
      • Support for Google PubSub as a streaming source
      • Support for Google Cloud Storage as a streaming target
      • Rollover support for ADLS Gen2 and Amazon S3 target connectors
    • Databricks. Additional transformation support on Databricks in streaming mode
    • Amazon-managed Streaming for Kafka (MSK) certification
    • Connectivity
      • Latest Apache and Confluent Kafka support
      • New target connectivity: Support for Cassandra and Kudu targets
      • Hive target enhancements
    • PAM
      • Databricks 5.5
      • Google Dataproc 1.5
      • Cloudera CDP Public Cloud 7.2 (AWS and Azure)
      • Cloudera CDP Private Cloud 7.1.4,7.1.5
      • Microsoft Azure HD Insights 4.0
      • Cloudera CDH 6.3.4 (with EBF)
      • Hortonworks HDP 3.1.5
      • Amazon EMR 5.29

     

    Data Engineering Quality (DEQ)

    PAM

    • Databricks integration:
      • You can run mappings with the following Data Quality transformations in an Azure Databricks or AWS Databricks environment: Address Validator, Case Converter, Classifier, Consolidation, Decision, Key Generator, Labeler, Match, Merge, Parser, Rule Specification, Standardizer, Weight Based Analyzer
      • You can create and run profiles on the Databricks cluster in the Informatica Developer and Informatica Analyst tools. You can perform data domain discovery and create scorecards on the Databricks cluster. Profiling of Databricks Delta table (JDBC) and objects in Azure ADLS Gen 2 in Azure Databricks clusters
      • Supported versions: Databricks 5.5 and 7.3
    • Google Dataproc 1.5
    • Cloudera CDP Public Cloud 7.2 (AWS and Azure)
    • Cloudera CDP Private Cloud 7.1.4, 7.1.5
    • Microsoft Azure HD Insights 4.0
    • Cloudera CDH 6.3.4  (with EBF)
    • Hortonworks HDP 3.1.5
    • Amazon EMR 5.29, 6.1

     

    Informatica Data Quality (IDQ)

    New pre-built package of Data Quality Accelerators for Japan

    • Contains rules for:
      • Address: multi-line address, multi-line address with geocoding, postal code and prefecture validation
      • Contact: Driver license, first name, last name and others
      • General: Date validation
      • Dictionaries: prefecture, town, postal code, area code for telephone, Romanized first name and last name, city, name by gender in Kanji and others
    • Update to data domain glossary to include national IDs

     

    Enterprise Data Preparation (EDP)

    New Features

    • Directly prepare on cloud data warehouses, cloud databases, and other JDBC V2 compliant relational data sources without duplicating data to the lake
    • Enhanced project centric user experience
    • Copy projects to quickly share the prepared recipes and datasheets with your team
    • Support for multi-byte characters
    • Performance improvements for the user interface
    • Enhanced Security

     

    PAM

    • Amazon EMR 6.1
    • Cloudera CDP Private Cloud 7.1.4
    • Microsoft Azure HD Insights 4.0
    • Hortonworks HDP 3.1.5
    • Cloudera CDH 6.3.4 (with EBF)

     

     

    Enterprise Data Catalog (EDC)

    Discovery

    • Profiling pushdown to Databricks: Support for pushdown to Databricks cluster for column profiling and data domain discovery.
    • Enhancements to similarity discovery such as grouping resources, reducing false positives, and computing similarity on enabled features.

     

    Re-architecture of Enterprise Data Catalog

    • Deprecated support for installation on internal and external Hadoop clusters.
    • Replaced HBase with MongoDB as the metadata store.
    • Nomad by HashiCorp introduced as the orchestration framework.
    • Support to back up metadata store, search database, and other stores either separately or in parallel.

     

    User Experience enhancements

    • Search result page. Enhanced with the following features to improve user experience in searching and identifying assets of interest:
      • Enhanced search bar with customizable search pre-filters.
      • Simplified search result filter pane, search result asset content layout, and pagination.
      • Additional information pane to show important details about the selected asset.
    • Asset notification. Improvements include filtering and export options.
    • Clone a resource configuration. Accelerates the creation of resources with similar type and settings.
    • New Catalog Administrator contextual walkthroughs. Accelerates the onboarding of Catalog Administrator:
      • Introduction to Home page
      • Create a Resource
      • Create a Custom Attribute
      • Create a Data Domain
      • Overview of Security and Permissions Management
    • New Enterprise Data Catalog contextual walkthroughs. Provides an overview of the following features:
      • Application Configuration
      • Business Term Overview
      • Data Domain Overview
      • Curate Data Domains
      • Resource Overview

    Analytics

    • Data Asset Analytics enhancements
      • Expose and document the Data Asset Analytics views that allow users to connect to report type datasets through any business intelligence tools.
      • Data Asset Analytics repository support for the following databases:
      • Microsoft SQL Server Named instance.
      • Oracle RAC with SCAN type connection.
      • Enhancements to improve user experience.

     

    • Data Flow Analytics (Technical Preview):
      • Data Flow Analytics for data mapping insight and discoveries with AI/ML to accelerate data modernization, improve data mapping efficiency, and reduce operational cost
      • Data Flow Analytics for PowerCenter mappings automates the following discoveries:
      • Similar mapping groups and the representative mapping within each group.
      • Duplicate mappings
      • Mapplet candidates
      • Reusable transformation candidates
      • User-defined functions candidates

     

    Technical preview functionality is supported for evaluation purposes but is unwarranted and is not production-ready. Informatica recommends that you use in non-production environments only. Informatica intends to include the preview functionality in an upcoming release for production use, but might choose not to in accordance with changing market or technical circumstances. For more information, contact Informatica Global Customer Support.

     

    Scanners

    • Advanced Scanners integration:
      • Install Advanced Scanners. which are bundled with the Enterprise Data Catalog installer binary files.
      • Implemented native models for the following Advanced Scanners:
        • Code: Oracle, SQL Server, Teradata, IBM DB2, Netezza, and Sybase.
        • BI: SAS, Microsoft SSAS, and SSRS.
        • Legacy: Cobol and JCL.
        • ETL: Oracle Data Integrator, Talend DI, IBM DataStage, and Microsoft SSIS.
        • Support for connection-less configuration for the dependent systems.
      • Support for reference resources.
      • Support for connection assignments with other resource end-points.
    • Axon scanner enhancement: Ability to filter lifecycle statuses for each object types.
    • Added support for S3 compatible filesystem that is Scality Ring certified.
    • New Snowflake scanner with advanced view SQL parsing and cross resource connection assignment capabilities.
    • New Advanced scanner for Oracle Data Integrator and Talend DI.
    • SAP S/4 HANA scanner that works for both SAP ECC and SAP S/4HANA is now available with support for data profiling.

     

    PAM Updates

    • Scanners:
      • Informatica Data Quality 10.5
      • Informatica Platform 10.5
      • Informatica PowerCenter 10.5
      • Business Glossary 10.5x
      • Erwin 2020Rx
      • Tableau 2020.x
      • SAP Business Objects 4.3
      • Oracle Data Integration 12.x
      • Talend DI 7.x
    • Deployment
      • Embedded deployment only (no support for external Hadoop cluster deployment)
      • Support added for RHEL 8.x
      • Support added for RHEL 7.9
      • Support added for Suse 15
      • Support added for Suse 12 SP5

     

    Informatica Metadata Manager

    • Support for Microsoft Azure SQL DB
    • Upgrade to latest cumulative version of MITI 10.1.0
    • Security bug fixes, and TPL upgrades to latest versions
    • Support large lineage export with more than 5K lineage nodes

     

    Data Privacy Management (DPM)

    Privacy enhancement

    • Data Stores, Location, and Subject Requests drill-down pages on the Privacy Dashboard
    • Filters on the Privacy Dashboard to enable Privacy Analysts to find content faster

     

    Domain discovery enhancements with Discovery Agent

    • Treats compressed files as folders, and files within compressed files are scanned and reported as individual files
    • Supports scans for image files (OCR Scans) - PDF with images, JPG, JPEG, BMP, TIFF, PNG
    • Supports scanning of emails in Office 365

     

    Installer, Infrastructure & Supportability Enhancements

    • Hadoop cluster/infrastructure has been removed from the product and is replaced with MongoDB. MongoDB gets installed as part of the product installation.
    • Support for Silent Installation of DPM through a Silent Installer
    • Enhanced SAML support

     

    PAM Updates:

    • Support for PostgreSQL as a repository for Data Privacy Management
    • Support for RHEL 8.0
    • Support for SuSe Linux 15.0
    • Support added for Suse 12 SP5
    • Lineage support for Informatica 10.4.1 and 10.5
    • Support for Axon 7.1
    • Domain Discovery and Subject Registry support for HDFS & Hive on CDP 7.1.1

     

    Test Data Management

    Enhancements

    • Support for encryption in the Data Masking transformation in the PowerCenter Designer and the Developer
    • Performance improvements in format preserving encryption rules
    • Improved dictionary usage in substitution masking
    • Support for UNICODE data in dictionaries in substitution masking
    • Support for Conditional Data Generation for XML files
    • Support for MinOccurs and MaxOccurs values in an XSD for Data Generation with XML files

     

    Connectivity updates

     

    • Support for PostgreSQL as a repository for Test Data Management
    • Support for PostgreSQL as a repository for Test Data Warehouse
    • Support for Oracle Exadata for the Test Data Management repository
    • Support for Oracle Exadata for the Test Data Warehouse repository

     

    Ecosystems and Connectivity

    Amazon

    • S3 – Directory-level partitioning support for complex files in mappings that run on the Spark engine
    • S3 - Support for Scality RING Amazon S3 compatible storage
    • Redshift - Partitioning support for Amazon Redshift mappings that run in the native environment

     

    Microsoft Azure

    Azure Synapse

    • Support for ADLS Gen2 as a temporary storage in the native environment or on the Spark or Databricks Spark engine
    • Table and schema override support for sources and targets
    • Staging file compression (Gzip) support to load data to the target for mappings that run on the Spark engine
    • Support for reading case-sensitive data from the database
    • Parquet file format support on staging for mappings that run in the native environment or on the Spark or Databricks Spark engine
    • Datetimeoffset, Date, and Smalldate data types support for mappings that run on the Databricks Spark engineADLS Gen2 as a temporary storage (Native/Spark/Databricks)

     

    ADLS Gen2

    • Directory-level partitioning support for complex files in mappings that run on the Spark engine
    • Support for wildcard characters while reading file names (Technical preview)
    • Support for recursive read from subdirectories (Technical preview)
    • JDBC V2
    • Support for the Create Target option
    • Support for dynamic mapping

     

    • JDBC V2
      • Support for the Create Target option
      • Support for dynamic mapping
      • Support for SAP HANA
    • Snowflake
      • Read and write support to Snowflake data warehouse Google Cloud Platform as staging
      • Audit support for Snowflake read operations for mappings that run in the native environment or on the Spark engine  
    • Google
      • Cloud Storage - Support for File Name port
      • Pub/Sub - (DES) Reader
      • BigQuery - ODBC pushdown enhancement
    • SFDC
      • SFDC API v50.0 is certified
    • SFMC
      • Support for dynamic Mapping     
    • Relational:
      • Support for PostgreSQL v12
      • Support for Azure SQL DB CDC
    • No-SQL:
      • Cassandra ODBC driver is upgraded to the latest version
    • Enterprise Data Warehouse
      • Support for Greenplum V6
    • Hadoop:
      • Complex File Connector – Support for partitioned files
      • HBase - HBase on CDP 7.1 support for mappings that run on the Spark or Blaze engine
      • PowerExchange for Kudu – New adapter that supports write operations

     

    PowerExchange CDC and Mainframe

    PAM Changes

    • CICS/TS V4.3 had been deprecated
    • CICS/TS V5.6 has been added
    • Db2 LUW V11.5 has been added
    • Open SSL version has been upgraded

     

    IBM i Improvements

    • IBM i (i5/OS) supports SSL connections

     

    Informatica Platform

    • Sequential File Name can be overridden at run time

     

    PowerExchange Command Improvements

    • LISTTASK enhanced reporting displays additional  entities, such as User-id, File Name, Task/Subtask etc.

     

    B2B/ISD

    ISD

    • Support on Databricks
    • Directory partitioning for Parquet, Avro, and ORC
    • New PDF preprocessor (very specific use case support)
    • Dynamic mapping support
    • Model generation based on XSD

     

    DT

    • performance enhancement: to address validations of large files and concurrency
    • "nillable" support in XSD

    --------------------------------------------------------------------------------------------------------

    Release Notes & Product Availability Matrix (PAM)

     

    PAM:

     

    Release Notes:

     

    What’s New and Changed:  https://docs.informatica.com/data-engineering/shared-content-for-data-engineering/10-5/what-s-new-and-changed.html