The IICS Summer 2019 release offers several new capabilities that include integration patterns, expanded connectivity, and platform functions. Highlights are listed below.
- Machine learning-based transformation recommendations allow you to get suggestions for the next best transformation based on the mapping you are working on.
- Improved support for custom, local parameter file locations, support for fully parameterized SQL queries, ability to overwrite connections at runtime using parameter files.
- Support for invoking taskflows using the RunAJob command line utility and using file listeners. These can be passed as parameters to data tasks for listener-driven events.
- New and improved taskflow canvas with a more compact layout improving design experience.
- The SQL transformation now supports ad hoc SQL queries (in addition to Stored Procedures).
- Intelligent Structure Discovery:
- Enhancements to existing ISD models with data from additional sample files (JSON & XML) to quickly adapt to data drift.
- The Structure Parser transformation can now update or change the associated intelligent structure model without breaking unchanged ports mappings which enhances mapping flexibility and allows for fast adaptation to changes in incoming files.
- Simplified design time experience with the ability to add a prefix or a suffix to existing node names using the bulk action functionality.
- Enhanced parsing engine support with the ability to discover and parse AVRO files.
- Mass Ingestion:
- Additional sources and targets: ADLS GEN1 as source, ADLS GEN2 as source and target, Snowflake as source and target.
- Support for custom actions during file transfer: compress/decompress, encrypt/decrypt
- Integration with Enterprise Data Catalog (EDC): select Mass Ingestion source from EDC.
- Filter files to transfer by file size and date.
- Optimization for large file handling – resume from point of failure.
- Developer productivity enhancements: new compact canvas, improved diagnostics and debugging
- Improved Process and Guide canvas compact layout improving design experience.
- New validation panels help diagnose service connectors and connection errors.
- Fault details are now available with every step to diagnose runtime errors and improve debugging.
- Guides: Improved page layout and Salesforce Lightning theme
- Guide Screen step designer has been revamped to provide more real estate to design guides.
- Salesforce Lightning theme support is now available for guides and the Guide Launcher.
- Source control and deployment automation
- New CLI utility automates export and extract of design assets to any source control system. The package, import and publish capabilities of the CLI provides the ability to deploy full deployments, packages and patches across environments.
- Sample Jenkins pipeline to help developers adopt continuous integration practices for Application and Data Integration, and help operations staff with the automation of continuous deployments.
- Expanded pub/sub and messaging capabilities with Salesforce Platform Events and RabbitMQ to compliment the current support for AMQP, JMS, AWS SNS/SQS, Azure Service Bus/Event Hub and Kafka.
- New support for Salesforce Platform Event and Push Topic events enabling customers to deliver secure, scalable and customizable event notifications within Salesforce or from external sources.
- New RabbitMQ native connector providing support for durable queues.
- CLAIRE-enabled privacy data leak identification: Privacy data identification in API content, masking or blocking of sensitive personally identifiable information. (This feature is in preview mode).
- Ability to customize the structure of API URLs.
- Support for the internationalization of API, group, and organization names.
- Governance enhancement: Support for predefined and custom roles with granular privileges that promote governance.
- Exchange files with your partners using Cloud AS2.
- Get insights into your daily gateway activities and statuses with the new dashboard.
- Asset Dependency feature providing “used by” and “uses” relationship information across all asset types and services for impact analysis.
- Common Explore enhancements: Changes made by the user to the Explore column settings are now automatically persisted for that user for various filtered views such as “all assets,” “projects & folders,” etc.
- More control and flexibility in managing users: Grant/deny service level access to a user through assign services capability. Disable users to prevent user logins for both native and SAML authentication mechanisms. REST APIs for user management – users, groups and roles.
- Ability to start/stop Secure Agent services to optimize computing resources and isolate the need to restart a single service from others.
- New native connectors to connect to CDM/Power BI, Snowflake (V2), MongoDB, Adobe Experience Platform, Ariba (V2), and Cassandra.
- Updates to existing connectors: Microsoft (MSD 365 Operations, Cosmos DB, ADLS Gen2 and Gen 1, SQL DW), Google (Google Big Query, Google Storage), Amazon (S3, Redshift), Oracle NetSuite, Salesforce, SAP (Concur V2, S/4 HANA) and Greenplum
New IICS Services
The IICS Summer 2019 release offers the following new services:
Streaming Ingestion (included in Data Ingestion Service)
- Ingestion of streaming data from logs, Apache Kafka and IoT sources.
- Supports AWS & Azure ecosystems as targets of Ingestion: Amazon Kinesis Streams, Kinesis Firehose, Amazon S3 and Azure EventHub
- Supports lightweight transformations at the edge.
- Real-time monitoring of streaming ingestion jobs.
Integration at Scale (included in Data Integration Service)
- Run data integration jobs at scale on a fully managed spark serverless cluster.
- Support for AWS ecosystem: S3 & Redshift
- Auto-tune and scale fully-managed Kubernetes clusters.
- Operational Insights is now available for IICS cloud services to help with efficient infrastructure monitoring and alerting. IICS cloud services users can monitor the health of a) Secure Agents, b) services running on each Secure Agent, c) Runtime environments and d) subscribed cloud services.
- Email Alerts can be set on services and Secure Agents for unavailability or excessive resource consumption.
- Enables business users to manage enterprise reference data in an easy-to-use, configurable, business-friendly user interface.
- Provides versioning, collaboration and complete life-cycle management of reference data.
- REST APIs for managing data and meta-data, enabling automation and system integration.
- Improved stability and performance.
- Enable business users to create rule specifications to perform a variety of Data Quality actions including completeness, validation and standardization.
- Create and manage value lists for data quality operations using dictionaries. Dictionaries can be used to identify, validate and standardize data as part of a rule specification.
- Embed Data Quality rules in Data Integration mappings to support Data Quality activities in typical data integration scenarios such as data warehousing, data migrations and more.
The IICS Team