erwin announce release 9.7 of the Metadata Management solutions erwin Web Portal and erwin Data Governance
- HARVESTED MODEL CHANGE DETECTION avoids systematically creating a new version of the content upon scheduled harvesting of large database, data modeling, data integration, or business intelligence servers. This is automatically performed as side effect of any new harvesting by a systematic and efficient (fast) model comparison with the previously harvested metadata This dramatically reduces the needed database space by creating fewer versions, and enable the possibility of reliable notification only if a change really occurred. Note that in previous versions, the incremental harvesting already offered systematic and automatic reuse of unchanged models (e.g. data model) from a large a multi-model server (e.g. data model repository)
- SUBSCRIPTION / NOTIFICATION mechanism on select changes at both the repository model level (e.g. a new version of a data store is harvested) and at the model object level (e.g. a terms changed or ready to approve/review in a Business Glossary under Workflow). Anyone may be assigned stewardship roles and thus will be notified as new imported content is harvested, with links back to the new object and the ability to compare using the newly re-written powerful comparator, and the ability to identify impacts of change for any architecture or configuration of assets.
- CUSTOM ATTRIBUTES on any harvested metadata (models) from data stores, data modeling, data integration, and business intelligence tools. Therefore allowing to tag any metadata (from Hive tables to Data Models down at the column/field level) for custom properties such as a company confidential level that can be read for external products on security enforcement and compliance. Note that in previous versions, these same custom attributes could already be applied to authored metadata (models) such as terms of Business Glossaries or tables/columns of Data Models.
- BUSINESS GLOSSARY WORKFLOW for the business users behind the Metadata Explorer UI. Note that in previous versions, a (customizable) Workflow was already available in the Metadata Manager.
- DATA INTEGRATION BROWSING AND LINEAGE for the business users behind the Metadata Explorer UI, allowing them to browse the data integration jobs (from DI/ETL/ELT tools as well as SQL Scripts) and analyze the summary data flow lineage of their execution. Note that in previous versions, the browsing (open) and full detailed data flow lineage of any data integration models (DI/ETL/ELT and SQL scripts jobs) were already available in the Metadata Manager (and still are).
- DATA CONNECTION STITCHING MAJOR IMPROVEMENTS now offering to stitch by column position (needed in SQL insert statements), and smart case aware stitching (as some database’s name spaces like schema/table/column are case sensitive while others are not)
- BULK DATA MAPPING at the table level.
- MODEL COMPARE/MERGE: The comparison facility has been completely re-written to include comparison for every level of detail for those models with the same profile (e.g., data model from one technology and data model from another). Even entirely different contents (e.g., a data model, or and Glossary) may be compared, only at a lessor level of granularity (basically at the granularity of stitching, e.g., schema, table, column). Finally, for physical data models (including documentable models based upon harvested database structures) one may use a powerful merge feature again with full control down to any level of granularity.
- DATA MODELING Major improvements and new features involving the physical data modeling capabilities centered around two major use cases:
- A data documentation tool enabled through physical data modeling based upon the structure harvested from existing data stores of the of data warehouse and data lake (including traditional RDBMS and Hadoop Hive big data)
- A data requirements tool for new data stores to be defined such as self service DI to new Hadoop Hive tables in the data lake.
Also includes new advanced support for physical data model diagram editing, relationships definition, all the way to export to data modeling tools, business intelligence design, etc., and comparison/compliance with live database / big data hive implementations..
- REST API SDK major enhancements with many new features
- LIGHTWEIGHT MODELS: Model content (such as a harvested databases or data models) can be stored in the repository as a lightweight model (just the XML file), or fully expanded (as both XML file and fine grained repository objects). When retaining many historical versions of a model, using lightweight models saves repository space, and also avoids slowing down the search by not indexing historical repository objects. Lightweight models cannot be directly used in a Configuration or in a Mapping. However, the lightweight model of a data store (such as a RDBMS or Hadoop Hive) can be documented with a Physical Data Model (PDM) for data model diagramming and semantic linking to a Business Glossary (BG). Such a PDM can of course be used in any Mapping or Configuration to be exposed to business users in the Metadata Explorer. Note that lightweight models can immediately (without any loss of performance) be opened in the Metadata Manager (to browse metadata or trace lineage within that model), Compared (with the Model Comparator to analyze the difference between versions), Exported (for example to BI design tool).
- UPGRADE NOTES: The Business Glossary Workflow has been significantly improved in the metadata explorer. Therefore, before upgrading to this version, it is strongly recommended that all business glossaries under workflow have published terms only (this can be achieved by listing all draft terms and either publishing or reverting them to the previously published state). Otherwise, the upgrade process will systematically publish all draft terms in their current state (which can be undesirable if the current state is not what you would like to publish). Finally, after the upgrade, you will need to manually re-enable the Workflow on these Business Glossaries.