Full Asset Deduping
One of the primary values the Vulcan Platform provides to cybersecurity professionals is the ability to have a single source of truth for all aggregated VM data in a single platform. The Vulcan Platform refines, aggregates, and deduplicates the ingested data to produce a view of consolidated assets.
Asset Deduping in the Vulcan Platform is a mechanism that deduplicates assets ingested from multiple CMDB (Asset Inventories) and Scanners, providing a clean aggregation of the Cyber Security attack surface.
How is Asset Deduping achieved?
Vulcan achieves Asset duplication by crossing complex merge criteria and identifying duplications across the data ingested from the different sources.
The merging mechanism is designed to avoid disassembling and reassembling Vulcan assets. It adds new data to the existing structure if the merging criteria are met.
The merging strategy and combination criteria are configurable and can be set according to customer needs with the assistance of the Vulcan Cyber Customer Success team.
Merging Criteria
Asset Type | Deduping Criteria |
Hosts | Cloud Instance ID* Mac Addresses Hostnames External IP FQDNS IP |
Websites | URL |
Code Projects | Repository Name |
Cloud Resources | Native ID |
Images | Digest Repository + Image tag |
*Cloud Instance ID is a crucial identifier and a block rule; assets with different cloud instance IDs won't merge.
Information Order
Information Order refers to the prioritized sequence in which connectors or sources are considered when merging assets. It determines the hierarchy of data sources and influences the merging process by specifying which connector's data will be the unified asset's data. The first connector in the Information Order is given priority during the merging process. It serves as the baseline for Vulcan Asset creation.
Subsequent connectors are considered in the order specified, and their data is added to the existing Vulcan Asset.
The Information Order determines which details from merged assets are displayed in the user interface. When data variations exist among merged assets, the Information Order determines which details are retained and displayed, clarifying the chosen data source.
The Information Order is configurable, allowing users to define a specific order for each property or criterion.
Initial Sync and Processing
Before You Start
Having a well-defined merge strategy in place during onboarding is recommended for a smoother merging process. This can be established together with the Vuclan Customer Success team.
Optimal Connector Sequence for Seamless Merging - Best Practice
For a seamless and trouble-free integration experience with Vulcan Cyber Platforms, we highly recommend that customers consider the order in which they connect their connectors.
We recommend starting with the Asset Inventory connector.
Why start with Asset Inventory Connector?
Rich and Significant Data: The Asset Inventory connector typically holds the most significant and richest asset details. Connecting it first provides a solid foundation and accurate baseline for Vulcan Asset creation.
Solid Baseline for Vulcan Assets: Vulcan Assets are built upon the data received from connectors. Initiating the integration with the Asset Inventory connector ensures that Vulcan Assets start with comprehensive and reliable information.
Mitigating Merging Issues: Customers who connect scanners or non-inventory connectors first may encounter merging issues, especially when making changes to assets on their side. Starting with the Asset Inventory connector minimizes the chances of such merging complications.
Cloud Instance ID Identifier: The Asset Inventory connector, when configured with the Cloud Instance ID identifier, offers a distinct and reliable reference point for merging. This identifier is crucial for deduplication and helps create a robust merging strategy.
Avoiding Future Integration Challenges: Connecting the Asset Inventory connector first establishes a robust merging foundation. This proactive approach helps avoid potential challenges in future asset management and integration activities.
Native Asset Data vs. Vulcan Asset in the UI
When initiating the first sync with an asset inventory connector for a new environment, the system processes the native asset data received on an asset. It creates a Vulcan Data out of it. The Vulcan Platform distinguishes between native data (Native Assets Data) ingested from the connector and Vulcan data (Vulcan Asset).
Native Asset Data
Native Asset Data is all the data collected on assets during syncing connectors. Assets data is usually retrieved from different sources (several connectors) and processed by the Vulcan Platform. The result of processing, merging, and deduping is a Vulcan Asset.
The native asset data can be viewed in the Asset Details card of the Vulcan Asset.
Vulcan Asset
A Vulcan Asset is a consolidated representation of data from various sources (native data) that undergoes a merging process to form unified Vulcan data. The merging occurs based on specified criteria, such as Cloud Instance ID, Mac addresses, external IPs, and other parameters, depending on the asset type and merging criteria.
The Vulcan Asset is a central entity that encapsulates information from different connectors or sources. This merging process is likened to building a puzzle, where pieces of data are added to create a complete picture, forming a Vulcan host. The concept of 'information order' is crucial in determining how unmerged assets should be combined, establishing a hierarchy for merging based on specific criteria.
In the Vulcan Platform, the Assets Page presents a list of Unique Assets (Vulcan Assets) with an indication of the sources of each asset. The "Source" is the connectors/vendors. from which the asset was retrieved and ingested into the Vulcan Platform. Across all asset types, the Vulcan Platform identifies the same asset from inventory and scanner sources, merges the asset details and data, and presents it only once as a unique asset on the assets list on the Asset Page.
Proactive Asset Detach
Proactive Asset Detach capability ensures that merged assets remain accurate and relevant, streamlining your asset management workflow.
Dynamic Merge Criteria Compliance: The Vulan Platform re-evaluates merged assets against updated merge criteria with each merger run. If assets no longer align with the current criteria, they are proactively detached, maintaining the integrity of asset grouping.
Reduced Need for Manual Intervention: This automated check fosters a more self-sustaining asset management system without the need for human intervention.
Out-of-the-Box Integration: 'Proactive Detach' is a built-in feature accompanying the asset deduplication process. This ensures that without additional configuration, your assets are constantly evaluated and adjusted to reflect the most up-to-date criteria.
Assets can detach if one of the following occurs:
Modification in Merge Criteria | The asset detaches proactively when there is a change in the merge criteria. This detachment aligns with the updated merge strategy, ensuring that assets are reevaluated and organized based on the revised criteria. |
Changes in Asset Details | The asset may detach if there are changes to its details from the connector, such as the asset being removed or merged via a specific criterion (e.g., hostname) and the corresponding details are altered. This detachment reflects real-time changes in the asset's status or characteristics, maintaining accuracy within the Vulcan Cyber Platform.
Note: This reason does not apply to the asset that originally and initially built the Vulcan Asset (if it was the first connector to introduce it to Vulcan). |
Any asset detachments and merging history are indicated in the Asset Details card.
Expected Behavior
Vulcan constantly improves the Asset Deduping mechanism to accurately refine the ingested Native Assets Data from multiple sources and produce a clean list of Unique Assets; Vulcan Asset.
When improvements for better accuracy occur, the SPR could be affected. The change in SPR happens because the number of Unique Assets reduces accordingly. Nevertheless, this is an expected and wanted system behavior.
The order of connectors influences the merging process. The Vulcan Asset's ID and other identifying criteria are determined by the first connector that completed processing with the Vulcan Platform.
A Vulcan Asset is created once the Asset's identifiers are retrieved from the first synced connector (usually an Asset Inventory connector). Any additional asset data retrieved from other connectors is added to the Vulcan Asset (AKA, asset deduping) if the merging criteria are met.
FAQs
What can the user do to ensure smooth merging before syncing connectors?
Having a well-defined merge strategy during onboarding is recommended for a smoother merging process. This can be established together with the Vuclan Customer Success team. In addition, we highly recommend starting with an Asset Inventory connector.
What is the default merging strategy?
The default merging strategy for Hosts assets -
cloud_instance_id,mac_addresses,[external_ips+hostname+fqdns],[ips+hostname+fqdns],[hostname+fqdns],[external_ips+fqdns],[external_ips+hostname],hostname
The default merging strategy for Code Project assets -
name
The default merging strategy for Website assets -
name
The default merging strategy for Image assets -
name
The default merging strategy for Cloud resources assets -
name
What happens after adding additional connectors with asset data?
If the merging criteria are met, the system merges data from different connectors into a single Vulcan Asset. If the merging criteria are not met, the Vulcan Platform concludes that this is not the same asset, and it creates a new unique asset.
Why do I only see Vulcan host information in the UI?
The UI displays Vulcan host information, and native data is stored separately.
How does the system decide whether to merge a new connector with an existing Vulcan host?
Based on the configured merge criteria, the system checks if the new data can attach to the existing Vulcan host.
Can I customize the merge criteria for each asset type?
Yes. You can configure merge criteria for each asset type using available criteria fields. This is achieved with the help of your Customer Success Manager at Vulcan.
Can customers change the information order for their connectors?
Yes, customers can configure the information order for each property based on their priorities.
What happens if two or more connectors are configured at the same time?
The system considers the natural order of sync stages to determine the information order. The first connector that completes processing defines the baseline of the Vulcan Asset. This cannot be modified.
How does proactive detach work in Vulcan's asset deduping?
Proactive detach occurs when the merge criteria change, leading to assets detaching based on the new criteria.
Why does outdated data sometimes persist even after a connector is removed?
Outdated data might persist if the Vulcan host is considered the template (Vulcan Asset Baseline) and the system doesn't detach data that appears to match.