The search mechanism in ThreatConnect® uses a combination of direct and indirect search algorithms to find data based on a given input. Depending on certain characteristics of the search term (e.g., size and complexity), different search methodologies are utilized to return the most relevant data possible back to the user. There are two main parts to this mechanism: (1) “exact”-matching algorithms that search for Indicators and other intelligence data based on a “direct hit” to a known item summary or a pattern; and (2) "potential"-matching algorithms that search for intelligence data by leveraging the Elasticsearch® engine. For the partial-matching part of the mechanism, all data, including document uploads, are searched to form a relevance-ordered result set based on a scoring system that filters out common words and phrases while prioritizing applicable matches.
The ThreatConnect search results also provide information of analytic value, including exact and potential matches in the user’s ThreatConnect instance and the ability to identify, create, and explore new Indicators.
Initiating a Search
To initiate a search, click the magnifying glass icon on the top navigation bar (Figure 1).
The Search flyout will slide in on the right-hand side of the screen (Figure 2).
Enter a term in the Search... bar at the top of the screen, drag and drop a file into the designated area or click that area to browse for a file, or click on a previously searched term under Search History (Figure 3).
To clear the search history, click the blue clear history text on the right-hand side of the Search History section.
Filters can be used to limit search results by owner, type, and location of match.
To filter results by owner, click the OWNERS button in the Filters section of the Search flyout. A dropdown menu will appear (Figure 4). Select owners to include, deselect owners to exclude, and click the APPLY button. The value in parentheses in the OWNERS button will change to show the number of owners selected (or ALL if all owners are selected).
To filter results by type (Indicator, Group, Tag, or Victim), click the TYPES button in the Filters section of the Search flyout. A dropdown menu will appear (Figure 5). Select types to include, deselect types to exclude, and click the APPLY button. The value in parentheses in the TYPES button will change to show the number of types selected (or ALL if all types are selected).
To limit the result set to show only data known to the user’s ThreatConnect instance, toggle the Local Matches Only switch in the Filters section of the Search flyout. Toggling the switch to “on” (orange) will limit the search results to objects that exist in an Organization, Community, or Source on the user’s ThreatConnect instance. Toggling the switch to “off” (gray) will allow the search results to return matches that may not yet exist in ThreatConnect, but do match a pattern for an Indicator type that could potentially be added to the user’s instance if desired.
Figure 6 shows the results of a search for badguy.com.
The Exact Matches section displays two types of results, each of which is represented by a different icon:
- objects that exist in an Organization, Community, or Source in the user’s ThreatConnect instance, represented by the appropriate Indicator or Group icon without a border; and
- objects that do not yet exist in an Organization, Community, or Source in the user’s ThreatConnect instance, but match a pattern for one of the Indicator types in ThreatConnect, represented by an Indicator icon with an orange border.
NOTE: The second type of object will not appear in the search results if the Local Matches Only switch is toggled to "on" (orange).
In Figure 6, the search results show that badguy.com is an exact match to an Indicator that exists in one Organization and one Community on the user’s ThreatConnect instance (i.e., the first type of results for Exact Matches). Information on Observations/False Positives and analytical information (in this case, from ThreatAssess, or from CAL if information from ThreatAssess does not exist for the Indicator) are also provided. Note that the icon associated with the exact match is an Indicator icon without a border.
Figure 7 shows the results of a search for badguys.com.
This search resulted in an exact match as well, but of the second type. The search term does not exist on the user’s ThreatConnect instance, but it did meet the pattern criteria for an Indicator of the Host type. Note that the icon associated with the exact match is an Indicator icon with an orange border.
Figure 8 shows the results of a search for bad guy.
This search resulted in an exact match for an Adversary Group called “Bad Guy”. All exact matches that are Groups will be of the first type, which means that all exactly matched Groups exist on the user’s ThreatConnect instance. Note that the icon associated with the exact match is a Group icon for the particular Group type (in this case, Adversary) without a border.
Exact matches can be sorted by Summary (name of the object—e.g., badguy.com), Type (e.g., Host, Tag), or Analytics (e.g., ascending or descending order by ThreatAssess score) by using the Sort Exact Matches By dropdown menu at the top right of the search results.
Potentially Related Matches
The Potentially Related Matches section shows objects in an Organization, Community, or Source in the user’s ThreatConnect instance that may be related to the search term. It shows only items that are in the user’s ThreatConnect instance. For example, Figure 6 shows two Email Address Indicators in one of the user’s Communities that may be related to the search term badguy.com. Figure 7 shows two URL Indicators and an Incident Group in one of the user’s Sources that may be related to the search term badguys.com. Figure 8 shows one Adversary Group, one Tag, one Host Indicator, one URL Indicator, and one E-mail Group (out of more than 1,000 results found) in various Organizations, Communities, and Sources on the user’s ThreatConnect instance that may be related to the search term bad guy.
Analyzing Search Results
The objects found in the search results can be explored further by clicking on the provided links.
For objects that exist in the user’s ThreatConnect instance, the Details by Owner section of a results entry will list one or more owners of the object. Each owner is a link that, when clicked, opens the Details flyout for the object on the left-hand side of the screen.
For example, in Figure 6, clicking on the Demo Organization link in the Exact Matches section will open the Details flyout for the badguy.com Host Indicator that exists in the user’s Organization named Demo Organization (Figure 9).
To view the Details screen for this object, click the Details icon at the top right corner of the Details flyout. To add this Indicator to a different owner, click the Plus icon at the top right corner of the Details flyout. A dropdown menu listing the owners to which the user has access will appear (Figure 10).
Click on an owner, and the Details flyout will change to show the Indicator in the new owner (Figure 11).
For Indicators that do not exist in the user’s ThreatConnect instance, the Details by Owner section of a results entry will provide a Learn more about it link that, when clicked, opens the Details flyout for the Indicator on the left-hand side of the screen (Figure 12).
This flyout can be used to view information that ThreatConnect’s Collective Analytics Layer (CAL™) has on the Indicator, if any, or to follow the Investigation Links for further exploration of the Indicator. To add this Indicator to an owner, click the Plus icon at the top right corner of the Details flyout or to the right of its results entry in the search flyout and select an owner from the dropdown menu that appears. Alternatively, clicking the Details icon at the top right corner of the Details flyout will open a Details screen for the Indicator (Figure 13).
Use the Indicator Analytics section to view any information CAL has on the Indicator, or follow the Investigation Links to explore the Indicator further. To add the Indicator to the user’s ThreatConnect instance, select an Owner from the Owner dropdown menu on the left and click the SAVE button. A new Details screen for the Indicator will open from which the Indicator can be configured further (Figure 14).
To search for multiple items at one time, the items may be entered one after another, with a space in between each item. For example, Figure 15 shows the results of a search for the following terms: bad.com terrible.com bad guy.
The search bar also supports line breaks. When search terms are entered with line breaks, no Potentially Related Matches will be returned. Only Indicators that are in the user’s ThreatConnect instance or satisfy a pattern for one of the Indicator types will be shown as Exact Matches. No matches to Groups in the user’s ThreatConnect instance will be shown, even if one of the search terms is an exact match for the name of a Group in the user’s instance.
For example, Figure 16 shows the results of a search for four separate terms: bad.com, terrible.com, bad guy, and dfdfdf.
The results returned show two exact matches: a Host Indicator for the term bad.com that is already in one Community in the user’s ThreatConnect instance and a Host Indicator for the term terrible.com that is not in the user’s ThreatConnect instance. Even though an Adversary Group called “Bad Guy” exists in the user’s instance, results for bad guy were omitted because they did not meet any Indicator patterns. Results for dfdfdf were also omitted because they did not meet any Indicator patterns.
Searching for Defanged Indicators
The ThreatConnect search engine automatically refangs Indicators. The term “defang” refers to the process of altering an Indicator so that a user cannot click on it by accident and navigate to a malicious website. The term “refang” is the reverse. It refers to the process of taking a defanged Indicator (e.g., bad[.]com) and returning it to its original state (bad.com). For example, if bad[.]com is entered into the search engine, results for bad.com will be provided.
Table 1 lists all of the defanged character sequences recognized by the ThreatConnect search engine and their corresponding refanged versions.
|Defanged Character Sequence||Refanged Version|
NOTE: In the last three rows, “.” is any character (e.g., “x” or “X”). For example, hxxp:// would be refanged as http://.
CAL™ is a trademark of ThreatConnect, Inc.
Elasticsearch® is a registered trademark of Elasticsearch BV.