TABLE OF CONTENTS
- Known File Filter (KFF)
- Introduction to the KFF Architecture
- Components of KFF Data
- About the Organization of Hashes, Hash Sets and KFF Groups
- About Pre-defined KFF Hash Libraries
- NIST NSRL
- NDIC HashKeeper
- Installing KFF
- Importing a CSV using the KFF Import Utility
- Verifying a File Using the KFF Import Utility
- Removing Pre-defined KFF Libraries Using the KFF Import Utility
- Using the KFF Utility in FTK Central
- Running KFF Against a Case
- Reviewing KFF Results in a Case
Known File Filter (KFF)
KFF (Known File Filter) is a utility that compares the file hash values of known files against the files in your case. The known files that you compare against may be the following:
- Files that you want to ignore, such as operating system or application files.
- Files that you want to be alerted about, such as malware or other contraband files
The hash values of files, such as MD5, are based on the file’s content, not on the file name or extension. This helps you identify files even if they are renamed.
Using KFF during your analysis can provide the following benefits:
- Immediately identify and ignore 40-70% of files irrelevant to the case.
- Immediately identify known contraband files.
Elements of Known File Filter
Introduction to the KFF Architecture | |
Components of KFF Data | |
About the Organization of Hashes, Hash Sets and KFF Groups | |
About Pre-defined KFF Hash Libraries | |
NIST NSRL | |
NDIC HashKeeper | |
Installing KFF | |
Importing a CSV using the KFF Import Utility | |
Verifying a File Using the KFF Import Utility | |
Removing Pre-defined KFF Libraries | |
Running KFF Against a Case | |
Reviewing KFF Results in a Case |
Introduction to the KFF Architecture
There are two distinct components of the KFF architecture:
- KFF Server - The KFF Server is the component that is used to store and process the KFF data against your evidence. After you install the KFF Server, you import your KFF data into it.
See KFF Server section.
- KFF Data - The KFF data are the hashes of the known files that are compared against the files in your case. The KFF data is organized in KFF Hash Sets and KFF Groups. The KFF data can be comprised of hashes obtained from pre-configured libraries (such as NSRL) or custom hashes that you configure yourself.
See KFF Data section.
Components of KFF Data
Item | Description |
Hash | The unique MD5 or SHA-1 hash value of a file. This is the value that is compared between known files and the files in your case. |
Hash Set | A collection of hashes that are related somehow. The hash set has an ID, status, name, vendor, package, and version. In most cases, a set corresponds to a collection of hashes from a single source that have the same status. |
Group | KFF Groups are containers that are used for managing the Hash Sets that are used in a case. KFF Groups can contains Hash Sets as well as other groups. Cases can only use a single KFF Group. However, when configuring your case, you can select a single KFF Group which can contain nested groups. |
Status | The specified status of a hash set of the known files which can be either Ignore or Alert. When a file in a case matches a known file, this is the reported status of the file in the case. |
Library | A pre-defined collection of hashes that you can import into the KFF Server. You can use the following pre-defined libraries:
For law enforcement users, you can also use Project Vic libraries. See About Pre-defined KFF Hash Libraries section. |
About the Organization of Hashes, Hash Sets and KFF Groups
Hashes, such as MD5, SHA-1, etc., are based on the file’s content, not on the file name or extension.
You can also import hashes into the KFF Server in .csv format.
For FTK-based products, you can also import hashes into the KFF Server that are contained in .tsv, .hke, .hke.txt, .hdi, .hdb, .hash, .nsrl, or .kff file formats.
Hashes are organized into Hash Sets. Hash Sets usually include hashes that have a common status, such as Alert or Ignore.
About Pre-defined KFF Hash Libraries
There are pre-configured hash sets currently available for KFF that come from federal government agencies and are available in KFF libraries.
The following pre-defined libraries are currently available for KFF and come from federal government agencies:
- NIST NSRL (The default library included in the KFF installer package)
- NDIC HashKeeper (An optional library)
- DHS (An optional library)
For law enforcement users, you can also use Project Vic libraries.
Use the following information to help identify the origin of any hash set within the KFF.
- The NSRL hash sets do not begin with “ZZN” or “ZN”. In addition, in the AD Lab KFF, all the NSRL hash set names are appended (post-fixed) with multi-digit numeric identifier. For example: “Password Manager & Form Filler 9722.”
- All HashKeeper Alert sets begin with “ZZ”, and all HashKeeper Ignore sets begin with “Z”. (There are a few exceptions. See below.) These prefixes are often followed by numeric characters (“ZZN” or “ZN” where N is any single digit, or group of digits, 0-9), and then the rest of the hash set name. Two examples of HashKeeper Alert sets are:
- “ZZ00001 Suspected child porn
- “ZZ14W”
- An example of a HashKeeper Ignore set is: “Z00048 Corel Draw.
- In 1.81.4 and later there are two sets named “DHS-ICE Child Exploitation JAN-1-08 CSV” and “DHS-ICE Child Exploitation JAN-1-08 HASH”.
- In AD Lab there is just one such set, and it is named “DHS-ICE Child Exploitation JAN-1-08”.
Once an investigator has identified the vendor from which a hash set has come, he/she may need to consider the vendor’s philosophy on collecting and categorizing hash sets, and the methods used by the vendor to gather hash values into sets, in order to determine the relevance of Alert (and Ignore) hits to his/her case. The following descriptions may be useful in assessing hits.
NIST NSRL
The NIST NSRL collection is described at: http://www.nsrl.nist.gov/index.html. This collection is much larger than HashKeeper in terms of the number of sets and the total number of hashes. It is composed entirely of hash sets being generated from application software. So, all of its hash sets are given Ignore status by Exterro staff except for those whose names make them sound as though they could be used for illicit purposes.
The NSRL collection divides itself into many sub-collections of hash sets with similar names. In addition, many of these hash sets are “empty”, that is, they are not accompanied by any hash values. The size of the NSRL collection, combined with the similarity in set naming and the problem of empty sets, allows Exterro to modify (or selectively alter) NSRL’s own set names to remove ambiguity and redundancy.
NDIC HashKeeper
NDIC’s HashKeeper collection uses the Alert/Ignore designation. The Alert sets are hash values contributed by law enforcement agents working in various jurisdictions within the US - and a few that apparently come from Luxemburg. All of the Alert sets were contributed because they were believed by the contributor to be connected to child pornography. The Ignore sets within HashKeeper are computed from files belonging to application software.
During the creation of KFF, Exterro staff retains the Alert and Ignore designations given by the NDIC, with the following exceptions. Exterro labels the following sets Alert even though HashKeeper had assigned them as Ignore: “Z00045 PGP files”, “Z00046 Steganos”, “Z00065 Cyber Lock”, “Z00136 PGP Shareware”, “Z00186 Misc Steganography Programs”, “Z00188 Wiping Programs”. The names of these sets may suggest the intent to conceal data on the part of the suspect, and Exterro marks them Alert with the assumption that investigators would want to be “alerted” to the presence of data obfuscation or elimination software that had been installed by the suspect.
Note: The basic rule is to always consider the source when using KFF in your investigations. You should consider the origin of the hash set to which the hit belongs. In addition, you should consider the underlying nature of hash values in order to evaluate a hit’s authenticity.
Installing KFF
Downloading the Latest KFF Installation Files
You can download the latest KFF installation files and guides from https://Exterro.com/product-download.
Determining Where to Install the KFF Server
Where you install the KFF Server depends on the application and environment you are running.
- For AD Lab, Enterprise and FTK Central applications, the KFF Server is generally installed on a different computer than that runs the main application.
- For large environments, it is recommended that the KFF Server be installed on a dedicated computer.
Installing Cassandra
To install Cassandra:
- If required, install 64-bit Java 8.
- Navigate to AccessData_Casandra_Installer.exe.
- Run AccessData_Cassandra_Installer.exe as an administrator.
- If required, install Python 2.7.
- On the Welcome page, click Next.
- Review and accept the license terms and click Next.
- Verify or change the Destination Folder and click Next.
- If needed, configure Remote Access.
- Select Enable Remote Access.
- In the RPC_Address field, enter the IP address of the computer you are installing on. For example, 10.10.10.10.
- In the Native Transport Port Number field, leave the default 9042.
- Click Next.
- If you enabled Remote Access, set the User Credentials for the service and click Next.
- Click Install.
- Click Finish.
Cassandra and Firewalls
During the installation, if you check the box to Enable Remote Access, the installer creates an inbound exception rule for the port entered in the Cassandra installer (if the rule has not already been created).
The rule has the following attributes:
- name = AccessData Cassandra Remote Access Port
- direction = in
- program = “<install directory>\Cassandra\bin\daemon\prunsrv.exe”
- local port = 9042 (or whatever the user entered)
- protocol = tcp
If you uninstall Cassandra, the installer checks to see if Enable Remote Access was checked during install, and if it was, the installer looks for the above firewall rule using the 5 listed attributes, and if it finds the rule, it removes it from the firewall.
Manually Configuring Remote Setting for Cassandra
In some situations, Cassandra needs to be configured to enable Remote Access.
During the installation of Cassandra there is the option to Enable Remote Access and then set the RPC_Address (the IP address of the computer that Cassandra is installed on).
If you set these settings correctly during the installation, no further configuration is needed.
However, if you did not enable remote access or make a change, you can manually configure the remote settings for Cassandra.
Note: Use an editor that supports YAML files
To manually configure remote setting for Cassandra:
- Go to the location that you installed Cassandra.
- By default, it is “<Drive>:\Program Files\ AccessData\Cassandra”.
- Open the \conf folder.
- Edit the cassandra.yaml file.
- Search for rpc_address:
- Change the address from local host to the IP or DNS name of the computer running Cassandra. For example, change rpc_address: localhost to rpc_address: 10.10.10.10
- Search for native_transport_port:Verify that the setting is: native_transport_port: 9042 (or the port you are using)
- Save and exit the file.
- Restart the AccessData Cassandra service.
Configuring a Remote KFF Server
To configure a remote KFF Server:
- Navigate to the FTK-Central bin folder (typically "<Drive>:\Program Files\ AccessData\Forensic Tools\<version>\bin\").
- From the bin folder, open ADG.WeblabSelfHost.exe.config in a text editor.
- Find the line <add key=”KFFServerUrl” value=”localhost:9042” />. 9042 is the default port for Cassandra.
- If needed, change localhost to be the location IP address of your KFF server. For example, value=”10.10.10.10:9042”
- Save and close the file.
- Restart the QuincSelfHostService service.
Installing KFF Import Utility
To install KFF import utility:
- Navigate to KFF_Import_Utility.exe.
- Run KFF_Import_Utility.exe as an administrator.
- Click Next.
- Review and accept the license terms and click Next.
- Verify or change the Destination Folder and click Next.
- Click Install.
- Click Finish.
Importing a CSV using the KFF Import Utility
You can import Hash Sets and KFF Groups by importing a custom CSV file.
To import a CSV using KFF import utility:
- Open the KFF Import Utility.
- Click the Browse button and locate the CSV that you want to import.
- Click Open.
- Enter package, vendor, version, etc.
- If you installed Cassandra enabling Remote Access, in the Server address field, you must enter the computer’s IP that has Cassandra installed on it, even if it is on the same computer as the import utility. Otherwise, leave it as localhost.
- Click Import.
- When complete, click OK.
Verifying a File Using the KFF Import Utility
You can verify Hash Sets and KFF Groups to ensure the correct file is being imported.
To verify a file using KFF import utility:
- Open the KFF Import Utility.
- Click the Browse button and locate the file that you want to import.
- Enter set name, package, vendor, version, and set status.
- If you installed Cassandra enabling Remote Access, in the Server address field, you must enter the computer’s IP that has Cassandra installed on it, even if it is on the same computer as the import utility. Otherwise, leave it as localhost.
- Click Verify.
- When complete, the Success window will appear, showing the following details:
- Group Count, Set Count, Hash Count, Photo DNA Count
- If you would like to open the log for further examination of the data, select Yes. If not, select No and the window will close.
Removing Pre-defined KFF Libraries Using the KFF Import Utility
You can remove a pre-defined KFF Library that you have previously imported. You cannot see or remove existing custom KFF data (your own CSVs or manually entered data).
To remove pre-defined KFF libraries using KFF import utility:
- On the KFF Server, open the KFF Import Utility.
- Select the library that you want to remove.
- Click Remove.
Using the KFF Utility in FTK Central
You can use the KFF Utility in FTK Central to create and import hash sets as well as create groups. The functionality from the stand-alone KFF utility has been carried over.
Warning: Apache Cassandra must be installed and configured for this feature to work
Creating a Hash Set
To create a hash set:
- From the home page, click on the Settings button
from the top-right corner.
- Navigate to the System Management tab.
- Select the Hash Sets section.
- Click Create Hash Set.
- The Create Hash Set prompt is displayed.
- Enter a Name.
5. Enter any one of the following Override Status.
- No Override
- Ignore
- Alert
6. Enter a Package name.
7. Enter a Vendor name.
8. Enter a Version.
9. Click Save.
Note: From the Hash Sets section, you can click on the Edit or Delete button to edit or delete hash sets respectively.
Importing a Hash Set
To import a hash set:
- From the home page, click Settings from the top-right corner.
- Navigate to the System Management tab.
- Click Hash Sets.
- Click Import Hashes.
- The Import Hashes prompt is displayed.
- Click Add File(s).
- Enter the location path in the Server field.
- Select the required hash file to be imported.
- Select anyone of the below Default Status:
- Alert
- Ignore
10. Click Import Data.
- The Import Summaries page will be displayed.
The progress of the import operation will be displayed.
Importing a Hash Set from Review Mode
To import a hash set from Review mode:
- From the Grid, select the records.
- Right-click on a selected record.
- Select Export.
- Select any one of the following options:
- Checked – This will export the file in native format.
- All to CSV – This will create a list of files with general metadata information.
- Click OK.
- From the home page, click Settings from the top-right corner.
- Navigate to the System Management tab.
- Click Hash Sets.
- Click Import Hashes.
- The Import Hashes prompt is displayed.
10. Click Add File(s).
11. Enter the location path in the Server field.
12. Select the required hash file to be imported.
13. Select anyone of the below Default Status:
- Alert
- Ignore
14. Click Import Data.
Creating a KFF Group
To create a KFF group:
- From the home page, click Settings button from the top-right corner.
- Navigate to the System Management tab.
- Select the KFF Groups section.
- Click Create KFF Group.
- The Create KFF Group prompt is displayed.
- Enter a Name.
- Select any one of the below provided Override Status:
- No Override
- Ignore
- Alert
- Enter a Package name.
- Enter a Vendor name.
- Enter a Version.
- Click Save.
Note:From the KFF Groups section, you can click on the Editor Delete
button to edit or delete KFF groups respectively.
Associating Hash Sets to KFF Group
To associate hash sets to KFF group:
- From the home page, click Settings button
from the top-right corner.
- Navigate to the System Management tab.
- Select the KFF Groups section.
- Click on the + button against the required KFF group to which the hash set should be associated.
- Check a specific Hash Set.
- Click Associate.
The hash set will now be associated with the selected KFF group.
Running KFF Against a Case
To run KFF import utility:
- From the Process Evidence page of case creation, click Customize Options.
- Select Document Content Analysis. Select KFF.
- Click the drop-down list.
Note: Alternatively, during review, select the desired items, right-click > Additional Analysis > Customize Options
5. Click Apply.
6. Click Process Data or Run Analysis.
Reviewing KFF Results in a Case
KFF results are displayed in Review.
You can use the following tools to see KFF results:
- KFF Columns. See Using Quick Columns section.
- KFF Facet Filters. See Using Facet Filters section.
KFF Facet Filters
You can use the following KFF facets:
- KFF Vendors
- KFF Groups
- KFF Statuses
- KFF Sets
Within a facet, only the filters that are available in the cases are available. For example, if no files with the Alert status are in the case, the Alert filter will not be available in the KFF Status facet.
To apply KFF facets:
- In the Grid, open the Filter Facets.
- Expand KFF.
- Select your chosen facets.
KFF Columns
You can use KFF Quick Columns and sort on KFF values. For example, you can sort on the KFFStatus column to quickly see all the files with the Alert status.
- From the Review, click the Quick Column menu.
- Select KFF.
- The following columns will appear:
- KFF Status
- KFF GroupName
- KFFSet
Column | Description |
KFF Status | Displays the status of the file as it pertains to KFF. The three options are Unknown (0), Ignore (1), and Alert (2).
|
KFF GroupName | Displays the name created for the KFF Group in the case. |
KFFSet | Displays the KFF Hash Set to which the file belongs. |