Digital forensics is the collection, examination and analysis of data from computers, networks and other devices to find out what happened, while keeping the data from being changed. The data used to prove this is called digital evidence. This article explains the basics of collecting evidence after a security incident, based on NIST SP 800-86 and other publicly available good-practice guides. It is general knowledge, not legal advice.
Why digital evidence is easy to damage
NIST SP 800-86 defines this work as applying science to identify, collect, examine and analyze data while preserving its integrity and keeping a strict chain of custody (a record of who held the evidence and when). The problem is that digital data changes easily. NIST states that every action on a system, whether taken by a person or by the operating system, will almost certainly alter some volatile data. Volatile data is data that is lost when a computer is powered down or as time passes, such as the memory (RAM) of a machine that is still running.
RFC 3227 (a 2002 Internet best-practice document on evidence collection and archiving) warns that it is all too easy to destroy evidence, even without meaning to. Evidence can therefore be damaged by the team that rushes to respond, not only by attackers who erase their traces.
If evidence may be used in legal or internal disciplinary proceedings, NIST advises deciding this before collection starts, with the analyst or management working with legal advisors. If it is unclear whether evidence needs to be preserved, it should generally be preserved by default.
The four principles from the UK police guide (ACPO)
The ACPO Good Practice Guide for Digital Evidence (March 2012) was agreed by the Association of Chief Police Officers for use by police forces in England, Wales and Northern Ireland. It is not Thai law, but it is a public good-practice guide with easy-to-read principles. It sets out four principles, which we summarize in our own words:
- Do not change data that may be relied on: no action should alter data that may later be relied upon in court
- If original data must be touched, a competent person does it: that person must be able to explain why and what the effects were
- Record every step: keep an audit trail that lets an independent third party repeat the process and reach the same result
- Someone is in charge: the person leading the investigation is responsible for making sure the law and these four principles are followed
The collection steps
NIST SP 800-86 divides this work into four phases: collection, examination, analysis and reporting. This article focuses on the first, collection, because evidence damaged at this stage cannot be repaired in the later ones (our view).
For the collection phase, NIST recommends four steps:
- Identify data sources: work out which machines, devices and logs are involved, including sources outside the organization
- Plan the collection: decide what to collect first, based on the likely value of the data, how fast it can disappear, and the effort needed
- Acquire the data: copy the data and keep the original secure
- Verify integrity: compare the hash of the original with the hash of the copy to show the data has not changed
Throughout, NIST says to log every step taken and every tool used so that other analysts can repeat the process, and to note or photograph what is on the screen before touching the machine. NIST also advises organizations to prepare data in advance, for example by configuring auditing on operating systems, centralizing logs, taking regular backups and using security monitoring. For logs, see Log Management: What to Log and How Long to Keep It.
What to collect first: the fastest-vanishing data
NIST says to prioritize collection by three factors: the likely value of the data to the investigation, its volatility (how fast it can disappear), and the effort required. In many cases volatile data should be collected before persistent data, but NIST cautions that some seemingly persistent data also changes quickly, such as log files that are overwritten as new events occur.
RFC 3227 gives an example order for a typical system, from the most volatile to the least volatile:
- CPU data: registers and cache, the fastest to disappear
- State of the running system: the routing table, ARP cache, process table, kernel statistics and memory
- Temporary files: the temporary file system
- Disk: data stored persistently on the machine's drives
- Remote logging and monitoring data: data relevant to that system that is stored elsewhere
- Physical configuration and network topology: how cables and devices are connected
The decision that has to be made quickly concerns a machine that is still running. NIST states that powering it off, or even disconnecting it from the network, can remove the chance to collect important information. RFC 3227 adds that cutting the network may trigger a deadman switch (a mechanism that wipes evidence when it detects the machine has left the network). NIST also warns that collecting from a running machine carries its own risks: files may change, and a rootkit (hidden malware that can return false information) may make the collected data untrue. NIST therefore advises writing the decision criteria down in advance so that whoever is on duty can decide immediately.
This can collide with the containment steps that are common in incident response, such as isolating a machine from the network. Our view: do not make this decision for the first time during an incident; write the criteria into the incident response plan in advance. See Incident Response Plan Under NIST SP 800-61 Rev. 3 for response planning and How to Prevent Ransomware: 5 First Steps for Organizations for prevention steps.
Chain of custody, copies and hashes
A chain of custody is a record that can answer who found, collected, passed on and examined each piece of evidence, when, and how it was stored. NIST explains that this record helps avoid allegations that evidence was mishandled or tampered with.
RFC 3227 says at least these four things need to be documented:
- Discovery and collection: where, when and by whom the evidence was discovered and collected
- Handling and examination: where, when and by whom it was handled or examined
- Custody: who held it, during what period, and how it was stored
- Transfers: when custody changed, when and how the transfer happened, including shipping numbers if it was sent
For copying, NIST describes two approaches. A logical backup copies only directories and files, and does not capture deleted files. A bit stream image is a bit-for-bit copy that includes free space and slack space on the disk, and it needs more storage space and time. If evidence may be needed for prosecution or disciplinary action, NIST advises making a bit stream image of the original media, labeling it, storing the original securely, and doing all analysis on the copy.
While copying, analysts can use a write-blocker (a hardware or software tool that stops a computer writing to the storage media connected to it) so the original is not altered. NIST also warns that bit stream imaging tools should generally not be used to copy an entire physical disk from a live system, because files and memory are changing constantly and the copy cannot be validated. A bit-for-bit copy of the logical areas (logical volumes) of a live system can be made and validated.
To show that a copy matches the original, use a hash (also called a message digest: a short value computed from the data, which changes completely if even one bit of the data changes). NIST gives this order: compute and record the hash of the original before imaging, compute the hash of the copy afterwards and compare, then compute the hash of the original again to confirm the imaging did not alter it, and document all results.
A note: SP 800-86 was published in 2006 and gives MD5 and SHA-1 as example hash algorithms, which are older ones. Our view: choose an algorithm that is still trusted today, such as SHA-256, and check with your tool vendor or a specialist before setting it as an organizational standard.
Standards, law and the limits of this article
ISO/IEC 27037:2012 is an international standard with guidelines for the identification, collection, acquisition and preservation of digital evidence. The full text is paywalled, so we have not read it and do not describe its details. What we know comes only from the publicly available scope description, which says the standard guides individuals in common situations in evidence handling and helps organizations with disciplinary procedures and with exchanging potential evidence between jurisdictions.
SP 800-86 was published in August 2006. NIST's abstract says it is not an all-inclusive guide to a digital forensic investigation and is not legal advice, and it recommends consulting management and legal counsel before applying it. Technology has changed a great deal since then, so tools and technical details should be checked with a specialist before use.
Which digital evidence Thai courts will accept, and how it must be collected to be usable, are questions of Thai law. We have not checked them and we do not give legal advice. Ask legal counsel or the organization's legal team before collecting evidence in an incident that may lead to a case.
RFC 3227 also raises privacy: follow the privacy rules and guidelines of the company and of the legal jurisdiction, and do not collect from areas you would not normally have reason to access, such as employees' personal files, unless there is sufficient indication of a real incident. If personal data under the PDPA (Thailand's Personal Data Protection Act) is involved, ask the legal team or the organization's data protection officer.
Common mistakes
- Powering off or rebooting immediately: without first deciding whether to collect volatile data, so the contents of memory are lost
- Browsing files on the original machine: instead of making a copy and examining the copy, which can change the data and file times
- Running programs that change the access time of every file: RFC 3227 gives tar and xcopy as examples
- Trusting the programs on a compromised machine: instead of running evidence-gathering tools from protected media
- Skipping the copy and the hash comparison: so there is no proof that what was examined matches the original
- Not noting who did what and when: which leaves no chain of custody and means others cannot repeat the steps
Damages evidence
- Power off at oncememory contents are lost
- Browse the originaldata and file times may change
- Trust local programsa compromised machine may mislead
Protects evidence
- Decide criteria in advanceknow when to collect before shutdown
- Work on copies, compare hashesthe original stays untouched and provable
- Record every step and holderothers can repeat and check
Start today
- Name the decision-maker: say who decides whether to collect volatile data, and write the criteria down in advance
- Prepare a chain-of-custody form: use the four RFC 3227 headings as a template
- Prepare tools and storage: for example a write-blocker, storage for copies and a hashing tool, tested to work
- Centralize logs ahead of time: so that data about each machine is kept off the machine
- Talk to legal and an outside specialist in advance: so you know which incidents call for whom (our view)
To see where your organization stands, try the self-assessment.
References and further reading
- NIST SP 800-86: Guide to Integrating Forensic Techniques into Incident Response: source of the four-phase process, the collection steps, chain of custody, copying and hashes (August 2006 edition)
- RFC 3227: Guidelines for Evidence Collection and Archiving: order of volatility, things to avoid, and what to document in a chain of custody
- ACPO Good Practice Guide for Digital Evidence, March 2012 (PDF): the four principles; this is a copy held on a third-party website, not on an ACPO site
- ISO/IEC 27037:2012 (ISO): the ISO standard page; the full text is paywalled, and our tools could not open this page directly
- ISO/IEC 27037:2012 (EVS): the publicly available scope description used for the summary in this article
- ENISA: Electronic evidence, a basic guide for first responders: a guide for incident response teams; we read only the description page, not the report
- NIST SP 800-61 Rev. 3: Incident Response Recommendations: the incident response guidance that forms the context for forensics; we read only the abstract page