Users may believe that when they delete a file on their hard drive, the document no longer exists. However, IT professionals understand that the data itself may remain.
Yet even experienced IT professionals may not understand the differences between different types of hard drive file erasure, data overwrite standards, or when those methods might fail to delete data from a hard drive. Depending upon the sensitivity of the information, an IT professional may need to pursue more sophisticated levels of deletion or even destroy the hard drive itself.
We get into a bit about how hard drives and flash drives store data, but for those security pros tasked with digital forensics and compliance responsibilities, the discussion is anything but academic.
Basic Data Deletion
If a hard drive (or USB drive) will continue to be used, deleting a file through the operating system works just fine. However, users should be reminded to empty the Trash folder on their desktop; otherwise, the file simply has been moved to the Trash folder.
Once the user empties the Trash folder, the operating system removes the file information from the storage directory. The portion of the storage previously recorded as occupied by the file will be flagged as available for rewrite.
It is important to note that the file data itself is still present on the hard drive until the hard drive overwrites the data with new data. Over time, though, any deleted file should eventually be overwritten by new data as long as the storage device remains in use.
Also read: Top 9 Data Loss Prevention (DLP) Solutions
Hard Drives and Their Deletion Methods
Users store data locally on hard drives (magnetic platters or flash memory), USB drives (flash memory), or shared network drives. Network drives may be virtual or physical, cloud-based or local-server based, and either a single hard drive or a redundant array of independent disks (RAID) with data written on multiple hard drives.
While the information here primarily focuses on physical drives, the same principles apply to virtual drives. Whether a virtual drive is part of a larger physical hard drive, encompasses multiple physical hard drives, or exists purely in computer memory, at some point the virtual drive’s data will likely be written to flash memory or a physical platter for storage.
To understand how more thorough deletion methods work on any drive, we need to understand technical details about hard drive data storage. Specifically, we should have a basic understanding of how files are written to storage, the types of hard drive technologies, and when data on the hard drive might be inaccessible to the operating system.
File-Write Basics
When an operating system needs to write a file to a drive, it examines the drive to locate a blank space. The space will consist of one or more sectors of space on the drive.
Hard drive sectors have varied in size over time, but current drives use between 512-byte and 4096-byte sectors. If the file does not take up all of the space in the sector, some of the sector will be blank. If the file later grows in size, it will take up additional sectors.
The operating system will then maintain a directory of files and their associated locations on the drive. Since sectors do not need to be continuous, once a hard drive has been used for a while, files may be written in many different locations and become inefficient. Although, some hard drive cleanup applications can reorganize or rewrite the files on the drive to maximize continuous sectors.
Simple deletion of a file removes the file from the operating system’s directory of files. However, it does not replace the data written in the file’s various sectors on the hard drive.
Also read: Top GRC Tools & Software
Magnetic vs. Flash Drives
Physical hard drives use one of two different technologies: magnetic platters or flash memory. The difference between these two technologies becomes relevant in deletion based on the difference in which the technologies store data and manage aging of the media.
Magnetic Platter Hard Drives
Magnetic hard drives store data to sectors on the hard drive by magnetizing the platters within the drive. Some sectors on the drive will be allocated to the firmware that manage the hard drive and communicate with the operating system.
As the drive ages, the magnetization will become less pronounced, and the hard drive heads that read the data will struggle to tell the difference between zeros and ones in the binary file. Once the binary data falls below the predetermined threshold for error, the hard drive will flag that sector as failing and copy the data to a new sector of the hard drive.
The magnetic drive firmware will not usually notify the operating system about reassigned bad sectors. Instead, the managing firmware for the hard drive will reroute operating system calls for reassigned sectors to the new sector automatically.
Flash Memory Hard Drives
Flash drives program transistors on the floating-gate memory chips within the hard drive or USB flash drive. The memory is written into cells, which are equivalent in size and function to the sectors on a magnetic hard drive. A separate chip or a portion of a memory chip will be allocated to the firmware that manages the flash drive and communicates with the operating system.
As the drive ages, the flash drive management algorithm counts the number of reads/writes to each memory cell. Once a predetermined number of read/write functions have been performed, the cell is considered to be at risk for failure, the cell will be flagged as bad, and its data will be copied to a new cell.
As with magnetic drives, the firmware will not inform the operating system about bad cells or copying data to the new cells. Instead, the management software will invisibly manage the process and reroute data requests to the new locations.
Inaccessible Drive Data
The operating system will typically be unable to access significant portions of every hard drive. The managing firmware for the hard drive is protected from access and overwrite, and a portion of the hard drive will also be reserved by the firmware as future replacements for bad sectors.
Bad sectors also are generally unavailable to the operating system. Data recovery tools can use the drive firmware to attempt to read data from failing sectors, but this requires specialized equipment and software.
See our guide to the Best Ransomware Removal and Recovery Services, a market that overlaps with data recovery tools.
Some sectors on a hard drive can also be hidden from the operating system or assigned to different operating systems. For example, a computer might be set up to boot in either Linux or Windows, and each operating system might access different portions of the hard drive formatted with different file system technologies.
Less commonly, people may intentionally create hidden sectors on the drive to hide data. Sometimes this is for benign reasons, but law enforcement often finds this technique is used to hide data associated with illegal activities.
Also read: Best Digital Forensics Tools & Software
Reformatting a Drive
When reassigning a computer to a new user or removing a hard drive to put it in a different machine, we may want all prior data to be inaccessible to the new user. In this case, we often reformat the hard drive using the operating system.
Reformatting the drive will delete all file directories on the hard drive and make all sectors visible to the operating system available for file storage. However, this level of data erasure may not delete all data on the drive.
Hidden sectors will not be recognized by the operating system, and those sectors may escape reformatting. Bad sectors will also not be accessible for reformatting.
Data Overwrite Methods
When a company or individual upgrades laptops, phones, or digital video recorders, older devices will usually still be of use to other people. While this equipment may be sold or donated, previous owners don’t want their data to be accessible to new users.
Reformatting hard drives can be sufficient, but to achieve higher levels of certainty various drive erasure standards are available. Each technique will overwrite new data over all of the available sectors of the hard drive.
Of these standards, ATA Secure Erase provides the most efficient data erasure standard because it uses a single pass to overwrite data on the hard drive. This erasure method is generally considered acceptable for most purposes, and the chance for a successful data recovery after a Secure Erase process is almost zero.
Keep in mind that to overwrite data on a 1TB hard drive might take four to six hours for a single pass—even longer if the drive is old or in failing condition. Most people will not benefit from the additional security of three to seven overwrite passes and will be satisfied by Secure Erase.
When Erasure Might Fail
Even the most robust data overwrite algorithm may fail to erase all of the data on a hard drive.
Most software-based hard drive erasure will access the drive through an operating system. These applications cannot recognize, access, delete, or overwrite data contained in bad sectors or hidden sectors.
Hardware can be purchased to directly access hard drives, but some hardware simply runs a light version of Linux or Windows and will have the same limitations for deletion as a computer. Buyers needing to overwrite bad sectors must verify the tool or software used will access the data at the firmware level of the hard drive.
Some hard drives may be inaccessible because of firmware-level hard drive passwords. In these cases the password must be removed before accessing, deleting, or overwriting any data.
Other times, the controlling firmware may be corrupted, or the hard drive may be experiencing failure for the hard drive sectors containing the firmware. In this case, the managing firmware will not be available to facilitate overwrite of the drive.
Some data recovery experts can recover or restore firmware and be able to read data from failed drives. Even though the drive may have failed, the magnetic platters or memory chips may contain data that could be read by very determined people willing to invest a lot of time or money.
Total Destruction
When the data must be destroyed with absolute certainty, the storage media must be destroyed. For magnetic hard drives, degaussers will destroy the usability of the drive for all sectors—including the firmware sectors.
Shredders can be used on both magnetic and flash hard drives and will chop the drives into pieces too small to be useful for practical data recovery. Drills can also render drives generally unreadable, but data recovery experts might extract data from undestroyed sections of a platter or memory chip.
Many people will likely never need the assurance of total destruction. However, there will always be use cases, such as national security, where they require hard drives to be overwritten first and then also physically destroyed to guarantee inaccessibility of the data.
Some Perspective
There is a big difference between the possibility of data extraction and the likelihood of data extraction from an erased hard drive. For most, a secure erase or hard drive reformat will be more than sufficient to eliminate a reasonable chance of data recovery.
Specialized multi-pass deletion requires the purchase of hardware or software solutions, and the deletion will consume IT time and resources to execute. The level of deletion selected should be appropriate for the use case, the value of the data, and the resources available.
Read next:
