NTFS
Content Leader: Jess Garcia - Last Updated: November 7, 2006
Gral. Info.
-
NTFS: New Technology File System
-
NTFS makes an exact copy of the boot sector in the last sector of the volume
-
File fragmentation info is contained in the MFT and is not immediately overwritten during formatting or deletion (as in FAT).
-
The MFT grows and grows and grows, and can eventually become fragmented.
-
Records are reused as they are freed. That's why you don't see many deleted files in NTFS.
-
The timestamps are kept in GMT in the disk. The time shown will be dependent on the Time Zone configuration of the system mounting this filesystem.
-
A duplicate of the boot sector is located at the logical center of the disk.
-
Everything on a NTFS filesystem resolves to an entry or file, including the Volume Boot Sector.
-
As part of the NTFS formatting procedure, 16 records are created and reserved for system files:
-
11 (0-10) as the first 11 records of the MFT.
-
5 (11-15) are reserved
-
Cluster Size & Sectors/Cluster vs Part. Size
Part.Size (G) Sectrs/Cluster Dflt Cluster Size Dflt (K)
<= 0.5 1 0.5
> 0.5 to 1.0 2 1
> 1.0 to 2.0 4 2
> 2.0 to 4.0 8 4
> 4.0 to 8.0 16 8 (NT3.51- only)
> 8.0 to 16.0 32 16 (NT3.51- only)
> 16.0 to 32.0 64 32 (NT3.51- only)
> 32.0 128 64 (NT3.51- only)
-
Reason for not allowing more than 4K big clusters in NT3.51+: Compression is not supported on file systems with cluster sizes over 4K
-
WNT only uses the cluster table above if you are creating a _new_ NTFS partition. If you are converting a partition from FAT to NTFS, WNT will always make the NTFS volume use the smallest clusters: 512 bytes (this applies to the partition where you initially install WNT which must always be FAT and is later converted).
Format
0 511
+-----------------+-----+--------------+---------------------------+
| Vol.Boot Sector | MFT | System Files | File Area |
+-----------------+-----+--------------+---------------------------+
Volume Boot Sector
-
Volume Boot Sector (512b):
-
Begins in the 1st sector of the partition.
-
0 3 11 36 84
+--------+------+---------+---------+----------------+---------------+
|JumpInst|OEM ID| BPB | Ext BPB | Bootstrap code |EndOfSectorMark|
+--------+------+---------+---------+----------------+---------------+
0 (0x00) [3] Jump Instruction
3 (0x03) [8] OEM ID
11 (0x0B) [25] BIOS Parameter Block
36 (0x24) [48] Extended BPB
84 (0x54) [426] Bootstrap Code
511(0x01FE)[2] End of Sector Marker (0x55AA)
* BIOS Parameter Block (BPB):
Block of data that contains fundamental info about the volume itself.
· Identifies the volume as an NTFS partition,
· Includes volume label and its size, etc.
· An extended BIOS parameter block contains additional info about the
volume such as the location of the key metadata files.
0x0B [2] Bytes Per Sector
0x0D [1] Sectors Per Cluster
0x0E [2] Reserved Sectors
0x10 [3] always 0
0x13 [2] not used by NTFS
0x15 [1] Media Descriptor
0x16 [2] always 0
0x18 [2] Sectors Per Track
0x1A [2] Number Of Heads
0x1C [4] Hidden Sectors
0x20 [4] not used by NTFS
* Extended BIOS Parameter Block (Extended BPB):
0x24 [4] not used by NTFS
0x28 [8] Total Sectors
0x30 [8] Logical Cluster # for $MFT
0x38 [8] Logical Cluster # for $MFTMirr
0x40 [4] Clusters Per File Record Segment
0x44 [4] Clusters Per Index Block
0x48 [8] Volume Serial Number
0x50 [4] Checksum
* Volume Boot Code:
Small block of program code that instructs the system on how to load the
OS. This code will be specific to whichever Windows is installed on the
system. It will generally load NTLDR, the NT loader program, and then
transfer control to it to load the rest of the OS. This code is also
present in the partition as a system (metadata) file.
-
The NTFS has numerous attributes that don't exist in typical Unix file systems -- you have multiple streams, SACLs, DACLs, and so forth.
-
NTFS is a journaling system (transactional). Changes are written in a LOG file before being performed.
Master File Table (MFT)
-
MFT starts from offset 0x4000, non-fragmented (TBC ???)
-
The MFT can be located by the signature FILE\x2A (WNT/W2K) or FILE\x30 (WXP)
-
Once a file has marked for deletion, it will be reused when a new entry needs to be created.
-
0-15 Special Records:
0-10 $MFT, $MFTMirr, $LogFile, $Volume, $AttrDef, $., $Bitmap, $Boot,
$BadClus, $Sec, $UpCase
11-15 Reserved
16- Files & Dirs
-
+---+---------------+---------------+----------------+---------------+----+
|HDR| Standard Info | File/Dir Name | Sec.Descriptor | Data or Index | |
+---+---------------+---------------+----------------+---------------+----+
· Record Header:
-
The three most useful attributes include the:
* Standard Information attribute with the date and time stamps
* File Name attribute that lists the file name and also includes the MFT
record number of the parent folder
* Data attribute that actually contains the data if the file is small
enough, or contains data runs that point to where the data is located in
the partition
-
[4] "FILE" identifier: FILE\x2A -> WNT/W2K ; FILE\x30 -> WXP
[2] Offset to update sequence (2 bytes)
[2] Size of update sequence (2 bytes)
[8] $LogFile Sequence Number (LSN) (8 bytes)
[2] Sequence Number (2 bytes)
[2] Reference Count (2 bytes)
[2] Offset to Update Sequence Array (2 bytes)
[2] Flags (2 bytes)
[4] Real size of the FILE record (4 bytes)
[4] Allocated size of the FILE record (4 bytes)
[8] File reference to the base FILE record (8 bytes)
[2] Next Attribute Id (2 bytes)
· Starting on the word "FILE" for 48 bytes
Offset Length
16 2 Number of times the MFT record has been used
22 2 Flags for the MFT record (e.g. file/dir)
0x00 File, deleted
0x01 File, allocated
0x02 Folder, deleted
0x03 Folder, allocated
24 2 Allocated size of the MFT record
28 2 Total size of the MFT record
42 6 MFT checksum
first 2 bytes: checksum of the 1st sector of the
1st MFT sector (bytes 510-511) ????
2nd 2 bytes: checksum of the 2nd sector of the
2nd MFT sector (bytes 1022-1023) ????
3rd 2 bytes ????
4 SI
4 Attributes Length
0 -> Resident ; 1 -> Non-resident
4 Filename Length
4 Data Length
...
32 4 Date & Time entries (CWMA)
4 Filename Attribute (8.3)
Filename (Windows Long Filename). Absent if name is 8.3
Data (if it is resident)
+ The end of the MFT record is identified by 8 bytes: FF FF FF FF XX XX XX XX
+ "Special" Records:
· All the "special" records (0-15) have the same FWAM times, that of the
formatting moment (these are not modified by the OS).
0 $MFT Master File Table
1 $MFTMirr MFT Mirror
Mirrors just the 4 first records of the MFT:
$MFT, $MFTMirr, $logFile, $Volume
It's located in the middle of the drive.
2 $LogFile Circular FIFO File
Transactional db for recovery
It's very hard to elliminate evidence from it: most
progs can't even touch it
You can find MFT records, index buffers, link files.
3 $Volume Info about NTFS version and volume name. MFT resident.
4 $AttrDef Attribute Definition Table
Contains a list of all the attrs used on the Volume
(Best viewed at 160 chars)
5 $. Root Folder
Points to the top of the dir. tree
6 $Bitmap Bitmap file for allocation clusters.
Tracks every cluster on the volume.
1 bit/cluster (0:available; 1:allocated)
7 $Boot 16 sectors used by the Volume Boot Sector
· Vol. Serial #: Offset: 72b ; Length: 4
(Right-to-Left)
8 $BadClus Bad Clusters - It's actually empty, it's
just a holder.
$BadClus $Bad A second stream where the actual Bad
Cluster data resides
9 $Quota NTFSv4. Intended to implement quotas but never used.
$Secure NTFSv5.
$Secure:$SDH Used to check new sec attrs against previously defined
ones.
$Secure:$SDS Contains the actual sec info attrs.
$Secure:$SII Used to obtain the sec descriptor attr for a file or
folder that has been accessed
10 $UpCase Converts uppercase to lowercase chars to the matching
uppercase Unicode chars
-
Resident Files:
-
600 bytes is the max size of a resident file.
-
Files either are resident or not.
-
The OS will try to create RFs whenever possible, it's more efficient.
-
If a file is resident and grows it will become non-resident. The reverse process does never occur.
-
Depending on the contents a MFT record can have space available for data: RD
-
RD must belong to that MFT record
-
RD does not have file slack.
-
RD does not start at the begining of the sector.
-
Typical examples: boot.ini
Files
+---+------------------+-------------------+--------------------+---------+
|Hdr|Standard Info Attr|File Name Info Attr|Sec Descriptor Attr |Data Attr|
+---+------------------+-------------------+--------------------+---------+
-
All files in NTFS consist of attributes:
ID Name
Standard Info Includes information such as timestamp and link count.
(10 00 00 00)
Attribute List Lists the location of all attribute records that do not
(20 00 00 00) fit in the MFT record.
File Name A repeatable attribute for both long and short file
(30 00 00 00) names. The long name of the file can be up to 255
Unicode characters. The short name is the 8.3,
case-insensitive name for the file. Additional names,
or hard links, required by POSIX can be included as
additional file name attributes.
Volume Version
(40 00 00 00)
Security Descriptor Describes who owns the file and who can access it.
(50 00 00 00) Contains file data. NTFS allows multiple data atts per
file. Each file typically has one unnamed data attr.
A file can also have one or more named data attrs,
each using a particular syntax.
Volume Name Vol. Label. Used only in the $Volume system file.
(60 00 00 00)
Volume Information Vol. Version. Used only in the $Volume system file.
(70 00 00 00)
Data
(80 00 00 00)
Index Root Used to implement folders and other indexes.
(90 00 00 00)
Index Allocation Used to implement folders and other indexes.
(A0 00 00 00)
Bitmap Used to implement folders and other indexes.
(B0 00 00 00)
Symlink
(C0 00 00 00)
HPFS extended attributes information
(D0 00 00 00)
HPFS extended attributes
(E0 00 00 00)
Object ID A volume-unique file identifier. Used by the distributed
link tracking service. Not all files have object ids.
Logged Tool Stream Similar to a data stream, but operations are logged to
the NTFS log file just like NTFS metadata changes.
This is used by EFS.
Reparse Point Used for volume mount points. They are also used by
Installable File System (IFS) filter drivers to mark
certain files as special to that driver.
-
Standard Information Attribute:
-
Always resident.
-
Follows the MFT record hdr.
-
Contents:
-
MACW times & dates
-
MS-DOS file permissions
-
Pointer to the secure file (which contains file's security descriptor) (NTFS v5)
-
[8] File Creation Time
[8] File Last Modification Time
[8] File Last Modification Time for File Record
[8] File Access Time for File Record
[4] DOS File Permissions 0x20 in our case Archive Attribute
-
Always follows the standard info attr.
-
Sometimes 2 File Name attrs are found in a MFT record:
-
Long file name
-
DOS-truncated name
-
-
File Name
-
File's Parent
-
Same 4 dates as the SI Attr.
-
Logical Files size (not including file attrs)
-
File perms flags (DOS-like): read-only, hidden, system, archive, compressed, directory
-
File Name (usually Unicode)
-
File Type
-
File Size
[8] File Reference to the Parent Directory
[32] File Modification Times
[8] Allocated Size of the File
[8] Real Size of the File
[8] Flags
[1] Length of File Name
[1] File Name Space
[var] File Name (Length of File Name * 2 bytes)
-
Security Information Attribute:
-
-
Attribute Type (4 bytes) (e.g. 0x80)
-
Length including header (4 bytes)
-
Non-resident flag (1 byte)
-
Name length (1 byte)
-
Offset to the Name (2 bytes)
-
Flags (2 bytes)
-
Attribute Id (2 bytes)
-
Starting VCN (8 bytes)
-
Last VCN (8 bytes)
-
Offset to the Data Runs (2 bytes)
-
Compression Unit Size (2 bytes)
-
Padding (4 bytes)
-
Allocated size of the attribute (8 bytes)
-
Real size of the attribute (8 bytes)
-
Initialized data size of the stream (8 bytes)
-
Data Runs ...
-
Follows either the File Name Attribute (NTFS v5) or the Security Information Attribute (pre-NTFS v5).
-
Data can be resident or non-resident.
-
Data Runs:
-
The data attr in the non-resident case contains pointers to the clusters allocated to the data, called Data Runs.
-
The 1st data run always stores the starting location of the data, and the number of clusters used within that segment.
-
2nd and suceeding DRs contain the distance from the prior data run and the number of contiguous cluster in each segment. This value can be + or -
e.g. 31 0A D0 01 01 21 08 88 13
3 -> (refers to D0 01 01)
1 -> (refers to 0A)
0A -> 10 clusters used in the first data run
D0 01 01 -> Starting cluster (66,000)
2 -> (refers to 88 13)
1 -> (refers to 08)
08 -> 8 clusters
88 13 -> distance from the previous cluster (D0 01 01)
-
Facts:
-
Every file identifies its parent through the file name attr.
Index Root, Dir. Entries, Buffers
-
Index Root: Folders
-
Index Entries: Entries in that folder
-
Buffers: Non-resident entries
Filesystem Operations
Filesystem Operations: File Creation
-
When a file is created:
-
1. An entry is created in the MFT
-
2. BitMap is updated
-
3. Data is stored in clusters
-
4. Entries are created in the index.
File Deletion
-
1. MFT entry is marked as deleted
-
2. BitMap is updated to allocate the disk space
-
3. Index buffer is deleted
MAC Times Updates
-
Facts:
-
NTFS updates the atime only if the updated atime would be an hour or more later than the previous atime.
-
When a file is copied, the mtime of the target file is the same as the original file, while the atimes and ctimes behave as expected. This can make a file appear as though it was created after it was modified.
-
lstat()ing a file does not change the MACtimes:w
Template "Boot Sector NTFS"
// Template by Paul Mullen, pcguru@the-answer.com
// last modified Jul 16, 2000
// To be applied to sector 0 of an NTFS-formatted
// logical drive or to the mirror copy of the boot
// sector, which will be located near then end of
// the partition.
// See Microsoft KB article Q153973, "Recovering NTFS boot Sector"
// http://support.microsoft.com/support/kb/articles/Q153/9/73.asp
description "Boot sector of an NTFS partition"
appliesto disk
sector-aligned
requires 0x00 "EB" // bytes 0 to 2 are
requires 0x02 "90" // JMP instruction
requires 0x03 "4E 54 46 53 20" // ID must be "NTFS"
requires 0x1FE "55 AA" // "magic" signature
begin
read-only hex 3 "JMP instruction" //00
char[8] "SystemID"
//03
uint16 "Bytes per sector" //0B
uint8 "Sectors per cluster" //0D
uint16 "Reserved sectors" //0E
hex 3 "(always zero)"
//10
read-only hex 2 "(unused)" //13
hex 1 "Media descriptor"
//15
read-only hex 2 "(unused)" //16
uint16 "Sectors per track" //18
uint16 "Heads"
//1A
uint32 "Hidden Sectors"
//1C
read-only hex 4 "(unused)" //20
read-only hex 4 "(always 80 00 80 00)" //24
int64 "Total sectors"
//28
move -8
hex 8 "Total sectors (hex)" //28
int64 "Start C# $MFT"
//30
int64 "Start C# $MFTMirr"
//38
uint32 "Clust per MFT rec" //40
uint32 "Clust per index block" //44
uint32 "Serial number (int)" //48
move -4
hex 4 "Serial number (hex)" //48
goto 0x1FE //boot load code follows
read-only hex 2 "Signature (55 AA)" //1FE
end
