NTUSER.DAT journals and logs are not cleaned offline

Messages
86
Reaction score
3
MSDN image files from MS are clean of NTUser LOG1, LOG2, BLF, and REGTRANS files found in user directories and in "Windows\System32\Config" directory. Whenever I use NTLite for offline imaging, it generates mentioned file types and preserves them. I have to manually remove them.

Is that normal? All images are affected - Install.wim, Boot.wim (both indexes), and WinRE.wim.
 
There's no functional need to purge any registry logs, they're a natural byproduct of updating hives.

And if you browse a a clean retail ISO (since I don't have a MSDN copy), the original images aren't "cleaned up" either.
- boot.wim (Default User)
- boot.wim (Windows\System32\config)
- install.wim (Default User)
- install.wim (Windows\System32\config)
- WinRE (Default User)
- WinRE (Windows\System32\config)
 
There's no functional need to purge any registry logs, they're a natural byproduct of updating hives.

And if you browse a a clean retail ISO (since I don't have a MSDN copy), the original images aren't "cleaned up" either.
- boot.wim (Default User)
- boot.wim (Windows\System32\config)
- install.wim (Default User)
- install.wim (Windows\System32\config)
- WinRE (Default User)
- WinRE (Windows\System32\config)
Such logs don't show up in official images. It seems unprofessional/messy to leave such logs behind. I think I saw $RECYCLE.BIN generated in NTLite ISO files as well.
 
Just loaded (with NTL) a vanilla UUpdump updated image 22621.2134 and the only ntuser(.dat) file is placed in \User\default.
So the assumption would be, that all other ntuser.dat, logs, etc. must be system generated after account generation (and control?) and mandatory for Windows to work after install.
 
Think about it.
If create a new user on a Windows install, it give meaning to have NTUser LOG1, LOG2, BLF, and REGTRANS to transfer silmilar settings to a new user (profile) generated - but same time control admin/ user privileges and other stuff.
 
Offline image is supossed to be generic/non-specific to any user. Tweaks and changes are applied on machine level and/or to default/all users, which becomes specific user when image is online (installed) and account is created, unless specified otherwise in some unattended settings.

In this case, I am talking about taking official ISO, applying updates, cleaning up, rebasing, and saving ISO without any other changes. In such a case, logs to default user profile should not be preserved.
 
If they dont need to be there and MS iso's dont have them i would like to have them removed.
I like to keep my updated(captured) "gold" image as damn near close to what ms intended as possible, no tweaks or removals so i can take that image in whatever direction i want, default or tweaked depending on the need.
 
Last edited by a moderator:
Most of them is ActivationStore.dat.Log1 or 2.
An some for the MS Apps keeped.
 
If they dont need to be there and MS iso's dont have them i would like to have them removed.
I like to keep my updated(captured) "gold" image as damn near close to what ms intended as possible, no tweaks or removals so i can take that image in whatever direction i want, default or tweaked depending on the need.
Exactly my point.
 
Oh i know there is crap in an ms iso, i agreed with garlin earlier. I do the cleanups once i have captured my image.
 
When you modify a registry hive, Windows updates the in-memory copy and then writes a transaction log. For performance reasons, you don't want to flush (resync) every change to disk immediately. Especially considering how large, and how often changes are made to SYSTEM or SOFTWARE hives.

In the event of a system crash before the pending changes are synced, Windows can use transaction logs to restore the missing edits. You don't get to control this activity, it's done whenever Windows decides to:

Flush strategies in the Windows registry
The Windows implementation of the registry deals with two entities: a registry hive in memory and a registry hive on a disk. When a registry hive is mounted, it's being read or mapped into memory from a disk. When you change something in a mounted registry hive (e.g. when you create a new key or modify a value), these changes happen directly in memory. The process of writing these changes back to a disk is called a hive flush.

Before Windows 8.1 and Windows Server 2012 R2, the flush process writes modified data (also known as dirty data) to a transaction log file first (overwriting modified data from previous flush processes), then the flush process writes the same data to a primary file. If a system crash (e.g. power outage) occurs when writing to a transaction log file, a primary file will remain consistent, because no data was written to this file during the failed flush operation; if a system crash occurs when writing to a primary file, a copy of modified data from a transaction log file will be used to finish writing to a primary file, thus bringing the file back to the consistent state. Before writing to a primary file, the flush process will invalidate its header to record the inconsistent state, and after modified data was successfully stored in a primary file, the flush process will validate the header (so it's possible to tell whether a primary file is consistent or not by examining its header). The flush process for a hive is triggered by a kernel at regular time intervals or by a userspace program using the RegFlushKey() routine. This flush strategy involves writing the same data twice, thus reducing the performance of an operating system.

In Windows 8.1 and Windows Server 2012 R2, a new flush strategy was implemented: when the flush process is triggered for a specific hive (either by a kernel or by a userspace program) for the first time or after the status of a transaction log file has been reset, a log entry with modified (dirty) data is written to a transaction log file, and the header of a primary file is invalidated. When the flush process is triggered again, a new log entry with modified data is appended to a transaction log file, and a primary file remains untouched. If a system crash occurs, a transaction log file will contain log entries required to recover the consistency of a primary file and to bring it to the up-to-date state. When all users (local and remote) become inactive, or when a hive starts unloading (e.g. during the full shutdown), or when an hour has been elapsed since the latest write to a primary file, the reconcile process will write all modified data to a primary file, validate its header, and reset the status of a transaction log file (next flush processes will be overwriting old log entries with new ones). The new flush strategy has performance improvements based on the significant decrease of the number of disk writes.

Much of this isn't officially documented, it's all from security researchers.

When you work with offline images, there is no host OS to actively reconcile the data. Therefore you have leftover registry overhead sitting around in the captured filesystem. Assuming these are active hives, eventually Windows should clean them up after normal operation.
 
MSDN image files from MS are clean
They are not

en-us_windows_11_consumer_editions_version_22h2_updated_july_2023_x64_dvd_f69501d4.iso

If the ISO generated by UUP dump does not have this then it may be that the script that generates the ISO (which is not official by Microsoft) removes the logs.

NTLite added a feature that compresses registry files (a feature that I requested in a thread here on the forum but I haven't been able to test it yet). This greatly reduces their size, but this may only come into play when removing components. In my thread I suggested removing the logs as well, but the changelog only says "Registry defragmented after removals".

In the same thread mentioned, I posted a script that I use by pasting the path of the image mounted by NTLite into it (you need to mount the image and click on "New session" to ensure that there is no registry hive loaded) that cleans the logs and compresses the registry files in the image.
 
.Net cache files are large, 300MB +, cache and temp files? untouched iso's are quite dirty.
 
.NET caches are dynamic regenerated by ngen. There's no point clearing them. Any disk space you saved will be consumed again.
 
I know but a bit from here and a bit from there all adds up especially when you are limited to 4.3GB formatted dvd disc.
Trim the fat. Mount and cleanup + shrink the boot.wim too.
 
For the record, you're not "compressing" the hive as much as "defragging" it. For performance, the in-memory store will be implemented on something like a heap object. You can't directly do garbage collection on a memory-mapped hive file. Saving it under a different file instance will allow skipping over the "dead" entries.
 
Back
Top