Software Generalist

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

InitOnceExecuteOnce API, synchronous (p. 197) - new in Vista and W2K8, http://msdn.microsoft.com/en-us/library/ms683493(VS.85).aspx

InitOnceBeginInitialize / InitOnceComplete API, asynchronous (p. 197) - code examples for one-time initialization: http://msdn.microsoft.com/en-us/library/ms686934(VS.85).aspx

Dynamic worker threads (pp. 199 - 200)

ALPC (pp. 202 - 206) - ALPC is used extensively in terminal services environments. Here is the sample output from WinDbg to mine for ALPC wait chains in frozen systems and blocked services: 

1: kd> !process 0 ff

[...]

THREAD fffffa8006ef7060  Cid 0350.279c  Teb: 000007fffffa4000 Win32Thread: fffff900c22904f0 WAIT: (WrLpcReply) UserMode Non-Alertable
    fffffa8006ef73f0  Semaphore Limit 0x1
Waiting for reply to ALPC Message fffff880104eecf0 : queued at port fffffa80060d4c80 : owned by process fffffa8004c39040
IRP List:
    fffffa8006d21c60: (0006,03a0) Flags: 00060030  Mdl: 00000000
    fffffa8005f876c0: (0006,03a0) Flags: 00060030  Mdl: 00000000
Not impersonating
DeviceMap                 fffff88000007450
Owning Process            fffffa80057844d0       Image:         svchost.exe
Attached Process          N/A            Image:         N/A
Wait Start TickCount      10908131       Ticks: 18864 (0:00:04:54.750)
Context Switch Count      12660                 LargeStack
UserTime                  00:00:00.671
KernelTime                00:00:00.578
Win32 Start Address 0×000007fefe51fdec
Stack Init fffffa6006249db0 Current fffffa6006249670
Base fffffa600624a000 Limit fffffa6006243000 Call 0
Priority 13 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP          RetAddr           Call Site
fffffa60`062496b0 fffff800`016a36fa nt!KiSwapContext+0×7f
fffffa60`062497f0 fffff800`0169835b nt!KiSwapThread+0×13a
fffffa60`06249860 fffff800`016cd4e2 nt!KeWaitForSingleObject+0×2cb
fffffa60`062498f0 fffff800`01916d14 nt!AlpcpSignalAndWait+0×92
fffffa60`06249980 fffff800`019137a6 nt!AlpcpReceiveSynchronousReply+0×44
fffffa60`062499e0 fffff800`0190330f nt!AlpcpProcessSynchronousRequest+0×24f
fffffa60`06249b00 fffff800`016a0ef3 nt!NtAlpcSendWaitReceivePort+0×19f
fffffa60`06249bb0 00000000`774d756a nt!KiSystemServiceCopyEnd+0×13 (TrapFrame @ fffffa60`06249c20)
00000000`029decd8 00000000`00000000 0×774d756a

1: kd> !alpc /m fffff880104eecf0

Message @ fffff880104eecf0
  MessageID             : 0x053C (1340)
  CallbackID            : 0xC26264 (12739172)
  SequenceNumber        : 0x0000031F (799)
  Type                  : LPC_REQUEST
  DataLength            : 0x0048 (72)
  TotalLength           : 0x0070 (112)
  Canceled              : No
  Release               : No
  ReplyWaitReply        : No
  Continuation          : Yes
  OwnerPort             : fffffa800a7804d0 [ALPC_CLIENT_COMMUNICATION_PORT]
  WaitingThread         : fffffa8006ef7060
  QueueType             : ALPC_MSGQUEUE_PENDING
  QueuePort             : fffffa80060d4c80 [ALPC_CONNECTION_PORT]
  QueuePortOwnerProcess : fffffa8004c39040 (svchost.exe)
  ServerThread          : fffffa800aa8c700
  QuotaCharged          : No
  CancelQueuePort       : 0000000000000000
  CancelSequencePort    : 0000000000000000
  CancelSequenceNumber  : 0×00000000 (0)
  ClientContext         : 0000000004e87390
  ServerContext         : 0000000000000000
  PortContext           : 0000000005615340
  CancelPortContext     : 0000000000000000
  SecurityData          : 0000000000000000
  View                  : 0000000000000000

1: kd> !alpc /p fffffa80060d4c80
Port @ fffffa80060d4c80
  Type                      : ALPC_CONNECTION_PORT
  CommunicationInfo         : fffff88007f66ba0
    ConnectionPort          : fffffa80060d4c80
    ClientCommunicationPort : 0000000000000000
    ServerCommunicationPort : 0000000000000000
  OwnerProcess              : fffffa8004c39040 (svchost.exe)
  SequenceNo                : 0x00000000 (0)
  CompletionPort            : fffffa80057d8040
  CompletionList            : 0000000000000000
  MessageZone               : 0000000000000000
  ConnectionPending         : No
  ConnectionRefused         : No
  Disconnected              : No
  Closed                    : No
  FlushOnClose              : Yes
  ReturnExtendedInfo        : No
  Waitable                  : No
  Security                  : Static
  Wow64CompletionList       : No

  Main queue is empty.

  Large message queue is empty.

  Pending queue has 192 message(s)

    fffff88012c7e030 0000201c 0000000000013f88:00000000000154fc 0000000000000000 fffffa8008573a30 LPC_CANCELED
    fffff88009a98cf0 0000221c 0000000000000b04:00000000000109b4 fffffa800b533bb0 fffffa8008ee7bb0 LPC_REQUEST
    fffff880129b9cf0 00001ffc 00000000000067d8:000000000000a85c fffffa800a032060 fffffa800a408060 LPC_REQUEST
    fffff8800ed62cf0 00001ea4 0000000000012c1c:0000000000013238 fffffa800afbcbb0 fffffa800bbf1060 LPC_REQUEST
    fffff88011fa7cf0 000014ec 0000000000000b04:0000000000013a38 fffffa80072c6bb0 fffffa800b4d2700 LPC_REQUEST
    fffff8801001e980 0000159c 000000000000b25c:0000000000003004 fffffa8006ebcbb0 fffffa8009c25060 LPC_REQUEST
    fffff88009a56cf0 00001f94 0000000000012940:0000000000015478 fffffa800a75d700 fffffa800b4b8060 LPC_REQUEST
[...]
    fffff880129aa640 000018f0 000000000000d31c:00000000000147c0 fffffa800ab9bbb0 fffffa8006ffb560 LPC_REQUEST
    fffff88008b29ac0 0000152c 000000000000fa5c:000000000000faa4 0000000000000000 fffffa800abadbb0 LPC_CANCELED
    fffff88009eaa460 000005dc 000000000000e13c:000000000000e3d4 fffffa800921b630 fffffa800a191060 LPC_REQUEST

  Canceled queue is empty.

The .NET Developer’s Guide to Windows Security by M. Brown:

Developing code in non-privileged way: two logons (p. 35)

whoami command (p. 37) - whoami /all

netsh command (p. 37)

runas /netonly (p. 39)

Nested runas commands (p. 40)

Debugger Users group (p. 41) 

Run - install assumption (p. 46) - Got an idea for a bugtation: “You must assume that your program will be” coded “by one person and” debugged “by another!”

Least privilige for installer design (p. 46)

Software Requirements & Specifications by M. Jackson:

Indicative mood of formalizations (p. 9)

Application domain vs. generic domain and app. domain vs. environment (p. 9)

Structured Analysis of DeMArco / Gane / Sarson - the danger of thinking in terms of a machine when looking at the existing system (p. 10)

Costruction of difficulty from simplicity in software development, shorter vs. longer problem description spans (pp. 12 - 14) - lesson: create one structural UML diagram that covers all possible problem spans

Different views on the origin of software development disasters (pp. 14 - 15) - the view from memory dump analysis engineer: study crash dump analysis patterns (by applying bijectionism) and learn from them. Software artifacts are memory dumps. Software development is memory change.

Software technology as technology of description (p. 17) 

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

InitializeCriticalSectionAndSpinCount (p. 187) - API description: http://msdn.microsoft.com/en-us/library/ms683476(VS.85).aspx

Tricky behaviour of keyed events (p. 188)

0: kd> dt _ETHREAD
nt!_ETHREAD
  +0x000 Tcb : _KTHREAD
  +0x330 CreateTime : _LARGE_INTEGER
  +0x338 ExitTime : _LARGE_INTEGER
  +0×338 KeyedWaitChain : _LIST_ENTRY
  +0×348 ExitStatus : Int4B
  +0×348 OfsChain : Ptr64 Void
  +0×350 PostBlockList : _LIST_ENTRY
  +0×350 ForwardLinkShadow : Ptr64 Void
  +0×358 StartAddress : Ptr64 Void
  +0×360 TerminationPort : Ptr64 _TERMINATION_PORT
  +0×360 ReaperLink : Ptr64 _ETHREAD
  +0×360 KeyedWaitValue : Ptr64 Void
  +0×360 Win32StartParameter : Ptr64 Void
  +0×368 ActiveTimerListLock : Uint8B
  +0×370 ActiveTimerListHead : _LIST_ENTRY
  +0×380 Cid : _CLIENT_ID
  +0×390 KeyedWaitSemaphore : _KSEMAPHORE
  +0×390 AlpcWaitSemaphore : _KSEMAPHORE

Safe and unsafe acquisition of fast and guarded mutexes (p. 189)

Gate primitive as an optimized event (pp. 189 - 190)

Executive resources (pp. 190 - 192) - These objects make analysis of synchronization issue easier because they have an owner thread:

0: kd> dt _ERESOURCE
nt!_ERESOURCE
  +0x000 SystemResourcesList : _LIST_ENTRY
  +0×010 OwnerTable : Ptr64 _OWNER_ENTRY
  +0×018 ActiveCount : Int2B
  +0×01a Flag : Uint2B
  +0×020 SharedWaiters : Ptr64 _KSEMAPHORE
  +0×028 ExclusiveWaiters : Ptr64 _KEVENT
  +0×030 OwnerEntry : _OWNER_ENTRY
  +0×040 ActiveEntries : Uint4B
  +0×044 ContentionCount : Uint4B
  +0×048 NumberOfSharedWaiters : Uint4B
  +0×04c NumberOfExclusiveWaiters : Uint4B
  +0×050 Reserved2 : Ptr64 Void
  +0×058 Address : Ptr64 Void
  +0×058 CreatorBackTraceIndex : Uint8B
  +0×060 SpinLock : Uint8B

0: kd> dt _OWNER_ENTRY
nt!_OWNER_ENTRY
  +0×000 OwnerThread : Uint8B
  +0×008 OwnerCount : Int4B
  +0×008 TableSize : Uint4B

There are some crash dump analysis pattern examples that involve executive resources: Deadlock (executive resources), High Contention (executive resources) and Wait Chain (executive resources)

Pushlocks are built on gates, pointer-size and have shared/exclusive mode (p. 192)

Lock convoys (p. 193) - This article explains them using critical section example: http://en.wikipedia.org/wiki/Lock_convoy

Shared vs. exclusive use of user mode critical sections (pp. 194 - 195)

Condition variables (p. 195) - New in Vista and W2K8. MSDN example of usage: http://msdn.microsoft.com/en-us/library/ms682052(VS.85).aspx. So it seems (if I understand it correctly) Windows implemented Monitor concept at OS level. I’m not accustomed to think in “high-level ” monitor terms when designing concurrency on Windows although it is a natural concept in Java. Even when I was programming in Java in the past I was thinking Windows primitives.

SRW (Slim Reader Writer) Locks to replace critical sections in user mode (p. 196) - Looks like new to Vista and W2K8

SRW + conditional variable as ideal combination (p. 196)

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

The .NET Developer’s Guide to Windows Security by M. Brown:

The principle of least privilege as software construction mindset (pp. 17 - 18)

Close a resource as you finish to avoid security intra-context leaks (p. 18)

Always think about failure (p. 23)

Multifactor authentication (p. 25) - Useful mnemonic HKM (Have, Know, Made of) 

Default modes in Kerberos and SSL: client to prove identity and server to prove identity (p. 26)

Did you complete identity proof by checking GUI details? (p. 26)

The luring attack (p. 27)

“Security Is a Process, Not a Product” by B. Schneier (p. 32) - Thinking analogically, regarding supportability, software engineers need to write maintainable code with a support engineer in mind.

Software Requirements & Specifications by M. Jackson:

Need to read this book to refresh my past software design and architecture knowledge

Software development as logic and math, as socio-ethical challenge, as labour employment relations, as problem in manufacturing control, as engineering (p. 1) - I see it as memory changes 

Distinction between machine and app domain: how and what (p. 1) - Modeling?

Requirements are in app domain (p. 2)

The importance of explicit and precise domain descriptions (p. 2)

Modeling is always incomplete, coincidental similarity of many OO descriptions to the real world (p. 3)

Specification as interface description between machine and app domain (p. 3)

Scope vs. Span: classes vs. areas (pp. 3 - 4)

Partial descriptions as separating concerns (p. 4)

Moods in description: optative (what we want) and indicative (is) (p. 4)

The importance of problem frames and multi-frame problems (pp. 4 - 5)

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

Queued spinlocks - spinning on a per-processor flag and FIFO ordering (p. 175) - Basic algorithm is described in this paper (section 2.1): Scalable Queue-Based Spin Locks with Timeout http://www.cs.rice.edu/~wns1/papers/2001-PPoPP-QBLTO.pdf

Executive resource is not a dispatcher object (p. 179) - Indeed it doesn’t start with _DISPATCHER_HEADER:

0: kd> dt _ERESOURCE
nt!_ERESOURCE
   +0x000 SystemResourcesList : _LIST_ENTRY
   +0x010 OwnerTable       : Ptr64 _OWNER_ENTRY
   +0x018 ActiveCount      : Int2B
   +0x01a Flag             : Uint2B
   +0x020 SharedWaiters    : Ptr64 _KSEMAPHORE
   +0x028 ExclusiveWaiters : Ptr64 _KEVENT
   +0x030 OwnerEntry       : _OWNER_ENTRY
   +0x040 ActiveEntries    : Uint4B
   +0x044 ContentionCount  : Uint4B
   +0x048 NumberOfSharedWaiters : Uint4B
   +0x04c NumberOfExclusiveWaiters : Uint4B
   +0x050 Reserved2        : Ptr64 Void
   +0x058 Address          : Ptr64 Void
   +0x058 CreatorBackTraceIndex : Uint8B
   +0x060 SpinLock         : Uint8B

0: kd> dt _KSEMAPHORE
nt!_KSEMAPHORE
   +0×000 Header           : _DISPATCHER_HEADER
   +0×018 Limit            : Int4B

Difference between a mutex in kernel and user (exported) modes (p. 181)

Some values in _DISPATCHER_HEADER can be ignored (depends on Type) (p. 185)

Meaning of _DISPATCHER_HEADER flags, table (p. 185)

_KOBJECTS, _DISPATCHER_HEADER.Type (pp. 185 - 186)

Looking wait queues manually (pp. 184 - 186) - I mistakenly subjected WinDbg to !list command passing the address of thread _KWAIT_BLOCK to it:

THREAD fffffa8003c0e8f0  Cid 0004.0070  Teb: 0000000000000000 Win32Thread: 0000000000000000 WAIT: (WrVirtualMemory) UserMode Non-Alertable
   fffff800019d97a0  Semaphore Limit 0x7fffffff
   fffff800019d9860  NotificationEvent
   fffff800019d9980  NotificationEvent
   fffff800019c8840  NotificationEvent
   fffff800019c8860  SynchronizationEvent
Not impersonating
DeviceMap                 fffff88000007400
Owning Process            fffffa8003bcd0b0       Image:         System
Attached Process          N/A            Image:         N/A
Wait Start TickCount      34801717       Ticks: 1683 (0:00:00:26.254)
Context Switch Count      12886
UserTime                  00:00:00.000
KernelTime                00:00:00.702
Win32 Start Address nt!MiDereferenceSegmentThread (0xfffff8000193caa0)
Stack Init fffffa6001947db0 Current fffffa6001947a60
Base fffffa6001948000 Limit fffffa6001942000 Call 0
Priority 18 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP          RetAddr           Call Site
fffffa60`01947aa0 fffff800`0186d28a nt!KiSwapContext+0x7f
fffffa60`01947be0 fffff800`0186c896 nt!KiSwapThread+0x2fa
fffffa60`01947c50 fffff800`0193cb36 nt!KeWaitForMultipleObjects+0x2d6
fffffa60`01947cd0 fffff800`01a8bfd3 nt!MiDereferenceSegmentThread+0x96
fffffa60`01947d50 fffff800`018a1816 nt!PspSystemThreadStartup+0x57
fffffa60`01947d80 00000000`00000000 nt!KiStartSystemThread+0x16

0: kd> dt _KTHREAD fffffa8003c0e8f0
nt!_KTHREAD
   +0x000 Header           : _DISPATCHER_HEADER
   +0x018 CycleTime        : 0x55dc0a75
   +0x020 QuantumTarget    : 0x69dfdff1
   +0x028 InitialStack     : 0xfffffa60`01947db0
   +0x030 StackLimit       : 0xfffffa60`01942000
   +0x038 KernelStack      : 0xfffffa60`01947a60
   +0x040 ThreadLock       : 0
   +0x048 ApcState         : _KAPC_STATE
   +0x048 ApcStateFill     : [43]  "8???"
   +0x073 Priority         : 18 ''
   +0x074 NextProcessor    : 0
   +0x076 DeferredProcessor : 1
   +0x078 ApcQueueLock     : 0
   +0x080 WaitStatus       : 0
   +0×088 WaitBlockList    : 0xfffff800`0198cfd0 _KWAIT_BLOCK
[…]

and it entered endless loop becoming unresponsive. CPU was spiking 50% (2 processor machine). I forced a crash dump in Task Manager (Spiking Thread pattern, http://www.dumpanalysis.org/blog/index.php/2007/05/11/crash-dump-analysis-patterns-part-14/):

0:000> kL
Child-SP          RetAddr           Call Site
00000000`0024bee8 000007fe`f507f0dd msftedit!CFormatRunPtr::GetFormat+0xa
00000000`0024bef0 000007fe`f507f54c msftedit!CTxtRange::SpanSubstring+0x101
00000000`0024bf50 000007fe`f5087d90 msftedit!CTxtRange::ItemizeRuns+0x2d8
00000000`0024c2c0 000007fe`f507dd0b msftedit!CRchTxtPtr::ItemizeReplaceRange+0x14c
00000000`0024c3c0 000007fe`f509d28f msftedit!CTxtRange::SetCharFormat+0xc77
00000000`0024c630 000007fe`f504f568 msftedit!CTxtSelection::SetCharFormat+0x23f
00000000`0024c770 000007fe`f50a3f84 msftedit!CTxtEdit::OnSetCharFormat+0x110
00000000`0024c910 000007fe`f50585b2 msftedit!CTxtEdit::TxSendMessage+0x23b0
00000000`0024cbb0 00000000`7769d53e msftedit!RichEditWndProc+0xcca
00000000`0024d5b0 00000000`7769b5b5 user32!UserCallWinProcCheckWow+0x1ad
00000000`0024d670 00000000`7769b649 user32!SendMessageWorker+0x64a
00000000`0024d700 00000001`3f9b3f9e user32!SendMessageW+0x5b
00000000`0024d750 00000001`3f9b41a0 windbg!RichEditAddRawText+0x13e
00000000`0024d7a0 00000001`3f9b47ef windbg!RichEditAddText+0x1b0
00000000`0024d840 00000001`3f977629 windbg!RichEditAddTextBuffer+0x13f
00000000`0024d9a0 00000001`3f9c8728 windbg!WinCommand::AddTextBuffer+0xb9
00000000`0024da10 00000001`3f9d3739 windbg!wmain+0x4b8
00000000`0024fad0 00000000`7776be3d windbg!_CxxFrameHandler3+0x291
00000000`0024fb10 00000000`778a6a51 kernel32!BaseThreadInitThunk+0xd
00000000`0024fb40 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

0:000> r
rax=0000000000000000 rbx=000000000008d64a rcx=000000000024bf10
rdx=0000000002603a00 rsi=0000000000000001 rdi=0000000000048b33
rip=000007fef50566f6 rsp=000000000024bee8 rbp=0000000000000000
 r8=000000000002b6e6  r9=0000000000000000 r10=000000000024bf90
r11=000000000024bf20 r12=00000000025e17d0 r13=000000000024bfd0
r14=0000000011461b46 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206
msftedit!CFormatRunPtr::GetFormat+0xa:
000007fe`f50566f6 394208          cmp     dword ptr [rdx+8],eax ds:00000000`02603a08=00042d7a

0:000> !runaway
 User Mode Time
  Thread       Time
   0:d64       0 days 0:01:12.899
   1:f38       0 days 0:00:25.880
   3:c58       0 days 0:00:01.669
   6:12e4      0 days 0:00:00.000
   5:a84       0 days 0:00:00.000
   4:e88       0 days 0:00:00.000
   2:f80       0 days 0:00:00.000

The .NET Developer’s Guide to Windows Security by M.  Brown:

Reading this book to refresh my knowledge of Windows security needed for my work on the next generation troubleshooting tools. I read the previous “Programming Windows Security” book more than 6 years ago

Data channel vs. control channel (p. 4)

Countermeasures in protection, detection and reaction (p. 7) - Troubleshooting tools: Supportability (code), Detection (monitoring) and Reaction (reporting, alerts)

Threat model (p. 12) - “Supportability model” in the sense of predicting issues and their servicing

Unqualifiability of “My System is Secure” () - “My System is Supportable”. An idea for a bugtation: “[…] it’s impossible to” support “a system that you don’t understand”. What about crash dump analysis then?

STRIDE (Howard, LeBlanc) (p. 13)

Repudiation as denial of an attack by an attacker (p. 13)

Compexity as a security enemy (p. 16) - An idea for another bugtation “compexity is the number-one enemy of” debugging. When I was thinking prioritizing supportability features and idea of supportlets came to my mind

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

Instancing the namespace in \Sessions\n (p. 167) - note that prior to Vista console session was 0 and was shared with services

DeviceMap _EPROCESS substructure (p. 168)

0: kd> dt _DEVICE_MAP
nt!_DEVICE_MAP
   +0x000 DosDevicesDirectory : Ptr64 _OBJECT_DIRECTORY
   +0x008 GlobalDosDevicesDirectory : Ptr64 _OBJECT_DIRECTORY
   +0x010 ReferenceCount   : Uint4B
   +0x014 DriveMap         : Uint4B
   +0x018 DriveType        : [32] UChar

Altitude object filtering concept (p. 170)

Incorrect sharing of memory example (p. 171) - although context switches emulate multiprocessor systems single-processor machines experience the same error conditions less frequently: http://www.dumpanalysis.org/blog/index.php/2007/04/14/race-conditions-on-a-uniprocessor-machine/

Spinlock illustration (pp. 173) - here is a “spinning” illustration in 3-dimensional abstract space: http://www.dumpanalysis.org/blog/index.php/2007/10/25/threads-as-braided-strings-in-abstract-space-part-1/

lock xadd and lock bts (pp. 172 - 173)

Spinlock busy wait CPU consumption (p. 174) - I had some cases and named a pattern called Dispatch Level Spin (not only applicable to spinlocks but to every loop at DPC level and higher(: http://www.dumpanalysis.org/blog/index.php/2008/01/25/crash-dump-analysis-patterns-part-44/

Pause instruction (p. 174) - Here’s a short description with disassembly example from Asmpedia: http://www.asmpedia.org/index.php?title=PAUSE

- Dmitry Vostokov @ SoftwareGeneralist.com -

Coming back to reading after holidays

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

Per session objects \Sessions\n\BaseNamedObjects (p. 158)

OpenXXX / CreateXXX race condition: solution CreateXXXEx (p. 159)

Most objects are temporary (p. 159) - I got an idea for a bugtation “Most bugs are permanent”

Name retention by handle access vs. reference retention by pointer access (pp. 159 - 160) - most objects have a few references but some are referenced a lot:

0: kd> !handle

[...]

002c: Object: fffffa80047db0d0  GrantedAccess: 000f01ff Entry: fffff880078c70b0
Object: fffffa80047db0d0  Type: (fffffa8003c0bdc0) Desktop
    ObjectHeader: fffffa80047db0a0 (old version)
        HandleCount: 61  PointerCount: 7534
        Directory Object: 00000000  Name: Default

[...]

Deferred object delete operation (p. 160)

Resource accounting (pp. 161 - 162) - it would be nice to have an example here using object header structures

Name squatting (p. 164)

Private namespaces in Vista, CreatePrivateNamespace for user apps, solution to name squatting attacks (p. 164)

CheckForOtherInstanceMutex (p. 166) - Windows API legacy: search for a window with the same class can be used for single instancing

Sound Mixer in Vista/W2k8 (p. 166) - this is the great feature I didn’t know about

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

Using Process Explorer diff. highlighting to see handle leaks (p. 151)

Similarity of handle tables to virtual-2-physical mapping tables on x86 (pp. 151 - 152) - Seems the theoretical handle limit value for x86 is 512 * 512 * 511 = 133,955,584 (if I didn’t miss anything)

Testlimit tool (pp. 152 - 153) - I try it for my book Software Defect Construction: Simulation and Modeling of Software Bugs (ISBN: 978-1906717759) and also for later crash dump analysis volumes

!devhandles WinDbg command, searching for open files (p. 155) - it looks like it is done through device prefix to a file name; I’ve done simple text search for a file name if known through all handle tables: http://www.dumpanalysis.org/blog/index.php/2008/05/30/who-opened-that-file/

!obtrace monitors more than !htrace (p. 156)

nt!_OBJECT_TYPE.Key is pool tag, needed to enable object reference tracing (p. 157)

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

TpWorkerFactory (thread pools), TmRm, TmTx, TmTm, TmEn executive objects (p. 137)

Object structure (p. 138) - I created a simplified UML diagram when I was investigating object signaling: http://www.dumpanalysis.org/blog/index.php/2008/03/06/signaled-objects/. Here’s another example of how to get an object header from the dispatcher address a thread is waiting for:

THREAD fffffa800579f060  Cid 0ee8.113c  Teb: 000000007ef9b000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable
     fffffa800538bc10  ProcessObject
Not impersonating
DeviceMap                 fffff88006972f30
Owning Process            fffffa8005ad1c10       Image:         WINWORD.EXE
Attached Process          N/A            Image:         N/A
Wait Start TickCount      29919495       Ticks: 4883905 (0:21:09:49.406)
Context Switch Count      3
UserTime                  00:00:00.000
KernelTime                00:00:00.000
Win32 Start Address 0×0000000073433619
Stack Init fffffa6004bc8db0 Current fffffa6004bc8940
Base fffffa6004bc9000 Limit fffffa6004bc3000 Call 0
Priority 10 BasePriority 8 PriorityDecrement 2 IoPriority 2 PagePriority 5
Kernel stack not resident.
Child-SP          RetAddr           Call Site
fffffa60`04bc8980 fffff800`0186d28a nt!KiSwapContext+0×7f
fffffa60`04bc8ac0 fffff800`0186e68a nt!KiSwapThread+0×2fa
fffffa60`04bc8b30 fffff800`01ae2368 nt!KeWaitForSingleObject+0×2da
fffffa60`04bc8bc0 fffff800`018670f3 nt!NtWaitForSingleObject+0×98
fffffa60`04bc8c20 00000000`75ec3d09 nt!KiSystemServiceCopyEnd+0×13 (TrapFrame @ fffffa60`04bc8c20)
00000000`049deec8 00000000`00000000 0×75ec3d09

0: kd> dt _DISPATCHER_HEADER fffffa800538bc10
nt!_DISPATCHER_HEADER
   +0×000 Type             : 0×3 ”
   +0×001 Abandoned        : 0 ”
   +0×001 Absolute         : 0 ”
   +0×001 NpxIrql          : 0 ”
   +0×001 Signalling       : 0 ”
   +0×002 Size             : 0×30 ‘0′
   +0×002 Hand             : 0×30 ‘0′
   +0×003 Inserted         : 0 ”
   +0×003 DebugActive      : 0 ”
   +0×003 DpcActive        : 0 ”
   +0×000 Lock             : 3145731
   +0×004 SignalState      : 0
   +0×008 WaitListHead     : _LIST_ENTRY [ 0xfffffa80`04ca25f8 - 0xfffffa80`0566bca8 ]

0: kd> dt _OBJECT_HEADER
nt!_OBJECT_HEADER
   +0x000 PointerCount     : Int8B
   +0x008 HandleCount      : Int8B
   +0x008 NextToFree       : Ptr64 Void
   +0x010 Type             : Ptr64 _OBJECT_TYPE
   +0x018 NameInfoOffset   : UChar
   +0x019 HandleInfoOffset : UChar
   +0x01a QuotaInfoOffset  : UChar
   +0x01b Flags            : UChar
   +0x020 ObjectCreateInfo : Ptr64 _OBJECT_CREATE_INFORMATION
   +0x020 QuotaBlockCharged : Ptr64 Void
   +0x028 SecurityDescriptor : Ptr64 Void
   +0×030 Body             : _QUAD

0: kd> dt _OBJECT_HEADER fffffa800538bc10-30
nt!_OBJECT_HEADER
   +0×000 PointerCount     : 55
   +0×008 HandleCount      : 5
   +0×008 NextToFree       : 0×00000000`00000005
   +0×010 Type             : 0xfffffa80`03bcd840 _OBJECT_TYPE
   +0×018 NameInfoOffset   : 0 ”
   +0×019 HandleInfoOffset : 0 ”
   +0×01a QuotaInfoOffset  : 0 ”
   +0×01b Flags            : 0×20 ‘ ‘
   +0×020 ObjectCreateInfo : 0xfffffa80`0489c010 _OBJECT_CREATE_INFORMATION
   +0×020 QuotaBlockCharged : 0xfffffa80`0489c010
   +0×028 SecurityDescriptor : 0xfffff880`06d9825b
   +0×030 Body             : _QUAD

Here is another example of how to get the name of an object if it exists (38 is the size of x64 _OBJECT_HEADER, _QUAD is 8 bytes):

0: kd> dt _OBJECT_HEADER fffffa80047da830
nt!_OBJECT_HEADER
   +0×000 PointerCount     : 175
   +0×008 HandleCount      : 116
   +0×008 NextToFree       : 0×00000000`00000074
   +0×010 Type             : 0xfffffa80`03c0b080 _OBJECT_TYPE
   +0×018 NameInfoOffset   : 0×20 ‘ ‘
   +0×019 HandleInfoOffset : 0×30 ‘0′
   +0×01a QuotaInfoOffset  : 0 ”
   +0×01b Flags            : 0×20 ‘ ‘
   +0×020 ObjectCreateInfo : 0xfffff800`01990fa0 _OBJECT_CREATE_INFORMATION
   +0×020 QuotaBlockCharged : 0xfffff800`01990fa0
   +0×028 SecurityDescriptor : 0xfffff880`0678a6fc
   +0×030 Body             : _QUAD

0: kd> dt _UNICODE_STRING fffffa80047da830-38+20
nt!_UNICODE_STRING
 “WinSta0″
   +0×000 Length           : 0xe
   +0×002 MaximumLength    : 0xe
   +0×008 Buffer           : 0xfffff880`00969660  “WinSta0″

Pointer count >= Handle count (p. 139)

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

int 2e is still available for legacy calls (p. 126)

rdmsr WinDbg command (p. 126)

Zw… as fake interrupts (p. 129) - here is another view (remember that there are ntdll!Nt… and nt!Nt… functions): http://www.dumpanalysis.org/blog/index.php/2007/04/10/yet-another-look-at-zw-and-nt-functions/

KiServiceTable entries on x64 (p. 132)

WinObj (pp. 133 - 134) - I also use !object WinDbg command to navigate the tree: 

0: kd> !object \
Object: fffff88000005d50  Type: (fffffa8003bacdc0) Directory
    ObjectHeader: fffff88000005d20 (old version)
    HandleCount: 0  PointerCount: 49
    Directory Object: 00000000  Name: \

    Hash Address          Type          Name
    ---- -------          ----          ----
     01  fffff88000005420 Directory     ObjectTypes
     03  fffffa8004bcd320 Event         NETLOGON_SERVICE_STARTED
     05  fffff88000952910 SymbolicLink  SystemRoot
     06  fffff880000e7900 Directory     Sessions
     07  fffffa8004b4d2b0 ALPC Port     MmcssApiPort
     08  fffffa8004839750 Event         DSYSDBG.Debug.Trace.Memory.270
         fffff880000138d0 Directory     ArcName
     09  fffff88000072bb0 Directory     NLS
         fffffa8004ce7cd0 ALPC Port     XactSrvLpcPort
     10  fffff880000079f0 Directory     GLOBAL??
         fffff880000e5ab0 Directory     Windows
         fffffa8004b80ee0 Event         LanmanServerAnnounceEvent
     11  fffff880000e7ab0 Directory     RPC Control
     13  fffffa80046bfd20 Event         EFSInitEvent
     14  fffffa8003eb5940 Device        clfs
         fffff88000962d20 SymbolicLink  Dfs
     15  fffffa8004690270 ALPC Port     SeRmCommandPort
         fffffa80047b3b00 Event         CsrSbSyncEvent
     16  fffff880000071c0 SymbolicLink  DosDevices
         fffffa80051ab8d0 Device        Cdfs
     17  fffff88000965b20 Directory     KnownDlls32
         fffffa8004c20060 ALPC Port     AELPort
         fffffa800487d620 Event         EFSSrvInitEvent
     18  fffff880000131e0 Key           \REGISTRY
         fffffa8004ca0a60 ALPC Port     WindowsErrorReportingServicePort
     19  fffff880061859f0 Directory     BaseNamedObjects
     21  fffff880000735c0 Directory     UMDFCommunicationPorts
         fffffa8004a67900 ALPC Port     SmSsWinStationApiPort
         fffffa8004251840 Event         UniqueInteractiveSessionIdEvent
     22  fffff88006117200 Directory     KnownDlls
         fffffa80054c6c80 Device        FatCdrom
         fffffa8005597b00 Device        Fat
     23  fffff8800008d2f0 Directory     FileSystem
         fffff88000005a20 Directory     KernelObjects
         fffffa8004059ad0 Device        Ntfs
     26  fffff88000005870 Directory     Callback
         fffffa800482fd90 ALPC Port     SeLsaCommandPort
     28  fffff88000009850 Directory     Security
     29  fffffa8004aeaa90 ALPC Port     UxSmsApiPort
     30  fffff880000135a0 Directory     Device
         fffffa80048776c0 Event         EFSSmbInitEvent
     32  fffffa8004876060 ALPC Port     LsaAuthenticationPort
     34  fffffa80046bf060 ALPC Port     SmApiPort
         fffff880066b2b00 Section       LsaPerformance
         fffffa80047aa840 Event         UniqueSessionIdEvent
     36  fffff8800008d4a0 Directory     Driver
         fffffa8004879eb0 Event         SAM_SERVICE_STARTED

0: kd> !object \Driver
Object: fffff8800008d4a0  Type: (fffffa8003bacdc0) Directory
    ObjectHeader: fffff8800008d470 (old version)
    HandleCount: 0  PointerCount: 76
    Directory Object: fffff88000005d50  Name: Driver

    Hash Address          Type          Name
    ---- -------          ----          ----
     01  fffffa800468e440 Driver        NetBT
         fffffa8004317ba0 Driver        PptpMiniport
         fffffa80042b49e0 Driver        usbuhci
         fffffa8003d64b90 Driver        Wdf01000
     02  fffffa8005532700 Driver        MYFAULT
         fffffa8004b82e70 Driver        mpsdrv
     03  fffffa800407f740 Driver        disk
         fffffa8004afe900 Driver        lltdio
         fffffa8004693ab0 Driver        PSched
         fffffa800453fc80 Driver        NDProxy
     04  fffffa8004c00e70 Driver        HTTP
     06  fffffa80042b02c0 Driver        usbehci
         fffffa80042a6a60 Driver        tunnel
     07  fffffa8003eba7c0 Driver        partmgr
     08  fffffa8004c50940 Driver        PEAUTH
         fffffa800430ee10 Driver        iScsiPrt
         fffffa80042d7dd0 Driver        b57nd60a
         fffffa8003c116f0 Driver        ACPI_HAL
     09  fffffa8004675e70 Driver        RDPENCDD
         fffffa800407e060 Driver        spldr
     10  fffffa800430ce10 Driver        Rasl2tp
         fffffa80040392c0 Driver        storflt
         fffffa8004269ad0 Driver        HidUsb
     11  fffffa8004dc5c00 Driver        AsyncMac
         fffffa8003bad7f0 Driver        PnpManager
     12  fffffa8004673610 Driver        Null
         fffffa800431f7e0 Driver        rdpdr
     14  fffffa80042dbbf0 Driver        Serenum
         fffffa8003ef27b0 Driver        CLFS
     15  fffffa80042f3d50 Driver        Serial
         fffffa80046729c0 Driver        RDPCDD
         fffffa8003ee97b0 Driver        KSecDD
         fffffa8003ebc7c0 Driver        volmgr
     16  fffffa8004333a30 Driver        umbus
         fffffa80040eb490 Driver        crcdisk
     17  fffffa80047b0e00 Driver        Win32k
     18  fffffa8004329730 Driver        mouclass
         fffffa80046894d0 Driver        Smb
     19  fffffa8003f47db0 Driver        msisadrv
     20  fffffa8004321510 Driver        kbdclass
     21  fffffa800407de70 Driver        volsnap
         fffffa800479ca60 Driver        mouhid
     22  fffffa80046c3550 Driver        nsiproxy
         fffffa8004673420 Driver        VgaSave
         fffffa8003bbfe70 Driver        WMIxWDM
     23  fffffa8003fab360 Driver        Wanarpv6
         fffffa8004678060 Driver        tdx
         fffffa8004671730 Driver        RasAcd
         fffffa800431d820 Driver        RasSstp
     25  fffffa8004319e70 Driver        RasPppoe
         fffffa80042b69c0 Driver        HDAudBus
     26  fffffa8004c68a00 Driver        secdrv
     27  fffffa80042dfe70 Driver        Parport
         fffffa800426a240 Driver        kbdhid
     28  fffffa8004b00a30 Driver        rspndr
         fffffa8004328a10 Driver        TermDD
     29  fffffa80046586a0 Driver        HdAudAddService
         fffffa8003f79710 Driver        pci
         fffffa800432c7d0 Driver        mssmbios
         fffffa8003eb87c0 Driver        volmgrx
     30  fffffa80042db370 Driver        cdrom
         fffffa8003ee3750 Driver        NDIS
     31  fffffa800432b420 Driver        swenum
     32  fffffa800433de70 Driver        usbhub
         fffffa8003f272a0 Driver        Tcpip
     33  fffffa80042aabc0 Driver        intelppm
         fffffa8003ec18f0 Driver        atapi
     34  fffffa800468d7c0 Driver        AFD
         fffffa800430a710 Driver        NdisTapi
         fffffa8003ec1e70 Driver        mountmgr
         fffffa8003ebee70 Driver        intelide
     35  fffffa8004c84db0 Driver        tcpipreg
         fffffa800465a060 Driver        ksthunk
     36  fffffa8004311640 Driver        NdisWan
         fffffa8003d96060 Driver        ACPI

It might be interesting to signal these events manually and see what happens:

0: kd> !object \KernelObjects
Object: fffff88000005a20  Type: (fffffa8003bacdc0) Directory
    ObjectHeader: fffff880000059f0 (old version)
    HandleCount: 0  PointerCount: 18
    Directory Object: fffff88000005d50  Name: KernelObjects

    Hash Address          Type          Name
    ---- -------          ----          ----
     00  fffffa8003c0f920 Event         MemoryErrors
     02  fffffa8003bfc510 Event         LowNonPagedPoolCondition
     04  fffffa80047be6b0 Session       Session1
     05  fffffa8003bef740 Event         SuperfetchScenarioNotify
         fffffa8003bef7c0 Event         SuperfetchParametersChanged
     06  fffffa8003c0b4e0 Event         BootLoaderTraceReady
     12  fffffa8003c0fa20 Event         HighCommitCondition
     14  fffffa8003bfb590 Event         HighNonPagedPoolCondition
         fffffa8003bfd590 Event         HighMemoryCondition
     21  fffff88000009060 KeyedEvent    CritSecOutOfMemoryEvent
     23  fffffa8003c0f9a0 Event         MaximumCommitCondition
     25  fffffa8003bfb510 Event         LowCommitCondition
     26  fffffa8003bfc590 Event         HighPagedPoolCondition
     28  fffffa8003c0e5d0 Event         LowMemoryCondition
     32  fffffa8003bfd510 Event         LowPagedPoolCondition
         fffffa80047ac520 Session       Session0
     34  fffffa8003bef6c0 Event         PrefetchTracesReady

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

SEH for 64-bit (p. 116) - here is an example from x64 W2K8

0:000> kn
 # Child-SP          RetAddr           Call Site
00 00000000`0016f358 00000000`76d8679a ntdll!ZwReadFile+0xa
01 00000000`0016f360 000007fe`fdedfc9a kernel32!ReadFile+0×8a
02 00000000`0016f3f0 000007fe`fdedfa3b advapi32!ScGetPipeInput+0×3a
03 00000000`0016f440 000007fe`fdede00d advapi32!ScDispatcherLoop+0×9a
04 00000000`0016f540 00000000`ffa11dca advapi32!StartServiceCtrlDispatcherW+0×176
05 00000000`0016f7e0 00000000`ffa124b2 svchost!wmain+0×110
06 00000000`0016f810 00000000`76d8be3d svchost!ScCreateWellKnownSids+0×301
07 00000000`0016f850 00000000`76ec6a51 kernel32!BaseThreadInitThunk+0xd
08 00000000`0016f880 00000000`00000000 ntdll!RtlUserThreadStart+0×1d

0:000> !exchain
9 stack frames, scanning for handlers...
Frame 0×01: kernel32!ReadFile+0×8a (00000000`76d8679a)
  ehandler kernel32!_C_specific_handler (00000000`76d7dcbc)
Frame 0×04: advapi32!StartServiceCtrlDispatcherW+0×176 (000007fe`fdede00d)
  ehandler advapi32!_GSHandlerCheck_SEH (000007fe`fdf6ab88)
Frame 0×06: svchost!ScCreateWellKnownSids+0×301 (00000000`ffa124b2)
  ehandler svchost!_C_specific_handler (00000000`ffa13ea4)
Frame 0×08: ntdll!RtlUserThreadStart+0×1d (00000000`76ec6a51)
  ehandler ntdll!_C_specific_handler (00000000`76ed9518)

VEH is user-mode only (p. 116) - Here is an article from Matt Pietrek about vectored exception handling: http://msdn.microsoft.com/en-us/magazine/cc301714.aspx

Debug object (p. 116) - coincidentally, today I wrote a case study where we can see how an exception dispatcher sends a message: http://www.dumpanalysis.org/blog/index.php/2009/08/11/stack-trace-collection-suspended-threads-not-my-version-special-process-main-thread-and-blocked-lpc-chain-threads-pattern-cooperation/. Also here is another case study showing the flow of exception processing originated from user mode and space: http://www.dumpanalysis.org/blog/index.php/2007/12/07/interrupts-and-exceptions-explained-part-6/

First and second chance exception handling (p. 117) - some time ago I was interested in this mystery and wrote this post: http://www.dumpanalysis.org/blog/index.php/2008/05/22/demystifying-first-chance-exceptions-part-1/

Wercon.exe (p. 120)

WER processing overview (pp. 120 - 125) - some time ago when Vista appeared I did experiments with TestDefaultDebugger.exe (similar to Accvio.exe): http://www.dumpanalysis.org/blog/index.php/2007/05/19/inside-vista-error-reporting-part-1/ 

LocalDumps registry key values (p. 123)

- Dmitry Vostokov @ SoftwareGeneralist.com -