Software Generalist

Comments in italics are mine and express my own views, thoughts and opinions

Windows Internals by M. Russinovich, D. Solomon and A. Ionescu:

WMI CIM Studio (pp. 321 - 322)

dynamic and static MOF classes (p. 323) 

WbemTest, BMF (binary MOF), Mofcomp.exe (p. 323)

Object keys as WMI class instance specifiers (\\computer\root\namespace:class_name.Key1=”…”, Key2=”…”, … ) (pp. 324 - 325)

WMI association classes (p. 325) 

WQL exampe (p. 327)

wmiprvse.exe as a WMI provider host (p. 327)

wmic.exe (p. 328)

Namespace level WMI secutiry (p. 329)

WDI, Windows Diagnostic Infrastructure and its instrumentation, DiagLog, SEM Scenario Event Mapper, on-demand diagnosis (pp. 329 - 330) - looks interesting, especially in the context of possible first fault software problem solving techniques (OpenTask has published a book on this topic: http://www.dumpanalysis.com/First+Fault+Software+Problem+Solving)

Advanced Windows Debugging by M. Hewardt and D. Pravat:

LRPC_CCALL(ADDRESS) vs. OSF_CCALL(ADDRESS) vs. DG_CCALL(ADDRESS) (pp. 389 - 390)

Undocumented MSRPC (p. 391) - there is an empirical technique to find LRPC server endpoint: http://www.dumpanalysis.org/blog/index.php/2008/07/11/in-search-of-lost-pid/

!lpc message (p. 393) - some additional scenarios can be found in patterns: http://www.dumpanalysis.org/blog/index.php/2008/12/17/crash-dump-analysis-patterns-part-42e/, http://www.dumpanalysis.org/blog/index.php/2007/11/29/crash-dump-analysis-patterns-part-9d/ and various case studies involving LPC chains: http://www.dumpanalysis.org/blog/index.php/pattern-cooperation/

_PS_IMPERSONATION_INFORMATION (p. 395) - Looks like on W2K8 x64 it is another bit union:

lkd> dt -r _ETHREAD
[…]
  +0×3b0 ClientSecurity   : _PS_CLIENT_SECURITY_CONTEXT
      +0×000 ImpersonationData : Uint8B
      +0×000 ImpersonationToken : Ptr64 Void
      +0×000 ImpersonationLevel : Pos 0, 2 Bits
      +0×000 EffectiveOnly    : Pos 2, 1 Bit

RPC cell debugging configuration (pp. 397 - 398)

Advanced .NET Debugging by M. Hewardt:

Lutz Roeder’s .NET Reflector (pp. 15 - 16)

Roberto Farah’s PowerDbg (pp. 17 -18)

MDA Managed Debugging Assistants (pp. 19 - 21) - looks similar to WDI (Windows Diagnostic Infrastructure) on-demand diagnostics for unmanaged code mentioned in Windows Internals book

CLI(+BCL) -> CLR (p. 24)

Rotor (p. 25) - looks like it has the same value as WINE for unmanaged code: http://www.dumpanalysis.org/blog/index.php/2006/11/16/how-wine-can-help-in-crash-dump-analysis/ 

- Dmitry Vostokov @ SoftwareGeneralist.com -

Resuming reading notebook after the long silence

Comments in italics are mine and express my own views, thoughts and opinions

Advanced Windows Debugging by M. Hewardt and D. Pravat:

LPC protocol was built with debugging in mind (p. 382)

LPC communication steps: NtCreatePort (s) -> NtListenPort (s) -> NtConnectPort (c) -> NtAcceptConnectPort (s) -> NtCompleteConnectPort (s) -> NtRequestWaitReplyPort (c) -> NtReplyWaitReceivePort (s) (pp. 383 - 384) - Actually LPC message ID is stored in _ETHREAD not in _KTHREAD, for example:+0×3f0 LpcReplyMessageId : 0xd50b74d

!lpc thread WinDbg command to show all LPC activity (pp. 386 - 387)

Concurrent Programming on Windows by J. Duffy:

Software Factories by J. Greenfield, et. al.:

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Didn’t have time today - resume reading tomorrow

- Dmitry Vostokov @ SoftwareGeneralist.com -

Resuming reading notebook and plan to fill it at least 3-4 times per week

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

Implicit or hidden function calls in high level languages, step-into command, GDB step (pp. 133 - 135) - t command in WinDbg

step-out command to bypass stepping through implicit functions, GDB finish command (p. 136) - gu command in WinDbg

temporary breakpoint in GDB, tbreak (p. 136) - also .step_filter in WinDbg to create the list of function to skip

conditional breakpoints and breakpoint commands (pp. 138 -139) - bp and ba commands in WinDbg have an additional parameter: command string; Also bp “J” and bp “.if” conditional variants in WinDbg

print strcmp(…) and $$0 in GDB (p. 139) - I think in WinDbg this can be done via .call and $callret

static constructors and initializers (pp. 140 - 143)

technique: slowing the program to have time to attach the debugger (p. 142)

Advanced Windows Debugging by M. Hewardt and D. Pravat:

.dbgdbg command (p. 368)

technique: using local kernel debugger when user debugger extension fails (p, 375)

Concurrent Programming on Windows by J. Duffy:

A thread may already begin or even finish by the time CreateThread returns (p. 89)

thread pseudo-handle from GetCurrentThread == -2 (p. 94)

thread handle from thread id - OpenThread (p. 95)

use CRT function only in threads created by _beginthread(ex) (p. 96)

_beginthread closes handle automatically by return of start routine (p. 97)

Software Factories by J. Greenfield, et. al.:

EAI to solve the problem of data flow and transformation between applications on different servers (p. 21)

Building applications from business process perspective (p. 22)

DCOM and CORBA are tightly coupled to component implementations (p. 23)

service-oriented architecture (SOA): loosely coupled coarse grained components, message interaction, protocol sequencing defined by contracts and negotiated constraints (SLA, service level agreements) (p. 25)

BPMS (business process management system) as logical analog to DBMS (pp. 25 - 26)

pi-calculus as theoretical platform for BPMS (p. 25)

CRM subprocesses: campaigns, sales force and leads, customer management, customer service and self-service (pp. 28 - 29)

Portals as encapsulation of user interaction (p. 30) - an idea of tool portals for troubleshooting; Also each troubleshooting domain is implemented as a troubleshooting process or service that encapsulates troubleshooting logic and required data access; troubleshooting facades working with older troubleshooting tools

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Didn’t have time today - resume reading tomorrow

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

DLL-specifc problems (pp. 122 -127) - For Windows, I’ve identified 2 patterns so far: http://www.dumpanalysis.org/blog/index.php/2008/04/22/crash-dump-analysis-patterns-part-59/ and http://www.dumpanalysis.org/blog/index.php/2008/06/12/crash-dump-analysis-patterns-part-59b/

-fPIC flag for GCC to generate position independent code for dynamic linking and -shared flag to generate .so files (p. 123)

ldd utility on UNIX to find required .so files (p. 124) - On Windows it can be Dependency Walker (depends.exe)

Incorrect search order problems (p. 124) - See also my pattern for Windows: http://www.dumpanalysis.org/blog/index.php/2008/06/19/crash-dump-analysis-patterns-part-64/

Analyzing loader issues, LD_DEBUG on UNIX (pp. 125 - 126) - On Windows it can be loader snap in option in gflags.exe and !dlls WinDbg command

shared library GDB command (p. 126) - corresponding WinDbg command lmv

break GDB command (p. 126) - corresponding b* commands in WinDbg

pending breakpoints (pp. 126 - 127) - corresponding deferred or unresolved breakpoints in WinDbg (bu command)

ptype and info function commands in GDB to list class methods and overloaded functions (pp. 130 and 131)

breakpoints in C++ templates, function signatures for instantiated templates vs. source code lines (pp. 131 - 132) - I need to check this with WinDbg

info breakpoints GDB command (p. 132) - corresponding WinDbg commands are bl and .bpcmds

Advanced Windows Debugging by M. Hewardt and D. Pravat:

Implicit CoInitializeSecurity with EOAC_APPID (p. 365)

ole32!g* variables (p. 366)

Client callback interfaces make gives a client a server role (p. 367)

DCOM failure logging registry keys (p. 367)

Concurrent Programming on Windows by J. Duffy:

Software Factories by J. Greenfield, et. al.:

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Stiil busy to read last 3 books. Hope to resume them tomorrow.

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

Linking order (pp. 113 - 114) - I need to check this with MS linker, how many passes does it have

Symbol clash as undetected multiple definitions (pp. 117 - 118) - I need to check this with MS linker

Symbol localization (suppressing exports) EDITBIN and LIB on Windows (p. 118) - Never used EDITBIN before. Will try

The possibility of crash due to linker mismatch or error (p. 120)

How to determine a compiler version from an object or library file? (pp. 120 - 122) - It looks like .obj files compiled by MS compiler contain all compilation options and the following string, for example: -compiler:”c:\Program Files (x86)\Microsoft Visual Studio 8\VC\bin\c2.dll”

Advanced Windows Debugging by M. Hewardt and D. Pravat:

Breaking the code path technique: freezing processes one by one and checking if this freezes the client (p. 355)

AccessCheck in RPCSS, machine access check -> machine launch check -> component-specific check for DCOM (pp. 357 - 361)

RPCSS service -> DcomLaunch service (p. 362)

Concurrent Programming on Windows by J. Duffy:

Software Factories by J. Greenfield, et. al.:

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Didn’t have time today. Will resume notebook on Friday.

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

An application as a set of processes (p. 103)

strace (Linux) and truss (Solaris) to trace OS calls (pp. 104 - 106) - On Windows we can trace API while debugging using WinDbg extensions: http://www.dumpanalysis.org/blog/index.php/2007/01/03/tracing-win32-api-while-debugging-a-process/. Process Monitor can also be used to trace API subsets like File and Registry I/O.

Compiler bugs (pp. 106 - 107)

Debugger and compiler incompatibilities (p. 107)

Link-time bugs (Chapter 9)

Missing symbols (pp. 112 - 113) - On Windows search strategies like Explorer Search and dumpbin can be useful for dynamic linking errors

Advanced Windows Debugging by M. Hewardt and D. Pravat:

The importance of lazy initialization and associated impersonation bugs (pp. 347 - 354)

Concurrent Programming on Windows by J. Duffy:

.NET programs are multithreaded from the start (gc runs on a separate thread) (p. 79) - We can see this in WinDbg when attaching it to a .NET app container

Thread as a virtual processor metaphor (p. 80)

Thread as an execution context (p. 81)

Non-local transfer of control; context switch, exception handling, hardware interrupt (borrows kernel stack), DPC and APC (pp. 84 - 85)

DPC and APC run in the context of the current thread (p. 85)

The differences of CLR (managed) threads: just additional CLR information is stored on per OS thread basis (p. 86)

Explicit threading: explicit thread creation (p. 87) with thread pool alternative (p.88)

Software Factories by J. Greenfield, et. al.:

Failure of enterprise data modeling during terminal era of 70s - 80s (p. 13)

Bifurcation of of software into personal and enterprise with the advent of PCs (pp. 15 - 16)

The notion of the phenomenon of cheaper products eventually displacing market leaders after improvement over time (p. 16)

Thick client: business rules on both tiers (p. 17)

The growth of packaged application industry and outsourcing after client-server era (pp. 18 - 21)

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

SE principles to counter SE constraints (cognitive, organizational and resource) (p. 2)

Software as a unique abstract structure (p. 7)

The notion of the intelligent behavioural metaphor (p. 7)

Inductive math-based methodology of theoretical software problems vs. deductive experiment-driven for empirical problems: both must be used for SE problems (1st Law of SE)  (p.

The need for mathematical modeling of software system architecture and software behaviours (p. 9)

- Dmitry Vostokov @ SoftwareGeneralist.com -

Resumed my reading notebook today after more than two week break caused by finishing my Windows Debugging book.

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

Deadlock roots (circular locking and protocol mismatches in produce-consumer scenarious (p. 93)

info thread command in GDB (0. 95), thread N command to switch current thread, info stack command - In WinDbg we use ~*kv in process dumps and !process 0 ff in kernel and complete kernel dumps. Beware of WOW64 though:
http://www.dumpanalysis.org/blog/index.php/2007/01/26/note-32-bit-stack-from-64-bit-dump/ and http://www.dumpanalysis.org/blog/index.php/2009/02/06/32-bit-stack-traces-from-x64-complete-dumps/

Threading analysis tools: Intel Thread Checker and Helgrind (pp. 96 - 98) 

Environment dependency as root cause for some problems (pp. 101 - 103)

Advanced Windows Debugging by M. Hewardt and D. Pravat:

The importance of system boundaries (pp. 338 - 340)

Logical error representations by subsystems (p. 341) 

Security descriptor for debugger created thread can have different DACL (pp. 346 - 347)

Security chapter is awesome - sheds light on what described in other Win32 security books

Concurrent Programming on Windows by J. Duffy:

Software Factories by J. Greenfield, et. al.:

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

I resume reading these books next Monday

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

How Profilers Work section (pp. 73 - 74) - good addition to How Debugger Work books. I think detouring can also used for profiling in real-time, like famous MOV EDI, EDI in Windows

2 core profiler areas: collecting and presenting data (p. 73)

profiler sampling (interrupt and collect stack traces) and instrumentation data collection approaches (p. 73) - The problem with sampling is that most threads wait during their life time, so it is more useful only for CPU-intensive apps

flat profile and call graph presentation techniques (p. 74)

Callgrind as part of Valgrind (pp. 81 - 82)

Advanced Windows Debugging by M. Hewardt and D. Pravat:

I resume this book reading from page 328 after very long pause.

!token -n (pp. 329 - 330) - command shows no impersonation if it fails to get one

SD pointer in an object header is a pseudo-pointer with the first 3 bits used for a different purpose (p. 331)

ISC/ASC client impersonation sequence for remote authentication (p. 335) - possible UML sequence diagram for my book Windows Internals Distilled

The importance of impersonation level in a token (pp. 337 - 338)

Concurrent Programming on Windows by J. Duffy:

Software Factories by J. Greenfield, et. al.:

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Didn’t have time today to read these 3 books because of the work on my next book ”Windows Debugging: Practical Foundations”

- Dmitry Vostokov @ SoftwareGeneralist.com -