Software Generalist

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

VTune profiling (pp. 82 - 83) - there are also Visual Studio profiler http://msdn.microsoft.com/en-us/magazine/cc337887.aspx, Visual Studio Team System profiling tools and Windows kernel profiler kernrate

TBB, OpenMP, MPI and Google MapReduce (p. 87) - I have a book about Intel TBB *Threading Building Blocks”, should read or browse it soon

Mimic the scheduling (p. 90) - One technique I found useful is when you suspect two separate racing functions we can patch function prolog with a return to see if the problem is gone

Using log file to localize race conditions (p. 91 - 93) - In Citrix we use CDF tracing based on ETW (http://msdn.microsoft.com/en-us/library/aa468736.aspx) as application-, service- and system-wide logging mechanism. It records TID and PID for every trace statement. All debug statements are actually written in a production environment if we choose to record certain or all categories of them:
http://support.citrix.com/article/ctx117426

stderr should be unbuffered (p. 91)

debugger tracepoints (p. 93) - stop, record data, resume - similar techniques exist in WinDbg/CDB/KD family of debuggers

Advanced Windows Debugging by M. Hewardt and D. Pravat:

Concurrent Programming on Windows by J. Duffy:

Software Factories by J. Greenfield, et. al.:

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Didn’t have time today to read these 3 books because of the work on my next book ”Windows Debugging: Practical Foundations”

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

How Profilers Work section (pp. 73 - 74) - good addition to How Debugger Work books. I think detouring can also used for profiling in real-time, like famous MOV EDI, EDI in Windows

2 core profiler areas: collecting and presenting data (p. 73)

profiler sampling (interrupt and collect stack traces) and instrumentation data collection approaches (p. 73) - The problem with sampling is that most threads wait during their life time, so it is more useful only for CPU-intensive apps

flat profile and call graph presentation techniques (p. 74)

Callgrind as part of Valgrind (pp. 81 - 82)

Advanced Windows Debugging by M. Hewardt and D. Pravat:

I resume this book reading from page 328 after very long pause.

!token -n (pp. 329 - 330) - command shows no impersonation if it fails to get one

SD pointer in an object header is a pseudo-pointer with the first 3 bits used for a different purpose (p. 331)

ISC/ASC client impersonation sequence for remote authentication (p. 335) - possible UML sequence diagram for my book Windows Internals Distilled

The importance of impersonation level in a token (pp. 337 - 338)

Concurrent Programming on Windows by J. Duffy:

Software Factories by J. Greenfield, et. al.:

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Didn’t have time today to read these 3 books because of the work on my next book “Windows Debugging: Practical Foundations”

- Dmitry Vostokov @ SoftwareGeneralist.com -

When will OS vendors introduce slotting fees for ISV to sell favorable distribution of quantum slots for thread scheduling of ISV forthcoming products? May be something is already hidden under the hood? When software becomes more complex in the future but hardware becomes less (this is a different topic to discuss because we have 3 possible scenarios, less, more and equal) we expect software slotting fees to appear.

- Dmitry Vostokov @ SoftwareGeneralist.com -�

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

Performance bugs include slow and fast program runs (p. 63) - Do I need to include them in my collection of patterns?

Upfront performance analysis before debugging performance problems (p. 64) - can be delegated to troubleshooting engineer

Measuring time (pp. 64 - 65) - Complex products sometimes write time for every trace statement like Citrix Common Diagnostic Facility (CDF) technology based on Event Tracing for Windows (ETW). Also memory dumps usually have various times saved inside. 

Varying CPU clock speed (p. 67)

Disconnect computer from network for clean testing to remove possible network influence on execution time (p. 68)

Concurrent Programming on Windows by J. Duffy:

Spin waiting causes CPU spikes and blocks threads (pp. 63 - 64) - See my real life patterns Dispatch Level Spin and Affine Thread http://www.dumpanalysis.org/blog/index.php/2008/01/25/crash-dump-analysis-patterns-part-44/ and http://www.dumpanalysis.org/blog/index.php/2008/06/27/crash-dump-analysis-patterns-part-68/

Busy-waiting for fine-grained concurrency to avoid scheduling costs (p. 65) 

Continuation passing style (CPS) as alternative to waiting - packaging the rest of computation into some code that we pass to an interface that runs the continuation when the wait condition is satisfied (p. 66)

(Event, predicate) -> race condition -> monitor (critical region, conditional variable) (p. 68)

Structured parallelism - coordination abstraction (cobegin, forall, futures) (p. 70)

Message passing (encapsulation of inter-thread states) (p. 71)

Shared memory as an optimization to message passing (p. 71)

Coordination and Concurrency Runtime - first-class message passing (MS Robotics SDK) (p. 73)

Software Factories by J. Greenfield, et. al.:

Didn’t have time to read today

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Didn’t have time to read today

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

Data structure instrumentation (pp. 54-55) - I devised a visual method called Colometric Computer Memory Dating http://www.dumpanalysis.org/blog/index.php/2008/04/16/computer-colometric-memory-dating-part-1/

Too much detailed data from memory profiler (p. 55) - This is true indeed in the case of UMDH or user mode stack trace db on Windows

Memory usage increase might also be because of not cleared buffers or containers after functional iterations (p. 61) - I might need to investigate this on Windows to add to my pattern collection

Concurrent Programming on Windows by J. Duffy:

Peterson’s algorithm simplifies DD’s (pp. 53 - 54)

Lamport’s Bakery algorithm, thread failure in critical region  doesn’t destroy liveness (pp. 54 - 55)

Most modern synchronization primitives are built using CAS, compare and swap or interlocked instructions  (p. 57)

Software concurrency algorithms may not work because of compiler optimizations and hardware read/write reordering (pp. 58 - 60)

Causal thread dependence and state dependence (p. 62)

Software Factories by J. Greenfield, et. al.:

Tools lag platforms (p. 4) - True also for T&D tools

Increased stakeholder expectations (pp. 4 - 5) - Does it affect customer expectations?  Same limits of troubleshooters pool. They are overwhelmed too, like developers

Agile development doesn’t scale (p. 5)

The need to encapsulate knowledge as reusable assets (p. 6) - The same need for T&D. Encapsulation of T&D patterns in T&D tools

Web services as progress in packaging and interfacing (p. 6) - Are Web services the nexte step for T&D tools? There are some companies that do live debugging remotely

Aspect-oriented methods to incorporate contextual into functional (p. 6) - Can we use this for T&D tools?  I need to elaborate on this: eliminate the need to rewrite T&D tools 

Software development challenges (p. 7) - Great parallels (word-to-word) with software troubleshooting challenges 

An idea of a fictitious company to show adoption of methods (p. 7) - This might be useful idea to borrow for a fictitious software support centre

Batch era (pp. 9 - 11) - Are we still in batch era of software troubleshooting?

Multiple islands of data (p. 10) - Good metaphor

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

Didn’t have time to read today

- Dmitry Vostokov @ SoftwareGeneralist.com -

Facts are actual states of affairs, worldly correlates to true propositions that make them true. (*) Suppose that a piece of code that you are reviewing contains a security flaw like a local buffer overrun. Is this security defect factual? According to the definition of a fact it is not. The proposition about buffer overrun can be true from a reviewer’s logic but could never happen in reality. This is further complicated by a potential simulability of that piece of code that might predict all possible facts about its execution on some machine.

(*) The Oxford Companion to Philosophy, 2nd edition, p. 287 

- Dmitry Vostokov @ SoftwareGeneralist.com -

What is the amount of pages you allocate to read in one go? Software background suggests to use 2-page chunks, 10-page chunks and 16-page chunks. I personally prefer 10 page chunks or 5 lists per book daily. Then it is manageable to read at least 5 books in parallel.

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

Strategy of memory profiling (p. 45) - good summary of a general approach broken down in several steps.

The need to have an estimated and / or expected memory usage value (p. 45) - customers always ask me the question whether the certain amount of VM consuption is normal

Phases of execution and measurements at phase boundaries (p. 47)

Memory growth as a function of input data (linear, logarithmic, quadratic, …, exponential) (p. 48)

Stop points (pp. 48 - 49) - I also recommend to dump process memory to a memory dump file at regular intervals.

top tool from Cygwin (p. 49)

Using Task Manager (pp. 50 - 51) - Actually there are several memory columns for processes and it is important to choose the most appropriate. For example, Commit Size also includes memory allocated in page file as well. The following picture shows various columns in Vista: 

Deallocation not releasing memory to OS (pp. 51 - 52) - On Windows it might not decrease the size of VM but might decrease the size of committed memory and reduce page file usage. I need to test this: writing this down in my hardback Idea: Blogger’s notebook. Also on Windows there can be several separate process heaps with their own lifetime semantics. 

Memory profilers: Massif, AQtime and mpatrol (pp. 53 - 54) - On Windows you can use Gflags and select user mode stack trace database and then use WinDbg: http://www.dumpanalysis.org/blog/index.php/2007/08/06/crash-dump-analysis-patterns-part-20a/

Concurrent Programming on Windows by J. Duffy:

Association of a critical region with static lexical scope (p. 43)

Orphaned lock (p. 45) - I have seen it many times. Looks like good candidate to be included in my crash dump analysis patterns.

Coarse- vs. fine-grained locks, false sharing (pp. 46 - 47)

CAS instructions (compare and swap) (p. 48)

Strict alternation algorithm - critical region implementation (pp. 49 - 50)

Dekker’s and Dijkstra’s algorithm - critical region implementation (pp. 50 - 53)

Out-of-order execution breaks DD’s algorithm (p. 53)

Software Factories by J. Greenfield, et. al.:

Problem: small market for vertical focus and domain knowledge concentration in end-user organizations (p. xviii)

Ideal product built by an individual is an exception (pp. xviii - xix) - I think this depends on the project size an complexity.

Tools help with mechanistic tasks (p. xix)

Generative Programming book (p.  xxiii) - I need to finish reading it, I started 5 years ago…

Tools automate and help to focus on creativity (p. xxvii) - T&D tools help in focusing on creative part of T&D  

Architecturally driven and contextually aware tools for specific domains (p. xxix)

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

I bought this book last year and started reading now to get the most possible broad overview of software and related disciplines.

Historical programming-language-centered approach to SE (p. xxxvi) - I propose a memory-centered approach in my forthcoming books about Memoretics and Memiotics including a foundational philosophy called Memoidealism where Software is an emergent and predictable artifact.

Architecture of the book (p. xxxix) - concept and example of a book architecture (especially for a foundational book where SE can be templetized to a different discipline)

Broad multidisciplinary foundations of software rather than just computer science (p. xlix)

The author plans to apply metaphors from other disciplines - very much like I started to do with philosophy and physics. I also coined the term metaphorical bijection to describe two-way application of metaphors, for example, from physics to software and from software to physics. 

- Dmitry Vostokov @ SoftwareGeneralist.com -

Recall that not so long ago slavery was morally acceptable and it took courage for some to step back and question that. (*) My point here is that we currently do not question what we do to software but in the future killing a computational process or even attaching a debugger to it would definitely cause certain moral questions to arise. So we should question what other contemporaries consider obvious and take for granted and reconcile new answers with future computational perspectives.

(*) Philosophy, by S. Law, p. 15-16

- Dmitry Vostokov @ SoftwareGeneralist.com -

John Earman was one of early proponents to re-cast philosophical questions about space and time as mathematical conjectures within space-time mathematical language. (*) Any computation is some kind of abstract space of moving instruction pointers against memory spaces guided by discrete time. Therefore, a software program can be considered as some space too. All possible software programs can be aligned on another abstract space dimension, for example, consider all possible variations of a memory space coding possible software programs.

(*) The Oxford Companion to Philosophy, 2nd edition, p. 225

- Dmitry Vostokov @ SoftwareGeneralist.com -