Software Generalist

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

An application as a set of processes (p. 103)

strace (Linux) and truss (Solaris) to trace OS calls (pp. 104 - 106) - On Windows we can trace API while debugging using WinDbg extensions: http://www.dumpanalysis.org/blog/index.php/2007/01/03/tracing-win32-api-while-debugging-a-process/. Process Monitor can also be used to trace API subsets like File and Registry I/O.

Compiler bugs (pp. 106 - 107)

Debugger and compiler incompatibilities (p. 107)

Link-time bugs (Chapter 9)

Missing symbols (pp. 112 - 113) - On Windows search strategies like Explorer Search and dumpbin can be useful for dynamic linking errors

Advanced Windows Debugging by M. Hewardt and D. Pravat:

The importance of lazy initialization and associated impersonation bugs (pp. 347 - 354)

Concurrent Programming on Windows by J. Duffy:

.NET programs are multithreaded from the start (gc runs on a separate thread) (p. 79) - We can see this in WinDbg when attaching it to a .NET app container

Thread as a virtual processor metaphor (p. 80)

Thread as an execution context (p. 81)

Non-local transfer of control; context switch, exception handling, hardware interrupt (borrows kernel stack), DPC and APC (pp. 84 - 85)

DPC and APC run in the context of the current thread (p. 85)

The differences of CLR (managed) threads: just additional CLR information is stored on per OS thread basis (p. 86)

Explicit threading: explicit thread creation (p. 87) with thread pool alternative (p.88)

Software Factories by J. Greenfield, et. al.:

Failure of enterprise data modeling during terminal era of 70s - 80s (p. 13)

Bifurcation of of software into personal and enterprise with the advent of PCs (pp. 15 - 16)

The notion of the phenomenon of cheaper products eventually displacing market leaders after improvement over time (p. 16)

Thick client: business rules on both tiers (p. 17)

The growth of packaged application industry and outsourcing after client-server era (pp. 18 - 21)

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

SE principles to counter SE constraints (cognitive, organizational and resource) (p. 2)

Software as a unique abstract structure (p. 7)

The notion of the intelligent behavioural metaphor (p. 7)

Inductive math-based methodology of theoretical software problems vs. deductive experiment-driven for empirical problems: both must be used for SE problems (1st Law of SE)  (p.

The need for mathematical modeling of software system architecture and software behaviours (p. 9)

- Dmitry Vostokov @ SoftwareGeneralist.com -

Comments in italics are mine and express my own views, thoughts and opinions

Developer’s Guide to Debugging by M. Wloka, et al.:

Strategy of memory profiling (p. 45) - good summary of a general approach broken down in several steps.

The need to have an estimated and / or expected memory usage value (p. 45) - customers always ask me the question whether the certain amount of VM consuption is normal

Phases of execution and measurements at phase boundaries (p. 47)

Memory growth as a function of input data (linear, logarithmic, quadratic, …, exponential) (p. 48)

Stop points (pp. 48 - 49) - I also recommend to dump process memory to a memory dump file at regular intervals.

top tool from Cygwin (p. 49)

Using Task Manager (pp. 50 - 51) - Actually there are several memory columns for processes and it is important to choose the most appropriate. For example, Commit Size also includes memory allocated in page file as well. The following picture shows various columns in Vista: 

Deallocation not releasing memory to OS (pp. 51 - 52) - On Windows it might not decrease the size of VM but might decrease the size of committed memory and reduce page file usage. I need to test this: writing this down in my hardback Idea: Blogger’s notebook. Also on Windows there can be several separate process heaps with their own lifetime semantics. 

Memory profilers: Massif, AQtime and mpatrol (pp. 53 - 54) - On Windows you can use Gflags and select user mode stack trace database and then use WinDbg: http://www.dumpanalysis.org/blog/index.php/2007/08/06/crash-dump-analysis-patterns-part-20a/

Concurrent Programming on Windows by J. Duffy:

Association of a critical region with static lexical scope (p. 43)

Orphaned lock (p. 45) - I have seen it many times. Looks like good candidate to be included in my crash dump analysis patterns.

Coarse- vs. fine-grained locks, false sharing (pp. 46 - 47)

CAS instructions (compare and swap) (p. 48)

Strict alternation algorithm - critical region implementation (pp. 49 - 50)

Dekker’s and Dijkstra’s algorithm - critical region implementation (pp. 50 - 53)

Out-of-order execution breaks DD’s algorithm (p. 53)

Software Factories by J. Greenfield, et. al.:

Problem: small market for vertical focus and domain knowledge concentration in end-user organizations (p. xviii)

Ideal product built by an individual is an exception (pp. xviii - xix) - I think this depends on the project size an complexity.

Tools help with mechanistic tasks (p. xix)

Generative Programming book (p.  xxiii) - I need to finish reading it, I started 5 years ago…

Tools automate and help to focus on creativity (p. xxvii) - T&D tools help in focusing on creative part of T&D  

Architecturally driven and contextually aware tools for specific domains (p. xxix)

Software Engineering Foundations: A Software Science Perspective, by Y. Wang:

I bought this book last year and started reading now to get the most possible broad overview of software and related disciplines.

Historical programming-language-centered approach to SE (p. xxxvi) - I propose a memory-centered approach in my forthcoming books about Memoretics and Memiotics including a foundational philosophy called Memoidealism where Software is an emergent and predictable artifact.

Architecture of the book (p. xxxix) - concept and example of a book architecture (especially for a foundational book where SE can be templetized to a different discipline)

Broad multidisciplinary foundations of software rather than just computer science (p. xlix)

The author plans to apply metaphors from other disciplines - very much like I started to do with philosophy and physics. I also coined the term metaphorical bijection to describe two-way application of metaphors, for example, from physics to software and from software to physics. 

- Dmitry Vostokov @ SoftwareGeneralist.com -

John Earman was one of early proponents to re-cast philosophical questions about space and time as mathematical conjectures within space-time mathematical language. (*) Any computation is some kind of abstract space of moving instruction pointers against memory spaces guided by discrete time. Therefore, a software program can be considered as some space too. All possible software programs can be aligned on another abstract space dimension, for example, consider all possible variations of a memory space coding possible software programs.

(*) The Oxford Companion to Philosophy, 2nd edition, p. 225

- Dmitry Vostokov @ SoftwareGeneralist.com -

The subtitle of this blog has changed from “All you need to know to become a successful software engineer” to “Connecting Software with Engineering, Science, Philosophy and Religion” to reflect the shift in content towards the broader spectrum of topics since its foundation almost a year ago. The original idea to provide short independent survey articles about various aspects of software engineering now becomes one of categories of posts on this blog.

- Dmitry Vostokov @ SoftwareGeneralist.com -

This is a new book planned by OpenTask with the some preliminary details:

The Variety of Software: The Richness of Computation (ISBN: 978-1906717544)

- Dmitry Vostokov @ SoftwareGeneralist.com -

The same process suddenly reveals a different law (*). Although in the preceding definition the notion of a process should be taken generally, in the software world it often happens when a running process or an operating system suddenly exhibits all sorts of strange, sudden and unexplainable behaviour after some time or when running in a different environment. Here we can consider an environment as a sort of law under which a process operates (Chambers) or a process output and interaction as a law (Babbage). As a consequence of this we can never know whether our program crashes or hangs in the future.

(*) The Oxford Companion to Philosophy, 2nd edition, p. 76

- Dmitry Vostokov @ SoftwareGeneralist.com -

Book abbreviations correspond to titles on the picture from this link:

Cooperative Multireading Revisited and Started

POSA2: Widespread use of non-OO algorithmic decomposition in network and concurrent systems.

POSA5: Evolution in modeling and design. Design illusion.

AMP: Mutual exclusion (both excluded) problem with deadlock-freedom (if both want, one eventually succeeds), starvation-freedom (lockout-freedom, one eventually succeeds) and waiting properties.

EIP: Patterns: validation of understanding, reference and common vocabulary. Pattern realization in specific environment. Pattern cohesiveness. 

MM: Detaching threads in pthreads is similar to closing the last handle in Win32. In pthreads it is possible to create detached threads.

SC: Creativity: breaking traditional patterns. Dichotomy of two brain hemispheres.

ASS: ASAP is close to NOW. One-stop shopping.

SM: The necessity of impact analysis to determine potential ripple effects on existing environment/system.

CC: Software construction is central guaranteed activity. Source code is always up to date.

RP: Pedagogical patterns. Patterns: forces - tension - resolution by balancing forces with compromise - consequences

MES: A5/x algorithms to encrypt GSM communications

OOM: Fundamental split between OO databases and object-relational (OQL vs. SQL3).

TIS: Application- vs. System-provided recovery.

NAT: Addressing in OSI network layer 3 - networkId(flat|subnetted|hierarchical):hostId(flat)

PT: Generative grammar as fixed-size recipe for sentence construction. Platonic grammar as complete language description.

CW: E-mail and IM redirectors. Session hijacking.

PNA: Saltzer: application name -> node address -> point of attachment -> route. DNS as hierarchical distributed DB to replace host table.

CA: Simple RISC non-pipelining implementation: IF (instruction fetch) -> ID/RF (instruction decode/register fetch, fixed-field decoding) -> EX/EA (execution/effective address) -> MEM (memory access) -> WB (write-back).

GC: Pointer reachability from computation roots. The set of live heap nodes as transitive referential closure. Accessible nodes subsume live set. Direct counting vs. tracing.

CS: Ripple-carry of full adders and carry-propagation delay in adder networks.

- Dmitry Vostokov @ SoftwareGeneralist.com -

Book abbreviations correspond to titles on the picture from this link:

Cooperative Multireading Revisited and Started

POSA2: Loose-coupling.

POSA5: Design as a sort of problem solving vs. seeing the beauty. Newness.

AMP: View of concurrent concurrency problems as physical. Coordination protocol and proof by contradiction.

EIP: Vendor-independent vocabulary.

MM: In Win32 when main() exits other threads exit too, in Java it is not.

SC: Fundamental conflict between management and practitioners: discipline/control vs. freewheeling methods.

ASS: Support as a profit center.

SM: The definition of software includes all artifacts.

CC: Software construction is part of many software development activities but in different proportions.

RP: Remoting styles on top of existing middleware (COM+, EJB).

MES: Phreaking: phone hacking. 2600 magazine.

OOM: COM+ components as actors (need knowledge about collaborators). 

TIS: Atomicity and durability as recovery techniques.

NAT: OSI Layer 1 is independent from the physical media.

PT: Multilevel view of the language: Word as sentences of token letters. Linguist vs. computer scientist vs. formal-linguist. The latter’s view of the language as the set of sequences of symbols with identity.

CW: Screenscrapers and alternate on-screen input of characters. Polling vs. hooking for keyloggers.

PNA: Control bits are treated as opcodes and create more code complexity. Single PDU leads to proliferation of protocols.

CA: RISC: load/store architecture.

GC: Suspension or closure: pair (function, binding of names to locations). The ability of a function to return closure is based on heap allocation.

CS: Requirements for ADA were set by a committee of users, not language designers. Abstraction tools: package, type (state values) and generic.

- Dmitry Vostokov @ SoftwareGeneralist.com -

Book abbreviations correspond to titles on the picture from this link:

Cooperative Multireading Revisited and Started

POSA2: Inherent and accidental complexities. Tactical and Strategic aspects of concurrent software (low level OS mechanism vs. threading model).

POSA5: design metaheuristic: tractable forward model (essence of agile methods).

AMP: Shared counter and mutual exclusion.

EIP: Loose coupling: send and forget.

MM: Every Java program is automatically concurrent. Class extension vs. interface implementation. No preemption guarantee in Java threading. Sleep statements are useful for testing.

SC: Fun as a motivator for building software.

ASS: Stress in support is due to chaos.

SM: Software maintenance: modification after delivery.

CC: Code construction as a distinct activity that is guaranteed to be done.

RP: Technology projections for patterns.

MES: 802.11i is a security enhanced version of 802.11(a|b|g).

OOM: The influence of WIMP on OO languages.

TIS: Isolation hides other transactions and concurrency and requires concurrency control techniques.

NAT: Troubleshooting from bottom to top.

PT: Timeless structures for a mathematician vs. structure dynamics for a computer scientist.

CW: Stages of a crimeware attack: distribution, infection, execution, data retrieval, data misappropriation.

PNA: Logical vs. physical address spaces in networking (like in OS). Paradigm of connectionless.

CA: Pipelining, pipe segment, throughput, processor cycle.

GC: User program as mutator of the graph connectivity for heap structures.

CS: Static (scope of variables) and dynamic links (call scope) for procedure activation records.

- Dmitry Vostokov @ SoftwareGeneralist.com -

Book abbreviations correspond to titles on the picture from this link:

Cooperative Multireading Revisited and Started

POSA2: To Read: Mind and Nature: A Necessary Unity by Gregory Bateson. Grouping patterns into key problem areas. Pattern language - interwoven related patterns.

POSA5: Design metaheuristics. Tractable inverse models.

AMP: Future advances in speed will come through parallelism via shared memory. The model of concurrent asynchronous shared-memory computability. The importance of liveness property.

EIP: Asynchronous messaging is important in application integration.

MM: Communication overhead in multithreaded program

SC: software construction -> problem-solving -> creativity. Conflict between creativity and discipline.

ASS: Support department is like hospital emergency room. Transforming chaos to dance.

SM: Software Maintenance as a distinct discipline. Theory of software chance.

CC: The importance of understanding software construction practices beyond the working environment and language.

RP: Myth that remote communication is transparent to applications. Decoupling of operation invocation in space and time.

MES: 802.1x port based access control for wired and wireless.

OOM: OO language Simula pre-dates structured programming. OO is natural for simulation modeling.

TIS: Transactions important for management workflows. Transaction concept is abstract concept.

NAT: Layered methodology for network analysis.

PT: Parsing is a process of structuring a linear representation according to a given grammar. Grammatical inference - a reverse process.

CW: Crimeware: social engineering or security exploits.

PNA: ARPANET was built by people with OS background not communications.

CA: Cost-performance-power trade-off. ILP/TLP/DLP Instruction/Thread/Data-level parallelism (multicores).

GC: Myths of garbage collectors - overhead.

CS: Access time: time (receipt) - time (initiation). For sequential access half rotation time is important.

- Dmitry Vostokov @ SoftwareGeneralist.com -