Re: [gentoo-dev] [PATCH 3/3] dev-vcs/hub: migrate to go-module.eclass

[Michael Orlitzky <mjo@gentoo.org>]

On 9/12/19 1:45 PM, Alec Warner wrote:
> 
> Er, I'm fairly sure computer *science* has not conclusively proven that
> dynamic binaries are somehow superior to static binaries.
> 

Please don't make me work this hard ever again.

The principal of modularity in software design goes back to at least
1970. In Parnas's famous 1971 paper[0], he cites "Designing Systems
Programs" by Gauthier and Pont from 1970, and suggests that the idea
goes back even further, to at least 1967 (Balzer & Mealy). On the very
first page of Parnas' paper, he says

  The major advancement in the area of modular programming has been the
  development of coding techniques and assemblers which

    (1) allow one module to be written with little knowledge of the code
        in another module, and

    (2) allow modules to be reassembled and replaced without reassembly
        of the whole system.

The second item has a clear interpretation in terms of static/dynamic
linking. Parnas concludes that

  ...one begins with a list of difficult design decisions or design
  decisions which are likely to change. Each module is then designed to
  hide such a decision from the others.

If you statically link to a library, then none of its implementation
details are hidden from you -- you need to "reassemble" your program
whenever the library changes.

If we jump way forward to 1979, the SICP[1] is basically a thousand-page
treatise on abstraction. But not only at one level of computation:

  Well-designed computational systems, like well-designed automobiles or
  nuclear reactors, are designed in a modular manner, so that the parts
  can be constructed, replaced, and debugged separately.

"Replaced" here is of course what I want to draw your attention to. But
also on the fact that abstraction and modularity don't just apply at one
level of software design and engineering -- it's a fractal. At the
lowest levels, we abstract machine code to assembler, assembler to
low-level languages, low-level languages to high-level languages,
high-level languages to functions and procedures, functions and
procedures to libraries, libraries to services, and services to
distributed APIs. The same principles that apply to a collection of
functions (a program) also apply to a collection of programs (an
operating system). The rule "don't copy and paste code" applies to your
linker just as much as it applies to the first program you wrote in
CS101, and for the same reasons.

As you commented on IRC, the cost in terms of man-power is that
someone's workload jumps from O(n) to O(m*n), because you have to
duplicate everything you do for every statically-linked dependency. And
you can find as much theory as you like on software modularity in papers
from the 1970s and 1980s, but the benefits are not only theoretical.
There's a mountain of empirical data that supports the theory. Some
choice quotes [2][3]:

  Poorly placed dependencies, especially those that link otherwise
  independent modules, may result in a cascade of unwanted and hard-to-
  detect indirect interactions. Our results suggest that purposeful
  actions to reduce such "rogue dependencies" can be effective (the
  redesign of Mozilla reduced propagation cost by over 80%).

  Our results confirm the existence of a relationship between component
  modularity and design evolution that is both statistically significant
  and large in magnitude... Tightly-coupled components are... less
  adaptable  via the processes of exclusion or substitution; they are
  more likely to  experience "surprise" dependency additions unrelated
  to new functionality, implying that they demand greater maintenance
  efforts;  and they are harder to augment, in that the mix of new
  components is  more modular than the legacy design.

The only difference between static linking and copy/pasting chunks of
code around is in who pays the price. If the programmer copies & pastes
code into his own program, he will eventually have to deal with the
mess. On the other hand, it he forces his program to be statically
linked, then it is the end user who is harmed by his decision.

In any case, the theory says that modularity is superior, and the
empirical data confirm it. The fact that you won't find a paper saying
"dynamic linking is better than static linking" is somewhat beside the
point, and only muddies the water. Linking is just one specific instance
of a problem that was solved 40 years ago.


[0]
https://www.win.tue.nl/~wstomv/edu/2ip30/references/criteria_for_modularization.pdf

[1] https://web.mit.edu/alexmv/6.037/sicp.pdf

[2] https://pubsonline.informs.org/doi/10.1287/mnsc.1060.0552

[3] http://www.hbs.edu/research/pdf/08-038.pdf