The Linker Is not a Magical Program

From WikiContent

(Difference between revisions)
Jump to: navigation, search
Current revision (02:53, 19 July 2009) (edit) (undo)
 
(5 intermediate revisions not shown.)
Line 1: Line 1:
-
Depressingly often (happened to me again just before I wrote this), the view a programmer has of the process of going from source code to an executable in a compiled language is:
+
Depressingly often (happened to me again just before I wrote this), the view many programmers have of the process of going from source code to a statically linked executable in a compiled language is:
# Edit source code
# Edit source code
Line 8: Line 8:
Step 3 is, of course, the linking step. Why would I say such an outrageous thing? I've been doing tech support for decades, and I get the following questions again and again:
Step 3 is, of course, the linking step. Why would I say such an outrageous thing? I've been doing tech support for decades, and I get the following questions again and again:
-
# The linker says "abc" is an unresolved symbol.
+
# The linker says <code>def</code> is defined more than once.
-
# The linker says "def" is defined more than once.
+
# The linker says <code>abc</code> is an unresolved symbol.
# Why is my executable so large?
# Why is my executable so large?
Line 15: Line 15:
A linker is a very stupid, pedestrian, straightforward program. All it does is concatenate together the code and data sections of the object files, connect the the references to symbols with their definitions, pull unresolved symbols out of the library, and write out an executable. That's it. No spells! No magic! The tedium in writing a linker is usually all about decoding and generating the usually ridiculously over-complicated file formats, but that doesn't change the essential nature of a linker.
A linker is a very stupid, pedestrian, straightforward program. All it does is concatenate together the code and data sections of the object files, connect the the references to symbols with their definitions, pull unresolved symbols out of the library, and write out an executable. That's it. No spells! No magic! The tedium in writing a linker is usually all about decoding and generating the usually ridiculously over-complicated file formats, but that doesn't change the essential nature of a linker.
 +
 +
So let's say the linker is saying <code>def is defined more than once</code>. Many programming languages, such as C, C++, and D, have both declarations and definitions. Declarations normally go into header files, like:
 +
 +
extern int iii;
 +
 +
which generates an external reference to the symbol <code>iii</code>. A definition, on the other hand, actually sets aside storage for the symbol, usually appears in the implementation file, and looks like this:
 +
 +
int iii = 3;
 +
 +
How many definitions can there be for each symbol? As in the film ''Highlander'', there can be only one. So, what if a definition of <code>iii</code> appears in more than one implementation file?
 +
 +
// File a.c
 +
int iii = 3;
 +
 +
// File b.c
 +
double iii(int x) { return 3.7; }
 +
 +
The linker will complain about <code>iii</code> being multiply defined.
 +
 +
Not only can there be only one, there must be one. If <code>iii</code> only appears as a declaration, but never a definition, the linker will complain about <code>iii</code> being an unresolved symbol.
 +
 +
To determine why an executable is the size it is, take a look at the map file that linkers optionally generate. A map file is nothing more than a list of all the symbols in the executable along with their addresses. This tells you what modules were linked in from the library, and the sizes of each module. Now you can see where the bloat is coming from. Often there will be library modules that you have no idea why were linked in. To figure it out, temporarily remove the suspicious module from the library, and relink. The undefined symbol error then generated will indicate who is referencing that module.
 +
 +
Although it is not always immediately obvious why you get a particular linker message, there is nothing magical about linkers. The mechanics are straightforward; it's the details you have to figure out in each case.
 +
 +
By [[Walter Bright]]
 +
 +
This work is licensed under a [http://creativecommons.org/licenses/by/3.0/us/ Creative Commons Attribution 3]
 +
 +
 +
 +
Back to [[97 Things Every Programmer Should Know]] home page

Current revision

Depressingly often (happened to me again just before I wrote this), the view many programmers have of the process of going from source code to a statically linked executable in a compiled language is:

  1. Edit source code
  2. Compile source code into object files
  3. Something magical happens
  4. Run executable

Step 3 is, of course, the linking step. Why would I say such an outrageous thing? I've been doing tech support for decades, and I get the following questions again and again:

  1. The linker says def is defined more than once.
  2. The linker says abc is an unresolved symbol.
  3. Why is my executable so large?

Followed by "What do I do now?" usually with the phrases "seems to" and "somehow" mixed in, and an aura of utter bafflement. It's the "seems to" and "somehow" that indicate that the linking process is viewed as a magical process, presumably understandable only by wizards and warlocks. The process of compiling does not elicit these kinds of phrases, implying that programmers generally understand how compilers work, or at least what they do.

A linker is a very stupid, pedestrian, straightforward program. All it does is concatenate together the code and data sections of the object files, connect the the references to symbols with their definitions, pull unresolved symbols out of the library, and write out an executable. That's it. No spells! No magic! The tedium in writing a linker is usually all about decoding and generating the usually ridiculously over-complicated file formats, but that doesn't change the essential nature of a linker.

So let's say the linker is saying def is defined more than once. Many programming languages, such as C, C++, and D, have both declarations and definitions. Declarations normally go into header files, like:

extern int iii;

which generates an external reference to the symbol iii. A definition, on the other hand, actually sets aside storage for the symbol, usually appears in the implementation file, and looks like this:

int iii = 3;

How many definitions can there be for each symbol? As in the film Highlander, there can be only one. So, what if a definition of iii appears in more than one implementation file?

// File a.c
int iii = 3;
// File b.c
double iii(int x) { return 3.7; }

The linker will complain about iii being multiply defined.

Not only can there be only one, there must be one. If iii only appears as a declaration, but never a definition, the linker will complain about iii being an unresolved symbol.

To determine why an executable is the size it is, take a look at the map file that linkers optionally generate. A map file is nothing more than a list of all the symbols in the executable along with their addresses. This tells you what modules were linked in from the library, and the sizes of each module. Now you can see where the bloat is coming from. Often there will be library modules that you have no idea why were linked in. To figure it out, temporarily remove the suspicious module from the library, and relink. The undefined symbol error then generated will indicate who is referencing that module.

Although it is not always immediately obvious why you get a particular linker message, there is nothing magical about linkers. The mechanics are straightforward; it's the details you have to figure out in each case.

By Walter Bright

This work is licensed under a Creative Commons Attribution 3


Back to 97 Things Every Programmer Should Know home page

Personal tools