[bakefile-devel] Linking the whole archive.

[Scott Bailey]

Vacalv,

I am proposing adding a tag and am looking for feedback.


Background Information
----------------------------------

The project I'm working on has a multitude of pieces:

1. A number of generic cross-platform support libraries (static
libraries)
2. A single project specific library (a shared library with static
linkage)
3. A number of plugables (modules) that depend on items 1 & 2 (shared
libraries with dynamic linkage)
4. A number of executable that depend on items 1 & 2 and may load any
number of item 3s.


Items 2, 3, and 4 share a repository while item 1 is included via
'svn:exterals' and by including it's *.bkls via the other's (i.e item 2's
bkl includes item 1's bkl).

Ultimately, I want is to include item 1 inside item 2. Iitems 3 and 4 then
depend only on item 2 and are smaller by losing the static linkage of item
1. The alternative for me is to build all item 1 as a shared static; but I
would prefer to have a single shared static. Regardless, I'd have the same
problem as all of the item 1's are currently independantly built. This may
not make sense for most projects, but for mine it does.

Currently, I can do this in GNU's Makefiles (via the Bake ldflags and ldlibs
tags), but vc8's IDE requires a tag where no support currently exists
("UseLibraryDependencyInputs"). I have not deeply researched the xcode2
solution, but I expect some option exists. I have an OSX machine I can use
next week.

The GNU linker flags are '--whole-archive' and '--no-whole-archive'. Maybe
OSX's linker takes '-all_load' as the appropriate option.


Proposal
-------------

I am suggesting adding a tag to allow this, maybe call it "all-objects" or
something. This tag would only be applicable to dll and module rules.

In the gnu mode it would simply add:
<ldflags>-Wl,--whole-archive</ldflags>
<ldlibs>-Wl,--no-whole-archives</ldflags>

In MS projects that support it, it would use setAttribute to generate the
correct output.

Not sure what xcode implementation is like yet.



Comments?

[Vaclav Slavik]

Post by Scott Bailey
Ultimately, I want is to include item 1 inside item 2. Iitems 3 and 4
then depend only on item 2 and are smaller by losing the static
linkage of item 1. The alternative for me is to build all item 1 as a
shared static; but I would prefer to have a single shared static.

This text would be much easier to understand if you didn't confusingly
use the word "shared"... it wasn't until "shared static" that you don't
use the term "shared library" in its usual sense of a "DLL" on ELF
platforms :-/

Post by Scott Bailey
I am suggesting adding a tag to allow this, maybe call it
"all-objects" or something.

It should be something with "library" in its name, because of the
relation to <library> (e.g. <embed-library> or something)...

Post by Scott Bailey
Comments?

How are you going to handle the formats that don't support this?

Regards,
Vaclav

[Scott Bailey]

Post by Vaclav Slavik
This text would be much easier to understand if you didn't confusingly

Sorry.

Post by Vaclav Slavik

Post by Scott Bailey
I am suggesting adding a tag to allow this, maybe call it
"all-objects" or something.

It should be something with "library" in its name, because of the
relation to <library> (e.g. <embed-library> or something)...

I understand the need to link it to libraries. Just couldn't think of any
decent names. I like <embed-libraries>.

I'm not discussing a replacement for <library>, this is a modifier like
<lib-path>. I would prefer to have <embed-library> replace <library>, but
I'm certain that Bakefiles can't do this across all of it's platforms:
potentially different behaviors on each platform. No good. It's an all or
nothing tag.

Post by Vaclav Slavik

Post by Scott Bailey
Comments?

How are you going to handle the formats that don't support this?

This is probably not possible for msvs2003 and earlier. It's NON-trivial
for MS Makefiles; requiring a lot of effort for a largely unused feature
(my change, not MS makefiles). I think it is better to have Bakefile print
an error (e.g. "Bakefile does not support embedding libraries on this
platform") and let the user decide what to do from there (i.e. change their
bakefile or add support).

The other option is generating a tag that adds custom tag/value pair defines
for projects (e.g. xcode2 and msvs2005). This would allow me to get the
same thing done and more. Although it's more work.

[Vaclav Slavik]

Hi,

Post by Scott Bailey
I understand the need to link it to libraries. Just couldn't think of
any decent names. I like <embed-libraries>.
I'm not discussing a replacement for <library>, this is a modifier
like <lib-path>.

It doesn't make sense that way -- surely you don't want to embed _all_
libraries, including system ones? It sounds better to explicitly include
the libraries you need included with <embed-library> instead of
<library>. Or is that not possible with VC++ project files?

Vaclav

[Scott Bailey]

Post by Vaclav Slavik

Post by Scott Bailey
I'm not discussing a replacement for <library>, this is a modifier
like <lib-path>.

It doesn't make sense that way -- surely you don't want to embed _all_
libraries, including system ones? It sounds better to explicitly include
the libraries you need included with <embed-library> instead of
<library>. Or is that not possible with VC++ project files?

This modifier applies to <library> tag, not <sys-lib>. It should be
possible to do <embed-library> on a VC++ project. I would ABSOLUTELY prefer
this, but I'm not certain I can muster the hours necessary. Let me
investigate and think about it.

SB

[Scott Bailey]

Post by Scott Bailey

Post by Scott Bailey
I'm not discussing a replacement for <library>, this is a modifier

Post by Scott Bailey
like <lib-path>.

It doesn't make sense that way -- surely you don't want to embed _all_
libraries, including system ones? It sounds better to explicitly include
the libraries you need included with <embed-library> instead of
<library>. Or is that not possible with VC++ project files?

This modifier applies to <library> tag, not <sys-lib>. It should be
possible to do <embed-library> on a VC++ project. I would ABSOLUTELY prefer
this, but I'm not certain I can muster the hours necessary. Let me
investigate and think about it.

xcode2's linker flag is '-all_load'. There is a '-noall_load' flag, but it
is obsolete and does nothing. I had hoped it would perform as the GNU
linker's '--no-whole-archive'.

While I enjoy the technical challenge of implementing a per library tag
(i.e. <embed-library>), it is more work than I can justify.

I have a perfectly acceptable method, using <ldflags> and <ldlibs>, in
xcode2 AND gnu, but no method for msvs2005prj. Ugh.

I'm done thinking about this for today. Comments?

[Scott Bailey]

Vaclav,

Per library embedding for VC2005 is not worthwhile. It's easy to do what I
need in Linux and OSX with <ldflags>.

So, I will add something like:

if cfg._embed_libs:
tool.setAttribute("UseLibraryDependencyInputs", bool2vcstr(True))

to the "VCLinkerTool" section generated inside msvs200xprj.py's
buildLinkerToolElement().

But what should trigger setting the bool? My first impulse is to add a tag
(e.g <embedlibs/>), but maybe it is better to set a variable (e.g.
EMBED_LIBS) -- much like BUILDDIR or MSVS_PROJECT_FILE.

Which is more desireable, or is there a better way?

Thanks,
SB

Post by Scott Bailey

Post by Scott Bailey
I'm not discussing a replacement for <library>, this is a modifier

Post by Vaclav Slavik

Post by Scott Bailey
like <lib-path>.

It doesn't make sense that way -- surely you don't want to embed _all_
libraries, including system ones? It sounds better to explicitly include
the libraries you need included with <embed-library> instead of
<library>. Or is that not possible with VC++ project files?

This modifier applies to <library> tag, not <sys-lib>. It should be
possible to do <embed-library> on a VC++ project. I would ABSOLUTELY prefer
this, but I'm not certain I can muster the hours necessary. Let me
investigate and think about it.

xcode2's linker flag is '-all_load'. There is a '-noall_load' flag, but it
is obsolete and does nothing. I had hoped it would perform as the GNU
linker's '--no-whole-archive'.
While I enjoy the technical challenge of implementing a per library tag
(i.e. <embed-library>), it is more work than I can justify.
I have a perfectly acceptable method, using <ldflags> and <ldlibs>, in
xcode2 AND gnu, but no method for msvs2005prj. Ugh.
I'm done thinking about this for today. Comments?

[Vaclav Slavik]

Post by Scott Bailey
tool.setAttribute("UseLibraryDependencyInputs",
bool2vcstr(True))

Note that all the official documentation I could find makes me think
that the behavior you want is just a side-effect of this setting, that
it is meant to do something else
(http://msdn.microsoft.com/en-us/library/024awkd1(VS.80).aspx):

Use Library Dependency Inputs

In a large project, when a dependent project produces
a .lib file, incremental linking is disabled. If there
are many dependent projects that produce .lib files,
building the application can take a long time. When this
property is set to Yes, the project system links in
the .obj files for .libs produced by dependent projects,
thus enabling incremental linking.

Post by Scott Bailey
Which is more desireable, or is there a better way?

A tag.

Vaclav