Re: test10-pre7

Re: test10-pre7

[Vladislav Malyshkin]

Hi, Peter,
You can easily remove duplicates in object files without sorting. You
can just use a shell written function.
This is an example of such function (bash written).
It removes the duplicates from the argument and prints the result to
stdout.
No sorting used.

# This function removes duplicates from a string
remove_duplicates() {
 if [ $# -ne 1 ]; then
  echo "usage: remove_duplicates string" 1>&2
  exit 1
 fi
 str=""
 for tmpvar1 in $1 ; do
  flag_found=0
  for tmpvar2 in $str ; do
   if [ "${tmpvar1}" = "${tmpvar2}" ]; then
    flag_found=1
    break
   fi
  done
  if [ "${flag_found}" -eq 0 ]; then
    str="${str} ${tmpvar1}"
  fi
 done
 echo "${str}"
 return 0
}

# This is a usage example.
x="`remove_duplicates \"a b c d e a b c\"`"
echo "$x"

Vladislav

---Peter Samuelson wrote:
>> And it should make all this FIRST/LAST object file mockery a total
>> non-issue, because the whole concept turns out to be completely
>> unnecessary.
>>
>> Is there anything that makes this more complex than what I've
>> outlined above?
>
>One thing. The main benefit of $(sort), which I haven't heard you
>address yet, is to remove duplicate files. Think about 8390.o, and how
>many net drivers require it. There are two ways to handle this:
>
>  obj-$(CONFIG_WD80x3) += wd.o 8390.o
>  obj-$(CONFIG_EL2) += 3c503.o 8390.o
>  obj-$(CONFIG_NE2000) += ne.o 8390.o
>  obj-$(CONFIG_NE2_MCA) += ne2.o 8390.o
>  obj-$(CONFIG_HPLAN) += hp.o 8390.o

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[Peter Samuelson]

[Vladislav Malyshkin <mal@gromco.com>]
> You can easily remove duplicates in object files without sorting.
> You can just use a shell written function.

This is true.  That was something I forgot to mention.  I have looked
at that as well, and it strikes me as even more of a hack than the
solutions I mentioned: it is yet another external shell process for
each invocation of Rules.make (ie each directory).  As I said before,
though, one man's hack is another man's clean design, so whatever.

Your function is rather long; try this one instead (untested):

  remove_duplicates () {
    str='';
    for i; do
      case "$str " in *" $i "*) ;; *) str="$str $i" ;; esac
    done
    echo "$str"
  }

I still think anything outside the makefiles that's needed to organize
the build process is a hack.  That includes scripts/pathdown.sh (yes, I
do have a scheme to get rid of it) and 2.2.18 scripts/kwhich (yes, I
did propose a working alternative).  It doesn't include scripts/mkdep.c
(which must do a lot of work as efficiently as possible),
scripts/Configure et al (which are really standalone programs), or
scripts/split-include.c (which is really a continuation of Configure).

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[Vladislav Malyshkin]

Also, the function remove_duplicates can be written using make rules and
functions.
Using functions "foreach" "if" from make and comparison you can easily
build
a function remove_duplicates in make, no shell involved.
so instead of $(sort) your will have $(remove_duplicates)
written entirely in make.
Vladislav

Peter Samuelson wrote:

> [Vladislav Malyshkin <mal@gromco.com>]
> > You can easily remove duplicates in object files without sorting.
> > You can just use a shell written function.
>
> This is true.  That was something I forgot to mention.  I have looked
> at that as well, and it strikes me as even more of a hack than the
> solutions I mentioned: it is yet another external shell process for
> each invocation of Rules.make (ie each directory).  As I said before,
> though, one man's hack is another man's clean design, so whatever.
>
> Your function is rather long; try this one instead (untested):
>
>   remove_duplicates () {
>     str='';
>     for i; do
>       case "$str " in *" $i "*) ;; *) str="$str $i" ;; esac
>     done
>     echo "$str"
>   }
>
> I still think anything outside the makefiles that's needed to organize
> the build process is a hack.  That includes scripts/pathdown.sh (yes, I
> do have a scheme to get rid of it) and 2.2.18 scripts/kwhich (yes, I
> did propose a working alternative).  It doesn't include scripts/mkdep.c
> (which must do a lot of work as efficiently as possible),
> scripts/Configure et al (which are really standalone programs), or
> scripts/split-include.c (which is really a continuation of Configure).
>
> Peter

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[Linus Torvalds]

Ok, how about this approach? It only works for the case where we do not
have the kind of multiple stuff that drivers/net has, but hey, we don't
actually need to handle all the cases right now.

We can leave that for the future, as the configuration process is likely
to change anyway during 2.5.x, and the multiple object case may go away
entirely (ie the case of slhc and 8390 will become just a normal
configuration dependency: you'd have a "CONFIG_SLHC" entry that is
computed by the dependency graph at configuration time, rather than by the
Makefile at build time).

This is the simplest rule base that I could come up with that should work
for both SCSI and USB:

	# Translate to Rules.make lists.
	multi-used      := $(filter $(list-multi), $(obj-y) $(obj-m))
	multi-objs      := $(foreach m, $(multi-used), $($(basename $(m))-objs))
	active-objs     := $(sort $(multi-objs) $(obj-y) $(obj-m))

	O_OBJS          := $(obj-y)
	M_OBJS          := $(obj-m)
	MIX_OBJS        := $(filter $(export-objs), $(active-objs))

Does anybody see any problems with it? Basically, we're sidestepping the
sorting, because neither SCSI nor USB need it. Making the problem simpler
is always good.

Now, the above won't work for drivers/net, but I think it will work for
just about anything else. So let's just leave drivers/net alone for now.
Simplicity is good.

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[H. Peter Anvin]

Followup to:  <Pine.LNX.4.10.10010311018180.7083-100000@penguin.transmeta.com>
By author:    Linus Torvalds <torvalds@transmeta.com>
In newsgroup: linux.dev.kernel
> 
> Does anybody see any problems with it? Basically, we're sidestepping the
> sorting, because neither SCSI nor USB need it. Making the problem simpler
> is always good.
> 
> Now, the above won't work for drivers/net, but I think it will work for
> just about anything else. So let's just leave drivers/net alone for now.
> Simplicity is good.
> 

I was going to ask to what extent we genuinely need sorting, and if we
might be better off trying to eliminate that need as much as possible.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[Peter Samuelson]

[hpa]
> I was going to ask to what extent we genuinely need sorting, and if
> we might be better off trying to eliminate that need as much as
> possible.

We don't need sorting.  We need removing of duplicates.  The GNU make
sort function removes duplicates as a side effect, which is why we want
to use it.

As has been discussed, there are lots of ways to remove duplicates.
You can spawn a shell script, you can keep track of each "common" file
with a tristate make variable, you can play with deeply nested if
statements, and rumor has it you may be able to write a custom GNU make
function (though I have doubts, as I have tried this before and
couldn't get anything to work).

To Keith, Michael and me, the cleanest way to remove duplicates is
$(sort).  Since some object files must *not* be sorted, we came up with
a simple, readable way to declare that certain things had to come in a
certain order -- the idea being that most of the time it would not be
needed.  Linus disagrees that our solution is simple, readable or
otherwise desirable.  That's basically the whole issue in a nutshell.

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[H. Peter Anvin]

Peter Samuelson wrote:
> 
> To Keith, Michael and me, the cleanest way to remove duplicates is
> $(sort).  Since some object files must *not* be sorted, we came up with
> a simple, readable way to declare that certain things had to come in a
> certain order -- the idea being that most of the time it would not be
> needed.  Linus disagrees that our solution is simple, readable or
> otherwise desirable.  That's basically the whole issue in a nutshell.
> 

I would tend to agree with Linus on that.  If that's truly what you're
doing, it would be rather nonobvious.

But the question, perhaps, is when does ordering matter.  I'm a little
concerned about things highly dependent on link ordering.

	-hpa

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Linus's poll variation

[Lyle Coder]

Hello,
Is someone working on Linus's poll variation discussed in this list a week
ago?

Thanks
Lyle

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[Peter Samuelson]

[hpa]
> I would tend to agree with Linus on that.  If that's truly what
> you're doing, it would be rather nonobvious.

Well, ok, opinion vs. opinion.  The thing is, userspace code almost
*never* needs to care about link order -- and, not counting boot loader
magic, kernel code didn't care about it either until 2.3.16 or so
(debut of __initcall).  The reason link order suddenly became an issue
is that people started cutting out explicit lists of init code like
net/Space.c and relying on ld.  All in all that was a Good Thing -- it
made code more modular, etc.

My point here is that in most of the world, people are not accustomed
to caring about link order.  So I don't see why it's seen as such a
horrible thing to now say "ok link order is once again something you
ideally should not care about, *but* if there's a really good reason
otherwise, add your entry to LINK_FIRST."  Because the link-order-
*does*-matter paradigm is less than a year old, here.

> But the question, perhaps, is when does ordering matter.  I'm a
> little concerned about things highly dependent on link ordering.

Agreed.  And that is another point: we (Keith, Michael and I wrote all
this expecting link order to usually *not* matter -- LINK_FIRST was
just to cover corner cases.  Then we get Linus back saying the whole
drivers/scsi/ must be ordered *just so* (presumably so people don't
have to use boot flags for their multiple scsi host adapters) which is
either an unproven claim or, since it's Linus, a fiat.  (Which is OK,
don't get me wrong, better him than me making these decisions!)

And as for "rather nonobvious" -- it's not as if our stuff isn't
documented, at least.  Keith's patch includes several paragraphs added
to Documentation/kbuild/makefiles.txt, a file every kernel developer
should read anyway so that they know how to write correct makefiles.

Oh well.  Life goes on.  We can do fine without LINK_FIRST, but
possibly at the cost of increased cruft in the rest of the makefile
system.  (That depends on taste, I suppose.)

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[Peter Samuelson]

[Vladislav Malyshkin <mal@gromco.com>]
> Also, the function remove_duplicates can be written using make rules
> and functions.  Using functions "foreach" "if" from make and
> comparison you can easily build a function remove_duplicates in make,
> no shell involved.

Could you please write me this function?  I am curious to see how you
do it.

I am also a bit skeptical.  About 3 months ago, I thought it would be
possible to do this, so I spent a few hours fiddling around and reading
documentation.  I failed; nothing I tried worked.

> so instead of $(sort) your will have $(remove_duplicates) written
> entirely in make.

That would make me happy.

Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: test10-pre7

[Vladislav Malyshkin]

Hi, Peter

> [Vladislav Malyshkin <mal@gromco.com>]
> > Also, the function remove_duplicates can be written using make rules
> > and functions.  Using functions "foreach" "if" from make and
> > comparison you can easily build a function remove_duplicates in make,
> > no shell involved.
>
> Could you please write me this function?  I am curious to see how you
> do it.
>
> I am also a bit skeptical.  About 3 months ago, I thought it would be
> possible to do this, so I spent a few hours fiddling around and reading
> documentation.  I failed; nothing I tried worked.
>
> > so instead of $(sort) your will have $(remove_duplicates) written
> > entirely in make.
>
> That would make me happy.

Absense of recursive macros in make makes the problem not that clear.
An alternative may be if to put \n into variable value,
so the make commands will be automatically generated
---- like this
remove_duplicates = $(2) := ; $(foreach tmpvar1,$(1),$(2) += $$(if
$$(findstring $(tmpvar1),$$($(2))),,$(tmpvar1)); )
$(call remove_duplicates, x y a a c c a b c,ABCD)
--- in this example the variable ABCD is set to the string without
duplicates
but make seems does not allow \n as a value and does not understand
several assignments in one line, so
A = B ; C = D
does not work as expected.

So the best what we can get without using shell is below (the code and
usage example)
The function $(call remove_duplicates,string with duplicates)
removes the duplicates from the string.

# joins four strings
join4 = $(join $(join $(1),$(2)),$(join $(3),$(4)))
# generates indexes
numbers = $(foreach x4,0 1 2 3 4 5 6 7 8 9,\
$(strip $(foreach x3,0 1 2 3 4 5 6 7 8 9,\
$(strip $(foreach x2,0 1 2 3 4 5 6 7 8 9,\
$(strip $(foreach x1,0 1 2 3 4 5 6 7 8 9,\
$(strip $(if $(findstring $(call join4,$(x4),$(x3),$(x2),$(x1)),0000),,\
$(if $(word $(call join4,$(x4),$(x3),$(x2),$(x1)),$(1)),\
$(call join4,$(x4),$(x3),$(x2),$(x1)))))))))))))
# generates indexes
numbers1 = $(wordlist 1,$(words $(wordlist 2,$(words $(1)),$(1))),\
$(call numbers,$(1)))
# Remove duplicates from a line of up to 10000 words
remove_duplicates = $(strip $(firstword $(1)) \
$(foreach tmpvar,$(call numbers1,$(1)),\
$(if $(findstring \
$(word $(tmpvar),$(wordlist 2,$(words $(1)),$(1))),\
$(wordlist 1,$(tmpvar),$(1))),,\
$(word $(tmpvar),$(wordlist 2,$(words $(1)),$(1))))))

f := x x y a b c d x e y jj jj j2 j2 j2 j7
all:
    echo '$(call remove_duplicates,$(f))'



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Linus's poll variation

[Dan Kegel]

Lyle asked:
> Is someone working on Linus's poll variation discussed in this list a week ago? 

I think the interface still needs some discussion.
The interface Linus proposed is IMHO oriented towards ease of 
kernel implementation, and doesn't appear to be easy to use
for all applications.
cf. recent posts by John Gardiner Myers <jgmyers@netscape.com>
e.g. 
http://boudicca.tux.org/hypermail/linux-kernel/2000week44/0966.html
http://boudicca.tux.org/hypermail/linux-kernel/2000week45/0212.html
IMHO kqueue()/kevent() is closer to what the apps want.

Mike Jagdis, however, has done some work on speeding up the
existing poll() system call, and has an eye on implementing
the /dev/poll interface.  See
http://boudicca.tux.org/hypermail/linux-kernel/2000week45/0266.html
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/